OpenChat 3.5 Seriously Killed ChatGPT (No, really!)

OpenChat 3.5 is the first 7B model that achieves comparable results
with ChatGPT!

OpenChat 3.5 (8192 CL) is released by Alignment Lab AI, and it (1) overcomes the challenges of larger models such as ChatGPT, (2) provides comparable performance against much bigger models, (3) is served with vLLM out-of-the-box, (4) can run on a consumer GPU with 24GB RAM.

Join our next cohort: Full-stack GenAI SaaS Product in 4 weeks!

In this article, I’ll walk you through:

• The context and explain what’s novel about the OpenChat framework
• The initial set-up with Hugging Face’s pipelines() to run simple experiments with OpenChat 3.5
• Serving OpenChat 3.5 with vLLM for application development
• How API server handle requests to generate responses

I’m running everything on Windows WSL2 with Nvidia RTX 4090 GPU.

The Problem with Supervised Fine Tuning and Reinforcement Learning From Human Feedback

Supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT) come with significant limitations:

• The purpose of SFT is to enhance LLMs’…