# The Two Fundamental Methods Shaping LLM Behavior How reinforcement learning and supervised fine-tuning make large language models more usable - **Source:** [O'Reilly Radar](https://www.oreilly.com/radar) - **Published:** 2026-08-12 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/0de4314a-7afc-4cf4-a50f-b6719c45a74f ## The purpose of post-training Post-training processes were developed to bridge the fundamental usability gap in large language models (LLMs). These processes make models behave in specific ways, producing more consistent and reliable outputs. Post-training is crucial for adapting a model's raw capabilities to specific tasks and interaction styles. ## What is reinforcement learning (RL)? Reinforcement learning enables models to learn through trial and error. The model generates a response to input, which is then graded with a reward signal. Positive rewards strengthen behaviors while negative ones weaken them. This method is particularly effective for tasks with objectively measurable correctness like code generation or math problems. ## How supervised fine-tuning (SFT) works Supervised fine-tuning operates on the principle of teaching by showing correct behavior examples. The model learns by imitating these examples. SFT is especially useful for generating natural and helpful responses in human interactions. This technique played a major role in developing models like ChatGPT. ## The role of verifiers The simplest way to provide reward signals in RL is through automated verifiers. These are functions that check whether generated code compiles or mathematical solutions are correct. Verifiers are fast, cheap, and reliable within their domains but only work for tasks that can be defined programmatically. ## Human feedback (RLHF) Humans can provide strong reward signals for subtle preferences hard to encode programmatically. However, involving humans in every training datapoint isn't practical. RLHF (Reinforcement Learning from Human Feedback) solves this by training a reward model that mimics human evaluations. ## How reward models are trained Reward models are typically trained using pairwise comparisons. Users are shown two model outputs and asked 'Which is better?' This yields more consistent results than absolute ratings. InstructGPT generated 200K-1.2M comparisons from ~33K prompts by ranking 4-9 outputs per prompt. ## LLMs as judges RLAIF (Reinforcement Learning from AI Feedback) uses LLMs as reward models. This approach scales better than human evaluation but inherits the judge model's biases. For example, a judge favoring verbose answers might train the model to be unnecessarily wordy. ## Efficient data labeling strategies InstructGPT's RLHF implementation showed labelers 4-9 different model outputs per prompt to rank. This generated 6-36 pairwise comparisons from a single ranking. Approximately 33,000 prompts yielded between 200,000 to 1.2 million comparisons for training the reward model. ## Importance of reward model quality During RL, the model optimizes against the reward model's scores. Therefore, better reward models produce more human-aligned models. A high-quality reward model is critical for preventing undesirable behaviors in the trained model. ## Advantages of comparative evaluations Using pairwise comparisons instead of absolute ratings (like 1-5 stars) significantly improves consistency in human evaluations. Even the same person may shift rating standards over time. The comparative approach enables more reliable training of reward models. ## Limitations in application domains RL and SFT techniques aren't suitable for all tasks. Verifying outputs like novel drug combinations might take years. Performance testing generated GPU code could require days. Such tasks necessitate trade-offs between speed and reliability when choosing verifiers. ## Role in modern AI development RL and SFT are fundamental tools for shaping LLM behavior. Nearly all post-training processes involve some combination of these approaches. The success of modern chat assistants like ChatGPT demonstrates the effectiveness of these techniques in aligning with human preferences. ## Sources covering this story - https://www.oreilly.com/radar/the-two-pillars-of-post-training-reinforcement-learning-and-supervised-fine-tuning/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._