Reward models and PPO versus a single elegant loss. How RLHF and DPO turn human preferences into model behaviour, and what each costs.
