Training Language Models to Follow Instructions with Human Feedback

Maya and Leo dig into the InstructGPT paper — the moment the human-feedback recipe grew from a summarization trick into the way assistants get made. They walk the pipeline as three stations and a punch list (the Apprenticeship, the Ranking Desk, the Governor, the Punch List), stage the scale-versus-feedback argument over the famous result that humans preferred a 1.3B-parameter aligned model to the 175B GPT-3 baseline, and close on why the feedback process itself — labelers, instruction sheets, audits — is the real product. Sources: • Training Language Models to Follow Instructions with Human Feedback: https://arxiv.org/pdf/2203.02155