A recipe for frontier model post-training

Apple, Meta, and Nvidia all agree -- synthetic data, iterative training, human preference labels, and lots of filtering. This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/frontier-model-post-training 00:00 Llama 3.1 post-training and the new normal for RLHF 01:18 A new standard pipeline 01:45 Human preference data 02:59 Scaling RLHF 05:03 Synthetic data 06:10 The new normal 06:51 Data quality is king 07:18 Apple confirms the new normal Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/frontier-rlhf/img_018.png Fig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/frontier-rlhf/img_020.png Fig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/frontier-rlhf/img_031.png Fig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/frontier-rlhf/img_033.png Fig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/frontier-rlhf/img_035.png This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.interconnects.ai/subscribe