EP052: GPT-4 Bar Exam and Visual Reasoning

GPT-4 is a large-scale, multimodal model developed by OpenAI capable of processing both text and image inputs to generate text outputs.Here is a short summary of its key aspects based on the technical report:Capabilities and Performance: GPT-4 exhibits human-level performance on a wide variety of professional and academic benchmarks. Most notably, it passed a simulated bar exam with a score in the top 10% of test takers, a significant leap from GPT-3.5, which scored in the bottom 10%. It also considerably outperforms previous large language models on traditional NLP benchmarks and demonstrates strong capabilities across multiple languages, including low-resource languages like Latvian and Welsh.Predictable Scaling: A core focus of the project was building deep learning infrastructure that scales predictably. Because extensive tuning was unfeasible for a model of GPT-4's size, OpenAI developed methods to accurately predict its performance and final loss by extrapolating from much smaller models trained with up to 10,000 times less compute.Limitations: Despite its advancements, GPT-4 shares limitations with earlier GPT models. It is not fully reliable and is prone to "hallucinations" (inventing facts or making reasoning errors), has a limited context window, and does not learn from its experience. Post-training processes have also been shown to reduce the model's calibration, meaning it can sometimes be confidently wrong in its predictions.Safety and Mitigations: The model's advanced capabilities introduce significant safety challenges, such as the potential generation of illicit advice, biased content, or disinformation. To mitigate these risks, OpenAI engaged over 50 domain experts for adversarial testing (red-teaming). They also fine-tuned the model using Reinforcement Learning from Human Feedback (RLHF) and a model-assisted safety pipeline that uses rule-based reward models (RBRMs) to steer the model away from harmful behaviors. These efforts decreased the model's tendency to respond to requests for disallowed content by 82% compared to GPT-3.5.