What does OpenAI's Sparse Autoencoder Reveal About GPT-4’s Inner Workings

This episode analyzes the research paper titled **"Scaling and Evaluating Sparse Autoencoders"** authored by Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu from OpenAI, released on June 6, 2024. The discussion focuses on the development and scaling of sparse autoencoders (SAEs) as tools for extracting meaningful and interpretable features from complex language models like GPT-4. It highlights OpenAI's introduction of the k-sparse autoencoder, which utilizes the TopK activation function to enhance the balance between reconstruction quality and sparsity, thereby simplifying the training process and reducing dead latents. The episode further examines OpenAI's extensive experimentation, including training a 16-million latent autoencoder on GPT-4’s residual stream activations with 40 billion tokens, showcasing the model's robustness and scalability. It reviews the introduction of new evaluation metrics that go beyond traditional reconstruction error and sparsity, emphasizing feature recovery, activation pattern explainability, and downstream sparsity. Key findings discussed include the power law relationship between mean-squared error and computational investment, the superiority of TopK over ReLU autoencoders in feature recovery and sparsity maintenance, and the implementation of progressive recovery through Multi-TopK. Additionally, the episode addresses the study’s limitations and potential areas for future research, providing comprehensive insights into advancing SAE technology and its applications in language models. This podcast is created with the assistance of AI, the producers and editors take every effort to ensure each episode is of the highest quality and accuracy. For more information on content and research relating to this episode please see: https://arxiv.org/pdf/2406.04093