Self-Improving AI and Human Co-Improvement for Safer Co-Superintelligence

This paper studies "co-improvement" as a safer and faster alternative to the current focus on "autonomous self-improving AI" for achieving superintelligence. This paper argues that instead of AI systems improving themselves without human intervention, the focus should be on building AI that actively collaborates with human researchers across all stages of the research pipeline, from ideation to evaluation and safety alignment. The authors propose that this bidirectional collaboration, leading to co-superintelligence, ensures that the resulting advanced AI is better aligned with human needs and values. They suggest creating new benchmarks and methods specifically designed to enhance the AI's research collaboration skills, contrasting this approach with views that minimize the future role of humanity.