Cracking the Cosmic Code: How Software Powers the Rubin Observatory

Imagine a telescope so powerful it could give us a whole new picture of the cosmos and help answer some of the biggest questions about the universe–if we can handle the data. From a mountaintop in Chile, the Vera C. Rubin Observatory is revolutionizing astronomy. The Observatory began capturing images of the entire night sky in June 2025, launching the most ambitious astronomical survey in history. This powerful telescope–with the world’s largest digital camera–generates 20 terabytes of data daily, creating a decade-long "movie" of the cosmos through its Legacy Survey of Space and Time (LSST) initiative. But without the development of innovative software, algorithms and computational systems, much of what the telescope is capturing would be unusable. In this episode, host Randy Scott talks with Carnegie Mellon University’s Professor Rachel Mandelbaum and Jeremy Kubica, who take us behind the scenes to reveal the computational innovations and interdisciplinary collaborations making this massive data collection scientifically useful. Through the LINCC Frameworks initiative, their team has developed innovative open-source software that enables scientists worldwide to analyze data from the telescope with unprecedented access, collaboration and scale. Professor Mandelbaum is the interim head of Carnegie Mellon's Department of Physics and CMU's lead for the LINCC Frameworks, and Jeremy Kubica is the Director of Engineering for the LINCC Frameworks. Our conversation explores groundbreaking technologies like algorithms for measuring galaxy shapes and gravitational lensing effects, software for detecting faint and distant moving objects in our solar system that we otherwise couldn’t see, and collaborative tools that bring researchers to the data rather than downloading massive datasets locally. Over the next ten years, this project will revolutionize our understanding of dark matter, the formation of our solar system, and the fundamental nature of the universe itself. We first reported on the Rubin Observatory in our podcast’s first season in the episode “Stellar Observations: AI’s Journey Into the Cosmos.” The universe is about to reveal its secrets—if our computers can keep up.