When LeJEPA Truly Learns a World Model

This episode explores the paper When Does LeJEPA Learn a World Model? and uses it to examine what should count as a genuine world model in latent predictive learning, contrasting JEPA-style representation prediction with generative reconstruction. It explains why good probe scores are not enough: the real standard is linear identifiability, where a single global linear map recovers the environment’s hidden state well enough to support planning and compositional generalization. The discussion centers on the paper’s main theorem that, under stationary additive-noise dynamics with Gaussian latent variables, LeJEPA’s alignment objective plus SIGReg recovers the true latent state up to an orthogonal rotation, and on the sharper converse result that this universal guarantee fails for non-Gaussian latents. Listeners get a rigorous argument for when latent models are truly learning the world’s coordinates instead of merely extracting features that happen to be useful on downstream tasks. Sources: 1. When LeJEPA Truly Learns a World Model https://arxiv.org/pdf/2605.26379 2. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations — Francesco Locatello, Stefan Bauer, Mario Lucic, et al., 2018 https://scholar.google.com/scholar?q=Challenging+Common+Assumptions+in+the+Unsupervised+Learning+of+Disentangled+Representations 3. Variational Autoencoders and Nonlinear ICA: A Unifying Framework — Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, Aapo Hyvarinen, 2019 https://scholar.google.com/scholar?q=Variational+Autoencoders+and+Nonlinear+ICA%3A+A+Unifying+Framework 4. On Linear Identifiability of Learned Representations — Geoffrey Roeder, Luke Metz, Diederik P. Kingma, 2020 https://scholar.google.com/scholar?q=On+Linear+Identifiability+of+Learned+Representations 5. Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning — Aapo Hyvarinen, Ilyes Khemakhem, Hiroshi Morioka, 2023 https://scholar.google.com/scholar?q=Nonlinear+Independent+Component+Analysis+for+Principled+Disentanglement+in+Unsupervised+Deep+Learning 6. Auto-Encoding Variational Bayes — Diederik P. Kingma, Max Welling, 2013 https://scholar.google.com/scholar?q=Auto-Encoding+Variational+Bayes 7. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning — Adrien Bardes, Jean Ponce, Yann LeCun, 2021 https://scholar.google.com/scholar?q=VICReg%3A+Variance-Invariance-Covariance+Regularization+for+Self-Supervised+Learning 8. LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics — Randall Balestriero, Yann LeCun, 2025 https://scholar.google.com/scholar?q=LeJEPA%3A+Provable+and+Scalable+Self-Supervised+Learning+Without+the+Heuristics 9. When Does LeJEPA Learn a World Model? — David Klindt, Yann LeCun, Randall Balestriero, 2026 https://scholar.google.com/scholar?q=When+Does+LeJEPA+Learn+a+World+Model%3F 10. LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels — Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, Randall Balestriero, 2026 https://scholar.google.com/scholar?q=LeWorldModel%3A+Stable+End-to-End+Joint-Embedding+Predictive+Architecture+from+Pixels 11. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — Mido Assran et al., 2025 https://scholar.google.com/scholar?q=V-JEPA+2%3A+Self-Supervised+Video+Models+Enable+Understanding%2C+Prediction+and+Planning 12. Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning — Aapo Hyvarinen, Hiroaki Sasaki, Richard E. Turner, 2018 https://scholar.google.com/scholar?q=Nonlinear+ICA+Using+Auxiliary+Variables+and+Generalized+Contrastive+Learning 13. Joint Embedding Predictive Architectures Focus on Slow Features — Vlad Sobal, Jyothir S V, Siddhartha Jalagam, Nicolas Carion, Kyunghyun Cho, Yann LeCun, 2022 https://scholar.google.com/scholar?q=Joint+Embedding+Predictive+Architectures+Focus+on+Slow+Features 14. Cross-Entropy Is All You Need To Invert the Data Generating Process — Patrik Reizinger, Alice Bizeul, Attila Juhos, Julia E. Vogt, Randall Balestriero, Wieland Brendel, David Klindt, 2024 https://scholar.google.com/scholar?q=Cross-Entropy+Is+All+You+Need+To+Invert+the+Data+Generating+Process 15. Identifiability of latent-variable and structural-equation models: from linear to nonlinear — Aapo Hyvarinen, Ilyes Khemakhem, Ricardo Monti, 2023 https://scholar.google.com/scholar?q=Identifiability+of+latent-variable+and+structural-equation+models%3A+from+linear+to+nonlinear 16. On the Identifiability of Sparse ICA without Assuming Non-Gaussianity — Ignavier Ng, Yujia Zheng, Xinshuai Dong, Kun Zhang, 2024 https://scholar.google.com/scholar?q=On+the+Identifiability+of+Sparse+ICA+without+Assuming+Non-Gaussianity 17. Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity — Emiliyan Gospodinov, Vaisakh Shaj, Philipp Becker, Stefan Geyer, Gerhard Neumann, 2024 https://scholar.google.com/scholar?q=Adaptive+World+Models%3A+Learning+Behaviors+by+Latent+Imagination+Under+Non-Stationarity 18. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors — Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long, 2023 https://scholar.google.com/scholar?q=Koopa%3A+Learning+Non-stationary+Time+Series+Dynamics+with+Koopman+Predictors 19. Simplifying Latent Dynamics with Softly State-Invariant World Models — Tankred Saanum, Peter Dayan, Eric Schulz, 2024 https://scholar.google.com/scholar?q=Simplifying+Latent+Dynamics+with+Softly+State-Invariant+World+Models 20. Structured World Models from Human Videos — Russell Mendonca, Shikhar Bahl, Deepak Pathak, 2023 https://scholar.google.com/scholar?q=Structured+World+Models+from+Human+Videos 21. AI Post Transformers: LeWorldModel: Stable Joint-Embedding World Models from Pixels — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-03-25-leworldmodel-stable-joint-embedding-worl-650f9f.mp3 22. AI Post Transformers: Causal-JEPA for Object-Level World Models — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-15-causal-jepa-for-object-level-world-model-311a8b.mp3 23. AI Post Transformers: Learning Latent Action World Models from Video — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-04-09-learning-latent-action-world-models-from-1570a4.mp3 Interactive Visualization: When LeJEPA Truly Learns a World Model