KumoRFM for In-Context Relational Learning

This episode explores KumoRFM, a 2025 proposal for a foundation model that can perform in-context learning directly on relational databases, aiming to handle tasks like churn prediction, fraud detection, recommendation, and forecasting without training a separate model for each schema and label. It explains how the approach represents warehouse data as heterogeneous graphs of rows and foreign-key relationships, using attention over local relational neighborhoods instead of flattening everything into handcrafted feature tables. The discussion focuses on the paper’s strongest claim, zero-shot transfer, and carefully separates true inference-time generalization from easier settings like continued pretraining on the target database or later fine-tuning on the target task. Listeners would find it interesting because the episode gets precise about what this system could change in enterprise ML, while also surfacing the practical caveats around task specification, temporal leakage, infrastructure cost, and how literal the any database, any task promise really is. Sources: 1. KumoRFM for In-Context Relational Learning https://kumo.ai/research/kumo_relational_foundation_model.pdf 2. Modeling Relational Data with Graph Convolutional Networks — Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Max Welling, et al., 2017 https://scholar.google.com/scholar?q=Modeling+Relational+Data+with+Graph+Convolutional+Networks 3. Heterogeneous Graph Transformer — Ziniu Hu, Yuxiao Dong, Kuansan Wang, Yizhou Sun, 2020 https://scholar.google.com/scholar?q=Heterogeneous+Graph+Transformer 4. Relational Deep Learning: Graph Representation Learning on Relational Databases — Matthias Fey, Weihua Hu, Kexin Huang, Jan Eric Lenssen, Jure Leskovec, et al., 2023 https://scholar.google.com/scholar?q=Relational+Deep+Learning%3A+Graph+Representation+Learning+on+Relational+Databases 5. Relational Graph Transformer — Vijay Prakash Dwivedi, Sri Jaladi, Yangyi Shen, Federico Lopez, Matthias Fey, Jure Leskovec, et al., 2025 (ICLR 2026) https://scholar.google.com/scholar?q=Relational+Graph+Transformer 6. One Model to Rule them All: Towards Zero-Shot Learning for Databases — Benjamin Hilprecht, Carsten Binnig, 2021 https://scholar.google.com/scholar?q=One+Model+to+Rule+them+All%3A+Towards+Zero-Shot+Learning+for+Databases 7. Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction — Benjamin Hilprecht, Carsten Binnig, 2022 https://scholar.google.com/scholar?q=Zero-Shot+Cost+Models+for+Out-of-the-box+Learned+Cost+Prediction 8. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second — Noah Hollmann, Samuel Muller, Katharina Eggensperger, Frank Hutter, 2022 (ICLR 2023) https://scholar.google.com/scholar?q=TabPFN%3A+A+Transformer+That+Solves+Small+Tabular+Classification+Problems+in+a+Second 9. Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data — Rishabh Ranjan, Valter Hudovernik, Mark Znidar, Carlos Guestrin, Jure Leskovec, et al., 2025 (ICLR 2026) https://scholar.google.com/scholar?q=Relational+Transformer%3A+Toward+Zero-Shot+Foundation+Models+for+Relational+Data 10. Relational In-Context Learning via Synthetic Pre-training with Structural Prior — Yanbo Wang, Jiaxuan You, Chuan Shi, Muhan Zhang, 2026 https://scholar.google.com/scholar?q=Relational+In-Context+Learning+via+Synthetic+Pre-training+with+Structural+Prior 11. OpenRFM: Dissecting Relational In-Context Learning — Zhikai Chen et al., 2026 https://scholar.google.com/scholar?q=OpenRFM%3A+Dissecting+Relational+In-Context+Learning 12. KumoRFM-2: Scaling Foundation Models for Relational Learning — Valter Hudovernik et al., 2026 https://scholar.google.com/scholar?q=KumoRFM-2%3A+Scaling+Foundation+Models+for+Relational+Learning 13. Retrieval & Fine-Tuning for In-Context Tabular Models — Valentin Thomas et al., 2024 https://scholar.google.com/scholar?q=Retrieval+%26+Fine-Tuning+for+In-Context+Tabular+Models 14. Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Models — Xumeng Wen et al., 2025 https://scholar.google.com/scholar?q=Scalable+In-Context+Learning+on+Tabular+Data+via+Retrieval-Augmented+Large+Language+Models 15. Exploring Fine-Tuning for Tabular Foundation Models — Aditya Tanna et al., 2026 https://scholar.google.com/scholar?q=Exploring+Fine-Tuning+for+Tabular+Foundation+Models 16. AI Post Transformers: Predictive Query Language for Relational Databases — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-08-predictive-query-language-for-relational-103e68.mp3 17. AI Post Transformers: KumoRFM-2 for Relational Learning at Scale — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-06-08-kumorfm-2-for-relational-learning-at-sca-13c996.mp3 18. AI Post Transformers: Why LightGBM Made Boosted Trees Fast — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-05-why-lightgbm-made-boosted-trees-fast-286a89.mp3 19. AI Post Transformers: How Induction Heads Emerge in Transformers — Hal Turing & Dr. Ada Shannon, 2026 https://podcast.do-not-panic.com/episodes/2026-05-03-how-induction-heads-emerge-in-transforme-a7bfcb.mp3