Update on LLMs - hope, hype and hallucination

Some 36 months after the release of ChatGPT, the verdict is still out on the role that large language models (LLMs) will play in biotech, pharma, and medicine. On paper, the range of tasks that LLMs can perform in biomedical research and healthcare is vast—excavating relevant information for drug discovery from mountains of scientific literature, designing novel proteins, transcribing doctors' notes, aiding diagnostic decision-making, and acting as patient-facing chatbots. But given the models’ propensity to hallucinate, we need to define how much error we can tolerate for different LLM use cases in the biomedical fields and create evaluation frameworks that allow us to apply the models confidently. In some cases, it might turn out that the time spent for human supervision of the model will outweigh the efficiency gain. In episode 25 of We’re doomed we’re saved we talk to idalab founder and mathematician, Paul von Bünau we discuss the promise and challenges of LLMs in the biomedical field and ask the question if we can ever stop them from hallucinating. Content and Editing: Louise von Stechow and Andreas Horchler Disclaimer: Louise von Stechow & Andreas Horchler and their guests express their personal opinions, which are founded on research on the respective topics, but do not claim to give medical, investment or even life advice in the podcast. Learn more about the future of biotech in our podcasts and keynotes. Contact us here: scientific communication: https://science-tales.com/ Podcasts: https://www.podcon.de/ Keynotes: https://www.zukunftsinstitut.de/louise-von-stechow Image: jo-coenen-studio-dries-2-6-yST9mzlMVLQ via Unsplash References: Excavating scientific literature 1. https://www.nature.com/articles/s41587-023-01788-7 2. https://idalab.de/insights/how-large-language-models-excavate-crucial-information-to-scale-drug-discovery 3. https://www.cell.com/molecular-therapy-family/nucleic-acids/fulltext/S2162-2531(23)00222-6#secsectitle0015 Unburdening Healthcare 1. https://www.nature.com/articles/s41591-023-02700-1 2. https://medium.com/mantisnlp/applications-of-llms-in-patient-care-83e07548dbb1#:~:text=Applications%20of%20LLMs%20in%20Patient%20Care%201%201.,6%206.%20Personalized%20Health%20Plans%20and%20Coaching%20 3. https://www.nature.com/articles/s41591-023-02700-1 4. https://arxiv.org/pdf/2303.16416 5. https://www.nature.com/articles/s41467-024-46411-8 6. https://link.springer.com/article/10.1007/s00330-023-10213-1 7. https://link.springer.com/article/10.1007/s00330-023-10213-1 8. https://link.springer.com/article/10.1007/s00330-023-10213-1 9. https://www.nature.com/articles/s41591-024-03097-1#Sec9 Reading molecular language 1. https://academic.oup.com/bioinformatics/article/36/4/1234/5566506 2. https://arxiv.org/abs/2106.03598 3. https://resources.nvidia.com/en-us-hc-biopharma/hc-solution-overview-5 4. https://www.nature.com/articles/s41592-024-02201-0#author-information 5. https://www.prnewswire.com/news-releases/ginkgo-bioworks-and-google-cloud-partner-to-build-next-generation-ai-platform-for-biological-engineering-and-biosecurity-301912283.html 6. https://www.nature.com/articles/s41598-024-61124-0 7. https://www.nature.com/articles/s41587-022-01618-2 8. https://www.biopharmatrend.com/post/835-14-companies-pioneering-ai-foundation-models-in-pharma-and-biotech/ 9. https://www.biorxiv.org/content/10.1101/2023.01.11.523679v1 10. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-024-05847-x 11. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-024-05847-x 12. https://arxiv.org/abs/2403.04197 13. https://arxiv.org/abs/2311.07621