Large Language Models in Production Round-table Conversation

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian.// AbstractUsing LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board jobs.mlops.community// MLOps Swag/Merchhttps://mlops-community.myshopify.com/// Related LinksLLM in Production Survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewformVirtual LLMs in Production Conference registration:https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13Chinchilla papers:https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556--------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerCatch all episodes, blogs, newsletters, and more: https://mlops.community/Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/Timestamps:[00:00] Round table success to Virtual LLM in Production Conference on April 13th![00:18] Register for the Virtual LLM in Production Conference now![00:44] LLM in Production survey[01:40] Lightning round of introduction of speakers[04:34] Large Language Models definition[09:17] What do we consider large?[10:35] Thought process in use cases production [14:30] LLM open source huge movements [16:50] Problems with qualifications[19:25] Production use cases frameworks directions[25:25] Open-source language models tokenizer[26:25] Language models democratization[29:25] Three categories for LLMs in Production[31:22] Latency at 2 levels[33:27] Defining production[34:57] Hitting the latency problems[38:20] Fundamental latency barrier[40:39] Latency use case requirement[44:25] Costs and the use cases[48:12] Product management involvement in costing[49:38] LLMs Hallucination definition[52:05] Building deterministic systems trust[55:21] Wrap up