Skip to content
Large Language Models in Production Round-table Conversation episode artwork

Mar 23, 2023 57 min

Large Language Models in Production Round-table Conversation

from Agentic Conversations (formally mlops.community) Hosted by Demetrios

LLM in Production Round Table with Demetrios Brinkmann, Diego Oppenheimer, David Hershey, Hannes Hapke, James Richards, and Rebecca Qian. // Abstract Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event! // MLOps Jobs board jobs.mlops.community // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related LinksLLM in Production Survey: https://docs.google.com/forms/d/e/1FAIpQLSerEryK4xHEZTq0hSu-sVmBHilOzaT71BfCQgXe_uIRgIah-g/viewform Virtual LLMs in Production Conference registration: https://home.mlops.community/public/events/llms-in-production-conference-2023-04-13 Chinchilla papers: https://paperswithcode.com/method/chinchilla, https://arxiv.org/abs/2203.15556 --------------- ✌️Connect With Us ✌️ ------------- Join our Slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Diego on LinkedIn: https://www.linkedin.com/in/diego/ Connect with David on LinkedIn: https://www.linkedin.com/in/david-hershey-458ab081/ Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hanneshapke/ Connect with James on LinkedIn: https://www.linkedin.com/in/james-richards-4baa73a7/ Connect with Rebecca on LinkedIn: https://www.linkedin.com/in/rebeccaqian/ Timestamps: [00:00] Round table success to Virtual LLM in Production Conference on April 13th! [00:18] Register for the Virtual LLM in Production Conference now! [00:44] LLM in Production survey [01:40] Lightning round of introduction of speakers [04:34] Large Language Models definition [09:17] What do we consider large? [10:35] Thought process in use cases production [14:30] LLM open source huge movements [16:50] Problems with qualifications [19:25] Production use cases frameworks directions [25:25] Open-source language models tokenizer [26:25] Language models democratization [29:25] Three categories for LLMs in Production [31:22] Latency at 2 levels [33:27] Defining production [34:57] Hitting the latency problems [38:20] Fundamental latency barrier [40:39] Latency use case requirement [44:25] Costs and the use cases [48:12] Product management involvement in costing [49:38] LLMs Hallucination definition [52:05] Building deterministic systems trust [55:21] Wrap up

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Large Language Models in Production Round-table Conversation

0:00 57:47

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Agentic Conversations (formally mlops.community)?

This episode is 57 minutes long.

When was this Agentic Conversations (formally mlops.community) episode published?

This episode was published on March 23, 2023.

Can I download this Agentic Conversations (formally mlops.community) episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.