Self-hosting & scaling models (Practical AI #243)
We're excited to have Tuhin join us on the show once again to talk about self-hosting open access models. Tuhin's company Baseten specializes in model deployment and monitoring at any scale, and it was a privilege to talk with him about the trends he is seeing in both tooling and usage of open access models. We were able to touch on the common use cases for integrating self-hosted models and how the boom in generative AI has influenced that ecosystem.
Embed this episode
Chapters
- 0:00 Welcome to Practical AI
- 0:43 Overview of Baseten
- 7:45 We've come a long way
- 9:06 Infrastructure & drones
- 12:45 Hosting vs making a product
- 15:54 Baseten's users
- 18:55 Keep your data
- 19:50 The approach to Baseten
- 22:28 Difference in day to day
- 25:51 Convenient auto-scaling
- 27:02 Gotta go fast
- 29:05 Where trends are going
- 33:09 Running on the edge
- 35:25 Solving future problems
- 39:39 Wrapping up
No transcript yet.
Frequently asked questions
How long is this episode of Changelog Master Feed?
This episode is 41 minutes long.
When was this Changelog Master Feed episode published?
This episode was published on October 31, 2023.
Is there a transcript available for this episode?
Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.
Can I download this Changelog Master Feed episode?
Yes. Use the download control on the episode player to save the publisher-provided media file.