Skip to content
Self-hosting & scaling models episode artwork

Oct 31, 2023 41 min

Self-hosting & scaling models

from Practical AI Hosted by Daniel Whitenack and Chris Benson

We’re excited to have Tuhin join us on the show once again to talk about self-hosting open access models. Tuhin’s company Baseten specializes in model deployment and monitoring at any scale, and it was a privilege to talk with him about the trends he is seeing in both tooling and usage of open access models. We were able to touch on the common use cases for integrating self-hosted models and how the boom in generative AI has influenced that ecosystem. Sponsors: Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs. Typesense – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster! Featuring: Tuhin Srivastava – GitHub, X Chris Benson – Website, GitHub, LinkedIn, X Daniel Whitenack – Website, GitHub, X Show Notes: Baseten Upcoming Events: Register for upcoming webinars here!

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Self-hosting & scaling models

0:00 41:07

Chapters

  1. 0:00 Welcome to Practical AI
  2. 0:43 Overview of Baseten
  3. 7:44 We've come a long way
  4. 9:05 Infrastructure & drones
  5. 12:45 Hosting vs making a product
  6. 15:53 Baseten's users
  7. 18:55 Keep your data
  8. 19:50 The approach to Baseten
  9. 22:28 Difference in day to day
  10. 25:51 Convenient auto-scaling
  11. 27:01 Gotta go fast
  12. 29:05 Where trends are going
  13. 33:08 Running on the edge
  14. 35:25 Solving future problems
  15. 39:39 Wrapping up

No transcript yet.

Requests help us decide what to transcribe next.

Share this episode

Similar podcasts

CISO Perspectives (public) N2K Networks This season on CISO Perspectives, host Kim Jones explores some of the challenges of leading through uncertainty. We explore the complexity of the changing nature of regulation and working with the federal government, the evolution of privacy and fraud, and how emerging technologies like AI and quantum computing are changing cyber. When you don’t know what questions to ask, you’re afraid to ask, or don’t know who to ask, CISO Perspectives provides the foundation for learning in this brave new world.

Frequently asked questions

How long is this episode of Practical AI?

This episode is 41 minutes long.

When was this Practical AI episode published?

This episode was published on October 31, 2023.

Can I download this Practical AI episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.