Collaboration & evaluation for LLM apps (Practical AI #253)
Small changes in prompts can create large changes in the output behavior of generative AI models. Add to that the confusion around proper evaluation of LLM applications, and you have a recipe for confusion and frustration. Raza and the Humanloop team have been diving into these problems, and, in this episode, Raza helps us understand how non-technical prompt engineers can productively collaborate with technical software engineers while building AI-driven apps.
Embed this episode
Chapters
- 0:07 Welcome to Practical AI
- 0:43 Origin of Human Loop
- 5:23 Types of designers
- 8:06 Tech & non-tech worklfow
- 11:47 What am i building?
- 13:19 Sponsor: Read Write Own
- 14:40 What can Human Loop do?
- 17:22 In-production feedback
- 18:32 Fine-tiuning jargon
- 21:43 Fine-tuning trends
- 24:11 Proliferation of open models
- 26:49 Sponsor: Changelog News
- 28:20 Different roles in the HL system / Collaborating in Human Loop
- 32:04 Production framework
- 34:40 Importance of evaluation
- 38:14 Surprising usecases
- 42:43 Exciting things that are happening
- 45:27 Outro
No transcript yet.
Frequently asked questions
How long is this episode of Changelog Master Feed?
This episode is 46 minutes long.
When was this Changelog Master Feed episode published?
This episode was published on January 23, 2024.
Is there a transcript available for this episode?
Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.
Can I download this Changelog Master Feed episode?
Yes. Use the download control on the episode player to save the publisher-provided media file.