Alignment is Real // Shiva Bhattacharjee // #260

Shiva Bhattacharjee is the Co-founder and CTO of TrueLaw, where we are building bespoke models for law firms for a wide variety of tasks.Alignment is Real // MLOps Podcast #260 with Shiva Bhattacharjee, CTO of TrueLaw Inc.// AbstractIf the off-the-shelf model can understand and solve a domain-specific task well enough, either your task isn't that nuanced or you have achieved AGI. We discuss when fine-tuning is necessary over prompting and how we have created a loop of sampling, collecting feedback, and fine-tuning to create models that seem to perform exceedingly well in domain-specific tasks.// Bio20 years of experience in distributed and data-intensive systems spanning work at Apple, Arista Networks, Databricks, and Confluent. Currently CTO at TrueLaw, where we provide a framework to fold in user feedback, such as lawyer critiques of a given task, and fold them into proprietary LLM models through fine-tuning mechanics, resulting in 7-10x improvements over the base model. // MLOps Jobs board jobs.mlops.community// MLOps Swag/Merchhttps://mlops-community.myshopify.com/// Related LinksWebsite: www.truelaw.ai --------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerCatch all episodes, blogs, newsletters, and more: https://mlops.community/Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with Shiva on LinkedIn: https://www.linkedin.com/in/shivabhattacharjee/Timestamps:[00:00] Shiva's preferred coffee[00:58] Takeaways[01:17] DSPy Implementation[04:57] Evaluating DSPy risks[08:13] Community-driven DSPy tool[12:19] RAG implementation strategies[17:02] Cost-effective embedding fine-tuning[18:51] AI infrastructure decision-making[24:13] Prompt data flow evolution[26:32] Buy vs build decision[30:45] Tech stack insights[38:20] Wrap up