Skip to content
Streaming Ecosystem Complexities and Cost Management // Rohit Agrawal // #302 episode artwork

Apr 4, 2025 48 min

Streaming Ecosystem Complexities and Cost Management // Rohit Agrawal // #302

from Agentic Conversations (formally mlops.community) Hosted by Demetrios

Streaming Ecosystem Complexities and Cost Management // MLOps Podcast #302 with Rohit Agrawal, Director of Engineering at Tecton. Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter // Abstract Demetrios talks with Rohit Agrawal, Director of Engineering at Tecton, about the challenges and future of streaming data in ML. Rohit shares his path at Tecton and insights on managing real-time and batch systems. They cover tool fragmentation (Kafka, Flink, etc.), infrastructure costs, managed services, and trends like using S3 for storage and Iceberg as the GitHub for data. The episode wraps with thoughts on BYOC solutions and evolving data architectures. // Bio Rohit Agrawal is an Engineering Manager at Tecton, leading the Real-Time Execution team. Before Tecton, Rohit was a Lead Software Engineer at Salesforce, where he focused on transaction processing and storage in OLTP relational databases. He holds a Master’s Degree in Computer Systems from Carnegie Mellon University and a Bachelor’s Degree in Electrical Engineering from the Biria Institute of Technology and Science in Pilani, India. // Related Links ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Rohit on LinkedIn: /agrawalrohit10 Timestamps: [00:00] Rohit's preferred coffee [00:34] Takeaways [01:25] Data Streaming [06:50] Optimizing for Speed [09:17] DuckDB vs Spark Flink [13:09] Optimizing Feature Engineering Best Practices [16:42] Checkpointing Frequency Guide [23:08] Streaming Tips and Tricks [30:00] Cloud Costs vs Human Costs [32:09] Race to Learn [39:24] Right-Sized Engineering Practices [42:06] Streaming Simplicity vs Complexity [47:02] Wrap up

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Streaming Ecosystem Complexities and Cost Management // Rohit Agrawal // #302

0:00 48:51

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Agentic Conversations (formally mlops.community)?

This episode is 48 minutes long.

When was this Agentic Conversations (formally mlops.community) episode published?

This episode was published on April 4, 2025.

Can I download this Agentic Conversations (formally mlops.community) episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.