03: Turning All Data at Grofers into Live Event Streams

In this week’s episode of The Data Stack Show, Kostas Pardalis connects with Satyam Krishna, a data engineer at Grofers, India’s largest low-price online supermarket. Grofers boasts a network of more than 5,000 partner stores, a user base with three million iOS and Android app downloads, and an efficient supply chain that allow it to deliver more than 25 million products to customers every month. Satyam offers insights into how he helped build the data engineering function at Grofers, how they developed a robust data stack, how they’re turning production databases into live event streams using Change Data Capture, how Grofers’ internal customers consume data, and the company made adjustments due to the pandemic. Topics of discussion included:Satyam moving from a developer to a data engineer (2:43)Describing Grofers’ data stack and data lake (6:41)Who is consuming data inside the company and what are some of their common uses specific to Grofers? (12:03)What are the biggest issues day-to-day as a data engineer? (18:21)COVID’s impact on business practices and the data stack (21:28)The big problem of data discoverability and metadata cataloging (27:44)Completely changing architecture to something that can scale up (33:16)The Data Stack Show is a weekly podcast powered by RudderStack. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.