EPISODE · Jun 30, 2025 · 11 MIN
Ensuring Data Quality at Petabyte Scale [Glassdoor]
from Snacks Weekly on Data Science · host Pan Wu
In this episode, we dive into how Glassdoor addresses the challenge of maintaining data quality at a petabyte scale. By treating data as a product, the engineering team built a centralized, scalable platform that enables proactive validation, continuous monitoring, and cross-team collaboration. From data contracts and static code analysis to LLM-based logic checks and anomaly detection, we unpack the key practices behind their approach.For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/glassdoor-engineering/data-quality-at-petabyte-scale-building-trust-in-the-data-lifecycle-7052361307a4
What this episode covers
In this episode, we dive into how Glassdoor addresses the challenge of maintaining data quality at a petabyte scale. By treating data as a product, the engineering team built a centralized, scalable platform that enables proactive validation, continuous monitoring, and cross-team collaboration. From data contracts and static code analysis to LLM-based logic checks and anomaly detection, we unpack the key practices behind their approach.For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/glassdoor-engineering/data-quality-at-petabyte-scale-building-trust-in-the-data-lifecycle-7052361307a4
NOW PLAYING
Ensuring Data Quality at Petabyte Scale [Glassdoor]
No transcript for this episode yet
Similar Episodes
Apr 22, 2025 ·32m
Feb 27, 2025 ·0m
Sep 20, 2024 ·57m
Aug 7, 2024 ·16m