Skip to content
Applying Machine Learning To The Problem Of Bad Data At Anomalo episode artwork

Jan 24, 2023 59 min

Applying Machine Learning To The Problem Of Bad Data At Anomalo

from AI Engineering Podcast Hosted by Tobias Macey

Summary All data systems are subject to the "garbage in, garbage out" problem. For machine learning applications bad data can lead to unreliable models and unpredictable results. Anomalo is a product designed to alert on bad data by applying machine learning models to various storage and processing systems. In this episode Jeremy Stanley discusses the various challenges that are involved in building useful and reliable machine learning models with unreliable data and the interesting problems that they are solving in the process. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Jeremy Stanley about his work at Anomalo, applying ML to the problem of data quality monitoring Interview Introduction How did you get involved in machine learning? Can you describe what Anomalo is and the story behind it? What are some of the ML approaches that you are using to address challenges with data quality/observability? What are some of the difficulties posed by your application of ML technologies on data sets that you don't control? How does the scale and quality of data that you are working with influence/constrain the algorithmic approaches that you are using to build and train your models? How have you implemented the infrastructure and workflows that you are using to support your ML applications? What are some of the ways that you are addressing data quality challenges in your own platform? What are the opportunities that you have for dogfooding your product? What are the most interesting, innovative, or unexpected ways that you have seen Anomalo used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Anomalo? When is Anomalo the wrong choice? What do you have planned for the future of Anomalo? Contact Info @jeremystan on Twitter LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Anomalo Data Engineering Podcast Episode Partial Differential Equations Neural Network Neural Networks For Pattern Recognition by Christopher M. Bishop (affiliate link) Gradient Boosted Decision Trees Shapley Values Sentry dbt Altair The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Applying Machine Learning To The Problem Of Bad Data At Anomalo

0:00 59:24

Chapters

  1. 0:10 Introduction and Guest Introduction
  2. 0:37 Jeremy Stanley's Journey into Machine Learning
  3. 3:41 Anomalo: Data Quality Monitoring
  4. 6:34 Machine Learning in Data Quality Monitoring
  5. 10:48 Approaches to Data Quality Monitoring
  6. 18:14 Challenges and Constraints in Data Quality Monitoring
  7. 22:30 Operational Aspects and ROI of ML Models
  8. 30:01 Managing ML Models and Infrastructure
  9. 34:34 Lessons Learned and Unexpected Challenges
  10. 41:38 Interesting Use Cases and Applications
  11. 45:21 Challenges in Data Quality Monitoring and ML
  12. 52:23 Future Plans for Anomalo
  13. 55:22 Testing and Benchmarking ML Models
  14. 58:23 Conclusion and Final Thoughts

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of AI Engineering Podcast?

This episode is 59 minutes long.

When was this AI Engineering Podcast episode published?

This episode was published on January 24, 2023.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this AI Engineering Podcast episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.