#180 - Data Engineering 101

Data engineering is a critical field in data science that involves preparing the "big data" infrastructure to be analyzed by data scientists. In this episode we are discussing the differences and how important each is with our guests. 👥 Guests --------------------- - Mahmoud Fettal: https://twitter.com/mahmoudfettal - Salim Jannah: https://www.linkedin.com/in/salim-janah - Omaima Khalil: https://twitter.com/BadQuinn3 ⏱️ Timeline --------------------- 0:00:00 - Introduction and welcoming 0:02:50 - What is data engineering? 0:08:43 - What are the key skills required for a data engineer? 0:16:40 - How does data engineering differ from data science? 0:20:00 - Data analyst vs data engineer vs data scientist 0:22:41 - What are the common tools used in data engineering? 0:28:57 - What are data pipelines? 0:34:54 - What challenges do data engineers face? 0:42:12 - Q&A 0:53:42 - How important is real -time data processing in data engineering? 1:02:35 - What is a data lake, and how does it differ from a data warehouse? 1:12:52 - How do data engineers use machine learning? 1:18:01 - Types of projects really involved with Data engineering 1:32:17 - What future trends should data engineers be aware of? 1:41:00 - Geeksblabla Picks 2:18:30 - Conclusion and Goodbye 🔗 Links --------------------- - Apache Airflow vs Mage.ai: https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf - Lakehouse paper: https://medium.com/odicis-data-engineering/apache-airflow-vs-mage-ai-in-data-engineering-745c040a05e8 - Open Source Agent for Data Analysis: https://pandas-ai.com/ - Simplifying Data Engineering and Analytics with Delta: https://www.packtpub.com/product/simplifying-data-engineering-and-analytics-with-delta/9781801814867 🎤 Hosts --------------------- - Meriem Zaid: https://twitter.com/_iMeriem 🔗 Follow us --------------------- Spotify: https://open.spotify.com/show/0UlTBXh7iH6x0HO6FgYzAD LinkedIn: https://www.linkedin.com/company/geeksblabla-community Facebook: https://www.facebook.com/geeksblabla Twitter: https://twitter.com/geeksblabla Instagram: https://www.instagram.com/geeksblabla GitHub: https://github.com/geeksblabla Visit our website: https://geeksblabla.community 🎙️ جيكس بلابلا هو بودكاست ديال الكوميونيتي فين كنديرو نقاشات شيقة و ممتعة على مواضيع مختلفة في عالم التكنولوجيا مع ناس مميزين من الكوميونيتي ديالنا. كنلتقاو كل نهار الأحد على 8 ديال الليل، وجهد راسك باش تتعلم و تستافد معانا فهاد النقاشات حول أحدث المواضيع التقنية بالدارجة المغربية. 🚀 #GeeksBlabla #darija #تكنولوجيا #المغرب #برمجة #مبرمجين_مغاربة #تقنية #بودكاست_مغربي #تعلم_البرمجة #مطورين #تكنولوجيا_المعلومات #مجتمع_البرمجة #تطوير_الويب #دروس_برمجة #تقنية_المعلومات