Building Resilient Data Systems for Modern Enterprises at Astrafy with Andrea Bombino

Efficient data orchestration is the backbone of modern analytics and AI-driven workflows. Without the right tools, even the best data can fall short of its potential. In this episode, Andrea Bombino, Co-Founder and Head of Analytics Engineering at Astrafy, shares insights into his team’s approach to optimizing data transformation and orchestration using tools like datasets and Pub/Sub to drive real-time processing. Andrea explains how they leverage Apache Airflow and Google Cloud to power dynamic data workflows. Key Takeaways:(01:55) Astrafy helps companies manage data using Google Cloud.(04:36) Airflow is central to Astrafy’s data engineering efforts.(07:17) Datasets and Pub/Sub are used for real-time workflows.(09:59) Pub/Sub links multiple Airflow environments.(12:40) Datasets eliminate the need for constant monitoring.(15:22) Airflow updates have improved large-scale data operations.(18:03) New Airflow API features make dataset updates easier.(20:45) Real-time orchestration speeds up data processing for clients.(23:26) Pub/Sub enhances flexibility across cloud environments.(26:08) Future Airflow features will offer more control over data workflows.Resources Mentioned:Andrea Bombino -https://www.linkedin.com/in/andrea-bombino/Astrafy -https://www.linkedin.com/company/astrafy/Apache Airflow -https://airflow.apache.org/Google Cloud -https://cloud.google.com/dbt -https://www.getdbt.com/Apache Airflow Survey -https://astronomer.typeform.com/airflowsurvey24Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.#AI #Automation #Airflow #MachineLearning