Vibe Check: Zero to Data Pipeline — Building with AI & dlt in 10 Minutes

Watch the video version for the demos and screen shares: https://www.youtube.com/live/GMsIhP3RIHgElvis Kaharu (Developer Advocate at dltHub) joined Melanie and co-host Cecil Phillip for a Vibe Check on building data pipelines in 10 minutes with dlt, the open source Python SDK for moving data from anywhere it lives. Elvis walked through how dlt fits between custom Python scripts and managed platforms, and showed how an agent can one-shot a pipeline when the SDK gives it strongly typed, declarative primitives to fill in.Topics covered:Why data pipelines are more than copying from point A to point B (schema normalization, incremental loads, schema evolution as a SEV)Scaffolding a project with uvx dlthub start and what gets generated for Claude (skills, roles, starter pipeline)Installing dlt toolkits for REST APIs, data quality, and file systemsOne-shotting a GitHub issues pipeline with Claude and the REST API skillSwitching destinations with a single line of code (DuckDB, Snowflake, Iceberg, S3, Hugging Face)Schema contracts with Pydantic and the gatekeeper pattern for handling bad rowsAttaching to a pipeline with Marimo notebooks to inspect data, schemas, and run data quality checksFailure introspection via checks.get_failures() and feeding failed rows into another dlt pipelineExporting dlt transformations as dbt models for regulated industries (health, finance)Where Ibis fits in (Python expressions that compile to SQL in Snowflake)Snowflake Cortex AI operators through Ibis as a way to shift left on AI workloadsSkills vs. MCP in the dlt setup, and how skills get tuned differently for Claude, Codex, and CopilotdltHub Pro for hosted pipelines, monitoring, and deploying Marimo notebooks as dashboardsIf you have ever wired up a brittle ingestion script, fought with a managed connector that did not quite fit, or wondered how agents should actually pull data into your warehouse, this episode is for you.Resources:dltHub: https://dlthub.com/dltHub on GitHub: https://github.com/dlt-hubIntroducing dltHub Pro: https://dlthub.com/blog/introducing-dlthub-proHugging Face + dlt for ML: https://dlthub.com/blog/hugging-face-dlt-mlElvis on GitHub: https://github.com/elviskahoroTemporal on GitHub: https://github.com/temporalio