Episode 57: Pipeline chaining, meta pipelines and automation (Part 2 of 2)

In episode 57 (March 2026), Phil is joined by Edmund Miller and Ben Sherman to discuss how recent and upcoming Nextflow language changes can make pipeline chaining and meta pipelines easier. The episode was split into two parts: this first part surveyed current solutions, listen to episode 56 to check that out.In this episode we focus on defining workflow “contracts” through clearer inputs (typed params in Nextflow 25.10 and proposed record types, including streamlined sample sheet/record ingestion from CSV/JSON/YAML or queries) and clearer outputs (workflow outputs leaving preview in 25.10, adding an entry-workflow publish section and an output block to publish channels, route files, and serialize channel contents into JSON/CSV/YAML index files). We discuss reducing direct params access inside modules, using records to pass validated param bundles, generating/leveraging nextflow_schema.json for external tooling like Seqera Platform to validate chains, exploring glue pipelines, command-line piping of Nextflow runs, and how lineage IDs could support more data-centric, automated workflows.00:00 Welcome01:18 Nextflow language02:09 Inputs and outputs04:14 Streamlining sample sheet parsing07:47 Placement of params12:03 Sharing workflows14:09 Outputs20:04 Workflow ouputs to chain pipelines25:56 Publishing channels32:03 Schema34:37 Schema + pipeline chaining37:13 Piping pipelines41:20 Nextflow Lineage45:56 Conclusion and wrap-up