Faster Coroutine Pipelines

Mike Spivey, University of Oxford, UK, gives the first presentation in the second panel, Functional Programming Techniques, in the ICFP 2017 conference. Coroutine pipelines provide an attractive structuring mechanism for complex programs that process streams of data, with the advantage over lazy streams that both ends of a pipeline may interact with the I-O system, as may processes in the middle. Two popular Haskell libraries, Pipes and Conduit, support such pipelines. In both libraries, pipelines are implemented in a direct style by combining a free monad of communication events with an interpreter for (pseudo-)parallel composition that interleaves the events of its argument processes. These implementations both suffer from a slow-down when processes are deeply nested in sequence or in parallel. We propose an alternative implementation of pipelines based on continuations that does not suffer from this slow-down. What is more, the implementation is significantly faster on small, communication-intensive examples even where they do not suffer from the slow-down, and faster even than comparable programs based on lazy streams. The continuation-based implementation may be derived from the direct-style implementation by algebraic reasoning.