Signals and Threads

In university Jacob Baskin studied at the intersection of computer science and economics, thinking about systems that incentivize people to express their true preferences. He put those ideas into practice at Google, where he worked on ad serving, before joining Jane Street’s database infrastructure team. In this episode, Ron and Jacob discuss Superstore, a distributed columnar database now central to Jane Street’s tech stack that Jacob began building practically the day he started. How do you support wide-ranging analytical queries while transactional writes stream in at the speed of trading systems? And what’s it like when your first design doc leads to an eight-figure hardware purchase? After building Superstore Jacob has since gone back to his roots, thinking about schemes for bidding on compute time as he works to optimize usage of the Hive, Jane Street’s massive compute cluster for research. You can find the transcript for this episode on our website. Some links to topics that came up in the discussion: Mechanism design, second-price auction MapReduce, BigTable, Google File System Vertica Apache Parquet CockroachDB Paxos BitTorrent

Nate Foster is a professor at EPFL in Switzerland in the Networked Systems Abstractions Lab, and a visiting researcher at Jane Street on the Networking team. In this episode, he and Ron consider what happens when you bring a software mindset to network engineering. Can you use programming language theory and formal methods to realize the dream of software-defined networks? Along the way, they discuss how hyperscalers have shaped networking hardware; the return (or not) of multicast; the ways ML workloads are reshaping the networking layer; and the success Jane Street has had using an early Internet protocol, BGP, together with a more declarative high-level specification language. You can find the transcript for this episode on our website. Some links to topics that came up in the discussion: P4 (Programming language Lenses (bidirectional transformation) OpenFlow Kleene algebra with tests NetKAT End-to-end principle Border Gateway Protocol “Stable Internet routing without Global Coordination,” aka the Gao-Rexford conditions Unison file synchronizer Barefoot Networks