Fault Tolerance and High Availability in Kafka Streams and ksqlDB ft. Matthias J. Sax

Apache Kafka® Committer and PMC member Matthias J. Sax explains fault tolerance, high-availability stream processing, and how it’s done in Kafka Streams. He discusses the differences between changelogging vs. checkpointing and the complexities checkpointing introduces. From there, Matthias explains what hot standbys are and how they are used in Kafka Streams, why Kafka Streams doesn’t do watermarking, and finally, why Kafka Streams is a library and not infrastructure. EPISODE LINKSAsk Confluent #7: Kafka Consumers and Streams Failover Explained ft. Matthias SaxAsk Confluent #8: Guozhang Wang on Kafka Streams Standby TasksHow to Run Kafka Streams on Kubernetes ft. Viktor GamovKafka Streams Interactive Queries Go Prime TimeHighly Available, Fault-Tolerant Pull Queries in ksqlDBKIP-535: Allow state stores to serve stale reads during rebalanceKIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instanceKIP-441: Smooth Scaling Out for Kafka Streams Skip to end of metadataJoin the Confluent Community SlackLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperUse 60PDCAST to get an additional $60 of free Confluent Cloud usage*SEASON 2 Hosted by Tim Berglund, Adi Polak and Viktor Gamov Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed Music by Coastal Kites Artwork by Phil Vo  🎧 Subscribe to Confluent Developer wherever you listen to podcasts. ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.👍 If you enjoyed this, please leave us a rating. 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.