How etcd Compaction Causes Kubernetes API Latency
from DevOps Daily with Fexingo: CI/CD, Kubernetes, and Modern Software Operations Hosted by Fexingo
etcd compaction is a routine maintenance operation that removes historical key-value versions, but when it runs, it can consume significant IO and CPU, leading to increased latency for API server requests. This episode dives into the mechanics: compaction triggers defragmentation of boltdb pages, causing pauses that can spike request latency from milliseconds to seconds. We discuss a real-world incident where a 500-node cluster saw API latency jump to 8 seconds during peak hours due to compaction misconfiguration. Then we cover mitigation strategies: tuning auto-compaction retention, scheduling defrag during low traffic, monitoring etcd metrics, and setting alerts for compaction duration. Understanding this often-overlooked performance pitfall is crucial for maintaining responsive clusters at scale. #Kubernetes #etcd #APIServer #Latency #Compaction #Defragmentation #Performance #Tuning #SRE #DevOps #Infrastructure #CloudNative #FexingoBusiness #BusinessPodcast #Technology #Operations #Cluster #Scalability Keep every episode free: buymeacoffee.com/fexingo
Embed this episode
No transcript yet.