How Kubernetes StatefulSet Rolling Update Breaks Persistent Volume Claims
from DevOps Daily with Fexingo: CI/CD, Kubernetes, and Modern Software Operations Hosted by Fexingo
In this episode of DevOps Daily, Lucas and Luna dive into a subtle but destructive Kubernetes pitfall: StatefulSet rolling updates that orphan PersistentVolumeClaims when the pod template changes. Using a real-world incident at a mid-sized fintech, they walk through how a simple image update triggered PVC detachment, stalled the rollout, and left the cluster in an inconsistent state. They explain the underlying mechanics — how StatefulSet's pod identity logic interacts with PVC templates, why the 'OnDelete' update strategy buys time but doesn't fix the root cause, and what monitoring signals to watch for. Lucas shares a concrete remediation: pre-binding PVCs via volumeClaimTemplates with stable names, and using a custom operator to handle canary updates safely. Listeners will learn one specific configuration pattern to prevent this silent failure, plus a diagnostic command to check for orphaned PVCs after any StatefulSet change. This is a focused, practical episode for anyone running stateful workloads on Kubernetes. #Kubernetes #StatefulSet #PersistentVolumeClaim #RollingUpdate #DevOps #CloudNative #StatefulWorkloads #K8sFailure #PodIdentity #VolumeClaimTemplates #OnDeleteStrategy #Fintech #IncidentResponse #KubernetesTroubleshooting #CI/CD #Technology #FexingoBusiness #DevOpsPodcast Keep every episode free: buymeacoffee.com/fexingo
Embed this episode
No transcript yet.