How Kubernetes Certificate Rotation Breaks Service Mesh mTLS
from DevOps Daily with Fexingo: CI/CD, Kubernetes, and Modern Software Operations Hosted by Fexingo
In this episode, Lucas and Luna dig into a production incident where a service mesh's mutual TLS certificates expired overnight, silently killing traffic between microservices. They walk through the failure mode: certificates that were set to rotate every 24 hours but got stuck because of a clock skew issue on one of the worker nodes. They explain how Kubernetes CertificateSigningRequest objects work, why the rotation loop can fail silently, and what happens when the mesh's control plane doesn't fail open. They also discuss practical mitigations, like monitoring certificate expiry as a first-class signal, setting up alerting on rotation failures, and using short-lived certificates with a well-tested renewal path. The episode includes a concrete checklist for anyone running a service mesh in production. #Kubernetes #ServiceMesh #mTLS #CertificateRotation #DevOps #SiteReliability #ProductionIncident #Microservices #Istio #Linkerd #KubernetesCertificates #TLS #ClockSkew #Monitoring #Alerting #FexingoBusiness #BusinessPodcast #TechOps Keep every episode free: buymeacoffee.com/fexingo
Embed this episode
No transcript yet.