(Virtual Talk) Quality on Autopilot: Scaling Testing in Uyuni (osc25)

How do you scale quality in a fast-moving open source project like Uyuni? In this talk, I’ll walk you through our journey adopting TestOps practices to bring automation, visibility, and resilience into our quality engineering process. We’ll explore how infrastructure automation with Terraform and Salt laid the foundation for consistent, on-demand testing environments. From there, we introduced test observability using Prometheus and Grafana to analyze historical trends and gain visibility into test behavior over time. A key part of our process includes tracking flaky tests and known issues through a custom system that connects our internal QE workflows directly with test reports. To accelerate feedback loops, we implemented smart test selection in Pull Requests, targeting only the affected components with relevant end-to-end tests. We also integrated synthetic monitoring of key product metrics into our daily pipelines, ensuring continuous insight into overall health. Thanks to designing our own DevContainers, we improved engineer's experience on their IDEs. We are also working on providing public AWS-based test reports reinforcing transparency. Our journey is a story of small, iterative changes that helped us align quality with development at scale. Licensed to the public under https://creativecommons.org/licenses/by-sa/4.0/ about this event: https://c3voc.de