Privilege delegation for rootless containers, what choices do we have? (asg2025)

Going for minimal containers with restricted system calls and unprivileged users is the usual Kubernetes approach these days, and it works great for most web apps. However, the development of more complex infrastructure extensions frequently hinders application functionality. While looking for a solution to deploy virtiofsd in an unprivileged container for KubeVirt, we stumbled on seccomp notifiers. Seccomp notifiers are a kernel feature which monitors syscalls and get notifications to a userspace application when a syscall is executed. Alternative options involved either the use of a custom protocol using UNIX sockets or the deployment of virtiofs as a privileged component alongside the unprivileged VM. After our evaluation, the seccomp notifier turned out to be the simplest solution among all the choices. Unfortunately, the main constraint is the monitor's resilience after a restart, such as after a crash or an upgrade. This limitation forced us to back up to one of the less elegant approaches. But there is hope how this could be solved! The session will explain why seccomp notifiers are a lean solution to avoid extra userspace communication and synchronization, the current limitations and possible future solutions to overcome today’s challenges. Our experience will teach audiences several methods for dividing their privileged infrastructure. Utilizing virtiofsd as an actual example and a target application for KubeVirt integration and deployment. We will discuss the difficulties of using rootless containers in this session, as well as the design patterns, technologies, and tactics we thought about and ultimately chose to maintain or reject. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2025/talk/SPGAXS/