Getting Your Hands Dirty with Video Output on Linux Using C++ (froscon2025)

Most people will use some video app to obtain video output on Linux. But video output is still generally steeped in mystery and viewed as a magic blackbox among the Linux community. However the Lumiera team were confronted with having to examine the magic blackbox themselves. In this talk we demonstrate several well established mechanisms that can be used to display video frames from a Linux desktop application. We explain the technical prerequisites to display video frames, discuss the challenges and we show some widely available technologies used to implement video display. Lumiera is a video editing application under development in C++ — which we use here to illustrate how raw video output is at the heart of a video processing render engine. Most video formats are compressed and packed into containers for performance and storage reasons. To edit a video it is essential to use a video in a raw format as a sequence of individual frames. Raw video is processed as buffers filled with pixel data. The pixels are encoded with respect to some colour space, and there are a variety of standard mechanisms available, which will be discussed in the talk. Handling raw video requires to access, process and transfer an enormous amount of data. While modern CPUs are more than capable of managing such tasks in real time, modern special purpose hardware (GPU) is geared towards mastering these steps with precision and speed. The downside is obviously that addressing the GPU requires hardware specific programming, i.e., slightly different software implementations for the various GPUs on the market. Fortunately the variations of the hardware implementations can be accommodated by a software stack comprising vendor specific micro code, a driver in the kernel, and several layers of adaptor libraries in userland. For this talk we focus on displaying direct video output from a C++ application that uses some long and well established standard frameworks that are widely available not only on GNU/Linux, but on various Unix flavours: XVideo SDL OpenGL Vulkan X Window System (aka X11, X) is a graphics protocol most widely used on Linux and even older than Linux itself. It has several implementations, e.g., Xorg server. There is an extension to X to accommodate video output which is capable of using video hardware to achieve optimum output while not taxing the CPU. This is known as the X Video Extension (aka "Xv"). The X protocol specifies only primitive drawing features, so in addition to using X other graphic layers can be used that provide the user with more high-level graphic tools: OpenGL and Vulkan. In this talk we would like to present a bare-bone application in C++ to play and display a sequence of raw video frames to illuminate the various aforementioned frameworks. Licensed to the public under http://creativecommons.org/licenses/by/4.0 about this event: https://programm.froscon.org/2025/events/3316.html