Unified Latents (UL): How to train your latents

This paper introduces Unified Latents (UL), a novel framework designed by Google DeepMind researchers to improve the efficiency of generative diffusion models. By integrating the encoder, prior, and decoder into a single cohesive system, the method creates a more principled approach to managing latent representations. A key innovation involves using a fixed amount of Gaussian noise during encoding, which simplifies the training process and allows for a tight bound on latent information. This setup provides users with specific hyper-parameters, such as the loss factor and sigmoid bias, to precisely balance reconstruction accuracy against the complexity of the modeling task. Experimental results across image and video datasets demonstrate that this architecture achieves superior performance with lower computational costs compared to established baselines like Stable Diffusion. Overall, the research highlights how jointly optimizing all components of the latent space leads to higher-quality generative outputs and more effective scaling.