Weekly news update - 27.3.2026

Cloud: Standardizing AI OrchestrationThe Cloud Native Computing Foundation (CNCF) has accepted llm-d into its Sandbox, providing a Kubernetes-native framework for high-performance distributed LLM inference. Supported by Google Cloud, IBM, and NVIDIA, this project allows architects to automate scaling and fault tolerance for AI workloads across vendor-neutral infrastructure.Cybersecurity: Hardening Model InferenceAs Large Language Models move into production, security is shifting toward resource-level isolation within the orchestration layer. By leveraging Kubernetes-native scheduling for model deployments, organizations can now enforce consistent security boundaries and identity-based access controls directly on GPU-accelerated clusters.AI/ML: The Efficiency of Disaggregated PrefillA major technical milestone in the latest inference frameworks is the implementation of disaggregated prefill and decode. This specific optimization separates the initial prompt processing from the token generation phase, significantly reducing "Time to First Token" (TTFT) and maximizing GPU utilization for enterprise-scale reasoning tasks. https://creators.spotify.com/pod/show/1ver9PijJI9yUM83Bvnjov/episode/5DLSJFGsguaX6mIT44OU0v/wizard