NVIDIA AI Releases Dynamo Snapshot: A CRIU-Based Fast Startup System for AI Inference on Kubernetes — 2026-06-05

## Short Segments Perplexity AI unveils a hybrid local-server inference orchestrator, enabling seamless AI task routing between personal devices and the cloud. Today, we'll explore how this innovation balances privacy, cost, and performance. Later, we'll dive into NVIDIA's Dynamo Snapshot, a breakthrough in reducing cold-start latency for AI inference on Kubernetes. Perplexity AI has introduced a groundbreaking hybrid local-server inference orchestrator at Computex 2026. This system automatically routes AI tasks between a user's local device and cloud-based models, optimizing for privacy, cost, and performance. The orchestrator, set to launch with Perplexity Computer in July 2026, uses a local AI model to evaluate tasks in real-time. It decides whether tasks involve sensitive data, require heavy computation, or can be handled on-device. This dynamic routing ensures that sensitive data remains local, while more demanding tasks are sent to the cloud. By acting as an "air-traffic controller" for AI tasks, Perplexity's system addresses enterprise concerns about data governance and operational efficiency. As AI models grow more capable, this hybrid approach offers a promising solution to balance the demands of accuracy, privacy, and cost. Microsoft's Fara tutorial shows how to run a browser-use agent in Google Colab with a mock OpenAI-compatible endpoint. This tutorial guides users through setting up Microsoft Fara in Google Colab, enabling a browser-use workflow from start to finish. By creating a small mock endpoint, users can test the agent loop that Fara uses for real tasks, including sending tasks, receiving model-style action responses, and executing those actions through the browser. This setup allows for flexible endpoint configuration, enabling connections to Azure Foundry, vLLM, LM Studio, or Ollama for real Fara-7B model use. Microsoft's Fara-7B, a 7-billion-parameter agentic small language model, is designed for computer use, predicting mouse and keyboard actions directly from screenshots. This compact model can run locally, reducing latency and enhancing privacy, making it a powerful tool for real-world web tasks. ## Feature Story NVIDIA's Dynamo Snapshot promises to revolutionize AI inference on Kubernetes by slashing cold-start times. This new checkpoint/restore system addresses a critical bottleneck in AI deployments: the lengthy initialization period that leaves GPUs idle and risks SLA violations during traffic spikes. Traditionally, cold-starting inference workloads on Kubernetes involves a multi-step process that can take several minutes, from pulling container images to loading model weights and warming up CUDA kernels. During this time, GPUs are allocated but remain idle, unable to serve requests or generate tokens. Enter NVIDIA's Dynamo Snapshot, which leverages CRIU (Checkpoint/Restore in Userspace) and NVIDIA's cuda-checkpoint tool to capture and restore the full state of an inference worker. This approach allows for sub-5-second initialization, a dramatic improvement over the previous multi-minute wait times. By enabling rapid scaling of inference replicas, Dynamo Snapshot helps prevent SLA violations during sudden demand spikes, ensuring that AI systems can respond swiftly and efficiently. The implications for enterprises running AI workloads on Kubernetes are significant. With Dynamo Snapshot, organizations can achieve greater operational efficiency and resource utilization, reducing the time and cost associated with idle GPUs. This development also enhances the scalability of AI systems, allowing them to handle fluctuating demand with ease. As AI continues to play a critical role in modern computing, innovations like Dynamo Snapshot are essential for maintaining performance and reliability in production environments. Looking ahead, NVIDIA's Dynamo Snapshot sets a new standard for AI inference on Kubernetes, offering a practical solution to one of the platform's most persistent challenges. As more enterprises adopt this technology, we can expect to see further advancements in AI infrastructure management, paving the way for even more efficient and responsive AI systems.