The AI Kubernetes Show

Secure your Kubernetes environment. Learn why zero trust cybersecurity is the only defense against AI agents and non-deterministic agentic software in your supply chain.

The future of software development isn't about single agents—it's about building AI agent fleets! Dive into this conversation with Okteto CEO Ramiro Berrelleza to understand how this shift is fundamentally changing platform engineering and accelerating developer productivity. In this episode of The AI Kubernetes Show, we sat down with Ramiro to discuss AI adoption and the need for constant experimentation in the current "Cambrian explosion" of AI tooling. Berrelleza highlights the move from single-threaded AI tools to large, asynchronous AI agent fleets, which solves the bottleneck of waiting for a single AI response. This agentic model is a game-changer, with some early adopters seeing a massive increase in output. Organizations need to adapt for AI-native workflows, because the focus on traditional metrics like measuring code production (lines of code, number of PRs) for AI is flawed. Instead, organizations should identify and focus their AI projects on their real constraints, such as slow CI workflows. Ramiro also addresses the disproportionate challenge of open source maintainer overload caused by AI-generated contributions, proposing a policy of "human-proof code." Finally, AI agents are presented as a powerful technical context multiplier for everyone from sales engineers to the CEO, significantly speeding up the onboarding process and improving communication across the organization. Read the blog post: Takeaways✓ The future is moving from single-threaded AI tools to "AI agent fleets" to solve productivity bottlenecks.✓ Traditional metrics like lines of code or PR count are now ineffective for measuring AI-driven developer productivity.✓ The new focus for AI investment should be on organizational bottlenecks, such as optimizing slow CI workflows.✓ Open source projects should adopt policies like "human-proof code" to manage maintainer overload from AI contributions.✓ AI agents can serve as a technical context multiplier, speeding up onboarding and improving organization-wide understanding of complex code.Hit the like button, subscribe for more content on platform engineering and AI, and ring the notification bell. What is the biggest productivity bottleneck you've solved with AI agents? Let us know in the comments!#AIAgentFleets #PlatformEngineering #DeveloperProductivity #Kubernetes #KubeCon #Okteto #AgenticAI #OpenSource #SoftwareDevelopment #TechTrends

Is your engineering org ready for the speed of AI? Grant Miller, CEO of Replicated, breaks down the intersection of AI and platform engineering, revealing why testing and validation are the biggest unsolved problems in the industry.In this episode of The AI Kubernetes Show, we sit down with Replicated CEO Grant Miller to discuss how the pace of AI is fundamentally reshaping software development. Miller argues that engineering velocity has become the core competitive differentiator and shares the concept of "leadership empathy," where leaders contribute to a pull request with AI to understand the new tools. This increased velocity, however, puts significant system pressure on platform engineering teams, leading to "Frankenstein-y" application footprints and a greater need for top-notch observability and optimized CI/CD pipelines to improve "iteration speed total."The unique distribution challenges of self-hosted AI applications and the difficulty of validating AI code generation, especially for templated infrastructure-as-code like Helm charts and Terraform. Unlike front-end code, the human validation loop for infrastructure-as-code is not intuitive, making the complexity of testing and validation the industry's most significant hurdle.Read the blog post: Takeaways✓ AI turns engineering velocity into the ultimate competitive advantage, requiring organizations to move incredibly fast.✓ Leaders must develop "leadership empathy" by using AI tools to understand the modern developer experience.✓ Rapid AI code generation can lead to complex, "Frankenstein-y" application architectures, increasing pressure on platform engineering for troubleshooting and observability.✓ The biggest challenge in AI-generated code is the lack of an intuitive validation loop for infrastructure-as-code like Helm charts.✓ Testing and validation are the key unsolved problems and future areas for discovery and job creation.Liked this podcast? Hit the like button, subscribe for more AI and platform engineering insights, and let us know in the comments: What is the biggest challenge your team faces with AI-generated code?#AI #PlatformEngineering #EngineeringVelocity #AIGeneratedCode #TestingAndValidation #Kubernetes #Replicated #TechPodcast #CloudNative

Is the AI boom a bubble, or is it a new technological wave? Dinesh Majrekar, CTO of Civo, breaks down the current state of software development, explains why data sovereignty is the paramount security concern, and details how AI's real value lies in increasing code auality, not just velocity.In this episode of The AI Kubernetes Show, Civo CTO Dinesh Majrekar tackles the AI bubble hype, suggesting it is a blend of market speculation and genuine, disruptive innovation, drawing a comparison to the historical hardware monopoly of IBM during the mainframe era. He dives into the challenge of data sovereignty in the age of large language models, explaining Civo's solution of using an "on-prem public cloud" to run an OpenAI-compatible endpoint on private GPUs. This approach ensures maximum security for sensitive data, like medical records, by guaranteeing the data "never leaving your building." We also discussed the flattening curve of open source LLM capabilities, noting that models like the Kimi K2 model are now matching and even beating proprietary benchmarks while using fewer resources.Majrekar challenges the prevailing focus on speed, arguing the true value for software development teams is in boosting code quality. He champions code generation as the best AI use case but stresses it must be a "partnership" where saved time is reinvested in tackling technical debt and strengthening the code base. This is important for managing deployment risk. Finally, he addresses the dilemma of non-deterministic outputs in deterministic processes, which engineers simply call "a bug," emphasizing that AI is not a universal solution.Read the blog post: www.buoyant.io/ai-kubernetes-episode/ai-bubble-or-bug-a-ctos-perspective-on-engineering-in-the-ai-eraKey Takeaways✓ Code Quality is the true benefit of integrating AI; the time saved on initial generation should be used to fix technical debt and strengthen code.✓ Achieving true Data Sovereignty requires running LLMs on private infrastructure (e.g., an on-prem public cloud) to keep data securely contained.✓ The non-deterministic outputs of LLMs can be considered a "bug" in core engineering processes that demand algorithmic certainty.✓ Code generation is the strongest AI use case, but developers must maintain ownership and set a high context standard for the LLM to follow.✓ Open source LLM capabilities are now "on par" with proprietary models.Hit the like button and subscribe to The AI Kubernetes Show for more AI content! What is your engineering team prioritizing with AI: velocity or quality? Let us know in the comments below!#AI #CodeQuality #DataSovereignty #SoftwareDevelopment #PlatformEngineering #Kubernetes #LLM

Is AI velocity breaking your DevOps processes? We sat down with Principal Platform Engineer Ahmed Bebars to discuss the critical balance of using AI to ship code faster while maintaining DevOps integrity in your platform engineering team.Ahmed Bebars, a principal platform engineer and CNCF ambassador, breaks down the AI's impact on SDLC and the transformation of platform engineering. In this The AI Kubernetes Show episode, he argues that AI is a powerful productivity tool that increases velocity, but teams must uphold established DevOps processes, including rigorous integration and regression testing, to prevent disruption and ensure quality. We discuss how Large Language Model (LLM) output is directly tied to input context, leading to the highly favored concept of spec-driven development, where humans guide the AI with precise specifications.The discussion also explores the challenge of adopting a new mindset for the non-deterministic nature of LLMs. Ahmed explains the difference between deterministic vs. non-deterministic LLMs and how tooling can be used to make the outputs predictable. For leaders looking at preparing platform engineering teams for AI, he advises embracing the technology, starting small, and focusing on hosting local LLMs and building agentic workflows for use cases like incident response triage and data gathering. Read the blog post: www.buoyant.io/ai-kubernetes-episode/maintaining-devops-integrity-in-the-age-of-ai-velocityFollow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Takeaways✓ How to use AI to increase code velocity without compromising established DevOps processes.✓ The critical role of context and spec-driven development for high-quality LLM output.✓ Understanding the engineering mindset shift required to work with non-deterministic LLM outputs.✓ Practical advice for preparing platform engineering teams for AI through local knowledge and agentic workflows.✓ The broader role of AI in the development ecosystem, including documentation, testing, and observability.If you found this discussion valuable, please like this video, subscribe for more insights into The AI Kubernetes Show, and hit the notification bell! What's the biggest challenge your team faces with integrating AI into your SDLC? Let us know in the comments below! #AI #DevOps #Kubernetes #PlatformEngineering #SDLC #SpecDrivenDevelopment #CNCF #KubeCon #TechTalk #OpenSource

Discover the dual nature of AI! Tech Lead Chris Khanoyan shares his view on the rapidly changing AI and data science landscape and the critical need for a technical foundation and the transformative power of AI accessibility for the deaf community. In this episode of The AI Kubernetes Show, we dive deep into the world of AI and data science with Chris Khanoyan, a tech lead and senior data scientist at Booz Allen. Chris highlights the rapidly changing data science landscape, noting the significant overlap between data scientists and data engineers. While auto-generated code has made coding more accessible to practically anyone, he stresses that a solid technical foundation remains critical for debugging and understanding the fundamental elements of a system.We covered the foundational challenge of data governance and the need for clean, trustworthy data. Chris explains the importance of establishing a data pipeline and provenance (where the data comes from and who owns the dataset) before training any Large Language Models (LLMs). He offers a core principle for starting any project: begin with the end in mind. We also explore the hurdles of overcoming data access and scarcity, which often require formal agreements with non-technical clients, especially in sectors like the federal government. Finally, as a deaf individual, Chris provides a unique perspective on AI accessibility. He discusses how AI assistance is easing the mental fatigue from constantly processing captions and the potential game-changer of AI-powered glasses for live captions, while also addressing the current security and data sensitivity barriers that prevent their widespread adoption.Read the blog post: www.buoyant.io/ai-kubernetes-episode/ais-technical-imperative-and-the-path-to-accessibilityFollow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Takeaways✓ A solid technical foundation is still vital for practitioners to manage bugs, even with the rise of AI code generation.✓ Data governance and establishing data provenance are primary challenges in successful AI implementation.✓ AI projects must always begin with the end in mind to effectively prepare and utilize data.✓ Workarounds for data scarcity involve combining and consolidating various datasets from different systems (on-prem and cloud).✓ AI accessibility tools, such as live captioning on glasses, offer a significant boost to productivity and ease mental fatigue, though data security remains a critical barrier.If you enjoyed this conversation on the technical imperative of AI, hit the Like button and subscribe for more expert interviews! Let us know in the comments: What is the single biggest data governance challenge your team is facing today? #AIandDataScience #DataGovernance #AIAccessibility #TechLeadInterview #DataProvenance #LLMData #GoogleCloud #DataEngineer #TechInterview #MachineLearning

Keith Maddox, co-lead of the Kubernetes AI Working Group, breaks down the architectural shifts and security challenges required to run enterprise AI agents at scale.In this The Kubernetes AI Show episode, we chat with Keith Maddox, senior principal software engineer lead at Microsoft and Istio maintainer, who shares his perspective on the convergence of data science, AI agents, and platform engineering on Kubernetes AI workflows. He details the organizational dissonance between traditional platform stacks and data science workflows and how the Kubernetes AI working group is working to create a seamless migration path. We cover advanced model specialization techniques like Low Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG), which are crucial for enterprise use cases driven by data privacy and liability concerns.Maddox also provides advice for platform owners, including the technical and non-technical strategies for LLM token spend management—recommending an egress gateway to centralize policy—and the importance of customer empathy with application developers. A major focus is the AI agent identity security gap, which falls between traditional human and machine identities. He strongly advocates for a zero trust AI mindset and immediate mitigation through agent sandboxing (using technologies like gVisor, KVM, or Wazet) and short-lived, ephemeral machine identities to manage the non-deterministic nature of LLMs.Read the blog post: www.buoyant.io/ai-kubernetes-episode/the-ai-tug-of-war-bridging-the-divide-between-platform-engineering-and-data-science Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/ Key Learnings✓ The core conflict is a "tug of war" over tech stacks between platform and data science teams.✓ Model specialization is necessary due to the high cost and lack of specificity of foundational models for enterprise applications.✓ Managing LLM costs requires centralizing policy through an egress gateway and open communication with development teams.✓ AI agents pose a new security challenge, requiring a move toward short-lived, ephemeral machine identities and agent sandboxing.✓ A "Zero Trust" mindset is the recommended security approach for non-deterministic AI agents and workflows.If you're building, deploying, or securing AI workflows, hit the Like button and subscribe for more deep-dive technical content! Let us know in the comments: What is the biggest challenge your team is facing with AI agent identity and security today? #PlatformEngineering #Kubernetes #AIAgents #LLMs #ZeroTrustAI #KubeCon #DataScience #TechSecurity #DevOps

KubeCon 2025 proved that platform engineering is essential, but the next big challenge for the cloud native community is sustainability. Learn from KubeCon Co-Chair Faseela what this future looks like and what it means for your platform.In this episode of The AI Kubernetes Show, we break down the key KubeCon NA 2025 takeaways with co-chair Faseela, a cloud native developer at Ericsson. She discusses the immense success of the Atlanta event and highlights the convergence of AI and Kubernetes, noting that platform engineering and AI were the hottest tracks based on talk submissions. Looking ahead, the focus shifts to designing platforms that are secure, cost-efficient, and sustainable.Fazila explains the "two-sided coin" of sustainability in the cloud native community: environmental sustainability—meaning engineers must actively consider how to design platforms to be optimized and efficient to minimize cloud native environmental impact—and human sustainability, which is all about tackling CNCF maintainer burnout. She details how the CNCF's TOC performs health checks on open source projects and encourages companies that are making use of that project to put resources under that project. This deep dive into the future of cloud native is essential viewing for all software engineers interested in responsible collaboration.Read the blog post: URLFollow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Takeaways✓ Platform Engineering and AI were the most popular technical tracks at KubeCon North America 2025 based on submission volume.✓ The future of cloud native requires engineers to design platforms that are secure and cost efficient and sustainable.✓ Sustainability in cloud native is two-fold: minimizing environmental impact and ensuring the "human sustainability" of the open source community by preventing maintainer burnout.✓ The CNCF's TOC performs regular health checks and encourages companies to dedicate resources to under-maintained open source projects.If you're building in the cloud native space, hit the like button, subscribe for more interviews, and leave a comment below! What is the biggest challenge your team faces when trying to implement a sustainable platform?#Kubernetes #CloudNative #PlatformEngineering #KubeCon #CNCF #Sustainability #AIandKubernetes #

Is your AI infrastructure safe? Marina Moore, research scientist and co-chair of CNCF Tag Security, talks about her research on AI agent isolation and how to build robust platform engineering security. Build it securely from the start!In this must-watch episode of The AI Kubernetes Show, we sit down with security expert Marina Moore to discuss the paradigm shift in AI-driven systems security. Moore shares her latest research on Securing Autonomous AI Agents by applying a "decompose" approach, which breaks down complex tasks into smaller pieces of work and enforces a security boundary with gated pathways for data flow. This strategy is a pragmatic solution for AI agent isolation and surprisingly results in minimal security performance overhead because the LLM inference processing is the system's slowest part.The goal is to build security in from the start rather than trying to "bolt on" security later. Moore explains how the CNCF Tag Security assessment process leverages a core threat modeling question—listing system actors and data flow—to help projects improve their architecture early. This discussion is for anyone involved in cloud native security assessment and the future of secure AI development, including actionable Kubernetes security best practices advice for both platform engineers and software developers.Read the blog post: www.buoyant.io/ai-kubernetes-episode/stop-bolting-on-security-the-key-to-reliable-ai-agent-systemsFollow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Takeaways✓  Security can be practically applied by breaking down autonomous work into smaller, isolated agents with secured, gated data flow.✓ Adding security layers has a small impact on performance because the LLM tool calls and inference processing are the primary system bottlenecks.✓ The simple act of enumerating all system actors, data flows, and potential attack vectors is a critical self-assessment that illuminates hidden connections for better design.✓ Integrating security early in the development lifecycle is more efficient and enhances overall system reliability compared to bolting it on at the end.✓ Focus on designing a "secure by design" infrastructure and establishing a secure baseline to enable safe experimentation with non-deterministic AI systems.If you found this video valuable, hit that like button, subscribe for more AI security content, and hit the notification bell! Let us know in the comments: What is the biggest platform engineering security challenge you are facing with AI agents today?#AI #Security #Kubernetes #AIAgentIsolation #CloudNativeSecurity #ThreatModeling #PlatformEngineering #CNCF #DevSecOps #LLMSecurity

Is your Kubernetes cluster ready for the next generation of AI workloads? Join Buoyant Tech Evangelist Flynn and Red Hat's Shane Utt from the AI Gateway Working Group as they reveal the critical shift from header-based to payload-based Kubernetes Networking for Agentic AI.In this episode of The Kubernetes AI Show, we dive deep into the challenges of running AI workloads on cloud native infrastructure with AI Gateway leaders, Flynn and Shane. They discuss how the AI Gateway Working Group evolved from the Gateway API inference extension (GIE) to focus on the average Kubernetes user who is often performing inference via egress outside the cluster. A key challenge is that AI models are fundamentally different from microservices—they are not interchangeable, making advanced routing a must.The conversation highlights a critical shift: networking infrastructure must now be retrofitted for payload processing for routing to ensure security, compliance (like GDPR), and proper model selection. We explore parallels between AI’s hardware demands and High Performance Computing (HPC), revealing the problem of low LLM hardware utilization and the talent gap between data scientists and network engineers. Finally, the co-leads detail the security implications of Agentic AI's autonomy, advocating for a defense-in-depth security strategy that includes multi-cluster routing and Model Context Protocol (MCP) servers with 'elicitation' to prevent accidental, destructive actions.Read the blog post: www.buoyant.io/ai-kubernetes-episode/agentic-ai-security-on-kubernetes-understanding-payload-routing-and-defense-in-depthFollow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Key Learnings ✓ Learn why AI models require advanced, non-interchangeable routing logic compared to standard microservices.✓ Discover how the shift to payload processing is essential for semantic routing, security, and GDPR compliance.✓ Understand why AI’s low hardware utilization (20-30%) is a known problem from High Performance Computing (HPC).✓ See how Agentic AI introduces a "deeply terrifying" security risk that demands a defense-in-depth security approach like MCP elicitation.If you're building or securing AI workloads on Kubernetes, you can't afford to miss this discussion! Subscribe to The Kubernetes AI Show for more foundational insights, like this one from the AI Gateway leaders. Hit the like button and let us know in the comments: What is the most critical challenge you are facing when building Kubernetes Networking for your Agentic AI applications?#KubernetesNetworking #AIGateway #AgenticAI #CloudNative #GatewayAPI #LLMs #K8s #Microservices #TechEvangelist

Is your organization ready for the rapid pace of AI adoption? We sat down with Rob Koch (Slalom Build, CNCF Deaf and Hard of Hearing Working Group Co-Chair) to discuss how engineers can master automation in the age of non-deterministic systems, focusing on real-world use cases in Kubernetes and platform engineering.In this episode of the AI Kubernetes Show, Koch provides essential insights on building a robust strategy for AI adoption. He emphasizes moving beyond a "move fast, fail fast" approach to focus on clear outcomes, which he describes as applying the engineering mindset to AI. A key discussion point is mitigating risk in non-deterministic systems by implementing strong guardrails, constraints, and human oversight—often by providing precise context and clear specifications.Koch shares a successful example of automating repetitive tasks using AI sub-agents in a database upgrade project. The system analyzed logs, diagnosed resource needs (checking for over/under-provisioning), and automatically drafted documentation, aligning with the "Don't Repeat Yourself" (DRY) principle. Additionally, he highlights the transformative power of AI for accessibility and the promise of LLMs in sign language recognition. Read the blog post: www.buoyant.io/ai-kubernetes-episode/the-engineering-mindset-for-ai-from-automation-to-guardrails Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/Takeaways✓ Strategic AI adoption starts with defining the desired outcome and working backward, not a "fail fast" approach.✓ Mitigate risk in non-deterministic AI systems by implementing guardrails, human oversight, and limiting context.✓ AI is best suited for automating repetitive tasks (DRY principle) to free up engineering time.✓ Subject matter expertise remains crucial for verifying AI output and detecting "hallucinations."✓ AI is a powerful tool for accessibility, helping bridge communication gaps (e.g., English syntax correction for ASL users).If you're tackling AI implementation or just getting started, hit the like button and subscribe for more deep dives into platform engineering! What is the biggest challenge your team faces when adopting AI? Let us know in the comments below! #AI #Kubernetes #PlatformEngineering #AIAutomation #Accessibility 

The rise of AI has dramatically increased code velocity. Join us as Comcast Platform DevOps Engineer Curtis Cook breaks down the shift to platform-level controls and a "zero trust" mindset to secure the future of Platform Engineering. In this episode of the AI Kubernetes show, Curtis Cook shares his expert insights on how the accelerating pace of code development driven by AI tooling is forcing platform teams to completely rethink code quality and security management. He explains that the massive amount of AI-generated code is like "hiring thousands of junior developers," creating a significant scaling problem. Drawing lessons from Kubernetes, Comcast is implementing platform-level controls to automate mundane tasks and enforce standards, adopting the cultural shift of "guardrails, not gates."Curtis dives into Security in the age of AI, where the concern moves beyond infrastructure because AI actively makes decisions, making securing AI workloads absolutely crucial. Comcast’s strategy involves a "zero trust by default" mindset and applying the least privileged access model to AI agents to combat risks like mitigating hallucinations and "context bloat." In the section "The non-deterministic world of AI," he discusses the profound change presented by non-deterministic AI outputs and security threats like prompt injection and model drift. This shift requires new testing methods like statistical validation instead of traditional unit tests. Curtis also highlights the importance of the CNCF community for aspiring platform engineers, noting how AI can help newcomers learn Kubernetes fundamentals.Find all resources and more in the blog post: Takeaways✓ The increase in code velocity from AI-generated code requires a move from policing individual commits to implementing robust platform-level controls.✓ AI is treated as a critical system at Comcast, secured with a "zero trust by default" mindset.✓ Mitigating hallucinations and context bloat is achieved by applying the least privileged access model to AI tools.✓ Non-deterministic AI systems introduce security threats and require advanced testing methods like statistical validation and confidence scoring.✓ The Kubernetes and CNCF community is essential for platform engineers, and AI-powered tools can help new members get started.What is your biggest security concern with generative AI in your software development lifecycle? Let us know in the comments below! If you found this discussion valuable, please like this video, share it with your team, and subscribe for more deep dives into Platform Engineering and cloud-native technology! #PlatformEngineering #GenerativeAI #Kubernetes #ComcastTech #CNCF #ZeroTrust #AIsecurity #CodeVelocity #DevOps 

The future of the service mesh is here! Don't miss this deep dive with Linkerd creator Oliver Gould on how to conquer the toughest challenges of running stateful AI workloads in production.In this episode of the AI Kubernetes Show, Buoyant CTO Oliver Gould reveals how Linkerd is adapting its battle-tested features to the new demands of the AI workloads era. He emphasizes that the cost of AI inference failures is extremely high, making network layer tools like Linkerd’s intelligent load balancing and fault tolerance features more critical than ever. The discussion zeroes in on the emerging MCP (Model Context Protocol), a stateful streaming protocol that poses unique challenges to traditional network tooling, particularly because critical information is buried in the payload, not the headers.Gould details the clash between MCP’s stateful nature and the prevalent Kubernetes microservice architecture, stressing the need to evolve infrastructure planning for auto-scaling and resource provisioning for GPUs. He shares a vision for Linkerd’s MCP support, which includes a load balancer mode optimized for streams and extending policy APIs for MCPRoute. Finally, Gould touches on the positive impact of AI tooling on developer productivity, sharing a personal anecdote about how Copilot and Cortex dramatically accelerated the diagnosis of a complex Go race condition, proving the value of using robots for data analysis and humans for high-value design.Read the blog post: Key Learnings/Takeaways✓ The high cost of AI inference failures makes network reliability (load balancing, retries) an enormous "cost lever" for organizations.✓ MCP is a stateful streaming protocol where success/failure information is in the JSON payload, making it opaque to most network headers-based tooling.✓ Running stateful MCP workloads at scale requires evolving stateless Kubernetes infrastructure around auto-scaling, GPU provisioning, and stable load distribution.✓ Linkerd is building a load balancer optimized for streams and extending policy APIs (like MCPRoute) to support the new protocol.✓ AI tooling, like Copilot and Cortex, significantly boosts developer productivity by automating boilerplate (like YAML) and quickly diagnosing complex issues.If you found this discussion valuable, hit the Like button and Subscribe for more insights from the AI Kubernetes Show! Let us know in the comments: What is the biggest stateful workload challenge you are currently facing in your Kubernetes cluster?#Linkerd #ServiceMesh #Kubernetes #MCP  #LoadBalancing #CloudNative #KubeCon

Don't miss this deep dive with Bloomberg's Alexa Griffith on how a financial firm is navigating the cultural and technological shift to Generative AI. Learn the critical role of platform engineering and Kubernetes in building a secure, scalable, and future-proof AI infrastructure.In this episode of the AI Kubernetes Show, Senior Software Engineer Alexa Griffith discusses Bloomberg's journey from predictive to Generative AI. She reveals how their early investment in Kubernetes back in 2016 for AI workloads led to open-source projects like KServe and the newer Envoy AI Gateway, built to manage traffic and unify APIs in a hybrid cloud environment. The conversation explores the massive challenge of wrangling the non-deterministic nature of LLMs, which dramatically affects cost and observability—a problem Bloomberg addresses by building a "deterministic cage" around the core models.Alexa details the new requirements for platform engineering teams, including the shift to token streaming and increased GPU usage. She introduces the concept of a Model Garden as a crucial layer of abstraction and enablement, providing developers with benchmarking data and features to make informed, use-case-driven choices. The discussion also covers critical security concerns, the use of LLMs as judges for evaluation, and the debunking of assumptions around RAG systems safety. Finally, she explains the evolution of performance metrics, highlighting the importance of time to first token and token flow for a good user experience in the era of agentic systems.Read the blog post: Key Learnings/Takeaways✓ Generative AI is an evolution, not a revolution, built on a strong predictive AI foundation.✓ Non-deterministic LLMs are managed by building a "deterministic cage" for control, cost, and observability.✓ The Model Garden is a key platform engineering concept for abstracting model complexity and empowering developers.✓ New performance metrics are critical, specifically time to first token (snappiness) and token flow (continuous experience).✓ The Envoy AI Gateway and the MCP protocol are essential for unifying disparate model APIs and managing agentic systems.If you enjoyed this conversation, hit the like button, subscribe for more content on Kubernetes and AI, and leave a comment below! What is the biggest platform engineering challenge you're currently facing with Generative AI?#GenerativeAI #AI #Kubernetes #PlatformEngineering #LLMOps #AIInfrastructure #KServe #CNCF #CloudNative #TechAtBloomberg

In this episode of The AI Kubernetes Show, we talked with Troy Connor, Senior Software Engineer at Microsoft and a maintainer of Porter and Kubernetes controller-runtime maintainer (two CNCF projects), about the balance between enterprise work and open source contribution and the impact of AI on software development and community maintenance. This post summarizes our discussion, live from KubeCon in Atlanta.

On this episode of The AI Kubernetes Show, we chatted with Diana Todea, Developer Experience Engineer at Victoria Metrics and Co-chair for the CNCF Neurodiversity Working Group. This post summarizes our discussion, which covered the impact of AI, generational shifts, and the role of Developer Experience (DevX).

In this AI Kubernetes Show episode, we chat with OpenSSF CTO and Linux Foundation Chief Security Architect Christopher Robinson, or CRob, as everyone calls him. We explore the potential and the inherent risks of integrating AI into the open source security landscape. 

Learn how structured observability data and the platform team enable fast AI troubleshooting. Balance AI automation with essential developer craftsmanship.

Learn how prompt engineering and a cutting-edge multi-agent workflow pipeline are reshaping software dev. Get the framework for safe AI integration.

Bailey Hayes discusses how AI is reshaping platform engineering, covering WebAssembly (Wasm) for scale and security, agentic workloads, non-deterministic programming, and essential API fuzz testing.