Skip to content
How One Engineer Reduced Incident Response with a Custom Chaos Engineering Tool episode artwork

Jun 10, 2026 9 min

How One Engineer Reduced Incident Response with a Custom Chaos Engineering Tool

from The Software Engineering Podcast with Fexingo: Code, Architecture, and Engineering Best Practices Hosted by Fexingo

In this episode of The Software Engineering Podcast, Lucas and Luna dive into chaos engineering — but not the abstract kind. They focus on a specific story: how a senior engineer at a mid-sized e-commerce company built a lightweight chaos simulation tool from scratch after existing solutions proved too heavy for their team. The tool intentionally injects latency into specific microservices during off-peak hours, and the team runs it against a staging environment that mirrors production. The result? They uncovered a cascading timeout failure in their payment service that would have hit Black Friday traffic. Lucas walks through the technical architecture: a Go-based control plane, a configurable experiment specification in YAML, and a simple dashboard that visualizes error budgets. Luna pushes back on the risk of running chaos experiments at all, and Lucas explains how gradual rollout and blast radius limits make it safe. The episode also covers how the team automated experiment scheduling using cron jobs and Slack notifications, turning chaos engineering from a quarterly exercise into a weekly habit. Listeners will come away with a concrete mental model for building their own lightweight chaos engineering practice — no vendor required. #ChaosEngineering #SoftwareEngineering #IncidentResponse #Microservices #LatencyInjection #GoLang #YAML #ErrorBudget #Resilience #Slack #CronJob #StagingEnvironment #Technology #FexingoBusiness #BusinessPodcast #CodeArchitecture #EngineeringBestPractices #TheSoftwareEngineeringPodcast Keep every episode free: buymeacoffee.com/fexingo

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

How One Engineer Reduced Incident Response with a Custom Chaos Engineering Tool

0:00 9:01

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of The Software Engineering Podcast with Fexingo: Code, Architecture, and Engineering Best Practices?

This episode is 9 minutes long.

When was this The Software Engineering Podcast with Fexingo: Code, Architecture, and Engineering Best Practices episode published?

This episode was published on June 10, 2026.

Can I download this The Software Engineering Podcast with Fexingo: Code, Architecture, and Engineering Best Practices episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.