Scaling Platform Engineering: Shopify’s Blueprint - OpenObservability Talks S4E08

In this episode, join us as we delve into the intricate world of Platform Engineering with Aparna Subramanian, Director of Production Engineering at Shopify. Discover how Shopify, a powerhouse in e-commerce, masters the art of scaling platform engineering. Gain invaluable insights into their strategies, innovations, and lessons learned while navigating the complexities of sustaining and evolving a robust infrastructure to support millions, even through special peak events like Black Friday and Cyber Monday. If you're keen on understanding the backbone of a thriving online platform, don’t miss out on this episode. Aparna started her career as a Software Engineer and has spent most part of her almost two decades of technology experience specializing in Infrastructure and Data Platforms. In her current role she leads Shopify’s Cloud Native Production Platform. Previously, she was Director of Engineering at VMware where she was a founding member of Tanzu on vSphere, a Kubernetes Platform for the hybrid cloud. She also serves as co-chair of the “CNCF End User Developer Experience” SIG and as member of the CNCF End user technical advisory board. The episode was live-streamed on 11 January 2024 and the video is available at https://www.youtube.com/watch?v=6ShtsTTUizI OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. https://www.youtube.com/@openobservabilitytalks   ⁠https://www.twitch.tv/openobservability Show Notes: 00:00 - Show intro & 2023 stats 01:49 - Episode and guest intro 04:15 - Shopify’s scale 06:09 - Shopify’s journey to Platform Engineering 08:56 - Shopify’s platform structure 11:49 - division of responsibility 13:51 - golden path vs flexibility 17:58 - balancing flexibility and abstraction 19:56 - platform group structure 23:28 - handling load spikes 28:55 - FinOps in Platform Engineering 38:38 - avoiding silos and the cultural aspect 41:13 - CNCF end-user SIG and community challenges 49:24 - KubeCon Paris and guest contact  51:03 - OpenTofu reached GA 53:33 - Isovalent acquired by Cisco 55:00 - year-end summary articles 57:07 - .NET Aspire released preview2 58:58 - Episode and show outro Resources: Shopify Engineering Blog https://shopify.engineering/ Performance wins at Shopify: https://www.shopify.com/news/performance%F0%9F%91%86-complexity%F0%9F%91%87-killer-updates-from-shopify-engineering CNCF End User SIG https://github.com/cncf/enduser-public OpenTofu has reached GA https://logz.io/blog/terraform-is-no-longer-open-source-is-opentofu-opentf-the-successor/?utm_source=devrel&utm_medium=devrel Observability in 2024: https://thenewstack.io/observability-in-2024-more-opentelemetry-less-confusion/ OpenTelemetry in 2024: https://www.apmdigest.com/2024-application-performance-management-apm-predictions-4 .NET Aspire preview2: https://devblogs.microsoft.com/dotnet/announcing-dotnet-aspire-preview-2/  Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: @horovits LinkedIn: in/horovits Mastodon: @horovits@fosstodon Aparna Subramanian ================= Twitter: @aparnastweets LinkedIn: https://www.linkedin.com/in/subramanianaparna/