Charting New Territory: OpenTelemetry Embraces Profiling - OpenObservability Talks S4E10

OpenTelemetry is expanding beyond the traditional “three pillars of observability” and introduces a groundbreaking addition to its signals - Continuous Profiling. The new Profiling Special Interest Group (SIG) that was formed to lead the topic has already made significant advancements, to be featured at KubeCon Europe. Join us in this special panel episode of OpenObservability Talks as we explore the significance of this new dimension in understanding application behavior, optimizing performance, and gaining deeper insights into your systems. Our expert guests, Felix Geisendörfer and Ryan Perry, members of the OpenTelemetry Profiling SIG, share their insights into how Profiling enhances the OpenTelemetry framework, and update on the work for open specification and implementation.  This special episode hosts a panel of two distinguished members of OpenTelemetry’s Profile SIG, and prominent members of the observability vendor ecosystem. Felix Geisendörfer is a Senior Staff Engineer at Datadog where he works on Continuous Profiling and contributes to the Go runtime. Before that he was working at Apple, co-founded Transloadit, contributed to node.js and inspired a generation of mad scientists to program flying robots with it. Ryan Perry is Principal Product Manager at Grafana Labs. He has built a career at various startups while actively contributing to open source projects and advancing open telemetry initiatives. Most recently he built Pyroscope, an open source continuous profiling project/company, which has been acquired by Grafana Labs. The episode was live-streamed on 7 March 2024 and the video is available at https://www.youtube.com/watch?v=iGM67RT12gQ OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.https://www.twitch.tv/openobservabilityhttps://www.youtube.com/@openobservabilitytalks   Show Notes: 00:00 - show intro 01:03 - episode and guests intro 04:02 - trends and advancements in the Profiling space 05:42 - from cost and performance into broader observability  11:27 - turning profile data into metrics 12:45 - runtime vs. full host profilers and eBPF use 18:44 - pprof JFR and other existing profile standards 21:19 - profile visualizations - from flame graphs to timeline view  22:37 - entrepreneur PoV on the profiling market 26:54 - OpenTelemetry adds profiles as a new signal 32:22 - OTel choosing a pprof extended standard   39:06 - discrete events vs. pre-aggregated data 41:09 - use cases for processing profiling data 44:19 - OTel Profiles reference implementation  49:11 - latest milestone and roadmap 54:44 - who’s involved in OTel Profiles 56:41 - how to follow OTel Profiles and the guests 59:34 - March community events and conferences 1:00:38 - Falco and CloudEvents projects reached CNCF graduation   1:01:59 - Prometheus and Linkerd latest releases 1:03:29 - Netflix open-sources bpftop CLI for eBPF app performance monitoring 1:05:15 - show outro Resources: Continuous Profiling: A New Observability Signal (previous episode): https://logz.io/blog/continuous-profiling-new-observability-signal-in-opentelemetry/?utm_source=devrel&utm_medium=devrel  OpenTelemetry extension proposal for adding Profiles: https://github.com/open-telemetry/oteps/pull/239 OTel Profile SIG notes: https://docs.google.com/document/d/19UqPPPlGE83N37MhS93uRlxsP1_wGxQ33Qv6CDHaEp0/edit#heading=h.63a4klfdbcob eBPF adoption in observability - github stats: https://www.linkedin.com/feed/update/urn:li:activity:7171044354667585537/  ProfilerPedia: https://profilerpedia.markhansen.co.nz/  Netflix releases bpftop CLI tool: https://netflixtechblog.com/announcing-bpftop-streamlining-ebpf-performance-optimization-6a727c1ae2e5 OpenTelemetry announces support of Profiles at KubeCon Paris 2024: https://opentelemetry.io/blog/2024/profiling/ Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: https://twitter.com/horovits  LinkedIn: https://www.linkedin.com/in/horovits/  Mastodon: @[email protected] Felix Geisendörfer =============== Twitter: https://twitter.com/felixge  LinkedIn: https://www.linkedin.com/in/felixg2/  Ryan Perry ========== Twitter: https://twitter.com/rperry_  Linkedin: https://www.linkedin.com/in/ryanaperry/