OCSF, Schemas, and the Convergence of Security and Data Engineering with Zach Schmerber

Zach joins Kostas and Nitay to trace how security data turned into a full-blown data engineering problem. He walks through his path from hospital networks with a sprawling attack surface, into tightly regulated banking, and then into big tech, where petabyte-scale volumes broke the Splunk-style stack he was hired to optimize and pushed him toward columnar storage and table formats like Iceberg and Delta.From there the conversation digs into OCSF, the Open Cybersecurity Schema Framework: why a producer-side schema would shrink the mountain of cleanup work defenders do today, how an unusually open community formed around it in a famously secretive industry, and why security schemas have to standardize semantics and values, not just structure. Zach, Kostas and Nitay also get into the overlap between security and observability stacks, the unknown-unknowns problem that forces teams to store data they may never query, the case for a metrics-plus-security semantic layer, and what changes when agents both need clean context and become the thing you have to monitor.Chapters00:00 Introduction and Zach's background in cybersecurity01:09 Early days in healthcare cybersecurity and data detection02:19 The rise of Splunk and schema on read in security03:07 Moving from healthcare to banking and regulated environments04:21 Transition to big tech and handling massive data volumes08:35 How Splunk's distributed compute works and scaling challenges10:45 The shift towards modern data stacks and table formats11:32 Introduction to the Open Cyber Security Schema Framework (OCSF)14:00 Community and collaboration in cybersecurity data standards15:36 The future of data schemas and standardization in security20:59 Behavioral analytics and threat modeling in cybersecurity23:28 The role of schemas in reducing security data complexity28:01 Convergence of security data and observability34:46 The impact of LLMs and AI on cybersecurity practices50:46 The future of OCSF and standardization in cybersecurity