Summit 2013: River - A data flow management infrastructure, by Harel Ben Attia

זו הרצאתו של הראל על ריבר (נהר) מהכנס האחרון. האזנה נעימה ותודה להראל! Starting from the algorithms which are at Outbrain's core, and ending with Internal and Customer reporting, the Outbrain backend is a data processing monster. As the company grows, our data processing needs grow as well, leading to very complex dependencies between the various processes. These dependencies form a growing challange, both from an operational viewpoint and from a development viewpoint. The Outbrain River infrastructure has been created in order provide a solution for this challenge. Outbrain River provides the following major features: Declarative job definitions Event-driven dependency management Decentralized development of data flows Ops-level managability Out-of-the-box support for JDBC and Hive/Hadoop, easily extensible to any other unit-of-work