Gustavo Alonso gave the following talk at MAKI Distinguished Lecture Series, TU Darmstadt, Nov. 7. 2013.
"Need for Speed: Building Big Data Platforms"
Big data and the NoSQL movement are just manifestations of a number of trends on the way data is processed and the type of services built on top of data. One the one hand, Internet, mobile telephones, social networks, on-line advertisement, and the proliferation of digital services have resulted in unseen demands on existing data platforms. On the other hand, the complexity of the queries, the need to support strict SLAs, and the wide variety of data types call for new approaches in the design of data processing engines. At the same time, the underlying hardware platforms are evolving at an amazing speed, changing many of the established assumptions about how to build software systems.
In this talk I will cover SwissBox, a long term, umbrella project at the Systems Group at ETH Zurich, where we are redesigning data processing engines from the ground up to adapt them to modern workloads. The project covers from hardware acceleration and operating system – database codesign, to radically new ways of organizing the engine's architecture and novel strategies to exploit multi-core hardware. Part of the system is already used commercially and the results obtained so far indicate the performance and the guarantees that can be provided with the new architecture exceed by orders of magnitude what can be achieved today.