The MaxStream Federated Stream Processing System

Project Description

Despite the availability of several commercial data stream processing engines (SPEs), it remains hard to develop and maintain streaming applications. A major difficulty is the lack of standards, and the wide (and changing) variety of application requirements. Consequently, existing SPEs vary widely in data and query models, APIs, functionality, and optimization capabilities. This has led to some organizations using multiple SPEs, based on their application needs. Furthermore, management of stored data and streaming data are still mostly separate concerns, although applications increasingly require integrated access to both. In the MaxStream project, our goal is to design and build a federated stream processing architecture that seamlessly integrates multiple autonomous and heterogeneous SPEs with traditional databases behind a common SQL-based declarative query interface and a common API in a way to facilitate the incorporation of new functionality and requirements.

Project Members

• Current: Nihal Dindar, Nesime Tatbul (ETH Zurich); Laura Haas (IBM Almaden); Renee Miller (University of Toronto).
• Former: Irina Botan, Roozbeh Derakhshan, Younggoo Cho, Kihong Kim, Chulwon Lee, Beomjin Yun (SAP Labs Korea); Girish Mundada (SAP Labs USA); Ming-Chien Shan, Ying Yan,  Jin Zhang (SAP Labs China).
  

Publications

• "SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems", Irina Botan, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Renee Miller, Nesime Tatbul, International Conference on Very Large Databases (VLDB'10), Singapore, September 2010.
• "Streaming Data Integration: Challenges and Opportunities", Nesime Tatbul, IEEE ICDE International Workshop on New Trends in Information Integration (NTII'10), Long Beach, CA, March 2010.
• "A Demonstration of the MaxStream Federated Stream Processing System", Irina Botan, Younggoo Cho, Roozbeh Derakhshan, Nihal Dindar, Ankush Gupta, Laura Haas, Kihong Kim, Chulwon Lee, Girish Mundada, Ming-Chien Shan, Nesime Tatbul, Ying Yan, Beomjin Yun, Jin Zhang, Demonstration Proposal, IEEE International Conference on Data Engineering (ICDE'10), Long Beach, CA, March 2010.
  
• "Federated Stream Processing Support for Real-Time Business Intelligence Applications", Irina Botan, Younggoo Cho, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Kihong Kim, Nesime Tatbul, VLDB International Workshop on Enabling Real-Time for Business Intelligence (BIRTE'09), Lyon, France, August 2009.
• "Design and Implementation of the MaxStream Federated Stream Processing Architecture", Irina Botan, Younggoo Cho, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Kihong Kim, Chulwon Lee, Girish Mundada, Ming-Chien Shan, Nesime Tatbul, Ying Yan, Beomjin Yun, Jin Zhang, Technical Report TR-632, ETH Zurich Department of Computer Science, June 2009.

More Information