Download
a new model and architecture for data stream management n.
Skip this Video
Loading SlideShow in 5 Seconds..
Aurora PowerPoint Presentation

Aurora

151 Views Download Presentation
Download Presentation

Aurora

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. A new model and architecture for data stream management Aurora

  2. The Topic • Aurora • The prototype • DBMS / SPE / DSMS • UI • The query language • The project • The authors

  3. The Authors • M.I.T. , Department of EECS and Laboratory of Computer Science • Michael Stonebraker • Brandeis University, Department of Computer Science • Daniel J. Abadi • Mitch Cherniack • Brown University , Department of Computer Science • Don Carney • Uğur Çetintemel • Christian Convey • Sangdon Lee • Nesime Tatbul • Stat Zdonik

  4. Talk Overview • Stream Processing Engines • SQuAl • Runtime • Related work

  5. Aurora Stream Processing Engines

  6. Stream Processing Engines • HADP vs DAHP • Events & Triggers • Continuous Queries • Real-time processing • Transient data • Lossy information

  7. Application Domains • Online Auctions • Network Traffic Management • Habitat Monitoring • Military Logistics • Immersive Environments • Road Traffic Monitoring • System Monitoring

  8. Aurora SQuAl (Stream Query Algebra)

  9. SQuAl Overview • Connection Points • Models • Continuous Query • View • Ad-hoc Query • Operators • Order-agnostic • Order-sensitive

  10. SQuAl Operators • Order-agnostic • Filter • Map • Union • Order-sensitive • BSort • Aggregate • Join • Resample

  11. Filter (Unordered)

  12. Map (Unordered)

  13. Union (Unordered)

  14. BSort (Ordered)

  15. Aggregate (Ordered)

  16. Join (Ordered)

  17. Resample (Ordered) • Based on RRDTool’s philosophy? • Paper: • Simple interpolation • Use The Force, Read The Source: • Average • Count • Sum • Max • Min • LastVal

  18. SQuAl: Example

  19. Aurora Runtime

  20. Query Optimization • Dynamic Continuous Query Optimization • Inserting projections • Combining boxes • Reordering boxes • Ad-hoc query optimization

  21. Real-time Scheduling • Timestamped Tuples • Train scheduling • Interbox nonlinearities • Intrabox nonlinearities • Superboxes • Introspection • Static • Run-time

  22. Handling overload • QoS specifications • Response times • Tuple drops • Values produced • Load Shedding • Not Implemented at the time

  23. Aurora Related work

  24. Related work • STREAM • Stanford University, 2000-2006 • Telegraph • UC Berkley, 2000-2007? • SASE • UC Berkley / Mass Amherst, 2006-2008? • Cayuga • Cornell University, 2005-2007? • PIPES • University of Marburg, 2003-2007? • NiagaraCQ • University of Wiscon-Madison, 1999-2002

  25. Aurora’s Evolution

  26. Complex Event Processing Today • Oracle • Oracle CEP • Microsoft • MS SQL Server StreamInsight • Open Source • OpenPDC • StreamBase • Aurora’s Grandchild • IBM • SPADE • Active Middleware Technology