Advanced media-oriented systems research: Ubiquitous capture, access, and interpretation

1. Advanced media-oriented systems research: Ubiquitous capture, access, and interpretation • Faculty involved with RI-related projects • Kishore Ramachandran, Mustaque Ahamad, Karsten Schwan, Richard Fujimoto, Ken Mackenzie, Sudha Yalamanchili, Irfan Essa, Jim Rehg, Gregory Abowd, Yannis Smaragdakis, Santosh Pande, Calton Pu, Ling Liu, ... • Federal funding • NSF RI, NSF ITR, DARPA, DOE • State funding • Yamacraw, GT Broadband Institute • Industry funding (equipment and personnel) • HP, Intel, Microsoft, IBM

2. MediaBroker and DFuse:Powerful support for emerging applications Kishore Ramachandran Jim Rehg, Phil Hutto, Ken Mackenzie, Irfan Essa College of Computing, Georgia Tech Kath Knobe, Jamey Hicks, Raj Kumar HP Labs Students: Rajnish Kumar, Matt Wolenetz, Ilya Bagrak, Martin Modahl, Bikash Agarwalla, Junsuk Shin, Arnab Paul, Sameer Adhikari, Nissim Harel, Hasnain Mandviwala, Yavor Angelov, David Hilley

3. www.amonline.net.au/invertebrates/mal/gallery/octopus.htm

4. Skiff Skiff • Distributed system components • sensors, actuators, embedded processors, PDAs, laptops, clusters… camera camera Sensor Fusion Sensors Actuators Unix / Linux / NT cluster Data Aggregators “OCTOPUS” DIAGRAM head / arms / tentacles

5. Killer App? • Application context • distributed sensors with varying capabilities • control loop involving sensors, actuators • rapid response time at computational perception speeds

6. Emerging Applications • Distributed Collaboration • Emergency Response • Collaborative search and rescue • Evacuation management • “Aware” Environments Key Requirements • Data distribution and Infrastructure adaptation

7. Application Characteristics • Physically distributed heterogeneous devices • Interfacing and integrating with the physical environment • Diverse stream types (low to high BW) • Diverse computation, communication and power capabilities (from embedded sensors to clusters) • Stream fusion/transformation, with loadable code • Resource scarcities • Dynamic join/leave of application components Two projects: • MediaBroker • DFuse

8. A Fusion Channel (a ‘Virtual Sensor’) Inputs (sensors or other fusion channels) Consumers (actuators or other fusion channels) F() . . . . . . Abstractions…abstractions • Goals: • Temporal guarantees, efficient streams, distributed synchronization, fusion support • Simplify application development • Key result: Fusion Channels • built on top of D-Stampede [ICDCS 2002]

9. The MediaBroker • An architecture for Data Distribution • Type management, Scalability, Stream “sharing” • A sample scenario for distributed collaboration • Physically distributed participants in a classroom setting • Interaction via a variety of devices • System takes care of any needed data transformations and distribution MediaBroker Infrastructure

10. audio video sonar bio fusion derivation sharing registry Re-publish discovery D-Stampede virtual sensors applications MediaBroker Infrastructure( ) • Fusion Channels and Virtual Sensors • Resource Discovery and Sharing • Sensor-Provided Attributes • Sensor-Derived Attributes • Scalability • Access Control and Resource Scheduling • Security

11. MediaBroker and Typed Streams • Experimental “type-lattice” registry, discovery, and resolution structure Fusion Channel Performs text-to-speech Fusion Channel Outputs A/V as MPEG, Incorporates text as caption Today’s Lecture:

12. DFuse [ACM SenSys 2003] • An architecture for Infrastructure Adaptation • A sample scenario for an aware environment • Field trip for a class! • Deployed power-constrained sensors • Dynamic wireless network consisting of the students’ PDAs • In-network stream filtering and aggregation

13. Sensors Collage Sink (Display) Filter Sample surveillance application task graph: filter and collage are the fusion functions. Cameras • Deployed on iPAQ farm! • Tested 4 cost-functions DFuse Fundamentals • Fusion Module: Deploys task graph on sensor network • Comprehensive API for fusion and migration • Low-overhead • Placement Module: Employs a self-stabilizing algorithm to place fusion points in the network • Energy and application aware cost functions • Localized decisions

14. EvaluationApplication Timeline showing Network Traffic

15. Applause!!! 