Middleware Platform for Sentient Computing Applications

1. Middleware Platform for Sentient Computing Applications Thirunavukkarasu Sivaharan, Maomao Wu, Gordon Blair, Adrian Friday, Paul Okanda. Computing Department, Lancaster University, UK 2nd MiNEMA Closed Workshop@ Lancaster, 1st Dec 2004

2. Overview of Presentation • Introduction • Sentient Objects • Research Challenges & Component Frameworks • Middleware Architecture • Sentient Vehicle Demonstrator • Conclusions Lancaster University

3. Introduction(2) EU FET Project : CORTEX • Universidade de Lisboa (Portugal) • Lancaster University (United Kingdom) • Trinity College (Ireland) • Universität Ulm (Germany) Aims • Middleware support for constructing distributed mobile proactive applications based on real-time sentient objects • Proposes sentient object model to support the construction of mobile, context aware, decentralised ,autonomus ,proactive and collaborative applications such as intelligent vehicles and smart buildings. • A middleware for networked embedded systems Lancaster University

4. Sentient Object Model(1) • Sentient Object Model • System consists of environment and a set of sentient objects • Sentient objects are capable of independently sensing the environment, derive context and infer autonomous actions • Sentinet objects communicate using event channels to establish higher level context and thus cooperate with each other Lancaster University

5. Sentient Object(2) Lancaster University

6. Autonomous sentient vehicle application in MANET • Autonomous navigation of vehicles from a source to destinations • Cooperating vehicles in MANET • Context aware vehicles Lancaster University

7. Some of the research challenges addressed • Suitable Communication Model for MANET • Routing in mobile ad-hoc environment • Context-awareness • End-to-End QoS and Fail safety • Run time and deployment time reconfigurations Lancaster University

8. Component Framework based Reflective Middleware • Publish-Subscribe Component framework (CF) • Multicast CF • Context CF • Resource Management CF Lancaster University

9. Why Component Framework based Middleware Platform? • Middleware is engineered as family of Component frameworks (CF) using Reflection and component technology • Each CF addresses specific research areas • Component Frameworks are highly configurable and dynamically reconfigurable (with the granularity of a component) • Clear separation of concerns • Adaptable to diversity of CORTEX applications • Reduction of memory footprint • CFs are implemented using Lancaster’s OpenCOM reflective component technology Lancaster University

10. Middleware Architecture Sentient Objects Sentient Objects Context CF- Sensor Fusion Inference Engine M I D D L E W A R E Programming Interfaces Publish-Subscribe CF- (for MANET) Timely Computing Base Group Communication CF-( Ad-hoc Multicast ) Payload Channel TCB control channel WLAN 802.11b (ad-hoc), Windows CE Middleware Configuration for MANET Lancaster University

11. Publish-Subscribe CF(1) • Communication model inspired by STEAM • Implicit event model • Sender & receiver based event filtering • Subscription Language supports subject, content & context based event filtering • Supports distance based context filtering & extensible to other contexts • XML based generic events • Events transported via selectable Multicast protocol Lancaster University

12. ISubscribe IDispatch IPublish Subscriber Dispatcher Publisher Notifier ISOAPMessaging IFilter IApplicationNotify SOAP Messaging IFilter Filter Filter ISOAPTransport Receptacle SOAPtoMulticast Interface IMulticast Multicast Publish-Subscribe CF(2) Lancaster University

13. Shared memory based IP Multicast Probabilistic Multicast Multicast CF • Underlying event Routing Protocol is based on multicast • The multicast protocol for ad-hoc networks is a probabilistic, stateless and multi-hop protocol • We offer this service in the form of a component framework. Lancaster University

14. Context CF (1) • Sensor capture and fusion • Multivariate Gaussian modelling • Bayesian networks • Dead-reckoning • Inference engine • A program that reasons about a set of rules (a knowledge base) in order to derive an output. • The knowledge is encoded as a set of production rules, contexts are represented as “fact”. • CLIPS – C Language Integrated Production System, its internal implementation is based on RETE net. Lancaster University

15. Context CF (2) • CLIPS rule sample • The paradigm facilitates uniform treatment of both context and QoS • Rules to trigger adaptations and actuations based on changes in measure of QoS data • CLIPS DLL and OpenCOM component for WinXP and WinCE (defrule rule-obstacle-near "CLIPS rule for obstacle near" (car-id (id ?id)) ?f1 <- (obstacle (distance near)) => (retract ?f1) (publish ?id stop) ) Lancaster University

16. End-to-End QoS Management and Fail Safety- Timeliness requirement • How can this be achieved? • Enforcing timely perceptions of the environment and timely actuations on it. • Which means timely event delivery and awareness of QoS of the event channels used for inter-sentient object communication • The key issue in uncertain and highly dynamic environments is that timing bounds for distributed actions may be violated because of timing failure Lancaster University

17. End-to-End QoS Management and Fail Safety-Timeliness Requirements • We model the uncertainty of timely event dissemination via event channels using a dependable timing failure detection service. • This service is provided by University of Lisboa’s Timely Computing Base (TCB) • TCB facilitates to construct distributed event channels with timing bound specification • This enables publisher or subscriber to be aware of the timing failures of event channels • Thus providing awareness of timing failure probability for a given required coverage • Fail safety is achieved by switching to fail-safe state as soon as QoS specifications are violated. Lancaster University

18. Autonomous Sentient Vehicles Demonstrator • Two Sub problems • Cooperative behaviour without human control • Autonomous vehicle navigation from a given source to pre-determined destination • Vehicles Objectives • Travel along a given path( virtual circuit-VC) defined by set of GPS waypoints and bearings. • Every vehicle that travels on the VC cooperate with other vehicles to avoid collisions and travel safely • Obey external roadside traffic lights. • Give way to pedestrians who cross the road. Lancaster University

19. Location aware Cooperating Sentient Vehicles Satellites Car publishes on Carcontrol channel: Event Packet: <car status, Location> Car publishes on Carcontrol channel: Event Packet: <car status, Location> Car subscribes to: CarControlChannel & Receives events from other cars Car subscribes to: CarControlChannel & Receives events from other cars IEEE 802.11b(ad-hoc) ---Event Channel---CarControlChannel Car A Car B 4m OC BEHIND OC CLOSE( 4m) OC FAR(4- 10m) OC VERY FAR OC BEHIND OC CLOSE Other Car’s location context w.r.t Car B Other car’s location context w.r.t car A OC – Other car Lancaster University

20. Non event publishing obstacle Ultrasonic sensors Ultra sound waves Pedestrian detection • Obstacle Sensing Service: Consumes raw ultrasonic sensor data and fuses using a suitable algorithm (reliable, timely-unreliable, Gaussian, …) to derive higher level obstacle distance context such as NEAR , FAR , NOOBJECT. Lancaster University

21. Inference Service Obstacle Sensing Service Location Sensing Service Direction Sensing Service GPS Fusion 2 Component Ultrasonic Fusion 2 Component GPS Fusion 1 Ultrasonic Fusion 1 Compass Fusion 1 CLIPS Inference Engine Speed Actuator Ultrasonic sensor Example: The Car Sentient Object & Context CF Steer Actuator GPS sensor Consume Produce Sentient object Sentient object Digital Compass sensor Interface receptacle Lancaster University

22. Sentient Vehicle Test Bed Lancaster University

23. Cont’d Lancaster University

24. Cont’d Lancaster University

25. Demo Settings Lancaster University

26. Waypoint 3 Waypoint 2 Traffic Light Waypoint 4 Waypoint 1 Virtual Circuit Lancaster University

27. Demo Video Lancaster University

28. Concluding Remarks • The sentient object model • has proved to be valuable programming abstraction for the development of real-time, cooperative, context-aware applications. • The component-Framework based Middleware approach • offers benefits of flexible configuration and reconfiguration of the middleware components • The middleware architecture • also provides the management of non-functional concerns such as timeliness and reliability properties. • Our middleware is reusable • we are keen to investigate the generality of our approach by applying our middleware to other application domains involving embedded autonomous components. Lancaster University

29. Thank You Questions Lancaster University