A Semantic S ituation A wareness Framework for Indoor Cyber-Physical Systems

1. Ph.D. in Engineering Dissertation Defense A Semantic Situation Awareness Framework for Indoor Cyber-Physical Systems Pratikkumar Desai Monday, 4/29/2013

2. Cyber-physical system e.g. Intelligent traffic management systems Networked embedded system e.g. wireless sensor networks Embedded systems e.g. thermostat

3. Cyber-Physical Systems Cyber : Computation, communication, and control that are discrete, logical, and switched. Physical : Natural and human-made systems governed by the laws of physics and operating in continuous time. Cyber-Physical Systems (CPS) : Systems in which the cyber and physical systems are tightly integrated at all scales and levels http://www.cs.binghamton.edu/~tzhu/

4. Disaster Management Military Drones Smart Grid CPS Examples Air Traffic Control Smart Home Traffic Management Remote Patient Monitoring

5. Motivation & Challenges (Situation awareness)

6. Motivation Situation: Actual fire at chair Mobile sensing platform

7. Motivation Event : Fire from temperature and CO2 data Event : Fire from temperature and CO2 data Mobile sensing platform

8. Motivation Uncertainty: Sensor data e.g. Due to resolution, calibration or robustness of sensors Mobile sensing platform

9. Motivation Incomplete domain knowledge e.g. Unknown sources in the environment Mobile sensing platform

10. Indoor location awareness Complex system Interoperability

11. Context “is a physical phenomenon,measured using sensors, and product of an event” Context Location Contextual situation awareness: “is a process of comprehending meaning of environmental context in terms of events or entities” Environmental context e.g. coordinates e.g. temperature, CO2, heart rate Location awareness: “is a process of identifying objects from raw spatial information and their relationship with the ongoing events” Contextual situation awareness Location awareness

12. Contextual situation awareness +Location awareness Raw environmental sensor data Raw spatial information Entities (High level abstractions) Object-Entity relationships Situation

13. Contextual Situation Awareness

14. IntellegO • Abductive reasoning • Crisp abstractions Quality-type Quality Entity 0 100 C Fire LowTemp temperature Domain Knowledge Base HighTemp DryIce 0 1000 ppm RoomHeater LowCO2 CO2 HighCO2 NormalCondition http://wiki.knoesis.org/index.php/Intellego

15. Fire DryIce RoomHeater RoomHeater

16. Motivation Incomplete domain knowledge e.g. Unknown sources in the environment Uncertainty: Sensor data e.g. Due to limitation, calibration or robustness of sensors Mobile sensing platform

17. Fuzzy abstractions µ 1160 ppm HighCO2 LowCO2 1 0.9 0.1 0 800 1200 ppm a Membership function

18. Fuzzy abductive reasoning Quality-type Quality Entity 0 80 120 C Fire LowTemp temperature HighTemp 500 °C DryIce 0 800 1200 ppm RoomHeater LowCO2 CO2 1160 ppm HighCO2 NormalCondition

20. Evaluation – Contextual Situation Awareness

21. Complex system Interoperability

22. Semantic Web • Semantic web: • Formally define the meaning of information on web. • Provide expressive representation, formal analysis of resources. • Ontology • Formally represents knowledge as a set of concepts within adomain and the relationships between pairs of concepts. • RDF (Resource Description Framework) • Graph-based language for modeling of information. • Allows linking of data through named properties. Predicate Subject Object InheresIn HighTemp Fire http://www-ksl.stanford.edu/kst/what-is-an-ontology.html

23. Contextual situation awareness (Semantic modeling) Perception process Observation process High-level Abstractions (entity) Raw Sensor Data SSN annotated Observations Low-level fuzzy abstractions (qualities) Fuzzy reasoning Fuzzification Rules SSN Ontology Fuzzy Inference rules ontology Domain Ontology

24. Indoor Localization

25. Indoor location awareness

26. Traditional Indoor Localization Techniques • Active Badge and Active Bat system. • RADAR: An In-building RF-based user location and tracking system. • RFID radar • Object tracking with multiple cameras • Computer vision based localization • Wireless Sensor Network RF Camera TDoA

27. TDoA (Time Difference of Arrival) RF Signal Ultrasonic Signal

28. Trilateration Number of nodes = 3. Outlier rejection and Multilateration

29. The Proposed Algorithm • Utilizes fusion of RSS (received signal strength) of RF signal and TDoA data for accurate distance estimation. • The algorithm stages:- • RSSI data training • Distance estimation • Localization • Uses TDoA as a primary distance estimation technique. • RSSI data is trained and converted into appropriate distance measurements. • The proposed algorithm can be used in absence of one or many TDoA links.

30. Initial Conditions • Distances between all beacons are known and fixed

31. Beacon B1 Transmit Data RSSI Link TDoA Link B1 B2 B4 B3 L

32. Beacon B2 Transmit Data RSSI Link TDoA Link B1 B2 B4 B3 L

33. Beacon B3 Transmit Data RSSI Link TDoA Link B1 B2 B4 B3 L

34. Beacon B4 Transmit Data RSSI Link TDoA Link B1 B2 B4 B3 L

35. Evaluation– Proposed Algorithm

36. Location Awareness

37. Bed-1 Chair-2 Bedroom-2 Bedroom-1 Desk-1 Fireplace-1 Indoor Environment Gym-1 Chair-1 Treadmill-1 Drawingroom-1 Sofa-1 Kitchen-1 Plant-1 Stove-1

38. Hierarchical mapping of the indoor environment

39. xsd:float POI is-a inLo:hasXmax xsd:float ChairPOI inLo:hasXmin has individual inLo:isLocatedIn xsd:float inLo:hasYmax Drawingroom-1 Chair-1 Defining Coverage space (Chair-1) inLo:hasYmin inLo:hasPOI has individual xsd:float inLo:hasZmax Drawingroom inLo:hasUnit inLo:hasZmin xsd:float is-a xsd:string StructuralComponent xsd:float

40. Raw location : ( x, y) = (190 cm, 570 cm)

41. Object-entity relationship Point of Interests Structural Components Entities isLocatedIn hasApplicableEntity hasIndividual hasApplicableEntity

42. Evaluation – Location Awareness Mobile-robot route (430,640) (140,640) Fireplace-1 (140,560) (430,560) Location (a) ( 190,570 ) (200,480) (60,480) Chair-1 Drawingroom-1 (760,360) (610,360) Sofa-1 Location (b) ( 630,325 ) (610,240) (760,240) (200,140) (60,140) (110,340) (180,110) Plant-1 (10,340) (180,10)

43. Location independent reasoning Fireplace Location aided reasoning

44. Comprehensive Framework

45. Comprehensive Framework (System level) Indoor Positioning System Domain Knowledge Semantic Object Identifier Location Awareness Fuzzy Abstraction Rules Environment Sensors Qualities Situation Entities Fuzzy Abductive Reasoning Spatial Reasoning Contextual Situation Awareness

46. Raw Location Data Comprehensive Framework (Semantic modeling) Indoor Location Ontology Semantic Location Identifier Optimized Situation Raw Physical Context Data SSN annotated Observations Low-level Fuzzy Abstractions (Qualities) High-level Abstractions (Entities) SSN Ontology Fuzzy Abductive Reasoning Rules Domain Ontology

47. Object coverage area Mobile robot path Temp: 150 °C CO2:1120 ppm Temp: 180 °C CO2:1160 ppm Location (a): (200,250,20) Location (b): (400,150,30)

48. Object coverage area Mobile robot path Fire: 0.80 Heater: 0.20 Fire: 0.90 Heater: 0.10 Location (a): (200,250,20) Location (b): (400,150,30)

49. Object coverage area Mobile robot path Fire: 0.80 Heater: 0.20 Fire: 0.90 Heater: 0.10 Location (a): Fireplace Location (b): Chair

50. Object coverage area Mobile robot path Fire: 0 Heater: 0 Fire: 0.90 Heater: 0.10 Location (a): Fireplace Location (b): Chair