Active

1. Active Active, a platform for building intelligent software Dr. Charles Baur (EPFL) Adam Cheyer (SRI International) Didier Guzzoni (EPFL) Active

2. Presentation Plan • Introduction • Problem space • Active proposition • Active Framework • Active Ontologies • Implementation • Methodologies • Applications • Conclusion Active Introduction

3. Motivation • Our information environment is rich and complex • Ubiquitous access to a wealth of data and services • Software and hardware industry constant innovations • UIs have not changed: Simple click-and-do approach not enough • Need for computer assistants • Interact naturally with humans • Can be delegated complex tasks • Observe, understand, anticipate and act Active Introduction

4. Intelligent Systems Understand Anticipate Resolve Interpret Plan action Observe Act Effectuate Communicate Listen Vision Sense Intelligent Systems : Naturally collaborates with human users to deliver services and contents through adaptable, efficient, multimodal user interfaces. Active Introduction

5. Difficult Task • Requires numerous AI techniques • Language processing • Plan execution • Dynamic service brokering (MAS) • Implementation is difficult • HCI components (speech recognition, vision, robotics) • Large teams of specialists • Different programming languages and platforms • Testing, debugging and maintenance is difficult • Performance is likely to be affected Lack of an integrated tool and methodology to easily and effectively build intelligent systems Active Introduction

6. Goal of Active • Provide programmers with an integrated framework and a methodology to build complex AI-based systems • Capable of encapsulating AI techniques • Language processing, plan execution and agent type techniques • Programmer friendly • Small teams • Based on popular programming languages (Java/Javascript) • Offers an IDE (code, test, debug and deploy) • Open and standard compliant • SOA-based (SOAP, REST, RMI) • Deployment (Java, J2EE) Active Active Framework

7. Basic Concept : Active Ontologies • Ontology : A data structure • Formal representation for domain knowledge • Classes, attributes, relations P movie P genre P actor P rating • Active Ontology : A processing environment • Processing elements arranged according to ontology notions • Communication channels Active Active Framework

8. rule set rule rule rule condition condition condition action action action Active Ontology : Processing • Production Rule System • Rules Sets • Rules (Conditions, actions) • Data store (facts) • Current state of the system • Evaluation Engine • Evaluation passes • Active Innovations • Organizing rules around Ontologies helps design and debug • Developer friendly rule language. Enhanced Java/Javascript with unification P movie Active Active Framework

9. Active Application Design • One or more Active Ontologies • Hosted on Active runtime • Typically : • Language processing • Plan execution • Dynamic service brokering • Service oriented (SOA) • Loosely coupled • Sensors (user interfaces, speech recognition, vision) • Actuators (robots, user interfaces) • Reusable • Dynamically swapped Active runtime services Active Active Framework

10. Active Ontology Active Ontology Active Ontology Active Ontology Implementation Active Editor • Active Server • Hosts Active Ontologies • Maintains a fact store • Runs evaluation engine • Extensions • Active Editor • IDE • Code, deploy, test • Pluggins • Active Console • Manages Active Server Active Server deploy Facts store debug Evaluation Engine monitor Active Console Active Implementation

11. Active Server • Runs and hosts Active Ontologies • Evaluation Engine • Fact Store • Implementation • Java application/J2EE webapp • SOAP / RMI interface • Rule language is Java/JavaScript enhanced by unification • Extensions • Encapsulate pre-compiled complex operations Active Server Active Ontologies Evaluation Engine Fact store Extensions SOAP/RMI interface services Active Implementation

12. Active Editor • IDE • Graphical editing of ontologies • Specialized concept and rule editors • Active ontology definition files saved locally (XML) • Active Server Connection • Deploy/undeploy edited ontologies • Integrated test/debug • Plugins • Automatically creates concepts and rules based on interactive wizards Active Implementation

13. Active Console • Management tool • Monitor and configure deployed Active Ontologies • SOAP/RMI interface • Query (read) panel • Construct complex queries to Active Server • Tabular result sets • Store (write) panel • Stimulates Active Ontologies by sending events to the server Active Implementation

14. Active Methodologies • Language Processing • Chart Parsing • Event based • Agent techniques • Delegated computing • Dynamic service selection • Plan execution • Process Execution Engine • Reactive Planning Active Methodologies

15. Language Processing : Grammar-Based • Grammar-based • Grammar based • Chart parsing • Advantages • Formal parsing (Mathematical expressions) • Deterministic • Disadvantages • Not flexible • Not robust to missing words • Not well suited for non-reliable input modalities (Speech recognition) Active Methodologies

16. Language Processing : Domain-Based “find action movies in San Francisco” • Implementation • Bottom Up • Leaves : Word set, regex • Nodes : Gather, Select • Context • Kept among utterances • Errors, Suggestions • Advantages • Robust to syntax • Ports well to different languages • Wizards • Easy modifications “nearby Chinese restaurants” Active Methodologies

17. Activity/Dialog Modeling • Modeling • Dialogs, Activities, Behaviors • Full-featured workflow management • State / Transitions • Flow instances, Instance space • Basic flow constructs • Start, End • Wait, Switch (branching) • Parallel, sequences • Active Implementation • Set of plugins (Editor Wizards) • Extension (Java/JavaScript) to access flow variables Active Methodologies

18. Dynamic Service Brokering • Delegated Computing • What instead of how or who • Service Registry • Service categories cross-ontology references • Service instances (providers) • Broker • Parallel, Sequential, Broadcast • Third-party meta-agents • Active Implementation • Specialized Active Ontology • Server Extension, IDE Wizards Active Methodologies

19. Prototypes / Demonstration • Information Retrieval Assistant • Meeting Organizer Assistant • Operating Room Assistant Active Prototypes

20. Active Server Language Processing Plan Execution Delegation Extensions Email SOAP City Information gmail SMTP/POP server Yahoo! Local Google Movies Opentable Information Retrieval Assistant Active Prototypes

21. gmail calendar gmail SMTP/POP server Meeting Organizer Assistant Active Server Language Processing Plan Execution Delegation Extensions IM SOAP Email Active Prototypes

22. Speech Recognizer Patient vital signs probes Context History Mouse 3D Hand/Head Tracker Gesture Recognizer User Interface Text to Speech Operating Room Assistant Language Processing Plan Execution Powered Endoscope Actions through Delegation Active Prototypes

23. Active Advantages • One platform to learn for programmers • Can build all three tiers of applications in Active • One language and tool to learn • Easier to debug, test and deploy • AI Encapsulation-Lowering the bar • Methodologies encapsulated as Active Wizards and Extensions • NLP, process execution, service brokering • Componentized reusable sensors and actuators • TTS, Speech, Gesture recognizers, vision systems • Open and Standard-based • SOA design • Ease of integration through SOAP and REST • Reuse of existing components in multiple applications Conclusion Active

24. Ongoing Work • Research Topics • Combine activity recognition with process execution • Implement and evaluate BDI-like behaviors with Active (goal and intention stack) • Active implementation and features • Scalability • Performance optimizations • Lightweight embedded Active Server • User Evaluations • Put some of our systems on-line • Measure feedback from surgeons (Intelligent Operating Room) Conclusion Active

25. Thank You ! • Questions ? • Suggestions ? • Remarks ? Conclusion Active