Assessment of Ontology-based Tools: A Step Towards Systemizing the Scenario Approach

1. EON’2002 Assessment of Ontology-based Tools:A Step Towards Systemizing the Scenario Approach AlainGiboin, Fabien Gandon, Olivier Corby, Rose Dieng INRIA Sophia Antipolis

2. Requirement and Position • Requirement: Making Ontology-based Tools (OBTs) acceptable • by industry(Angele and Sure, 2002) or • by end users in companies(Gómez­Pérez) • Position: To meet the acceptability requirement, designing/evaluating OBTs • according to the context of tool usage • through a more systematic use of scenarios (methods)

3. The Scenario Approach discussed in terms of How to evaluate? What does assessment mean? What can be assessed? Why assess? What to assess against? Who assesses, When, and Where? Adapted from Gómez­Pérez Discussion Frame

4. Evaluation to judge technically the features of the tools (Gómez­Pérez) Assessment to judge the usability and utility of the tools in companies (Gómez­Pérez) • Balancing technology with usage: • ‘focus on use first and foremost’(Jarke) • Balancing formality with informality(Buckingham Shum) What Does Assessment Mean?

5. Motivating Scenarios Story problems or examples which are not adequately addressed by existing ontologies + intuitively possible solutions to the problems (Grüninger and Fox) Ontology-engineering community Scenarios of Use Descriptions, often narratives, of what people (could) do and experience (e.g., problems) when using computer systems(Carroll) Syn.: Task scenarios, Interaction scenarios HCI and CSCW communities What Does Assessment Mean? Scenario-based Assessment

6. Ontology-design Tools Ontology Engineering Environments (OEE) Ontology Building tools Ontology Merging and Integration tools Ontology Evaluation tools Ontology Learning tools Ontology-based Annotation tools Ontology Storage and Querying (From: OntoWeb Working Group) Ontology-use Tools Uses of Ontologies in ApplicationsOntology-based Applications KM / Information Systems Knowledge Discovery E-Commerce Natural Language Applications Intelligent Information Integration Information Retrieval Semantic Portals (eCRM) Business To Business Business Process Modelling and Support Software Reuse ... Business Scenarios What Can Be Assessed?Ontology-based Tools

7. What Can Be Assessed?Tools Motivated by Scenarios OntoWeb Business Scenario (EON Experiment) • Ontology-based application: • End-user: ‘our travel agent in New York’ • Focus: A client ’s travelling (transport) and lodging (accommodation) • Scenario: e.g.:John’s Trip to Nueva York • John is travelling from Madrid to NY on April 5 th, 2002 to see the Statue of Liberty and continuing on to Washington on April 11 th. He plans to return to Madrid on April 15 th. He has selected two hotels belonging to the Holiday Inn chain in New York and Washington

8. Why Assess?Scenarios as Motives • Assessing to guarantee to end users in companies that OBTs are adapted to them and to their tasks • OBTs must fit scenarios of use, e.g. Travel Agency • Scenarios motivate the design • Scenarios motivate the end-users

9. What to Assess Against?Scenarios as Contextualized Frames of Reference • Evaluation should be performed against a frame of reference anchored in the real world (# context of use) • Contexualizing through scenarios (e.g, Travel Agency) • scenarios elicited from real-world observations • scenarios elicited from activity models based on real-world observations

10. FORM FIND-A-HOTEL HOTEL COMPANY FIND A HOTEL CITY STATE/PROV COUNTRY FIND HOTEL Decontextualized Ontology CHECK ROOM AVAILABILITY ARRIVAL DATE [MONTH] [DAY] [YEAR] [NUMBER OF] NIGHTS [NUMBER OF] PERSONS [NUMBER OF] ROOMS [AVAILABILITY] CHECK AVAILABILITY OBJECT ACTIVITY PERSON What to Assess Against?Scenarios as Contextualized Frames of Reference Contextualized Ontology

11. Who Assesses, When, and Where? Scenarios as Multifaceted Representations Shared by a Design-and-Use Community • Assessment performed • by users or designers-as-users, • at different stages of the OBT design cycle, • at the end user location, or in a similar context • Scenario: • a shared representation, • passing through the different stages of the design cycle, • which can be represented informally as well as formally, • and allows to always keep usage (the motive) in mind

12. Conclusion Systemizing the scenario-based approach: • A position backed up by our OBT design experience (CoMMA, European IST project) • An action point in the research agenda of the ontology-engineering community?