1 / 25

C é cile Paris Anne-Marie Vercoustre Stephen Wan PeterWilkins Ross Wilkinson Mingfang Wu

An Empirical Study of the Effect of Coherent and Tailored Document Delivery as an Interface to Organizational Websites. C é cile Paris Anne-Marie Vercoustre Stephen Wan PeterWilkins Ross Wilkinson Mingfang Wu. Outline. Problem statement System Description Experiment Discussion.

tannar
Télécharger la présentation

C é cile Paris Anne-Marie Vercoustre Stephen Wan PeterWilkins Ross Wilkinson Mingfang Wu

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Empirical Study of the Effect of Coherent and Tailored Document Delivery as an Interface to Organizational Websites Cécile Paris Anne-Marie Vercoustre Stephen Wan PeterWilkins Ross Wilkinson Mingfang Wu

  2. Outline • Problem statement • System Description • Experiment • Discussion

  3. Finding Information about a Corporation • Often done through the web, searching and browsing • Differences between results from such search-and-browse and printed brochures: • Printed brochures are coherent documents – and include more than just information about a specific topic • Printed brochures are typically tailored to an audience

  4. Challenges in accessing an organization’s website • Browse • know the organization’s structure • know the web data structure • otherwise -> users feel lost • Search • It works for the “right” query or someone with the “right” mental model how querying/searching works • Otherwise, it may cause zero results or too many results • Result lists is disorganized - each result is presented independently, although each result may be a piece of the overall picture

  5. Our research question and approach Can a tailored hypermedia system offering tailored and coherent information provide a good alternative to search-and-browse? • Approach: Given a modest user model and a query: • Present information that not only matches the query but also is tailored to the user. • Present other information that is relevant to the context of the query (here information about an organization), like printed brochures do (e.g., introduction, contact details, etc.) • Deliver the result as a concise and readable summary. • Note that the summary may not cover all information a user needs, but is an informative guide that will lead the user to the needed information.

  6. Our Tailor Hypermedia System: Percy Virtual Document Planner DataSources Data Interface VDP Data Interface Interface Specification Content Planner Discourse Rules UserModel Stereotype PresentationPlanner Presentation Rules Individual Grammar(s) Lexicon(s) Templates Stylesheets ‘Documents’ SurfaceRealiser

  7. Percy – user model • Based on a requirements analysis • Contains: • Stereotypical information • Current job position (e.g., scientist or CEO) • Industry sector (e.g. science, engineering, finance) • Individual information • Interests (as represented by a query term, e.g., language technology) • Preferred delivery channel (e.g web page, or hand-held device)

  8. Percy – discourse planner • Discourse rules or Schemas • They indicate which information to present and how to organize it • They are used by the discourse planner to build a discourse tree • Each rule extends a branch in the discourse tree • Coherence relations amongst siblings indicate how the information fits together

  9. PWeb C PPalm C Percy – presentation planner • Discourse tree (which indicates content and organization) is extended with presentation markup • Uses presentation plans

  10. PWeb C ~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~ Surface Realisation XSLT templates Percy – surface generator • Traversal of final discourse tree to generate document • Conversion from tree markup of layout to device dependent markup • Device dependent markup may include use of further style sheets

  11. Introduction to CSIRO Short introduction to the divisions that have the projects related to the query Show the expanded query Short description of the matched project PERCY – an example delivery to a scientist

  12. Link to the detailed description Link to the online demonstration Publications Contact person PERCY – an example delivery to a scientist

  13. PERCY – an example delivery to a CEO Short introduction of how to do business with CSIRO

  14. PERCY – an example delivery to a CEO The project description and business relevance

  15. Preliminary Evaluation • Question Is this coherent and tailored delivery is useful interface to organizational websites ?

  16. Possible evaluation criteria • User’s interactive experience • format • content • User’s attitudes towards the received message • intention • The influence of the delivery message • The further interaction with the corporate

  17. Experimental setting • Test systems: PERCY vs. search-and-browse mechanism: a website search engine (Panoptic) • Subjects: 20 university students • Search topics: • T1: Human computer interaction • T2: Mathematical modeling • T3: Image analysis • T4: Language technology

  18. Evaluation instruments • Post-system questionnaire (PSQ) • To get users’ opinion on the evaluation criteria independently on each test system • Exit questionnaire (EQ) • To get users’ opinion of the second system compared with the first system • All questions are evaluated on a 5-point Likert scale

  19. Experiment design • Latin square design to reduce the sequence effect of systems

  20. Post-system questionnaire – Measuring the content of the delivered information • Q1: The system provides me accurate information • Q2: The system provides sufficient information • Q3: The information provided by the system meets my need. • Q4: The system provides me comprehensive information

  21. Post-system questionnaire – Measuring the format of the delivered information • Q5: The structure of the presented information is clear to me. • Q6: I think the presented information is organized in a useful format. • Q7: I think the presented information serves well as a useful online brochure.

  22. Post-system questionnaire – Understanding the intention of the user • Q8: I would like to get more information on some specific projects presented. • Q9: I would like to get more information because I have not got the needed information.

  23. Exit-questionnaire – Evaluating users’ preferences focus on the search topic better provide more comprehensive info provide better explanation deliver higher quality info make search task easier attract attention better preference

  24. Summary of experimental results • With Percy, subjects can better focus on their tasks. • With Percy, subjects understand better why a piece of information is presented. • Subjects preferred Percy.

  25. Discussion • About tailored hypermedia (anything to say about the delivery method itself or limitations?) • Delivery – who is in charge, user/author? or it is situation dependent? • How to apply AH to a dynamic resource (or even a different domain) with minimum effort? • Is user’s expectation of AH system different from IR? • About the evaluation • What to compare – Is it fair to compare the adaptive system with the non-adaptive system? Or should we compare two adaptive systems? • Will the preference brings the enhancement of user performance? • (How to measure long term benefit?)

More Related