Help-Desk Systems

1. Help-Desk Systems Stephen Lee-Urban November 17, 2006

2. References • Experience Management: Foundations, Development Methodology, and Internet-Based Applications, Ralph Bergmann. • Chapter 9: Developing and Maintaining Experience Management Applications • Chapter 11: Experience Management for Self-Service and Help-Desk Support

3. Outline • Motivation / Goals / Background • The HOMER architecture • Developing & Managing EM* Applications • Evaluation of HOMER *EM = Experience Management

4. Motivation • Complexity of technology increasing • Operation, Maintenance, Repair • Probability of failure grows exponentially with complexity • Expertise in all features unreasonable • Problem solving experience distributed • Experience overlap is variable Experience Management System (EMS)

5. Goals of EMS • Create knowledge repository • Store problem solving experience • Simplify trouble-shooting complex technical domain • Domain is likely to change over time • Use knowledge repository • By varying levels of expertise • In time-critical situations

6. Complex problem solving Problem acquisition Development and Management Methodologies Experience evaluation and retrieval Reuse-related knowledge 1. Retrieve Experience base 4. Retain Experience presentation Experience adaptation Case Library 2. Reuse Background Knowledge 3. Revise Experience Management vs CBR Experience Management (Organization) BOOK CBR (IDSS) Slide: Dr. Munoz-Avila

7. Help Desk Support Levels • Bottlenecks at each level. • Problems recur. • Frustration for all parties, and wasteful of resources! • “Managerial” Aids to Help-Desk: • Inventory systems, trouble-ticket tools, call-tracking software • Don’t support diagnosis nor serve as knowledge repository • Key-user CBR • Help-Desk sys. • Specific technician • Vendor http://www.simpsonstrivia.com.ar/simpsons-photos/wallpapers/homer-simpson-wallpaper-brain-1024.jpg

8. Outline • Motivation / Goals / Background • The HOMER architecture • Developing & Managing EM* Applications • Evaluation of HOMER

9. The HOMER System • German: “HOtline Mit ERfahrung” • Developed as part of INRECA-II project • For CAD/CAM help-desk at DaimlerChrysler in Sindelfingen • Generic experience management architecture & set of related tools • Stores experience in CB for access, reuse and extension • Drastically decreases problem solving time http://www.aeria.phil.uni-erlangen.de/photo_html/portraet/griechisch/dichter/homer/homer3.JPG http://gattaca.com.ar/weblog/wp-content/homer.gif

10. Structure and Representation • Determines experience management approach • HOMER uses an object oriented approach to model experience. • Advantages of OO approach: • Structure of system to diagnose is representable in detail • Symptoms (attr.) easily related to owning object • Semantics of prob. description can be captured and used for selecting appropriate prior exp. • High retrieval accuracy can be achieved • Particularly suited for diagnosis help-desks w/ complex equipment  Can show many hard to diagnose faults • Can be used to help guide help-desk operator while describing and entering cases • Better domain model = easier maintenance and use • Disadvantages of OO approach: • Harder to create model • Additional effort in beginning of knowledge acquisition Representation should be discussed with system admin. Shallow?

11. Case Structure • Modeled according to approach used by help-desk operators to solve problems • Feasible for most complex help-desks Case: Complete path from prob. to solution • Topic: problem area (e.g. hardware) • Subject: failing object (e.g. printer) • Behavior: way (miss-) behaves (e.g crashes) • Minimum set of attribute-value pairs describing symptoms necessary to fault diagnosis. • In OO, all attributes are related to certain object. • Fault: problem cause • Remedy: problem fix • Represented as (hyper-) text

12. Experience Base Partitioning • Domain size & complexity + demanded accuracy & consistency = partitioning • Two kinds of cases: • Approved: experience reviewed by top-level experts. “Best Practice” for problem solving • Open: recently captured, pending validation (correctness, completeness, & utility) • Case base separated into: • Case Buffer – all open cases • Main Case Base – all approved cases All available to help- desk operators

13. HOMER Architecture • Client-Server • Shared domain model (DM) + case base (CB) • Eases maintenance of DM + CB • Server accessible via intranet / internet • “Fat” client reduces network traffic • Object-Oriented experience model • Not “shallow”  users are experienced • For non-trivial problems

14. Middle rights • Case maint. & case approval • Redundancy & consistency checks • Buffer to base • May modify value ranges of attributes • Lowest access rights • Daily user • Retrieval & acquisition • Highest rights • Creates & maint. domain and case model • +/- attr & concepts • Admin users and rights

15. The Server • CBR-Works server stores model & CB, • CBR-Works modeling tools provided by server for domain modeling, case & model maintenance, & initial case acquisition • Domain model snapshot…

16. Attributes Help Desk Case Hierarchical domain concepts • Problem, holds failure • Situation, holds symptoms • - Hierarchical, sub-concepts • - Structure helps maint. & retrieval • Loesung, holds solution • Administrativa, holds organizational & statistical information

17. The Client • Interface to server for case retrieval, case acquisition, & case browsing • Written in Java • “Hotline” component for help-desk operator • Only shows relevant information • Assists via two modes: user/system driven • “Case Browser” component for experience author • For access to CB and case buffer • Revision, extension, approval of “open” cases • Removal of outdated cases

18. Hotline Component (HC) questions • Used mainly during hotline operations • Supports help-desk operator during problem-solving process • Four main modes of execution • Problem description (initialization/refinement) • Situation description (user/system driven) • Solution retrieval (manual/automatic) • Retain w/ feedback answers to questions problems solutions (cases)

19. HC: Problem Description question (free text) • Gives initial info. on user’s problem • Answer following questions consecutively: • Problem’s topic? (e.g. output, software, …) • Topic’s subject? (e.g. output: plotter or printer) • Behavior? (e.g. “no output at all”) values problems value range units

20. HC: Situation Description • Problem determines relevant case structure (i.e. situation template) • Contains possible symptoms for prob. • Each symptom associated w/ question • Symptom attribute values complex/simple • Two modes of operation • User-driven: (no guidance, direct entry) • System-driven: • Shows sorted view of most relevant questions • Based on attributes with highest info gain

21. HC: Solution Retrieval • Experience retrieved based on problem and situation descriptions • Manual (batch) / automatic (per question) • Each row a solution, sorted by relevance (similarity) • Solutions viewed (read-only)

22. HC: Retain/Feedback • Retain case when it • Has new experience • Requires case model extension (e.g. new value to type) • Case entry interface allows • Final modifications (e.g. finish case descr.) • Annotating why op. thinks case should be retained

23. Case Browser Component • Used by experience author to manage CB • Works on both approved and open cases • Can perform following operations: • Case creation • Case copy • Case deletion • Case approval question (free text) values Problem, situation, solution, administrativa value range units

24. Outline • Motivation / Goals / Background • The HOMER architecture • Developing & Managing EM* Applications • Evaluation of HOMER

25. Developing & Maintaining EM Applications • IT companies require efficient, effective application development • Guidelines/Methods of implementing apps • Also seek to preserve past project experience • INRECA methodology • Targeted at industrial EM applications • Developed in the INCRECA-II European ESPRIT project (1996-9)

26. EM Model Problem Solving Cycle Knowledge Kernel • Knowledge Acquisition & Knowledge Maintenance • Technical, organizational & managerial aspects

27. Three Process Types • Technical • Describe development of system & required documentation • E.g. requirements analysis, system design, implementation and testing • Organizational • Address parts of business process in which software will be embedded • E.g. training end users, archiving request records, updating & maintaining the help-desk system • Managerial • Provides environment and services for developing software that meets product requirements and project goals • E.g. project planning, monitoring, quality assurance

28. Methodologies • Makes development an engineering activity, rather than an art • Use of methodology provides benefits: • Productivity, Quality, Communication, Management decision making • INRECA methodology based on software engineering principles, covering: • Project management (cost assessment, schedules, project plans, etc.) • Product/Deliverable specification • Product development and maintenance • Analysis/Organization of target env. for CBR system

29. INRECA • Basic philosophy: experience based construction of experience management applications • Self-application of principles of experience reuse • Experience modeled as structured text documents, hyper-text linked. • Origin: Software process modeling

30. Software Process Modeling • Well defined terminology to describe the engineering of a product • Process, “what”: basic step to carry out, transforming input into output. Defined by a goal, a set of alternative methods, input/output/modified products, & required resources (agent/tool) • Method (simple/complex) “how”: Detailed specification of a way to achieve a process’ goal • Product: goal of the process INRECA notation

31. INRECA Process Models • Make explicit all processes, products, methods, resources and interactions • Diagrams insufficient  Each element must be described in detail • Called a process model • Solid basis for project planning (e.g. effort calculable based on processes involved) • General vs. Specific descriptions • Common Generic/Cookbook vs. Project level

32. The INRECA Experience Base • Collection of processes, products, & methods common across EM apps. • Processes, products, & methods tailored to class of applications (e.g. help-desk). • Each class has recipe • Recipe: has process models describing how app. of that class be developed & maintained • Describes experience of an instance of a completed project • Project-specific info. (e.g. processes carried out, effort of processes, etc.)

33. The Common Generic Level (CGL) • Overview of top-level processes and products in Bergmann, chapter 9 • General enough to occur in many projects • Problem statement  “vision document”  goal checklist  feasibility study  detailed analysis of organization  project plan  software development  evaluation • Meanwhile, experience base can be consulted for all of the above processes • Consult book for thorough exposition

34. Go / No-go (analysis & elicitation) Organizational (planning) Implementation Maintenance

35. CGL: The Three Processes • Managing an AI / EM project differs from other IT projects • Concepts & technologies altogether new • Emphasize early awareness training • Ensure user-participation in iterative prototyping process • Technical processes very typical of software development projects • Organizational includes identifying perceived problems and opportunities at the human & organizational levels

36. Documenting in INRECA • Processes, products, & methods documented and stored in experience base • “Sheets”  structured page w/ all relevant information in a predefined format • Standardize documentation • Created as web pages for easy access & use • CGL has more than 150 linked sheets • 8 type of description sheets: [ generic | specific ] [process | product | simple method | complex method] • Contains references to respective input, output, and modified products of the process

37. Process Description Sheets • Recipe/Project Name • Process Name • Process Goal • Input/Output/Modified Products • Set of Applicable Methods (names) • Agents (e.g. personnel) • Required Tools • Administrative Information (e.g. sheet author, version, last modified)

38. Product Description Sheets • Module/Project Name • Product Name (e.g. “requirements doc”) • Product Description • Administrative Information

39. Simple Method Description Sheets • Module/Project Name • Method Name • Method Description • Administrative Information

40. Complex Method Description Sheets • Module/Project Name • Method Name • Method Description • Details • Links to a product flow description which contains the relevant sub-processes (byproducts) • Administrative Information

41. The INRECA Reuse Procedure • Recipes at cookbook level are most useful for building a new application • Even projects not fitting a recipe can use processes described in CGL

42. Don’t Forget to Remember! • After completing project, include it in the experience base  continuous improvement of the EM software development process

43. Tool Support for INRECA • Experience modeling methodology tool implemented in Visio® • Natural choice: shapes, hierarchical, HTML, VBA, database access • Knowledge modeling tools • Integrated in the CBR-Works tool (tec:inno GmbH 1999) • CBR-Works Concept & Type Hierarchy Editor for vocabulary modeling • Similarity modeling tools integrated in the Concept & Type editors • Also has an editor for adaptation and completion rules

44. Outline • Motivation / Goals / Background • The HOMER architecture • Developing & Managing EM* Applications • Evaluation of HOMER

45. Evaluation of HOMER • Evaluation performed by INCRECA-II project partners to identify benefits of EMS • Incoming calls monitored. Help-desk operator 1st solves conventionally, then with HOMER • No solution  new case created • Two month test period: 102 calls; 45 unsuitable for HOMER

46. Evaluation (II) • Of the 57 (/102) suitable problems • 18 solved (i.e. 32%) • Ave. resolution time w/out HOMER: 141 min. • Ave. resolution time w/ HOMER: 9 min. • Correct result a limited bias  HOMER sys. Mode • HOMER + CB transferred to a new site • Operators have no experience with the process chain so the cases are of great value • Initial knowledge acquisition gave operators insight into domain.

47. Evaluation (III) • Methodology recipe created and used • Impact: productivity, quality, communication, & management decision making • Customer and developer both benefited • Creation of project definition from scratch • Took three months • New development team reused recipe to define three new projects, each in < one week (12x speedup!) • Were sure all relevant aspects taken into account • Basic recipe available  developers focus on domain peculiarities  quality/detail of descriptions greatly enhanced

48. Evaluation (IV) • Development & testing of 1st prototype • About 6 months • Subsequent efforts: 2 weeks (13x speedup) • Less qualified personnel w/out sacrifice of quality • Useful as training tool for maintenance & use • Message: development of EM system a science, not art • Each process describable • Validity of approach defensible • Realistic expectations of effort by whom and when • Measurable project process

49. (Questions?) (Comments?) Thank You!