1. Schedule • Today • BORE lecture • discussion of projects • Tomorrow (1112 AVW) • Haiying on Answer Garden and associated systems • discuss projects • April 18 (1112 AVW) • Erica on learning from failure, case-based reasoning • April 25 (1112 AVW) • Weiwei on software agents

2. Building an Organizational Repository of Experiences (BORE) Dr. Scott Henninger CMSC 838y Department of Computer Science University of Maryland

3. Overview • BORE concepts • creating a repository of best practices • using process to deliver information • The BORE system and methodology • tailoring process to project needs • delivering resources and best practices • Possible projects • feature design • empirical studies • application of the tool & methodology

4. Overall Principles • Software development is a knowledge intensive activity • knowledge from many different domains • no longer a programmer and a programming language • exceeds the capacity of any individual to fully understand a given development effort • Much of the requisite knowledge is: • domain-specific • organization-specific • less project-specific than people believe • don’t have to start from a blank sheet of paper

5. Overall Goals • Tools supporting sustained learning communities • software development as a community of practice • capturing and using knowledge generated by the community • Knowledge management • more like building on existing knowledge • not just manage what already exists • have also used “organizational learning” • loaded term for MIS

6. BORE Overview • Building a Organizational Repository of Experiences (BORE) • using process to organize and manage knowledge • delivering best practices • capturing the context for using different techniques • support process diversity through rule-based tailoring • Process adaptation • deviation process when current rules/tasks don’t apply • create new activity, encode rationale in rules • allow subsequent projects to use tailored processes • i.e. follow the precedent • allow the SDM to grow “organically”

7. The Domain Lifecycle

8. The Domain Lifecycle • Model of how knowledge evolves (red) • incremental formalization • Model of how it is used (blue) • increasing levels of support for designers • Anderson Consulting • rules: find best practices • schools: package and teach the accumulated knowledge • tools: embed the knowledge in tools

9. Early BORE • Essentially an organizational memory • case-based architecture • capture project experiences • Case-based repository • hierarchical representation of project activities • instantiation of activities are a cases • problem and solution statement • document context-specific information • Individual experiences captured as cases • i.e. situation-specific knowledge • first step in the domain lifecycle

10. BORE Project Interface

11. Organizing Knowledge • Artifacts associated with cases • attach documents to a case • upload to server • download via HTTP • Resources Tab

12. Experiences with BORE • Used in software engineering classes (‘98-’99) • document project activities • no defined process • Pilot study at Union Pacific Railroad (‘98) • limited comments on usefulness • very little detail • Results were as expected - nothing • not much information captured • not enough detail to “reuse” the experience • didn’t understand what needed to be documented • viewed as an extra step • just a documentation medium

13. Process as Knowledge Management • Repository needed to be more proactive • more than a discretionary tool • mandated as part of everyday activities • KM as information retrieval • build a repository and let people search • people need to know that knowledge exists • …and when/where to look for it • KM as information delivery • i.e. a push model • agent model: watch people’s behavior • make inferences about what information they need • can be effective in limited domains • problem: how to infer people’s behavior

14. Process as KM (con’t) • Applying knowledge to the task at hand • Information Delivery • Did You Know: inappropriate timing • applicability conditions for knowledge • Critics: on-task delivery, too late for planning • rule-based paradigm • rules determine applicability conditions • BORE has a different model • define information sources at points in the process • use process to organize and manage knowledge • workflow as information delivery

15. Software Process Improvement Initiatives • Define the activities for a software project • standard definition • not a workflow, more a listing of activities • projects should use configuration management, for example • normally delivered as a document • or series of documents • High-level activities • must address the least common denominator • complexity is an issue • …when viewed as a monolithic document • little support for how the activity should be carried out

16. Examples of Process Activities • CMM key practices • “The project follows a written organizational policy for planning and implementing a risk management program.” • “A risk assessment is performed early in the project life cycle using a defined process.” • NASA Goddard • Develop a system, software and operations concept. • It is recommended that the Team develop system, software and operation concepts if they do not already exist. • Code New Modules • Team members shall code new modules from the low-level design, using the coding standards or conventions specified in the Software Management Plan (see paragraph 6.1). This activity also includes implementing tailoring and configuration of COTS/GOTS components.

17. Problems with Software Process • Not seen as a knowledge delivery and capture mechanism • no good way to organize knowledge generated by individual development efforts • BORE cases aim to do this • Often seen as a definition effort • review of the process is normally part of the process • I.e. process refinement is part of the process • …but few mechanisms to do this • Not a resource for developers • only managers need to be concerned with them • no support for developers, testers, etc.

18. BORE Process Goals • More than a conformance tool • also deliver information when it is needed • Capture and disseminate best practices • use process to show how others have performed the activities • building an organization-specific knowledge base • Flexible to meet different levels of detail • both managers and development personnel

19. Overall Approach • Process tailoring • define applicability rules for activities • “if there are significant technical risks” • “document the risks, perform prototyping activities” • Flexible software process • SDM as a repository of best practices • more detailed knowledge than SDMs • capture more than the least common denominator • supporting process diversity • tailor SDM to specific project needs • document project-specific issues • use assigned tasks to disseminate knowledge

20. Process Tailoring • Process captured as decision points • project requirements captured as question/answer pairs • assign activities based on answers

21. Process Tailoring • Allows flexible process definition • can go beyond one size fits all • different type of projects can incorporate more detail • without affecting other types of projects • Allows more complex process definition • project manager doesn’t need to know everything • just what is necessary for their project • Iterative disclosure of detail • any case can have process decisions • decisions define increasingly detailed project tasks

22. Modifying the Process Model • Tailor the SDM to specific project needs • deviation process when current rules/tasks don’t apply • i.e. a “breakdown” requiring new actions • new domain activity to describe the task • rules for tailoring (encode rationale for the deviation) • allow subsequent projects to use tailored processes • i.e. set a precedent • match problem to specific context • allow the SDM to grow “organically” • ...as new problems are encountered, technology evolves, etc. • Beyond the expert system paradigm • rules as a resource for human action • collaborative creation of rules

23. BORE Methodology

24. BORE Knowledge Editing • Create a domain • set of activities • rules for when activities are assigned to projects • Create activities • hierarchical set of activities • treated as a project in the Case Manager • Create rules • preconditions • actions

25. BORE Domains • Domain activities • case-based paradigm • “principles” contain general rules or knowledge • cases specialize the principles • domain activities play the role of principles • projects belong to a domain • domain defines all activities for that domain • Rule-based engine • preconditions: question/answer pairs • actions: assign tasks, question stack

26. Domain Activities • Same as editing a project • in “Domains” resource • Initial questions • Options tab

27. Creating Rules • Rules Manager • choose domain • edit existing rule or create a new one • separate windows for editing preconditions, actions

28. Problems With BORE Rule Editing • Difficult to edit existing rules • can’t find all rules having a given precondition • can find all rules with an action • No good way to get context • want to be able to see rule chains

29. Current System Status • BORE prototype • http://cse-ferg41.unl.edu/bore.html • ==> Bore v. 3.2 • log in as ‘guest’ • Just a prototype... • documentation is lacking • currently only works on Communicator • most works on IE, but some problems • no application security • current rule engine, interface need work • document management needs Web-based upload

30. Project Ideas • BORE feature designs • applying case-based planners • apply to pattern languages • Domain-Specific Design Environments • deviation rationale • experience factory and EMS • Case Studies • apply BORE to a software process • create a BORE domain, assess shortcomings • GSFC (??), CMM compliance • Empirical Studies • diary study of software development organization • contextual inquiries

31. Case-Based Planners • General idea • view process/workflow as a planning activity • retrieve past plans based on similarity measures • Collaboration with Nau et al. • some interest to apply their planner to BORE • and vice-versa • meeting on Thursday to explore further • Integration of rule and case-based system • some work on this already (SiN)

32. Pattern Languages • General idea • document known solutions to recurrent problems • also document the context • put patterns together in a pattern language • General idea is similar to BORE • similar documentation format • Usability patterns • currently a hot topic in HCI • integrates technical and aesthetic domains • Develop design for applying patterns • how BORE would need to change • some concrete examples

33. Domain-Specific Design Environment • Aka Domain-Oriented Design Environment (DODE) • environment for creating product families • third step in the domain lifecycle • similar to O-O frameworks • General process • developer steps through questions • some percentage of system automatically created • develop rest of the system • creating the requirements in rule Q/A pairs • when finished have another path through the rule system • metaphor: extensible wizard • Develop a DODE example • using JavaBeans or a similar component technology

34. Deviation Rationale • Design the process of deviating from the standard • use as “opportunity for learning” • have people document the rationale for the deviation • create new activities • Capture rationale in activities, rules • may want to have a text format • that is then turned into rules by BORE curators

35. Experience Factory • Integrating metrics into BORE • choosing metrics • tools to collect metrics • metric reporting • Look into integrating with EMS • EMS - packaging experiences and finding similar ones • some features complementary with BORE • Integrating BORE and EF ideas • comparative literature study • suggest some integration activities • maybe use CeBASE project to organize

36. Case Studies • Goddard ISC Software methodology • current focus on ISO 9000 compliance • pilot study using BORE • use in small project not on critical path • problem: can this be started within a week? • map CMM to Goddard ISC • Assess CMM compatibility • choose one of the CMM-SW models • detailed analysis of how BORE addresses the KPAs, etc. • how it would need to be extended • Review Process • elaboration on how reviews are conducted, etc. • guidelines for how and when knowledge should evolve

37. Empirical Studies • Diary study of software development organization • have people write down certain activities • focus on a key question • for ex: all knowledge management activities • analyze for trends etc. • Contextual inquiries • essentially an interviewing technique • people abstract in interviews • follow people around for a half day (or more) • record activities (video or audio) • transcribe activities and analyze • “people say they do x, but they do a lot to get there” • again, focus on a specific activity - knowledge management

38. BORE Project Process • Choose a topic • choices discussed today • Literature review • breadth-first search of literature • choose a subset to concentrate on • Requirements • document through use cases, scenarios, flow of events • use mock-ups • Design • UML activity and class diagrams • need some understanding of BORE system

39. Study Project Process • Choose a topic • choices discussed today • Literature review • learn about method, collect materials • choose study topic to concentrate on • Design and perform study • study software developers • populate BORE domain • Analysis • report results • discuss findings