Evolutionary object oriented software development and project management

1. Evolutionary object oriented software development and project management Wolfgang Hesse, University of Marburg Contents: 1 Introduction: The software project, people concerned, dependencies 2 Traditional life cycle models 3 The EOS model 4 Managing EOS projects 5 Summary

2. 1 Introduction: The software project, people concerned, dependencies Software project: Set of activities, limited in time, directed to achieve adetermined goal, yielding defined results which consist of (computer) programs and corresponding documents. (cf. [HKLR 84]) Developer Manager Software project User

3. Investigating software projects: an interdisciplinary task - New technical paradigms for development influence work design - of users and of developers themselves. - New work procedures lead to new requirements on methods and tools. - New (technically motivated) development procedures require new forms of project management. - Quality of work (and its results) depend on the kind of project management. --> Analysis and design of software development processes is an interdisci-plinary task. The project IPAS(Interdisciplinary project investigating the work situation in software development) was staffed by . computer scientists . work psychologists . sociologists and followed a holistic approach.

4. An empirical study on software engineering practice Topics of investigation: Study A: (by work psychologists and computer scientists): Communication and cooperation in projects User participation The software process Evaluation of methods and tools Exchange of information of software developers Object oriented methods and techniques, changeability of software, Implications for project management Study B: (by sociologists): Adherence to plans and settings Formal planning vs. project success Informal, self-organising processes of software developers Efficiency of current management practice Conditions for dynamic project management The IPAS project

5. Summary of IPAS results • Theory and practice, official guidelines and real work in software development often significantly deviate from each other. • Example: Development handbooks, phase models vs. actual processes • New technologies (like C/S or the internet) induce a new generation of methods and work procedures --> resulting in requirements for new project management techniques. • New development and management paradigms are necessary in order • (a) to harmonize official standards and actual practice, • (b) to adjust to new technologies and development techniques.

6. 2 Traditional life cycle and process models Classes of life cycle models(cf. [Boehm 1988], [Hesse/Merbeth/Frölich 1992]) sequential: - Code and fix - Phase and waterfall-like models (since 1970) - Transformation models (since 1975) non-sequential: - Models for prototyping and incremental development (since 1980) - Process-oriented and spiral models (Ch. Floyd 1985, B. Boehm 1988) - Evolutionary development models (M. M. Lehman 1980) Software development practice(as reported to IPAS): - official:predominantly phases/waterfall, sometimes prototyping or incremental - inofficial: “subversive”, all forms, much less waterfall, more prototyping, incremental or evolutionary development, but also "code and fix"

7. Software life cycle models .. .. are necessary for .. - structuring the development process, defining activities, (intermediate) results and quality assurance actions, - giving the developers better orientation, - setting up and controling milestones, intermediate goals, dates, - project documentation, evaluation, comparisons, etc. ..; .. but have their weaknesses since they are often .. - not flexible enough (too rigid phase schema, unable to cope with instable requirements) - too bureaucratic (e.g. producing redundant documents, cf. [Denert 1990]) - over-automated, de-motivating and not quality-oriented, - not encouraging products well suited for maintenance, extension, re-use, - not properly dealing with the phases of operational use, - separating the user’s from the developer’s worlds - restricted to single, stand-alone-projects

8. Software process models The process-oriented view (since ~ 1990) - considers SW development as a holisticprocess, which .. . does not follow a rigid phase schema, . takes other than just the time dimension into account, e.g. space, organisation, system architecture. . is built up from activities anditerations (development cycles), .. producing well-determined results, . defines the roles and workflows of the people involved Software process(cf. [Hum 89]) : The set of activities, methods, and practices that are used in the production and evolution of software. Software process architecture A framework within which project-specific software processes are defined. Software process model One specific embodiment of a software processes architecture.

9. Analysis 1 1 3 4 Design 7 9 Implementation 10 Operations & maintenance 4 29 projects 156 responses concerning the phase schema 39 reports of explicit loops

10. IPAS results on project management practice • Phase schema: prescribed in 61% of the projects, • - actually followed in 25% (from 100%) • - overlapping phases: 41% • - “Echternach procession”: 13 % • - anarchic:13 % • Formal planning and project success: are only loosely coupled. • Success of projects depends more on ability to adapt to new situations and requirements than on formal planning. • Informal (self-controlled)processesand "tacid work" contribute significantly to project success. • The "static perception”of project managers ignores mostly the dynamics of real development processes. • --> hidden conflicts, discrepancies • from: IPAS investigation of 46 software projects, cf. [Hesse & Weltz 1994]

11. Knowledge Transfer from project to project: Its importance is often underestimated or neglected. Over-estimation of tool capabilities: Problems and difficulties cannot completely be solved by new CASE tools or software architectures. Primarily, a better understanding of project management and inner-project communication processes is needed. IPAS results (cont'd) Resume: The so-called software crisis is rather a crisis of managementthan of development methods and tools.

12. 3 The EOS model Why YAM (yet another process model) ? - Traditional models do not meet nowadays requirements (cf “weaknesses ..”) - Existing process models are rather phase- or "waterfall-" than really “object-" or component-oriented (cf. the models of Shlaer/Mellor, Rumbaugh, Jacobson, Booch’s macro cycle, and even RUP  [Hesse 2001]) - Existing process models are often too bureaucratic and not (or hardly) scalable. - The aspect of software evolutionis hardly reflected. - Component-oriented, distributed and web-based SW development requires flexible and well-adaptable processes. - Project management needs more support than a waterfall structure + milestones can offer.

13. Ph1 Ph2 Ph3 .... Phase-oriented vs. ... ... component-oriented process S Legend: X1 X2 X3 Building block C21 C22 Phase or activity

14. Objects and features of the software process • What are the main objects the software engineer has to deal with? • - Building blocks of various sizes: • . systems • . components / subsystems • . classes • - .. organised in a hierarchy lead to a three level system development structure: • . (S.) System level • . (X.) Component level • . (C.) Class level • What are the features of those objects ? • - Attributes: • . Size, Responsible_person, Start_date_of_work, Delivery_date, ... • - Operations: • . Development activities: Analysis, Design, Implementation, Operational_Use

15. Development cycles • Each development cycle has the same structure and consists of • (.A) Analysis: Define requirements, build model, consult building block (BB) library • (.D) Design: Specify and construct BB’s • (.I) Implementation: Transform designed BB’s to code, test, integrate • (.O) Operational use: installation, acceptance test, usage, revision • Evolutionary development is supported by: • - Integration of operational use (incl. “maintenance” and revision) into development cycles • - Further development and re-use of components • - Dynamic project planning and control based on cycles and activities

16. Use & Operations Development environment Use environment Phases of a development cycle Analysis Design Implementation synthetic, verifyingactivities planning,analyticactivities

17. System Analysis System Op. Use SO SA SD SI System Design System Implementation Subsystem Op. Use Component Analysis XA XO XD XI Component Design Subsystem Implementation Class Analysis Class Op. Use CA CO CD CI Class Design Class Implementation Combining development cycles in a traditional way

18. C02 S X 1 K01 X 4 X 2 X 3 C21 C31 Typical EOS-like process structure Development cycles intertwined in time

19. Metamodel for EOS process elements (from: [Beyer, Hesse 2002])

20. UML activity diagram forsystem analysis (SA) phase

21. Principles: Management structure follows system structure Starting point: the EOS hierarchy levels . S-cycle: Global planning (project-wide) . X-cycles: Detailed steps (e.g. team work packages) . C-Cycles: Activities of single developers Differenciated units of planning and control (on each level) . 1st planning stage: development cycle as a whole . 2nd planning stage: phases within cycle Dynamic, situative planning - Rather informal planning, "stand by"-management - Situation-driven adjustment of plans - Frequent plan revisions 4 Managing EOS projects

22. “Object oriented” workpackages - Developers are primarily responsible for “objects” - not for activities. . Planning refers to objects rather than to activities: Clearly defined responsibilities . on S- and X-level: by development (&support) teams(with users participating whereever necessary) . on C-level: by single developers or users Transparent planning, reliable plan control - Continuous information of teams on the project status - Plan revisions at defined points of time (  revision points) Dynamic and adaptable cost and effort estimation . based on the EOS process structure, experience data and statistical regression methods ( [Sarferaz, Hesse 2000]) Management principles (cont'd)

23. R1 Revision points .. replace milestonesbut are much moredifferentiated and flexible C-cycle CA CD H H CA CD CI E E E XA J XA XD G G X-cycle XA XD XI D D D XA XD XI XO B B B B XA XD XI XO A A A A SA SD SI S-cycle t

24. EOS combines the ideas of evolutionary and object-oriented software development The development process is structured - by hierarchy levels(system, component/subsystem, class) - by phases (analyse, design, implement, operate) and activities Cycles and phases are linked in a systematic and orthogonal manner. Development cycles are planned and executed on demandand ina dynamic way. Project managers can plan and survey the project on every level of detail by means of revision points. Ongoing work: S. Sarferaz: "Methods and tool support for evolutionary, object oriented software development", forthcoming Ph. D. thesis, Univ. of Marburg Summary and outlook

25. [Beyer, Hesse 2002] Use of UML for software process modelling. Internal report, Univ. Marburg 2002 [Bittner, Hesse, Schnath 95] U. Bittner, W. Hesse, J. Schnath: Praxis der Software-Entwicklung, Methoden, Werkzeuge, Projektmanagement - Eine Bestandsaufnahme, Oldenbourg 1995 [Budde et al. 91] R. Budde, K. Kautz, K. Kuhlenkamp, H. Züllighoven: Prototyping - an approach to evolutionary system development, Springer 1991 [Denert 90] E. Denert (u. Mitwirkung von J. Siedersleben): Software Engineering - Methodische Projektabwicklung, Springer 1990 [Frese, Hesse 93] M. Frese, W. Hesse: The work situation in software development - Results of an empirical study, ACM SIGSOFT Software Engineering Notes, Vol. 18, No. 3, pp. A-65 - A-72 (1993) [Floyd, Reisin, Schmidt 89] Ch. Floyd, F.-M. Reisin, G. schmidt: STEPS to software development with users; in: C. Ghezzi, J. McDermid (eds.): ESEC ‘89, 2nd European Software Engineering Conference; LNCS 387, pp. 48-64, Springer 1989 [Hesse, Merbeth, Frölich 92] W. Hesse, G. Merbeth, R. Frölich: Softwaretechnik - Vorgehensmodelle, Projektführung und Produktverwaltung, Handbuch der Informatik Bd. 5.2, Oldenbourg 1992 [Hesse 96] W. Hesse: Theory and practice of the software process - a field study and its implications for project management; in: C. Montangero (ed.): Software Process Tech-nology, 5th Europ. Workshop EWSPT 96; Springer LNCS 1149, pp. 241-256 (1996) References

26. [Hesse 97a] W. Hesse: From WOON to EOS: New development methods require a new software process model; Bericht Nr. 12, Fachbereich Mathematik, Univ. Marburg; and: Proc. WOON ´96, 1st Int. Conf. on OO technology, St. Petersburg 1997 [Hesse 97b] W. Hesse: Improving the software process guided by the EOS model. In: Proc. SPI '97 European Conference on Software Process Improvement. Barcelona 1997 [Hesse 01] W. Hesse: RUP - A Process Model for Working with UML. Ch. 4 in: K. Siau, T. Halpin: Unified Modeling Language: System Analysis, Design and Development Issues. Idea Group Publishing 2001 [Hesse, Weltz 94] W. Hesse, F. Weltz: Projektmanagement für evolutionäre Software-Entwicklung; Information Management 3/94, pp. 20-33, (1994) [Humphrey 89] W. Humphrey: Managing the software process; Addison-Wesley 1989 [Lehman 80] M.M. Lehman: Programs, life cycles, and laws of software evolution, Proc. of the IEEE Vol. 68, No. 9 (Cat. no. 0018-9219), pp. 1060-1076 (1980) [Sarferaz, Hesse 00] S. Sarferaz, W. Hesse: CEOS – A Cost Estimation Method for Evolutionary, Object-Oriented Software Development . In.: R. Dumke, A. Abran (Eds.): New Approaches in Software Measurement. Proc. 10th Int. Workshop, IWSM 2000, Springer LNCS 2006, pp. 29-43 References (cont'd)