A Software Engineering Institute for the Victorian Software Industry A Re-useable Case Study*...

1. A Software Engineering Institutefor the Victorian Software Industry A Re-useable Case Study*... by Assoc. Prof. Karl Reed,FACS, FIE-Aust., MSc,ARMIT Chair IEEE-Computer Society Tech. Council on Software Engineering Governor, IEEE-Computer Society(1997-1999,2000-2002), Director, Computer Sys. & Software Engineering Board, ACS, Department of Computer Science & Computer Engineering, La Trobe University Hon. Visiting Professor, Middlesex University *A summary presented to the Sheffield-Hallam Univ. Feb. 2001

2. Why Were We Proposing VSEI? • Promote government policy in information technology ensuring support for an already successful industry • Ensure that the industry has a major research institute • Ensure ‘world’s best practice’ is understood and adopted here • Twenty years from now • a permanent structure delivering an increased competitive edge to a major exporting industry!

3. Australian Political Reality • Information Technology Sector --> Primary Industry share of GDP • Extensive Gov’t R &D for Primary Sector.. • 10 Divisions of CSIRO, State R & D inst., CRC’s • ~15 Govt.-Industry Funded R & D Corps for Primary Industry • Estimated relative short-fall ~A$700M.p.a (£300M.p.a) • Few Public sector large-scale “professional” research groups • Universities regarding R & D as their domain.. means of cross-subsidising teaching(?) How does this compare with the UK?

4. What was our strategy? • “Forcing” argument based on comparison with Primary Industry and Govt. rhetoric • Linkage with industry • Credible proposal …. to deliver world’s best practice • Twenty years from now • a permanent structure delivering an increased competitive edge to a major exporting industry!

5. Goals • Guarantee the long term technical viability of a major IT industry sector • the software and services industries • Provide equivalent R & D support as that available to overseas competitors • Provide professional research capability advancing the discipline

6. Industry-Academia Working PartyMeeting since May 1997 • Chairman • Karl Reed, La Trobe University • Deputy Chairman • Paul Radford, Managing Director Charismatek • Working party • Silvio Salom, Managing Director Adacel • Laurie Lock Lee, Manager Planning and Development BHPIT • Robyn Lawrie, Technical Director Charismatek • Alex Sawicki, Department of State Development/Multimedia Victoria • Gary Stoneham, Marketing Manager MITS • Sally Duncan, Project Manager Megatec • Barbara O’Brien, Quality Manager Megatec • Bill Jacobs, Technical Director TUSC Computer Systems • David Cleary, La Trobe University • Special advisor • Dan Marantz, Cobol Digital • Observers • The Preston Group • Whittle Programming • KCS Computer Services • Clive Finkelstein, Information Engineering

7. Academic Partners with industry links • Karl Reed, Amdahl Project (Case, Metrics, Evolvable Programming) • Tharam Dillon, La Trobe University OO, Relibility, Metrics • T.Y. Chen, Swinburne Univ. Tech., IVE/STC HK Testing and Software Quality • Paul Bailes, Univ. Queensland. Software Maintenance Centre • HK opportunity for participation

8. Not Seeking Silver Bullets • We’ve all heard of • goto-less programming, structured analysis and design, CASE, object orientation, CMM, SPIN • ??? • all trumpeted as ‘the solution’ • This proposal • “a considered holistic response to a series of difficult problems” • not a “quick fix”!

9. An Appropriate SolutionA Small to Medium Enterprise Bias • Only one of its kind in the world, other SEIs • client domains generally not the software industry • focus on embedded systems, telecom and defense aerospace • pre-occupied with process issues • Our objectives • commercial outcomes, deliverables to industry • not just academic • Funding for technology transfer

10. World’s Best Practice • Aggregated research teams ~ researchers 40 + ~20 PG's • Funded technology transfer • Industry driven collaboration (including management) • Cash driven budget (mostly Government) • Single point of control

11. Scale Comparable to World's SEI's • Overall • A$2M pa to A$30M pa (combined Esprit is much larger) • Fraunhofer Institute for Experimental Software Engineering • DM20Mpa (goal) • Centre de Recherche de Montreal (CRIM) • C$18M pa (total) • C$3M pa (software engineering)

12. International Funding Models • Overall • typically less than 50% non-government funding, often as low as 30%, sometimes zero • funds often obtained from government sources other than granting agency • funding models • dominant mode; core funding (up to 70%) guaranteed by government agency with allowances for ramp-up • 100% government funding • grant based funding

13. International Funding Models • Fraunhofer Institute for Experimental Software Engineering • Fraunhofer Gesellschaft 40% • regional government 10% • future goal; non-Fraunhofer funding to be 70% (ramp up conditions apply) • joint projects; seek 50% involvement (people preferred)

14. SEIs Around the Worlda Brief Summary • Client domain • generally not targeted at the software industry • mainly ‘embedded’ systems, telecom and defense aerospace (Korea and Taiwan and some Esprit projects are software industry) • Modus Operandi • networks, multi-project granting (CNRC, CRIM), academic driven, research driven, captive-client (SEI), in house, large-scale (Fraunhofer), etc

15. SEIs Around the Worlda Brief Summary • Areas of investigation • all areas of software engineering... • Technology transfer • seminars to technology trials and ‘leverage’ activities (often funded by agency), experiments (PIEs), intellectual property handover • Research styles • single project (Esprit), short, medium, long-term, team, consortia, distributed consortia

16. SEIs Around the Worlda Brief Summary • Outcomes • process improvement, methodology, tools, some product (Esprit varies enormously), intellectual property • Intellectual property • all models; shared by partners, client owned

17. SEIs Around the Worlda Brief Summary • Participants (primary funding sources) • single agencies (SEI, Fruanhofer, etc), multiple agencies (European SEI), state governments (Italy, Canada, Germany), multi-national programs (Europe), very few consortia with upfront industry funding

18. SEIs Around the Worlda Brief Summary • Research agenda determination • ‘agenda setting’; agency attempts to inject new technology and best of breed practice (NSF, SEI, DARPA) • ‘responsive’; industry set research agendas • ‘academic interventionist’; academia propose research projects • ‘joint’; academia and industry • ‘empiricist’; agendas derived from study of practice

19. SEIs Around the WorldFraunhofer Institute for Experimental Software Engineering • Funding • Fraunhofer Gesellschaft 40% • regional government 10% • future goal; non-Fraunhofer funding to be 70% • local government 50% of new building • special ramp-up conditions apply • Scale goal DM20M pa

20. SEIs Around the World,Centre de Recherche de Montreal (CRIM)Software Development Tools and Methods (SDTM) • Funding • about C$3M pa for software engineering • Quebec • Scale total CRIM C$18M pa • Client domain • embedded systems, aerospace, telecom, electricity supply, software tool builders

21. SEIs Around the World,Centre de Recherche de Montreal (CRIM)Software Development Tools and Methods (SDTM) • Modus operandi • joint academic-industry projects • studies of tools, methods etc for clients • specific research projects. • participation sought • technology monitoring • projects seem small

22. SEIs Around the World,Centre de Recherche de Montreal (CRIM)Software Development Tools and Methods (SDTM) • Areas of investigation • methodology improvement • re- and reverse engineering • improvement of practice, including metrics, standards and SQA • object orientation • domain-specific architectures, reuse • real-time systems, including formal methods

23. Some ESPRIT Research Projects • Legacy Assessment Workbench (LAW) • areas (aerospace, nuclear) • reengineering, metrics workbench for C legacy code (safety critical systems) • symbolic execution, formal methods • outcomes • prototypes for evaluation and adoption

24. Where Are We? Traditionalist’s View DISASTER IN WAITING! • Bowsers that are limited • Time-To-Market web-application deployment • 16 year old wunder-kinder throwing systems together NO PROBLEMS MATE! • poorly designed functionality Modernist’s View • rapidly deployed functionality • rapid evolution of systems to meet customer needs • conventional approaches being left behind • the old do not understand the new

25. To many surprises….!!!(nsf report on s/w research 1998) “F1. Current software has too many surprises. The sources of surprise are poorly understood.” • “F2. Key sources of software surprise include immature or poorly integrated software domain sciences, construction (product) principles, and engineering processes. Software research emphases have swung from process to product research, with weak coverage of domain sciences and integration.” Sources of surprises... Real and apparent ambiguity in the means of representation of systems, e.i. Languages (cf 3 pages of c++ with 3 pages of government regulations)(Reed, 2000)

26. No surprises….!!!(nsf report on s/w research 1998) “F1. Current software has too many surprises. The sources of surprise are poorly understood.” Sources of surprises... Real and apparent unpredictability in behaviour... “Teenagers have less trouble with PC software because they are adept at playing computer games” Charles Wright, editor Melbourne Age “green pages” computer section 2000 “Building ‘bots’ that play computer games with near human competence is not that hard” US researcher in AI….

27. By way of Illustration...Some Contradictions…… and confusion 1. Software Architecture.. ‘not immutable, not always determinable a’priori,multiple versions in one artefact, retrofitable…. Analog with “built” systems not clear. 2. Software Process.. CMM vs fine-grained process independent, Time To Market vs Planned Process, Phase incompletedness, Extreme Programming. 3. Software Process... Often mandated, but not followed… few detailed studies similar to production engineering (see Hess) 4. Re-use… not successful, yet components industry emerging 5. Engineering & SE.. Poor choices of analogues from traditional domains, e.g. “immutable components”

28. Some Contradictions…… and confusion (cont’d) 6. SWEBOK.. Organised body of knowledge opposed by leading SE players. 7. Prescriptive Design processes... only slowly beginning to appear, perhaps via UML. 8. Requirements Engineering... Cannot always be completed in advance..may be continuous part of the implementation process... 9. Software Crisis… yet increasingly, successful large-scale applications are ubiquitous 10. High Quality training for 30 yrs.. Yet each new s/w development wave starts with a blank mind, e.g. web-based computing 11. Documentation matters but.. It’s seldom actually done

29. Approaching Software Developers… • Technico-Commercial Drivers… the linkage • The goal is to find a high-level, one-line statement of pressing commercial issue that maps directly on to a “technology acquisition” (research) agenda (map idea to common concept base accessible to highest management) • Be able to show ROI after adoption costs (equipment + training) and productivity losses due to learning curves after adoption. (improved profit) • Show resolution of competitive advantage problems (beat off competitors, maintain market share) • Show new market opportunities due to new products/services • Show an economic benefit

30. Research-Commercial Mapping… Defining Relevance Typical SE Research Agenda Australia ~ 1997 Technico-commercial Drivers ¶ Impact of developments in run-time platforms ¶ Low-cost and evolving software ¶ User Interface Development ¶ Software Productivity ¶ Performance Predictability ¶ Software Product Quality Certification ¶ Time to Market 1.Re-engineering and Empirical Studies of s/w Practice, 2.Tools and Methodologies, and Design Representation, 3. Re-Use, 4. Evolving Software, 6. Object Oriented Dev. 7. Product Quality Measurement 8. Time-to-Market 9. Testing

31. The ANSEI Technico-Comercial Driver to Research agenda mapping

32. Stakeholders Research Experience Government Technical Board CEO Industry Experience Industry Bus Man Management Board Technical Support Academia Personnel Organisational Proposal

33. Research Agendas Providing SolutionsOver-Arching Goals • Our research outcomes are methodologies with the following properties, these in fact become research objectives • performance-based, predictable, development of systems to stated performance requirements • fine-grained-prescriptive, provide precise prescriptions for steps in development • improved technology and expertise for re-engineering of existing systems • study of existing systems and projects, using re-engineering, design recording

34. Research Agendas Providing SolutionsOver-Arching Goals • Issues of re-use, re-use intensive and re-use based methodologies • evolving software (current agenda of DARPA) • object oriented methodologies, will influence, and be influenced • engineering of interfaces, part of the methodology program • Plus, tools that reflect this!

35. Characterising Time to Market • Delivery schedules severely truncated compared to ‘normal’ (less than 50%) • How would we deal with this currently? • RAD/JAD, prototyping, ‘super programmers’ • What would the product be like? • slow, unreliable, incomplete expensive! • Research goal • convert this to a methodology capable of delivering quality products

36. Characterising Time to Market • Research problems • for a given project • a minimum feasible delivery time tdmin • cost = k tdmin • quality greatly reduced • competitive position increased! -n

37. Time to Market Project Attributes Attribute Standard Development (under control) Schedule controlled acceptable Cost controlled acceptable Quality high Customer high satisfaction Runtime minimal resources Design high quality Time to Market (current situation) truncated unpredictable uncontrollable high usually low usually resentful excessive poor Time to Market (target situation) truncated predictable acceptable predictable high high predictable high

38. Research Questions • Schedule • What is an optimal/reasonable project schedule? • Is it feasible to attempt a project within a specific timeframe? • Resources • How can available human resources best be allocated to ensure projects succeed?

39. Research Questions • Factors determining schedules • estimating • staff quality and experience • methodology • re-use • tools

40. Research Activities & Outcomes • Determine existing best practices • collaborative work with institute partners from industry and academia • Develop models, methods, tools and methodologies • Assess models, methods and tools • field assessments using real projects • internal assessments using institute projects • Disseminate knowledge gained • field application of methods and tools with institute partners, publication

41. Research Overview Issue Factors Approach Outcomes Schedule Impact of schedule Data collection Calibrated new reduction Process recording exemplar estimating techniques projects Codification of known models (eg Microsoft) Risk assessment Identification of risk indicators Tool assessment Resource allocation Impact on project Data collection Improved planning planning Process recording methods Impact of decomposition models and parallel implementation Staff Quality Skill identification, fine Improved selection grained classification techniques Experience Identification and recording Training need of experience identification Project structure

42. Research Overview Issue Factors Approach Outcomes Methodology Improved Analysis RAD/JAD High quality prototyping productivity Conversion of prototyping to product development Identify essential features of usability Application generators Ultra high skill levels Re-use Areas of How to achieve this? Components application What is re-use? Plans Assess current re-user Designs practices Test plans Importance of experience Tools CASE versus What do we really need here? Tool set selection lightweight Tool integration New tools Project management Analysis of existing tools Choice and planning Identify processes Languages Review and recommendation Process/design recording

43. 5. Current State of Knowledge and Practice-MS vs the Rest..(cont’d) history • Dijkstra and THE OS in the 70’s (the lesson?)... • “Five people as smart as Edgar Dijkstra can do anything” Reed, 1981 • The first Unix effort…(but what did it take for product versions) • OS/360, PL/I (the lesson?) (60’s).. • Very large teams can build large systems very quickly..~ x1000 person years • Total volumes of functionality (e.g. OS/360) may allow partitioning..TTM issue obscured..

44. Optimal task allocation, observed <1970 one or two people Feasibility Study Requirements Analysis Systems Analysis Program Design Programming Unit Test System Integration System Test Waterfall S/W Process Model / No need for ‘third-party” readable work products! / “Extreme programming”? / Private s/w process? (PeSP compliant?)

45. Time to Market Conclusion • Massive competitive advantage from time to market products with traditional high quality • Necessary elements of time to market focussed processes may be accessible • Detailed ‘research’ by experienced staff with access to current practice is most likely to be successful

46. The role of re-engineering.. S/W Archaeology and S/W Architecture.... / recovery of standard architectures / identification of s/w construction practices, e.g. shifts from one programming style to another / development of maintainability and evolvability classifications for -- § design methodologies § architectural styles / development of maintainability and evolvability classifications for architectural styles

47. component semantics and concept extraction.. The role of re-engineering.. Architecture issues for the S/W Archaeologist / identification of design approaches which ensure that conceptual architectures are transferred to implementation / identification of standard mappings from conceptual to actual architectures which occur using different design approaches on different problems

48. Conclusion • The proposal didn’t win..beaten by SEA.. (Problem ridden.. Consortium) • Is there a need for an SEI in the UK? • Surely more than one) • Do the arguments “map”? • How will the academic community react? (SE research in the US did not stop because CMU got the SEI…)

49. Technico-Commercial ProblemsFacing the Software Industry • Hardware-software run-time environments-adding GUI etc to legacy systems • ‘Evolving’ software dynamically • ‘Engineered’ user interfaces • Software productivity • Design to predicted performance • Time to market • Software quality • Testing • Object orientation • ‘Net-centric’ systems- autonomous communicatings "agents"

50. Technico-Commercial ProblemsFacing the Software Industry • Developments in hardware-software run-time environments • continuing improvements in price/performance + GUI • need to migrate legacy systems to these platforms • need to add functionality to legacy systems on existing platforms, e.g GUI, client-server • Re-engineering of legacy systems to conform to these constraints a major business opportunity