Classical Open Source Software Process Model

1. Classical Open Source Software Process Model Alan Kelon Oliveira de Moraes <alan@kelon.org>Software Quality and Processes – 2005.2 CIn-UFPE

2. Open-source software is not a new idea but only recently have technical and market forces converged to draw it out of a niche role. Eric S. Raymond The Cathedral & the Bazaar, 1999

3. open source companies had raised ~$900 million from venture capitalists since 1999 [Rivilin 2005]

4. Summary • What is open source and why should you care about it? • Classical open source process model • Remarks • References

5. Definitions and Motivations What and why?

6. What is Open Source? • Open Source software is the process of systematically harnessing open developmentand decentralized peer review to lower costs and improve software quality [Raymond 1999]

7. Basic idea behind open source • When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. • People improve it, people adapt it, people fix bugs. http://www.opensource.org [Open Source Initiative]

8. Open source license definition • Open source means more than source code availability: • The source must be available for redistribution without restriction and without charge • The license must permit the creation of modifications and derivative works • Must allow those derivatives to be redistributed under the same terms as the original work. [OSI] [Perens 1999] [O’Reilly 1999]

9. How is open source different? • The term open source generally refers to software that differs from its commercially sponsored counterpart in three ways • source code’s ready availability to prepare derivative works • collaborative production approach • community governance of production [West 2005]

10. It is still evolving... • The term “open source project” does not have a clearly defined stable and shared meaning comparable to “open source license” • The processes evaluated refer to “classical” open source projects [O’Mahony & West 2005]

11. Why should you care? • It’s important not to reduce the open-source debate to a question of NT versus Linux, or Microsoft versus the rest of the world • Understand how the lessons of open source can be applied to software development across the board. • The open-source process reflects a powerful global trend toward networked collaboration. • Printing press: spread of knowledge in Renaissance • Open Source: large scale cooperative development efforts today. [O'Reilly 1999]

12. Open wallets for open source software • In 1999 and 2000, according to VentureOne, venture capitalists invested $714 million in 71 open-source companies • Turbolinux raised $95 million • Linuxcare raised at least $80 million • Most of those projects collapsed… [Rivilin 2005]

13. Open wallets for open source software • Venture capitalists are embracing open-source technology companies again • $149 millions to 20 business in 2004 • At least three open-source start-ups raised $20 million in April 2005 • The big difference is the increased adoption of open-source software by corporate users today • Red Hat had $125 million in revenue in 2004 and now has a market capitalization around $2 billion. [Rivilin 2005]

14. New business models • JBoss's business model, built on selling support services, made sense [Rivilin 2005] • Open Source has effect on commoditization, system architecture and network effect, and the development practices associated with software as a service[O'Reilly 2005]

15. Classical Open Source Process Model A brief survey

16. The Cathedral and the Bazaar • Development style of Linux • Release early and often, and listen to your customers • Delegate everything you can • Be open to the point of promiscuity • Present a plausible promise [Raymond 1999]

17. Brooks vs. Linus • Brooks’ Law: ‘‘Adding more programmers to a late project makes it later.’’ • Linus’ Law: “Given enough eyeballs, all bugs are shallow” [Raymond 1999]

18. A case study of open source software development: the Apache server • Apache Server Process: decision making-oriented process • Emphasized decentralized workspaces and asynchronous communication • Email lists exclusively to communicate with each other, and a minimal quorum voting system for resolving conflicts [Mockus et al. 2000]

19. A case study of open source software development: the Apache server • There is no explicit system-level design, or even detailed design • There is no project plan, schedule or list of deliverables • In order to keep track of the project status, an agenda file is stored in each product's repository, containing a list of high priority problems, open issues, and release plans. [Mockus et al. 2000]

20. A case study of open source software development: the Apache server • There is no single development process, but these are developer's common actions: • Discovering that a problem exists • Determining whether a volunteer will work on it • Identifying a solution • Developing and testing the code within their local copy of the source • Presenting the code changes to the group for review • Committing the code and documentation to the repository. [Mockus et al. 2000]

21. A case study of open source software development: the Apache server • Quantitative analysis based on Developer mailing lists, CVS commit logs and Problem reporting database • From may February 1995 to May 1999 [Mockus et al. 2000]

22. A case study of open source software development: the Apache server • Almost 400 individuals contributing code • The top 15 developers contributed more than 83% of the MRs and deltas, 88% of added lines and 91% of deleted lines. [Mockus et al. 2000]

23. A case study of open source software development: the Apache server • Around 3060 different people submitted 3975 problem reports. • The top 15 problem reporters (only 3 also in the core) submitted only 213 or 5% of Prs • Almost 2600 developers submitted one report, 306 submitted two, 85 submitted three • The maximum number submitted by one person was 32. [Mockus et al. 2000]

24. A case study of open source software development: the Apache server • Time to resolve problem reports • 50% of PRs are resolved within a day • 75% within 42 days • 90% within 140 days [Mockus et al. 2000]

25. Descriptive Process Model for Open Source Software Development • Three basic views for software process models [Humphrey 1989] • The state view • The organizational view • The control view [Johnson 2001]

26. Descriptive Process Model for Open Source Software Development • State view • Closed prototyping • Iterative and incremental enhancement • Concurrent development • Large-scale peer review • User-driven requirements [Johnson 2001]

27. Descriptive Process Model forOpen Source Software Development • Organizational view • Decentralized collaboration • Trusted leadership • Internal motivation • Asynchronous communication [Johnson 2001]

28. Descriptive Process Model for Open Source Software Development • Control view • Informal planning • Tiered participation • Modular design • Ubiquitous tool support • Shared information space [Johnson 2001]

29. An overview of the Software Engineering Process in the Mozilla project • The importance of tools • Bugzilla • Tinderbox • Bonsai • LXR [Reis and Fortes 2002]

30. An overview of the Software Engineering Process in the Mozilla project • Great deal to develop and to document the Mozilla software process • Rely heavily on communication tools such as mailing lists, irc, news, bugzilla • “Bug-driven” development {bugzilla} • Requirements • A message thread is started on a public newsgroup regarding a change in functionality • When a person has a specific idea for a change a bug will often be directly filed with no newsgroup discussion • “You can get a feature in if you’re willing to write code for it” -- Boris Zbarsky, Mozilla developer • Code review and inspection {bonsai} • Nightly build {tinderbox} [Reis and Fortes 2002]

31. An overview of the Software Engineering Process in the Mozilla project • Effective version control • A well-defined protocol for integrating source code changes • A high degree of accountability for people who integrate code • High modularity • Good communication channels • Documentation not always up to date [Reis and Fortes 2002]

32. Traditional SE: Elicitation Analysis Specification and modeling Validation Communicating and managing Open Source Post-hoc assertion Reading, sense-making, accountability Continually emerging webs of discourse Condensing and hardening discourse Global access to discourse Understanding the Requirements for Developing Open Source Software Systems [Scacchi 2002]

33. Free Software development lifecycle [Senyard and Michlmayr 2004]

34. Bazaar style development lifecycle [Senyard and Michlmayr 2004]

35. Quality assurance under the open source development model • User participation in open source projects is high • up to 20% of the changes for almost 50% of the projects • discovering 20–40% of the faults in 20% of the projects. • Change management processes and tools are at the cutting edge of large-scale collaborative software development • Documentation is not a high priority for most projects: ‘‘TODO’’ and install guide • Little investment in utilizing testing techniques and tools, relying mainly in community testing [Zhao and Elbaum 2003]

36. Open source software maintenance process framework [Koponen and Hotti 2005]

37. Open source software maintenance process framework [Koponen and Hotti 2005]

38. Concluding remarks Past, present and future

39. Past • Hacker community activity • Self-contained moviment

40. Present • Open source gets momentum • Developing F/OSS is different than software engineering [Scacchi] • not better, not worse, but different and new • more social, more accessible • F/OSS systems don’t need and probably won’t benefit from classic software engineering [Scacchi]

41. Abstract generic process models • Specification-based models • Waterfall • Incremental • Evolutionary development models • Prototyping • Spiral • Open Source* [Sommerville 1996]

42. So an open source process model is… • Evolutionary/maintenance-oriented development model • Not just a technical development process • Collaborative effort • Based on community government • User-driven requirements • Modular architecture/design without UML stuff • Heavily dependent on communication tools because it is distributed, asynchronous and discontinuous • Indeed it is not a traditional software engineering process

43. Future • Open source is still in its infancy today • OSS projects need to be more professional • Open Source Software Process • Do not to move into Process Paralysis [Yourdon 1987]

44. [Neus et al. 2005]

45. [Neus et al. 2005]

46. Something else?

47. References • O’Mahony, S. and West, J. 2005. What makes a project open source? Migrating from organic to synthetic communities. Academy of Management conference, Technology and Innovation Management division, Honolulu, August 2005. • O'Reilly, T. 1999. Lessons from open-source software development. Commun. ACM 42, 4 (Apr. 1999), 32-37 • O'Reilly, T. 2005. The Open Source Paradigm Shift. In: Feller, J., Fitzgerald, B., Hissam, S. A. and Lakhani. K. R. eds. Perspectives on Free and Open Source Software. The MIT Press, 461-482. Also available in http://tim.oreilly.com/articles/paradigmshift_0504.html since June 2004 • Open Source Initiative. http://www.opensource.org • Perens, B. .1999. The Open Source Definition. In: DiBona, C., Ockman, S., and Stone, M. eds. Open Sources: Voices from the Open Source Revolution. O'Reilly and Associates, 171-188. • Raymond, E. S. “The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary”. O'Reilly and Associates, 1999. Also available in http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ since May 1997 • Rivilin, G. 2005. Open Wallets for Open-Source Software. The New York Times, April 27, 2005. • West, J., Jury, A., Sengupta, S., and Penberthy, S. 2005. Bookshelf. IEEE Softw. 22, 4 (Jul. 2005), 114-118.

48. References • Johnson, K. A. “Descriptive Process Model for Open-Source Software Development”, Master Thesis, Univ. Calgary, Alberta, 2001. • Koponen, T. and Hotti, V. 2005. Open source software maintenance process framework, ACM SIGSOFT Software Engineering Notes, v.30 n.4, July 2005 • Mockus, A., Fielding, R. T., and Herbsleb, J. 2000. A case study of open source software development: the Apache server. In Proceedings of the 22nd international Conference on Software Engineering ( Limerick, Ireland, June 04 - 11, 2000). ICSE '00. ACM Press, New York, NY, 263-272. • Reis, C. and Fortes, R. P. M. 2002. An overview of the Software Engineering Process in the Mozilla project. In Proceedings of Workshop on Open Source Software Development, Newcastle, 155-175 • Scacchi, W. 2002. Understanding the Requirements for Developing Open Source Software Systems. In IEE Proceedings Software, volume 148, number 1, pp. 24-39. • Senyard, A. and Michlmayr, M. 2004. How to Have a Successful Free Software Project. In Proceedings of the 11th Asia-Pacific Software Engineering Conference (Apsec'04) - Volume 00 (November 30 - December 03, 2004). APSEC. IEEE Computer Society, Washington, DC, 84-91 • Zhao, L. and Elbaum, S. 2003. Quality assurance under the open source development model. J. Syst. Softw. 66, 1 (Apr. 2003), 65-75.

49. References • Neus, A., Scherf, P. and Subjects, R. 2005. Opening minds: Cultural change with the introduction of open-source collaboration methods. IBM Journal of Research and Development, Volume 44, Number 2, Page 215. • Sommerville, I. 1996. Software process models. ACM Comput. Surv. 28, 1 (Mar. 1996), 269-271. • Humphrey, W. S. 1989 Managing the Software Process. Addison-Wesley Longman Publishing Co., Inc. • Yourdon E. A Game Plan for Technology Transfer, Tutorial: Software Engineering Project Management. R.H. Thayer, ed. Computer Society Press, 1987