Anders Sigfridsson, Anne Sheehan,

1. Mixing research methods to unveil work practices of dispersed Open Source communities: lessons learned from the PyPy study Anders Sigfridsson, Anne Sheehan, Gabriela AvramInteraction Design Centre,University of Limerick, Ireland

2. Open Source Communities and GSD • For academic research in the area of GSE, Open Source communities are a good case of successful utilization of the competencies of dispersed, heterogeneous groups of developers (Crowston & Scozzi 2002) • Opportunity: publicly available archives of email lists, chat transcripts, and project documentation – rich source of information • Methodological challenges: • rich source of data, easy to access for quantitative studies • qualitative analysis of this body of material can be a cumbersome task • potential for misinterpretation due to the lack of context • on-site observations of work practices or formal interviews are usually difficult • Sprints provide researchers with the opportunity of performing qualitative studies that go beyond interviews at distance • Complex tapestry of heterogeneous actors and components that require a combination of methodological lenses to understand and explain (Sack 2006) • Our methods inspired by ethnography, virtual ethnography (Hine 2000)

3. The PyPy community • Was initiated in early 2003 with the objective of implementing a Python interpreter using Python itself and is one of the four large language implementation projects within the Python community • 300-500 people subscribe to the mailing lists, ~50 people have commit rights, and the core group consists of ~10 developers • Received partial funding from the EU between December 2004 and March 2007, enabling funding of sprint participation and a few people to work full time (both developers and project admins) • Sprint systematically, about every 6 weeks or so, always in different locations to accommodate as wide a group of participants as possible

4. The PyPy study: research purpose • An exploratory study aimed at investigating the collaborative work practices of this particular Open Source community • Focused on understanding how sprint-driven development is achieved and its implications on collaboration in the community Communities of Practice (Source: Wenger 1998)

5. Analysis of Limerick Sprint (August 2006) • Direct observation and in situ conversations • Project administrator organized workshop for local researchers • Video recordings of daily status meetings and workshop

6. The preliminary conclusions • a situated learning perspective could help account for the value of having sprints in an otherwise distributed project. • Sprints facilitate situated learning in the project, allowing developers to come together and work through hands on collaboration using agile practices such as pair programming and daily scrum meetings, as well as to socialize and form stronger personal relationships. • sprint-driven development also facilitates the expansion of the community through enculturation, enabling new members to both achieve necessary technical knowledge and the community membership needed to contribute to the project. • In the next phase, the study became more investigative in nature. The aim: to get an in depth understanding of what sprints mean for developers who participate in the project.

7. Analysis of Trillke Sprint (March 2007, Germany) • Joined PyPy as a newcomer and went to sprint to work with them as a regular participant, getting an insiders view • Participant observation and in situ conversations • Audio recordings of all group meetings

8. Analysis of online activities (August 2006-January 2008) • Extending back retrospectively as well as following events continuously • Identifying and following in-depth cases of newcomers approaching the community and beginning to participate • Structured analysis of how sprints add to the dispersed collaboration Excerpts from mailing lists Public archives

9. Email questionnaires (May 2008) • Sent to a representative core group • Meant to provide extra support for our analysis, not to be an exhaustive survey

10. Initial conclusions • Sprints facilitates situated learning and subsequently enables closer collaboration online because it allows developers to: • collaborate using hands-on agile approaches such as pair programming and daily status meetings • socialize and form stronger personal relationships • This also helps to facilitate the expansion of the community through enculturation: • it helps new members to achieve both the necessary technical knowledge and the community membership needed to contribute to the project Limerick Sprint Trillke Sprint

11. Discussion • Our research questions were not defined from the start. This has meant that our research focus has been tuned to what was important in this particular context • The findings, although not generalizable, contribute significantly to our ability to explain what is going on in this particular case. • During the study, our focus shifted to how do the sprints extend the dispersed collaboration and on how they allow for enculturation of newcomers. • we knew the context well enough to decide on what was important. We had spent roughly 6 months following the online activities after the initial observation of a sprint, therefore we knew both what goes on at sprints and were aware of the general • The importance of studying the dynamics of the community • While working up the data, we have discovered how one of the sources of material seems richer than the others. • combining different research methods is not straightforward and will require the researcher to do some tailoring.

12. Suggestions for the debate • The initial choice of methods and the general research approach seem to be very important for the evolution of research projects. • There is a need to learn more about which methods are suitable for which kinds of questions / research interest. • The evolution seems to be an important aspect of consideration and self-critique in the context of researching into GSD. • What are the general shortcomings / benefits of exploratory and evolutionary research designs compared to more statistical approaches with predefined hypothesizes? (I.e., why is it important for the GSD community to take into account this kind of research instead of doing experiments and such?)

13. Thank you! gabriela.avram@ul.ie anders.sigfridsson@ul.ie anne.sheehan@ul.ie