8 th October 2007

1. SERE: Software Engineering for the Research Environment 8th October 2007 int i, j, k; while… Viva presentation Lancaster University Andre Oboler Computing Department Lancaster University

2. Thesis focus • This thesis presents a new process based approach to software engineering designed to meet the needs of academic Computer Science researchers. • The core objective was to examine whether software engineering approaches could be adapted for the research environment so that they gain acceptance and enable improvement of the research process.

3. Limitations in scope Academic research environment: • While the approach and tools may have practical application in an industrial R&D environment, this is outside the scope of this work Computer Science Research: • While some process descriptors are generic in nature, the focus of this work is for research that involves the development of experimental software • We assume a graduate level of computer science background amongst the researchers, our approach seems to build on this and focus it on the needs of research

4. Hypothesis • A RAISER-based approach (building on our past work) can improve the research process in computer science academia. • The null hypothesis is that a new approach will make no difference.

5. Outcomes • The data from this thesis leads us to reject the null hypothesis on the basis that: • Empirical data shows an improved outcome • Qualitative feedback showed a positive experience • Process descriptors demonstrated benefits • The process perspective aids researchers in reflection and enables further improvements

6. Contributions from this work • The Process Model for Supporting Research • The Personal Process Model • Formal Technical Reviews for research • Documentation standard for research • The Research Process Optimisation Framework • The high level framework for CARE  First International Workshop on Improving Research Productivity

7. PhD Process Review • This research was undertaken in an evolutionary manner. • Though necessary to development the SERE framework, the final cycle (and post experiment analysis) leaves SERE much more mature than it was in the early cycles. • Though unavoidable, three years of data on the process as it stands now would be an improvement over the data we have • With plenty of scope for further development, even given a further three years the approach would continue to evolve • Inclusion of PhD candidates in the participant sample would be another improvement, but was not possible in the scope of this research • The cyclic nature of the work led to some difficulties in thesis structure, however the current mixed structure resolves these nicely (covering both the cycles and the big picture)

8. Experimental difficulties • By necessity participation was voluntary +ve gives a true picture on adoptability -ve smaller than idea sample sizes • Need to provide full support myself +ve first hand experience with all the data -ve limiting factor on sample size and what could be done • Timing not always ideal but dictated by circumstances • Interview data not full processed • No budget for transcription & very time consuming • Some interviews pre-selected then transcribed (other not transcribed) • Getting survey responses from students was a problem • Limiting the thesis scope has been difficult as the work touched upon many interesting and related topics.

9. Future developments • Expansion of provision to PhD students • Examination of the impact of a full Software Development Lab • Examination of specialised group templates for Personal Process Models • Examination of effectiveness and adaptation to industrial Research & Development • Further development of CARE • Create of new and more specialised process descriptors for different research areas

10. Viva discussion There are further slides if you wish on: • Motivation and reply to detractors • The nature and needs of research • The research process optimisation framework • Improving knowledge transfer • The Personal Process Model • This research’s Personal Process Model

11. Motivation and reply to retractors

12. Objections One may ask, why bother? • Current approaches have been working for decades • This seems like a lot of extra work • Experienced researchers KNOW what they are doing • The struggle and reinvention of ideas is part of the learning process

13. Response • Increased pressure on science to deliver • Increased pressure on students to complete faster • Increased competition for resources • Computer science is not yet mature • Information Technology is now a part of life, academia needs to be more scientific to hold value • We can and should do better.

14. Nature and needs of research

15. Research is… • “original investigation undertaken in order to gain knowledge and understanding”[1] • “creative work undertaken on a systematic basis in order to increase the stock of knowledge” [2]

16. Systematic yet Creative? • Reducing wasted effort is a type of improvement that does not impact on creativity • Not all effort is “wasted effort” so not all automation is “safe” – some might reduce scientific creativity • Remember: research is creative work undertaken on a systematic basis, creativity and a systematic nature must coexist

17. The Research Process Optimisation Framework

18. a b c d e f The Research Process Optimisation Framework Effort is the volume filling the pipe per Standard Knowledge Unit Knowledge (SKU) Reflects the fact that some parts of the research process take more effort than others, and the amount of this that is “wasted” also varies.

19. …as a bar chart Our aim should be to improve the Effort : Knowledge ratio, either by reducing (wasted) effort or by increasing the amount of knowledge that can be gained from that effort.

20. Improving knowledge transfer

21. Improving knowledge transfer • How do we improve knowledge transfer? • Capture process information • Share it • Develop it systematically • Allow feedback • We must also ensure that: • Creativity is not inhibited • “Wasted effort” is not increased (overly bureaucratic solutions provide one counter examples that both adds waste and inhibits creativity)

22. Approaches to knowledge transfer Artifacts: • Research journal • Electronic research journal (so searchable) • Coding Guidelines • Personal Process Model (PPM) Process: • Technical Reviews

23. Personal Process Model

24. Personal Process Model • Uses a a java source code file to set out the research process as a set of classes • Documents the key parts of the research process using comments • Encapsulate and relate ideas using class relationships • Generate external documentation, class diagrams and hypermedia using dOxygen • Monitor changes in ideas / plans using diff • Discuss changes with supervisor and colleagues, gain their wisdom MY STUFF

25. Key advantages - researcher • No context switching between coding environment and research management environment • Builds on exiting software engineering knowledge • Uses an existing shared language for representation • Diagrams are easy to read and follow, but don’t need to be created / updated by hand • Html output is easy to navigate and easy to share • Process Comments can link to internal code documentation MY STUFF

26. Key advantages - research For research as a whole, this approach means: • More data is captured about the particular research • More data is shared about research approaches • Researchers share best practise and improve upon it MY STUFF YOUR STUFF

27. This research’s Personal Process Model

28. My research model

29. My research model Next slide will expand this part

30. Zooming in… We are discussing this (please forgive the recursion)

31. References [1] "RAE Circular 2/00" vol. 2006: Higher Education Research Organisation, 2000 [2] OECD, The Measurement of Scientific and Technological Activities - Frascati Manual 2002 : Proposed Standard Practice for Surveys on Research and Experimental Development Frascati, France: Organisation for Economic Co-operation and Development, 2002.