Usability Measurement of Eclipse IDE's Refactoring Features: HCI Research Insights

1. Usability Measurement of Eclipse IDE for Refactoring feature Research Methods in Human Computer Interaction Shahnewaz A. Jolly Instructor: Dr. Saul Greenberg 30th November, 2009

2. Presentation Outline • Introduction • Background and Motivation • Focus • Planning the study • Conducting the study • Analysis and Findings • Problems and Limitations • Conclusion and Future Work

3. Introduction • Open source IDE • Java • C/C++, Python, Perl, PHP

4. Refactoring • Changing program structure • Categories • Changing name and physical location of coding • Changing local organization of coding at class level • Changing coding within a class

5. Background • High-impact refactoring based on architecture violations [Bourqun07] • Aligning development tools with the way programmers think about code changes [Boshernitsan07] • Breaking the barriers to successful refactoring: Observations and tools for extract method [Murphy-Hill08] • Gathering Refactoring Data: a Comparison of Four Methods [Murphy-Hill08] • How We Refactor, and How We Know [Murphy-Hill09]

6. Motivation • Why RENAME is used most • Why programmers prefer doing refatoring manually

7. Focus • Refactoring • Changing name and physical location of coding • Changing coding within a class • Documentation

8. Planning the study • Designing Tasks • Renaming method • Moving class • Changing parameters • Deleting method • Including factory method • Extracting class • Having set and get method

9. Planning the study (cont.) • Setting Procedure and Time • Contacting Users • Minimum 2 months experience • 90 minutes for the study

10. Planning the study (cont.) • Planning Equipment Source: www.cclonline.com Source: ecx.images-amazon.com Source: av.watch.impress.co.jp Source: www.dailyefl.com

11. Conducting the study • Informal Introduction • Focus, Procedure and Duration • Right of the Participants • Permission for Equipments • Consent form Source: jobsearchingblog.com

12. Conducting the study (cont.) • Pre-test questions • Tasks • During-test questions • Break time • Post-test questions Source: lh5.ggpht.com

13. Analysis – Interface Problems • Just right-click doesn’t work Figure: Available options after right click on the method name

14. Analysis – Interface Problems (cont.) Figure: Available options after selecting the whole method name

15. Analysis – Interface Problems (cont.) Figure: Available options from menu

16. Analysis – Interface Problems (cont.) • Option names are not clear enough Figure: The option name doesn’t represent delete function

17. Analysis – Documentation Problems • Example and Screenshots are not provided • Search option is not good

18. Analysis – Documentation Problems (cont.) Figure: List of information after searching by ‘delete method’

19. Interesting Findings • Performance of the participants Figure: Average task time of the participants in minutes

20. Interesting Findings (cont.) • Getting proper answer Table: Performance on Task 1

21. Interesting Findings (cont.) • Reluctance in using documentation • Reluctance in giving up tasks Source: www.fred.org

22. Problems and Limitations • Getting users • Denial of users • Task design • Thinking aloud Source: www.chesskids.com

23. Conclusion • Other option is available • Users don’t rely on refactoring • Documentation is not helpful

24. Future Work • Working in a group • Modifying tasks Source: xianrenaud.typepad.com

25. References [1] F. Bourqun and R. K. Keller. High-impact refactoring based on architecture violations. In CSMR ’07: Proceedings of the 11th European Conference on Software Maintenance and ReEngineering, pages 149–158, Washington, DC, USA, 2007. IEEE Computer Society [2] M. Boshernitsan, S. L. Graham, and M. A. Hearst. Aligning development tools with the way programmers think about code changes. In CHI ’07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 567–576, New York, 2007. ACM [3] E. Murphy-Hill and A.P. Black. Breaking the barriers to successful refactoring: Observations and tools for extract method. In ICSE ’08: Proceedings of the International Conference on Software Engineering, pages 421–430, 2008 [4] E. Murphy-Hill, A. P. Black , D.Dig, C. Parnin Gathering Refactoring Data: a Comparison of Four Methods, Proceedings of the 2nd Workshop on Refactoring Tools, New York, NY, USA, 2008, ACM [5] E. Murphy-Hill, C. Parnin and A. P. Black How We Refactor, and How We Know It – 31st International Conference on Software Engineering, pages 287-297, Washington, DC, USA, 2009. IEEE Computer Society

26. References (cont.) [6] http://en.wikipedia.org, Wikipedia, the free encyclopedia [7] http://www.eclipse.org, Eclipse.org home [8] http://www.ibm.com, IBM – United States