1. Topic Presentation: Task Analysis Article: MacKinnon, K. (2012). Context matters: The value of analyzing human factors within educational contexts as a way of informing technology-related decisions within design research. International Journal of Computer-Supported Collaborative Learning, 7(3), 379–397. Lisa Royse February 25, 2015 Human Computer Interaction Seminar

2. Article Overview • MacKinnon’s field is design-based research • Uses Cognitive Work Analysis (CWA) to identify constraints while designing a new application • Focused on major research component of a 2-year consecutive Master of Teaching program • Identified constraints to inform development of an online research mentorship program to support students carrying out their major research projects • Proposes the process as a method for organizing and reporting contextual information

3. Cognitive Work Analysis (CWA) • Guides the design of useful technologies in the work place • Guides analysis of human-information interaction to inform design of systems • Considers the “actors” that use the system (Fidel & Pejtersen, 2005)

4. Cognitive Work Analysis • Vincente (2004) uses CWA to link social and technological structures • Design begins with identifying a problem; then tailoring the technology to specific human factors • “Human-tech” • Narrows understanding of human behavior to five levels: • Political • Organizational • Team • Psychological • Physical

6. Applying Cognitive Work Analysis The CWA procedure includes two phases before proceeding with design: • Data collection and analysis of constraints and • Review of constraints to make decisions about which constraints will be addressed and corresponding design implications

7. Methods • Documentation analysis • Extended, semi-structured interviews with key stakeholders • Questionnaires • Informal observations and discussions with stakeholders

8. Results • Interview transcripts used to perform a content analysis using the five Human-Tech constraint categories. • Summary of human factor constraints identified in the MT program

9. Results • 14 of the faculty constraints and 10 of the student constraints were addressed • Used three criteria to identify most meaningful constraints • Relevancy to constraint • Relative importance of the constraint • Likelihood of impact

10. Results

11. Classify the Constraint Activity Human - tech Ladder Constraint Classification Students often look to each other for support and advice Team constraint There is a perception among some students that the Psychological research project is not valued in the program constraint Students are expected to conduct the literature review, data Organizational collection and write up for their research projects during constraint their second year Students have little contact with the university and their Team constraint cohort peers during practicum Faculty claim that some students are better able to carry out Psychological the research projects than others constraint Organizational constraints: program priorities and expectations, course expectations, admission criteria, scheduling, job descriptions Team constraints: how groups of people work together to make decisions and carry out goal-directive activity; communication Psychological constraints: personal competencies, understanding, priorities, skills, feelings

12. Conclusions • CWA was helpful in identifying and organizing contextual information during the design research process • Process does not reveal surprising or unusual information but provides a way to make the learning context explicit and accessible to others • Process provides a systematic way to report contextual constraints that helped designer prioritize meaningful constraints

13. References Anderson, T., & Shattuck, J. (2012). Design-Based Research: A Decade of Progress in Education Research? Educational Researcher, 41(1), 16–25. Fidel, R., & Pejtersen, A. M. (2005). Cognitive work analysis. In K. E. Fisher, S. Erdelez, & L. McKechnie (Eds.), Theories of Information Behavior (88-93). New Jersey: Information Today, Inc. MacKinnon, K. (2012). Context matters: The value of analyzing human factors within educational contexts as a way of informing technology-related decisions within design research. International Journal of Computer-Supported Collaborative Learning, 7(3), 379–397. Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 96–115. Reeves, T. C., (2006). Design research from the technology perspective. In J. V. Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds). Educational Design Research. (pp. 86-109). London: Routledge. Vicente, K. J. (2004). The Human Factor: Revolutionizing the Way People Live with Technology. Psychology Press.

14. Questions and Feedback?

15. Design-Based Research (DBR) Characteristics of DBR studies: • Situated in real educational context • Focuses on design and testing of a significant technological intervention • Involves multiple iterations • Partnership between researcher and practitioner • Evolves design principles • Similar to action research • Practical impact on practice (Anderson & Shattock, 2012)

16. Design-Based Research (DBR) (Reeves, 2006, p. 59)

17. Cognitive Work Analysis (CWA) (Fidel & Pejtersen, 2005)

19. Results—Design Implications • Students were organized into small online groups and a research mentor oversaw each group • Students were grouped according to research interests • Online system allowed for “just-in-time” assistance • Students could post questions to their mentor, request feedback on drafts, and receive support from fellow students • Students were able to see notes posted by other students regarding design process, providing opportunity to collaborate

20. Problem MacKinnon Addresses • Design research offers few methodological guidelines about how descriptions of the learning context ought to be developed and reported. This makes it unclear how specific information about learning context relates to design decision-making.