Implementing the Scholarship of Teaching and Learning (SOTL)

1. Implementing the Scholarship of Teaching and Learning (SOTL) We will: -Define SOTL -Identify types of SOTL projects -Examine steps of SOLT projects -Identify resources to develop SOTL projects -Evaluate potential SOTL projects Scott Cottrell, Ed.D. Assistant Professor scottrell@hsc.wvu.edu 304-293-0410

3. What is the SOTL? The scholarship of teaching and learning is an investigation of a problem relating to teaching or learning. The study of the problem is realized through methods appropriate to disciplinary epistemologies (e.g., quasi-experimental design, case studies), and the results are communicated to peers for critical reflection (e.g., journals, conference presentations, online delivery).

4. What is Considered Scholarship? www.aamc.org/mededportal PowerPoint Presentations Web Sites Virtual Patients Lab Guides CD-ROMs Cases (PBL, OSCE, Team Learning, CPC, etc) Faculty Development Resources

5. Why is SOTL important? •  SOTL breaks down barriers between research and teaching. •  SOTL yields an opportunity for teachers to frame their existing responsibilities as an investigation. •  SOTL allows teachers to present educational reforms and innovations to peers for critical reflection. • SOTL helps satisfy expectations of research in an academic institution. (AAMC- Advancing Educators and Education: Defining the Components and Evidence of Educational Scholarship) •  SOTL focuses on student learning and its improvement.

6. What is the purpose of research? • Describe- pharmacy students’ academic success across demographics • Predict- e.g., residents’ professional behavior in future practice • Improve- e.g., effectiveness of intervention to improve teaching skills of nursing faculty • Explain- subsumes all three

7. Theoretical Framework Scholarship Seeks to Explain Theoretical Development Research Questions Assessment Seeks to Describe Data Analysis DataCollection ResearchDesign Distinction Between SOTL and Assessment Cottrell, S.A. (2006). A Matter of Explanation: Assessment, Scholarship of Teaching and their Disconnect with Theoretical Development. Medical Teacher. 28(44), 305-08.

8. What is a theory? • A theory is an explanation of events in terms of the structures and processes that are presumed to underlie them. • Theory consists of constructs that specify how the constructs are related.

9. What is a construct? • A construct is a type of concept used to describe a structure of process that is hypothesized to underlie particular observable phenomenon. • e.g., residents’ professionalism, teaching skills, motivation, leadership

10. Important Caveat Don’t confuse theory with principles!! For example, adult learning principles are not a theory. Most researchers use theory much like a drunkard uses a light post – more for support than for illumination.

11. Example of Confusing Principles with Theory Educators should encourage students to exercise an innate ability to self-assess and evaluate their ability to perform a task. • This belief has set the stage for several studies that examine students’ self-assessment skills in a variety of learning contexts.

12. Assumption is not Supported with Theoretical Support “How do they know what they don’t know”

13. Self Assessment? Don’t Bother • Kruger and Dunning's research, for example, argues that learners who have clear deficits and would most likely benefit from self-assessment tend to be poor self-evaluators relative to learners who are doing well. • Self-Reflection? - A potential construct for exploration, which is not related to accuracy of evaluation skills.

14. The Essentials of SOTL • Identify a problem relating to teaching and learning • Articulate research questions 3. Adequate preparation/literature review • Identify an appropriate research design to answer your question • Analyze the assessment results to address the research problem • Connect results with extant research findings 7. Dissemination

15. The Essentials of SOTL • Identify a problem relating to teaching • and learning? • Examples: • We need to improve residents’ teaching skills. • How should we implement 360 evaluations? • Are self-evaluations helpful??

16. Articulate Research Questions 2. Define exactly what is it you want to know? Example: What impact does on online course have on nursing students’ interest in teaching, knowledge of educational theory, and assessment skills?

17. The Essentials of SOTL 3. Adequate preparation Informing investigation with existing literature and a theoretical framework offers several advantages, including: Direction Focus Coordination

18. Essentials of SOTL • Identify an appropriate research design to • answer your question. 4. Identify an appropriate research design to answer your question Things to consider: How are your questions framed? A good question lends itself well to a particular research design. Disciplinary epistemologies (e.g., quasi-experimental design, case studies)

19. Essentials of SOTL • Identify learning expectations by developing • measurable learning objectives. • Examples include: • The resident will be able to communicate complicated material to medical students. • The resident will be able to identify three strategies to partner with health care managers and health care providers to assess, coordinate, and improve health care. • The medical student will be able to demonstrate compassion for a patient, as evidenced by sitting down to communicate important information, maintaining direct eye contact and listening/responding to questions.

20. Essentials of SOTL Operationalize: Identify existing or create new assessment methods to capture evidence that the residents acquired the learning objectives. Examples include: Patient evaluations of the resident Chart reviews Written exams Student evaluations Journals Clinical exams

21. Credible Comprehensive Reliable Valid Content Concurrent Predictive Construct Consequential Useful/Practical in your program SOAP OperationalizeConsiderations for Assessment Tools:

22. SOAP Example of a Blueprint for Articulating Objectives ? = Don’t Know or Potential opportunities to incorporate 23

23. Essentials of SOTL • Analyze the assessment results to answer the • research problem.

24. Contributing Scholarship 6. Connect results with extant research findings. Otherwise, your results are isolated.

25. Essentials of SOTL 7. Dissemination - Effective communication of work to intended audiences helps move forward what is collectively known about the phenomenon. - It also invites reflective critique to improve quality of future work.

27. Examples of SOTLat WVU Dr. Paul Ogershok recognized that teaching basic information about pediatric medicine was little fun for both faculty and students. Therefore, he designed a game to help students acquire a strong foundation of pediatric medicine. He wanted to know if students learned from the game, and whether students reflected that the game was a useful learning opportunity. As evidenced by student comments and observations on evaluation forms, it was determined that the game was a valuable and fun way to help students comprehend important information. .

28. Where do I Publish?? • http://www.mlb.ilstu.edu/ressubj/subject/educat/sotl.htm • Multidisciplinary: Journal of Continuing Education in Nursing Journal of Nursing Education Journal of Instructional Psychology Biochemistry and Molecular Biology Education Chemical Educator Journal if the International Association of Medical Science Educators

29. Quality of Research? • What population was it based? • Does it clearly explain constructs? • Does it generate questions? • Is it practical?

31. Submitting Abstracts:Every Word Counts 1. Identify clearly the purpose of you work. Why is it important? 2. Rely on literature to frame an argument. • Explain the methodology like I am a three-year old? • Results: Just the facts. • Discussion: Was purpose achieved? Limitations? How does it contribute to literature? • Conclusion: How does your work contribute to scholarship? New directions for research? What is the “take-home” message?

32. Activity/Exercise Identify at least one problem you would like to address about teaching and learning. Considerations: What specific objectives would you like students/residents/faculty to learn? What assessment methods would help you determine whether the objectives were acquired? What are your thoughts about methodology?

33. References Beattie DS.  Expanding the View of Scholarship (Editorial).  Academic Medicine, Vol 75, N0. 9 / September, 2000, pp. 871-876. Bloom, B.S. (ed.). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. White Plains, N.Y.: Longman, 1956. Boyer, E.L. Scholarship reconsidered: Priorities of the professoriate. Princeton, N.J.: Carnegie Foundation for the Advancement of Teaching; 1990. Cottrell, S.A. (2006). A Matter of Explanation: Assessment, Scholarship of Teaching and their Disconnect with Theoretical Development. Medical Teacher. 28(44), 305-08. Cottrell, S.A. & Jones, B.A. (2002). Researching the scholarship of teaching and learning: An analysis of current curriculum practices. Innovative Higher Education, 27, 3, 10-16. Cross, K.P. Classroom research: Implementing the scholarship of teaching. San Francisco: Jossey-Bass; 1996. Huba, M.E. & Freed, J.E. Learner-centered assessment in college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon; 2000. Fincher RE, Simpson DE, Menin SP, Rosenfeld GC, Rothman A, McGrew MC, Hansen PA, Mazmanian PE, and Turnbull JM.  Scholarship in  Teaching: An Impreative for the 21st Century.  Academic Medicine, Vol 75, N0. 9 / September 2000, pp. 887-894.  Palomba, M.B. & Banta, T.W. Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass; 1999. Shulman, L.S. (1998). Course anatomy: The dissection and analysis of knowledge through teaching. In Hutchings, P. (eds.), The course portfolio: How faculty can examine their teaching to advance practice and improve student learning. (pp. 5-12). Washington, D.C.: American Association for Higher Education.