Workshop: Formative Assessment Institute for Excellence in Education Summer Teaching Camp 2013

1. Workshop: Formative AssessmentInstitute for Excellence in Education Summer Teaching Camp 2013 Edith Gurewitsch Allen, M.D. Mona Mohamed, M.D., Ph.D. Julianna Jung, M.D. Presenters have no relevant disclosures or conflicts of interest

2. Objectives • differentiate formative from summative assessment • demonstrate techniques for formative assessment of learners • develop and solicit formative assessment of: • communication skills • information literacy • strategic planning/critical reasoning • attitudes and disposition [professionalism] • Formative assessment for course or program refinement

3. What is Assessment? • Establishing clear, measurable expected learning outcomes • Ensuring sufficient opportunities to achieve those outcomes • Systematic gathering, analysis and interpretation of evidence to determine extent to which (how well) learning matched expectations • Using the resulting information to understand and improve learning

4. General Purpose of Assessment • Defines what learners will regard as important – Brown 2001 • high value placed on marks and grades; • ultimately students focus their efforts on assignments and what they will be tested on • Ensures that what is important is learned ASSESSMENTS MUST ALIGN WITH EDUCATIONAL GOALS

5. Planning Our Teaching Traditional Paradigm New Paradigm What do students need to know? critical understandings, theories, models; knowledge, skills and attitudes How will we assess their learning? How do we structure the educational experience to ensure that they learn? • What topics should we teach? • How do we test that the topics were learned? • What are the criteria for determining sufficiency of learning (what earns an A, B, C, etc)? Learner-Centered Teacher-Centered

6. Characteristics of Assessment Summative Assessment Formative Assessment Objective > Subjective Comparative, Relational Solitary Conclusive, “Add-on”/Post Hoc Retrospective: were goals met? • Subjective > Objective • Personal, Reflective • Interactive, Social • Midstream, Guiding • Prospective: are goals being met?

7. Understand goals and characteristics of excellent work Relate new learning to prior experiences Spend significant time studying and practicing, using and applying new knowledge in some way Diverse learning styles are respected Engage in multi-dimensional, real-world tasks, and interact with others Assessments are learning activities in their own right Reflect on what and how they learned, seeing coherence in their learning Students Learn Most Effectively When They: Evidence-based Strategies to Promote Lasting Learning

8. Universally Valued Learning Goals • Communication Skills • Information Literacy • Strategic Planning/Problem Solving • Professional Attitudes/Dispositions

9. In-Class Techniques Assessing Communication Skills Drawing Exercise

11. Communication Skills ENCODING DECODING Pay attention Receive information without first anticipating the response to be given back Ask questions to confirm or revise understanding • Understand the objective: why are you communicating? • Understand the audience: who are they? What do they need to know? • Anticipate reactions or potential sources for confusion; plan the message • Seek feedback to confirm understanding

12. In-Class Techniques Information Literacy “Headbandz”

13. Information Literacy • Recognize the need for information • Identify what information is needed • Find the information • Evaluate information critically for credibility and relevance • Use information to answer question/solve problem • Use information legally & ethically

14. In-Class Techniques Strategic Planning/Critical Reasoning Think-Aloud Exercise

15. www.websudoku.com

16. Metacognition • Use efficient learning techniques • Discuss and evaluate problem-solving strategies used • Critically examine the bases for arguments • Correct or revise reasoning • Form efficient plans for completion • Evaluate effectiveness of decisions/actions

17. In-Class Techniques Assessing Professional Attitudes/Disposition Opinion Polling

18. When a trainee fails to perform a task as expected, it is most often because: • s/he doesn’t value the task or see it as personally relevant. • s/he does not understand what is being asked of her/him. • s/he is overburdened or fails to plan for timely completion. • s/he lacks required knowledge or skill to complete the task.

19. When an educator deviates from a professional standard, it is most often because: • s/he opposes or disagrees in general with the standard. • s/he is unaware of the standard. • practical realities make the standard difficult or impossible to follow. • s/he deems the standard to be inapplicable or inappropriate for the particular situation.

20. How Students Can Contribute to Assessment – Suskie, 2009 • Identify the purpose of the assessment and ensure that they are relevant and worthwhile • Articulate learning goals, their importance and value • Design tools (rubrics, surveys, portfolios) • Advise on how to share results with students • Review logistics of assessments for feasibility and undue burden

21. Group Activity:Reducing Medication Error GROUP A: Root Cause Analysis GROUP B:Polypharmacy Atrovent Prednisone Claritin Xanax Vasotec Timoptic Avandia Neurontin Augmentin Protonix Bumex • 42 y.o. male, limited English, history afib, on verapamil 250 mg three times daily • Admitted from ER for ORIF wrist fx • 24 hours later developed atrial fibrillation, which resulted in congestive heart failure

22. Group Activity:Developing Formative Assessments • What “key” knowledge should your learners demonstrate through this task? • Which “universal” skills (communication, information literacy, strategic planning/critical reasoning, attitude/disposition) will you focus on? To what extent (percentage, points)? • Which criteria will you use to rate the adequacy of their proposal? • What challenges/feedback will you provide to advance their learning beyond the initial effort?