Module 6: Assessment

1. Module 6: Assessment IITE Professional Development Course Lucknow University (6/4/2010) Professor Tim Keirn timkeirn@csulb.edu

2. A Review: Standards-Based Approaches and Learning Outcomes • Programme learning outcomes • Course learning outcomes • Program curricular map w/ sequenced papers for two certifications • Physical Science: Teacher Ed, Chemistry, Mathematics and Physics • Biology/Life Science: Teacher Ed, Chemistry, Botany and Zoology

3. Review Continued • Learning outcomes for each paper • Design an example of a lesson within a paper that is: • Inquiry-based • Aligned to a paper specific learning outcome • Engages students with materials from the web

4. Review Continued • Design an assessment that is aligned to the inquiry-based lesson and the specified paper learning outcome • Design a rubric for the aforementioned specific assessment • Publish materials to the portal on the web

5. General Introduction to Assessment • Do students learn what faculty believe they are teaching? How do you know? On what evidence do you substantiate your claims? • As an employer of a candidate with an upper second B.Sc from Lucknow in e.g. Botany -- what do I know ‘they know’ and what do I know ‘they can do’? • What more do they know and what more can they do than someone with a ‘lower second’ and compared to some one with a ‘first’?

6. Introduction to Assessment • Can I assume that someone who did the same paper with Vivek ‘knows and can do’ the same as a student of Nalini? • If so -- how can you substantiate these claims? • Think-Pair-Share Strategy: Identify and discuss the origins of three weaknesses in the current means by which students are assessed at Lucknow University • This may not be an exhaustive list!

7. 3 Weakness of Current Assessment

8. Traditional Assessment • Traditional assessment is inseparable from traditional modes of teaching and learning • PH.D. provides discretion as to what is taught • Stand and deliver • Design assessment to measure knowledge retention • Assign marks based on the ‘volume’ of knowledge retained • PH.D provides discretionary authority to assess the ‘volume’ itself

9. Problems with Traditional Assessment • Serve to discriminate between students as opposed to demonstrating competencies • Almost always measures the reproduction of factual knowledge • Little if any variance in both the method of assessment and the modality of learning • Assessment is never deployed as a learning tool • The secret handshake • Blame the learner, not the teacher

10. Problems with Traditional Assessment (Cont) • Assessment is infrequent and heavily weighted (high stakes) • Summative over formative assessment • Limited measurement of teaching efficacy: • Did the instructor get the content ‘across’? • Did the students read and ‘remember’ the book?

11. Alternative Forms of Assessment • Standards-, disciplinary- and inquiry-based approaches to teaching and learning require a different approach to assessment • Seek to measure: • Thinking and skill > factual retention • Production and application of knowledge > reproduction of knowledge • What is learned (aligned to SLO) > What is taught

12. Alternative Assessment (Cont) • Standards-based assessments: • Are designed to measure task competence and degrees of proficiency > ranking and discriminating between students • Are done in multiple forms to measure multiple modalities of learning • Are learning tools in support of instruction and are transparent to students • Are on-going and used to support reflection and improvement in teaching practice

13. Alternative Assessment Practicum • In disciplinary groups -- design a draft of both a formative and summative assessment aligned to specific student outcome from a paper in the programme • Specify the SLO • Discuss what dimensions of a task are specifically measured in your standards-based assessments

14. SLO

15. Different Forms of Assessment and Methodologies • Formative Assessments • Aligned to learning outcome and to summative assessment • Should provide appropriate feedback to student in preparation for the summative assessment • Provide appropriate feedback to instructor about the efficacy of the pedagogic methodology • Monitoring for comprehension in lecture • Think-pair-share • Short prompts

16. Other types of formative assessment • Multiple-choice quizzes • Short exercises and prompts • Meeting the challenge of marking • Be specific about nature of feedback and limits of time • Peer evaluation • Rubrics

17. Multiple Choice Questions • Design questions that assess thinking and skill > factual content • Bloom’s taxonomy • Develop ‘justified’ multiple choice questions that demonstrate thinking and process • Develop distracters that demonstrate & identify student (mis)understandings • Questions that task students to substantiate or challenge claims

18. Bloom’s Taxonomy • Bloom’s pyramid and active verbs • Recall (list) • Application (show) • Analysis (compare) • Synthesis (predict) • Evaluation (dispute and/or substantiate)

19. Authentic Assessment • Performance assessments tied to authentic disciplinary-tasks -- students produce knowledge as opposed to reproducing knowledge • Laboratory practicum • Research projects • Assessment constructed as a problem • Evaluating the validity of different interpretations and conclusions and their evidentiary basis • Counterfactual questions and prompts

20. Rubrics - A Scoring Guide that Provides Criteria to Describe Levels of Student Performance • The advantages of using rubrics: • Instructors marks more accurately, reliably and quickly • Requires greater accuracy about the criteria of student performance • Serves as a learning tool and provides better feedback to students and makes the standard of performance explicit • Creates better reliability across sections

21. Challenges to Using Rubrics • Initially time-consuming (but in long-run saves time) • Difficulty to find exact language that distinguishes between levels of performance and establishes criteria • May require revision in initial implementation

22. Rubric Practicum • Identify the dimensions of competence in the task that can be both delineated and demonstrated in the student performance (aligned with SLO) • Holistic versus analytic (and the advantages of the latter within limits) • Weight and scale the dimensions within the task

23. Rubric Practicum Cont. • Establish criteria for competent performance of each specified dimension of the task • Establish a scale of criteria performance • How many clearly identifiable scales? E.g., • Competent and Not Competent • Not Proficient, Proficient, Excellent • Not Proficient, Developing, Proficient, Beyond Proficient, Exemplary • # of scales needs to be justified by clearly delineated performances of each dimension of the task

24. Rubric Practicum Continued • The ideal process • Create draft of rubric • Implement and refine with evaluation of samples of student work • Calibrate with other faculty • Mark!

25. Rubric Exercise • In disciplinary groups -- create a draft rubric for a laboratory practicum with three scales of performance for each dimension • Teacher education faculty -- to do the same but for a pre-service teacher’s design of a laboratory practicum

26. SLO: Laboratory Practicum