Incorporating Authentic Assessment in the Classroom

1. Incorporating Authentic Assessment in the Classroom Narrowing the Gulf Conference March/April 2011

2. St. Petersburg College Presenters • Dr. James Coraggio, Director, Academic Effectiveness and Assessment, St. Petersburg College • Dr. Carol Weideman, Mathematics Professor, Gibbs Campus, St. Petersburg College Narrowing the Gulf Conference

3. Assessment Basics Why do we assess? • To see how well we are doing • To confirm what we already know • To share our progress with others • To see where we can improve and change • In some cases to demonstrate what does not work J Narrowing the Gulf Conference

4. Assessment Basics Why do we assess? Source: http://www.c-pal.net/course/module2/pdf/Week1_Lesson5.pdf J Narrowing the Gulf Conference

5. Purpose of an Assessment J • “Clearly delineate between those that know the content and those that do not.” • To determine whether the student knows the content, not whether the student is a good test-taker. • Likewise, confusing and tricky questions should be avoided to prevent incorrect responses from students who know the material. Narrowing the Gulf Conference

6. Types of Assessments J • Objective assessments • Authentic assessment Narrowing the Gulf Conference

7. Objective Assessments C • Measure several types of learning (also levels) • Wide content, short period of time • Variations for flexibility • Easy to administer, score, and analyze • Scored more reliability and quickly Narrowing the Gulf Conference

8. Types of Objective Tests C • Written-response • Completion (fill-in-the-blank) • Short answer • Selected-response • Alternative response (two options) • Matching • Keyed (like matching) • Multiple choice Narrowing the Gulf Conference

9. Assignment Example I C • Multiple Choice Assessment • Assignment Objectives • Solving equations using addition and multiplication principles • Solving applied problems by translating to equations Narrowing the Gulf Conference

10. Assignment Example I C • Multiple Choice Assessment • Solve for x: 3(x + 8) = 4 (x – 4) a. -8 b. 40 c. 8 d. -40 • Write the sentence as an algebraic equation: The sum of 18 and a number is 5 a. x -18 = 5 b. 5 + 18 = x c. 18 = x + 5 d. 18 + x = 5 Narrowing the Gulf Conference

11. Issues with Objective Assessments C • Limited depth of content • Not able to reveal student misconceptions • Limited ability to test critical thinking skills • Students are ‘Test-wise’ Narrowing the Gulf Conference

12. Test-wise Students C • Are familiar with item formats • Use informed and educated guessing • Avoid common mistakes • Have testing experience • Use time effectively • Apply various strategies to solve different problem types Narrowing the Gulf Conference

13. Authentic Experiences J • Is the course aligned with the expectations for the student in the ‘real-world’? • Authentic Learning • Authentic Assessment Narrowing the Gulf Conference

14. Class Without Authentic Experiences J • Didactic instruction where students are presented with factual information from a text book • Assessment is primarily multiple choice items where students are expected to regurgitate factual information Narrowing the Gulf Conference

15. Class With Authentic Experiences J • Interactive learning environment where students not only learn facts but the relationship between the facts and the application of that information • Authentic assessment where students are able to model the applications of the discipline through simulations, projects, etc. Narrowing the Gulf Conference

16. Authentic Assessments Authentic assessments serve dual purposes of encouraging students to think critically and providing assessment data for measuring improved student learning. These assessment techniques fall into three general categories: criterion-referenced rubrics, student reports (reflection or self-assessments), and student portfolios. J Narrowing the Gulf Conference

17. Authentic Assessments J Authentic assessments include… • Criterion-referenced rubrics. Complex, higher-order objectives can be measured only by having students create a unique product, whether written or oral [in-class essays, speeches, term papers, videos, computer programs, blueprints, or artwork] (Carey, 2000). • Student Reflection.Written reflection is espoused to have several important benefits: it can deepen the quality of critical thinking, increase active involvement in learning, and increase personal ownership of the new learning by the student (Moon, 1999). • Student Portfolios. Collections of students’ work over a course or a program and can be an effective method of demonstrating student progress in the area of critical thinking (Carey, 2000). Narrowing the Gulf Conference

18. Assignment Example II C • Authentic Assessment: Painting a Room • Assignment Objectives • Solving Equations using addition and multiplication principles • Solving applied problems by translating to equations Narrowing the Gulf Conference

19. Assignment Example II C You have decided to paint your family room. Assume that one coat will be sufficient. Paint is currently on sale at Home Depot for $15.99 per gallon and one gallon will cover 400 square feet. Your family room is 10 ft by 14 ft and the ceiling is 9 ft above the floor. There are 3 windows, 30 in by 50 in and 2 doorways, 36 in by 7 ft high. Answer the following questions: How many square feet of walls do you need to cover? How much paint is needed? How much will it cost? Is there any additional information that would be helpful? Narrowing the Gulf Conference

20. Assignment Example II Room is 10 ft wide x 12 ft long x 9 ft high. 2 walls that are 10 ft x 9 ft = 90 ft x 2 = 180 sq ft 2 walls that are 14 ft x 9 ft = 126 x 2 = 252 sq ft Total Square feet = 432 sq. ft BUT: we need to adjust for windows and doors One gallon covers 400 sq ft; we need one gallon of paint Are we painting the ceiling? Do we need supplies (brushes, drop clothes, etc)? Do we need to add sales tax to paint cost? If so, what is rate? Paint cost is $15.99 C Write an equation for the square feet of wall. Let x = square feet of wall y = Total cost for paint Calculate the square footage. B. Using the given information and the equation from Part A, how much paint do you need? C. How much will the paint cost? D. What other information would be helpful? Narrowing the Gulf Conference

21. Assignment Example II 3 windows: 30 in x 50 in x 3= 4500/144 sq in per sq ft = 31.25 sq ft 2 doorways: 3 ft by 7 ft = 21 sq ft x 2 = 42 sq ft Total Square ft = 432 – 31.25 – 42 = 369.5 sq ft One gallon covers 400 sq ft; we need one gallon of paint Are we painting the ceiling? Do we need supplies (brushes, drop clothes, etc)? Do we need to add sales tax to paint cost? If so, what is rate? Paint cost is $15.99 C Write an equation for the square feet of wall. Let x = square feet of wall y = Total cost for paint Calculate the square footage. B. Using the given information and the equation from Part A, how much paint do you need? C. How much will the paint cost? D. What other information would be helpful? Narrowing the Gulf Conference

22. Rubrics C What is a rubric? • Scoring guidelines, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments Narrowing the Gulf Conference

23. Rubrics C SPC currently uses rubrics in such programs as… • College of Education • College of Nursing • Paralegal Studies Program Narrowing the Gulf Conference

24. Rubric Development Process Repeat as Needed J Re-examine the learning objectives to be addressed by the task Identify specific observable attributes your students should demonstrate Describe characteristics at each attribute Write narrative descriptions for each level of continuum Collect samples of student work Score student work and identify samples that exemplify various levels Revise the rubric as needed Narrowing the Gulf Conference

25. Assignment Profile J • Designed to provide consistency and accuracy as well as provide guidelines for the use • Rubric is an evaluation ‘tool’, but for a tool to be effective it must be in the correct situation or ‘job.’ It would be inefficient to use a machete to conduct heart surgery. • Rubric must be aligned to the most appropriate course assignment • The instructor is the assessment instrument not the rubric Narrowing the Gulf Conference

26. Assignment Example III J/C • Rubric: STA2023 Sampling Project • Assignment Objectives • Identify the sampling strategies commonly employed to collect data • Describe potential biases encountered with sampling strategies used in various statistical applications • Suggest strategies to avoid potential biases when using sampling to collect data for a statistical application Narrowing the Gulf Conference

27. Assignment Example III J/C Management at a retail store is concerned about the possibility of drug abuse by people who work there. They decide to check on the extent of the problem by having a random sample of the employees undergo a drug test. The lawyers for the retail store have assured the management that there are no legal issues with the proposed drug testing as long as the individual test results are not identified to a specific employee. Depending the extent of illegal drugs identified in the drug testing, drug counseling may be offered to all employees under the promise of complete confidentiality. You have been hired as the statistician who will design the sampling plan. The budget for the drug testing will cover the cost of 40 drug tests. Narrowing the Gulf Conference

28. Assignment Example III J/C  Management has proposed several different ideas about the best way to obtain the random sample of 40 employees who will be drug tested. There are currently 500 employees at this retail store. There are four classifications of employees: supervisors, full-time sales clerks, part-time sales clerks and maintenance staff. These sampling possibilities are listed below: • Select one of the employee classifications and sample all employees in that classification. •  Choose every fourth person who clocks in for each shift. •  Randomly select 10 employees from each classification. •  Each employee has a three-digit employee number. Randomly select 40 employees.  Narrowing the Gulf Conference

29. Assignment Example III J/C Answer the following questions regarding this scenario: • Define this problem in your own words. • Compare and contrast the four proposed sampling plans. • Select one of the proposed sampling plan that you feel is most appropriate for this situation and defend your choice. • Describe any weaknesses in your selected sampling plan. • Make suggestions on ways to improve/strengthen the sampling plan. You may include information not described in the scenario above. • Reflect on your own thought process after completing the assignment. “What did you learn from this process?” “What would you do differently next time to improve?” Narrowing the Gulf Conference

30. Assessment Rubric for CT Narrowing the Gulf Conference

31. Assessment Rubric for CT Narrowing the Gulf Conference

32. Assessment Rubric for CT Narrowing the Gulf Conference

33. Questions Narrowing the Gulf Conference

34. Assessment Basics C • Alignment of course objectives • Competency, clarity, bias, level of difficulty • Validity and Reliability Narrowing the Gulf Conference

35. Assessment Basics C Alignment • Everything needs to align (objectives through assessment) Narrowing the Gulf Conference

36. Competency C • Items should test for the appropriate or adequate level of knowledge, skill, or ability (KSA) for the students. • Assessing lower division students on graduate level material is an ‘unfair’ expectation. • The competent student should do well on an assessment, items should not be written for only the top students in the class. Narrowing the Gulf Conference

37. Clarity C • Clear, precise item and instruction • Correct grammar, punctuation, spelling • Address one single issue • Avoid extraneous material (teaching) • One correct or clearly best answer • Legible copies of exam Narrowing the Gulf Conference

38. Bias C • Tests should be free from bias… • No stereotyping • No gender bias • No racial bias • No cultural bias • No religious bias • No political bias Narrowing the Gulf Conference

39. Level of Difficulty J • Ideally, test difficulty should be aimed a middle level of difficulty. This can not always be achieved when the subject matter is based on specific expectations (i.e, workforce area). Narrowing the Gulf Conference

40. Trivial and Trick Questions J • Avoid trivia and tricks • Avoid humorous or ludicrous responses • Items should be straight forward, they should cleanly delineate those that know the material from those that do not • Make sure every item has value and that it is contributing to the final score Narrowing the Gulf Conference

41. Assessment Basics J Does one size fit all? • Assessments need to be valid • Assessments need to be reliable Narrowing the Gulf Conference

42. Validity J Does the assessment measure what it is suppose to measure? • “Validation is the process of accumulating evidence that supports the appropriateness of inferences that are made of student responses…” (AERA, APA, & NCME, 1999) Narrowing the Gulf Conference

43. Types of Validity Evidence J • Content Related - the extent to which a student’s responses to a given assessment reflect that student’s knowledge of the content area • Construct Related - the extent to which the responses being evaluated are appropriate indicators of the underlying construct • Criterion Related - the extent to which the results of the assessment correlate with a current or future event • Consequential – the consequences or use of the assessment results Narrowing the Gulf Conference

44. Reliability J Consistency of the assessment scores • Types of reliability… • Interrater Reliability – scores vary from instructor to instructor. • Intrarater Reliability – scores vary from a single instructor from paper to paper • A test can be reliable and not valid, but never valid and not reliable Narrowing the Gulf Conference

45. Reliability Concerns J Reliability Are the score categories well defined? Are the differences between the score categories clear? Would two independent raters arrive at the same score for a given student response based on the scoring rubric? Narrowing the Gulf Conference

46. Improving Scoring Consistency J • Provide grading rubrics or scoring criteria to students prior to assessment • Grade papers anonymously • Use anchor papers to define levels of proficiency for reference • Use multiple scorers • Calculate reliability statistics during training and grading Narrowing the Gulf Conference

47. Assessment Basics C • Multiple Measures • Always to good to implement multiple measures when possible • Ideally direct and indirect measures of competency Narrowing the Gulf Conference

48. Indirect Methods C • “indirect measures …help deepen the interpretation of student learning” (Maki, 2004). • SSI is a good example of an indirect measure. Narrowing the Gulf Conference

49. Incorporating Authentic Assessment in the Classroom Narrowing the Gulf Conference March/April 2011