Assessing Higher Order Thinking Skills

1. Assessing Higher Order Thinking Skills Vickie Mohnacky April 19, 2011

2. Assessing Higher Order Thinking Skills Today’s Objectives • To identify personal, social and academic student outcomes in the PreK-Grade 12 Programming Standards • To align student outcomes with different types of assessments

3. What do you want your students to learn as a result of gifted education? • http://www.twiddla.com/528716 What do I value the most? (As a teacher – as a human being)

4. What do we value most? • Following directions? • Timeliness? • Speed? • Accuracy? • Knowledge of content?

6. What do we value most? Complex thinking Originality Research Understanding systems Seeing connections Self Knowledge Perspective Abstract Concepts Leadership Civic Responsibility Empathy

7. How will we use the information? Teachers’ responses:

8. How will we use the information? Grade? Class rank? Determine entrance to program? Embarrass the student? Punishment? Predict success? Determine coursework? Measure growth/progress in learning? Plan instruction?

9. Assessment Uses

10. Formative Assessment An assessment FOR learning. Occurs while learning is forming. • Not high stakes/Not for accountability. • Used to adjust instruction/ improve student learning. • Not for report card grades. • Examples: • Teacher informal questioning • Warm-ups • Homework • K-W-L Chart

11. Benchmark/Interim Assessment An assessment OF learning – Interim points • Not high stakes. • Examples • End of chapter tests • Acuity http://wvde.state.wv.us/oaa/acuity.php • DIBELS • Benchmark testing

12. Benchmark/Interim Assessment An assessment OF learning – Interim points • Not high stakes. • Used to adjust overall curriculum, programming • More Examples: • Attitude scales • Interest Inventories • Established rubrics for class product and performances • Critical thinking tests/checklists (Cornell, Watson-Glaser, Test of Critical Thinking) • AP/IB course taking and performance

13. Summative Assessment An assessment OF learning; Sums up; Occurs at set points • High stakes. • Reporting and accountability • Used to evaluate the overall effects of programming • Examples • State Standardized Test (WESTEST 2) • NAEP (National Assessment of Educational Perf.) • PISA (Program of International Student Assessment • Act Explore/ACT/SAT

15. What are the ultimate student outcomes? Brian Housand

16. What is the ultimate outcome? Solve

17. Houston, we have a problem • http://www.youtube.com/watch?v=pbBS8sHrDgA&NR=1

18. The point is that we have to be clear in our objectives before we can differentiate instruction and properly assess our students’ attainment of those objectives.” Rick Wormeli 2006 Fair Isn’t Always Equal

19. Look at Your Data • What does it measure? • Will it give you the information you need?

20. 2010 Gifted Education Programming Standards Standards Focus on Student Outcomes Students identified with gifts and talents demonstrate important learning progress as a result of programming and services (Assessment Criterion 2.5)

21. Learning and Development Student Outcomes 1.1 – 1.3 Self-Understanding • Demonstrate self-knowledge of interests, strengths, identities and needs. • Possess developmentally appropriate understanding of how they learn and grow and recognize the influences of their beliefs, traditions and values on their learning and behavior • Demonstrate an understanding and respect for similarities and differences between themselves and their peer group and others in the general population.

22. Learning and Development Student Outcomes • Access resources from community to support cognitive and affective needs. • Receive support from family and communities who understand similarities and differences with respect to the development and characteristics of advanced and typical learners. 1.4 – 1.5 Awareness of Needs

23. Learning and Development Student Outcomes • Benefit from meaningful and challenging learning activities. • Recognize preferred approaches to learning and expand their repertoire. • Identify future career goals that match their talents and abilities and resources needed to meet goals. 1.6 – 1.8 Cognitive and Affective Growth

24. Curriculum Planning and Instruction Student Outcomes • Demonstrate advanced and complex learning as a result of using multiple, appropriate and ongoing assessments • Demonstrate important learning progress as a result of programming and services. 2.4 -2.5 Assessment

25. Curriculum Planning and Instruction Student Outcomes • Demonstrate growth commensurate with aptitude. 3.1 Curriculum Planning

26. Curriculum Planning and Instruction Student Outcomes • Become more competent in multiple talent areas and across dimensions of learning. • Develop their abilities in their domain of talent and/or area of interest. 3.2 – 3.3 Talent Development

27. Curriculum Planning and Instruction Student Outcomes • Become independent investigators. • Develop knowledge and skills for living and being productive in a multicultural, diverse and global society. 3.4 - 3.5 Instructional Strategies

28. Curriculum Planning and Instruction Student Outcomes • Benefit from programming that provides a variety of high quality resources and materials. 3.6 Resources

29. Learning Environments Student Outcomes • Demonstrate growth in personal competence and dispositions for academic and creative productivity (self-awareness, self-advocacy, self-efficacy, confidence, motivation, resilience, independence, curiosity, risk taking). 4.1 Personal Competence

30. Learning Environments Student Outcomes • Develop social competence manifested in positive peer relationships and social interactions. • Demonstrate personal and social responsibility and leadership skills. 4.2 – 4.3 Social Competence - Leadership

31. Learning Environments Student Outcomes • Value their own and other’s language, heritage, and circumstance. • Possess skills in communicating, teaming and collaborating with diverse individuals and across diverse groups. • Use positive strategies to address social issues, including discrimination and stereotyping. 4.4 Cultural Competence

32. Learning Environments Student Outcomes • Develop competence in interpersonal and technical communication skills. • Demonstrate advanced oral and written skills, balanced biliteracy or multiliteracy, and creative expression. • Display fluency and technologies that support effective communication. 4.5 Communication Competence

33. Specify clearly and exactly what it is you want to assess • Design questions and tasks that require students to demonstrate this skill or knowledge

34. Understanding By Design (UBD) 1. Identify desired results (what the student will know) 2. Determine acceptable evidence (how will the student demonstrate knowledge) 3. Plan instruction; Implement the design

35. “I began by seeing assessment as judging performance, then as informing teaching, and finally as informing learning.” • Carol Ann Tomlinson

36. Types of Assessment • Standard - • Selected Response (Multiple Choice) • Constructed Response (Short Answer) • Fill in the Blank • True/False • Performance-based or Product- • Rubrics • Rating scales • Checklists

37. High School Entrance Exam – 1928 http://wvde.state.wv.us/1928/

38. Standardized – Large Scale • Mass produced • Mass graded

39. Balanced Assessment System • Content – Knowledge • Reasoning • Performance Skill • Product

40. Standard/Objective: Type: Knowledge  Reasoning  Performance Skill  Product Learning Targets What are the knowledge, reasoning, skill or product targets underpinning the standard/objective? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets What knowledge would students need to master this skill? What reasoning proficiencies (if any) would students need to master this skill? What performance skills (if any) would students need to practice to master this skill? What products (if any) would students need to practice creating to master this skill?

41. Example from real life: Drive with skill. Type:  Knowledge Reasoning Skill Product Learning Targets What are the knowledge, reasoning, skill or product targets underpinning the standard/objective? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets • Know the law • Understand informal rules of the road • Understand what different parts of the car do • Read signs and understand what they mean • Understand what “creating a danger” means • Understand what “creating a hazard” means • Other? • Analyze road conditions, vehicle performance, and other driver’s actions • Compare/contrast this information with knowledge and past experience • Synthesize information and evaluate options to make decisions on what to do next • Evaluate “Am I safe?” and synthesize information to take action if needed. • Other? • Driving actions such as: steering, shifting, parallel parking, looking, signaling, backing up, braking, accelerating, etc. • Fluidity/automaticity in performance driving actions. • Other? None Since the ultimate type of target is a performance skill, there are no embedded product targets

42. Standard/Objective: SS.O.06.05.04: Interpret quotes of famous Americans from various periods of history Type:  Knowledge  Reasoning  Performance Skill  Product Learning Targets What are the knowledge, reasoning, skill or product targets underpinning the standard/objective? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets What knowledge would students need to master this standard? What reasoning proficiencies would students need to master this standard? None? None?

43. Standard/Objective: SC.O.7.1.11: Construct . . . charts, graphs and tables for various purposes . . . Type:  Knowledge  Reasoning  Performance Skill  Product Learning Targets What are the knowledge, reasoning, skill or product targets underpinning the standard or benchmark? Knowledge Targets Reasoning Targets Performance Skill Targets Product Targets What knowledge would students need to master this standard? What reasoning proficiencies would students need to master this standard? What performance skills would students need to practice to master this standard? What products would students need to practice creating to master this standard?

44. Informal Questioning • ConceptQuestions - • Why do we care about this? Why are we studying this? • Open ended • Broad, over-arching, outcome oriented questions • Will motivate students and target higher-order thinking . What is the big concept you are trying to uncover? Do the standards have any “big concept” words? • ContentQuestions– • What are the facts? What are the procedures? • Closed ended. • Will help strengthen and develop students’ understanding of the larger questions.

45. Informal Questioning Concepts: Conflict – How can conflict be resolved? (How could this particular conflict have been resolved?) Change - How can we cope with change? Friendship – What does it mean to be a friend? Freedom (of speech) – Is pure freedom of speech desirable in today’s world? Why or why not? http://www.esldiscussions.com/index.html Questions Resource

46. Teachers who do not specifically plan classroom discussion questions ahead of time to tap particular higher-order thinking skills, but rather ask extemporaneous questions “on their feet,” are likely to ask recall questions. Susan M. Brookhart, 2010

47. Checklists Checklist of items - the least complex form of assessment. Yes or No – Is it present or not? There is no value attached to the performance. All elements weighted the same. To determine a score, the total number of checked items or the percentage of total possible. No quality is attached unless specified in the item. Example, “three paragraphs required” or “neatly” or “500 words.” Meaning is then attached to the score. For example, what is the minimum score that would be considered proficient?

48. Checklists Example of basic speech skills:

49. Rating Scales Graphic rating scale on a continuum – example : Never Seldom Sometimes Usually Always • Turns in lessons on time • Uses correct capitalization • Completes projects

50. Rating Scales Numerical rating scale – example : 1 to 10 with 10 being the highest