Improving your classroom assessments

1. Improving your classroom assessments Dr. Patty LeBlanc Southeastern University

2. Agenda • Introductions • Objectives of the Seminar • Purpose of Educational Assessments • Educator and Educational Standards • Bloom • Test Blueprints • Item Writing Commandments

3. Item Critique Activity • Item Analysis • Example from grad students • Authentic Assessment • Rubrics • Summary • Action Plan • Questions?

4. Objective of the Seminar • To help teachers design valid assessments of instruction that inform instruction

5. What is the purpose of testing?

6. Participant Ideas

7. To encourage students and improve teaching • To assess the extent to which students have met the intended learning outcomes, and to look for and evaluate unintended outcomes • To recognize achievements and diagnose learning difficulties so that students learn to build on their strengths and overcome or cope with their weaknesses • To refine instruction and learning experiences in order to improve both individual and class learning • To help students set meaningful and realistic learning goals and assume responsibility for their own learning.

8. To communicate meaningful information to students, parents, and school authorities about student learning • To give realistic and helpful feedback about achievement, capabilities, behavior, attitudes, and dispositions • To put teachers, students, and parents/guardians in touch with one another about progress over time • To provide guidance for educational and vocational choices • To report learning achievement to school authorities such as school boards and governments

9. Florida Educator Accomplished Practices in Assessment The effective educator consistently: • Analyzes and applies data from multiple assessments and measures to diagnose students’ learning needs, informs instruction based on those needs, and drives the learning process; • Designs and aligns formative and summative assessments that match learning objectives and lead to mastery; • Uses a variety of assessment tools to monitor student progress, achievement and learning gains; • Modifies assessments and testing conditions to accommodate learning styles and varying levels of knowledge; • Shares the importance and outcomes of student assessment data with the student and the student’s parent/caregiver(s); and • Applies technology to organize and integrate assessment information.

10. Educational Standards • Florida Sunshine State Standards • Common Core Standards • Other

11. “Backward” Design • The standards dictate the lesson objectives… • The objectives dictate the instruction • The assessment must align with the objectives, standards, and instruction in order to be valid.

12. Blessed Assurance… • A fantastic way to ensure that your test items align with your objectives: A Test Blueprint!

13. Content Validity • Think matrix of instruction and assessment:

14. Bloom’s Taxonomy of the Cognitive Domain

15. Another blueprint…

16. Sung to the tune of the Farmer in the Dell • The teacher follows the standards… • The teacher creates the objectives… • The teacher writes the test… • The teacher teaches the lesson… • The students learn a lot… • The parents beam with pride…

17. Comunicación y Gerencia The Commandments of Test Item Writing Item Writing Commandments By W.James Popham • Click to add Text

18. General Item Writing Items

19. Thou shalt not • provide opaque directions to students regarding how to respond to your assessment instruments.

20. Thou shalt not • employ ambiguous or wordy statements in your assessment items. The term hypothesis, as used in research, as defined as: a. A conception or proposition formed by speculation or deduction or by abstraction and generalization from facts, explaining or relating an observed set of facts, given probability by experimental evidence or by factual or conceptual analysis but not conclusively established or accepted. b. A statement of an order or relation of phenomena that so far as is known is invariable under the given conditions, formulated on the basis of conclusive evidence or tests and universally accepted, that has been tested and proven to conform to facts. *c. A proposition tentatively assumed in order to draw out its logical or empirical consequences and so test its accord with facts that are known or may be determined, of such a nature as to be either proved or disproved by comparison with observed or know facts.

21. Better… The term hypothesis, as used in research, is defined as: a. An assertion explaining an observed set of facts that has not been conclusively established. b. A universally accepted assertion explaining an observed set of facts. *c. A tentative assertion that is either proved or disproved by comparison with an observed set of facts.

22. Thou shalt not • Provide unintentional clues as to the correct response. What meter is used to measure resistance? a. Voltmeterb. Ohmmeterc. Ammeterd. Oscilloscope

23. Thou shalt not • employ complex or give-away syntax in your assessment items. In item response theory, the one-parameter model assumes that each item ……. a. discriminates equally well.b. students perform equally well.c. students score the same across items.d. guessing affects all items the same.

24. Thou shalt not • use vocabulary that is more advanced than required. As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation? • a decrease in the labor force participation rate of women • a dispersing effect on population concentration • a downward trend in the youth dependency ratio • a broader base in the population pyramid • an increased infant mortality rate

25. Thou shalt • Use plausible distractors Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Warhol b. Flintstone *c. Michelangelo d. Santa Claus

26. Thou Shalt • Avoid using "always" and "never" in the stem or the distractors.

27. Thou shalt • Rarely us negative statements… Which of the following is NOT a symptom of osteoporosis? • decreased bone density • frequent bone fractures • raised body temperature • lower back pain

28. Binary Choice Items [True/False] • One of several types of Selected Response Items

29. Thou shalt not • Phrase items so that a superficial analysis by the student suggests a wrong answer. Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Warhol b. Flintstone *c. Michelangelo d. Santa Claus

30. Thou shalt NEVER use double negatives. True or False: There is no advantage to not using specific determiners in true-false items.

31. Thou shalt • Have an approximately equal number of items representing the two categories being tested. • Keep item length similar for both categories being tested.

32. Thou Shalt • Stick with absolute truths or falsehoods. True or False: Nearsightedness is hereditary in origin.

33. Thou Shalt Not • Use extreme qualifiers. • All sessions of Congress are called by the President. (F) • The Supreme Court frequently rules on the constitutionality of law. (T) • An objective test is generally easier to score than an essay test. (T)

34. Multiple Choice Items

35. Thou shalt • Write the stem so that it consists of a self-contained question or problem. California: a. Contains the tallest mountain in the United States b. Has an eagle on its state flag. c. Is the second largest state in terms of area. *d. Was the location of the Gold Rush of 1849.

36. Better… What is the main reason so many people moved to California in 1849? a. California land was fertile, plentiful, and inexpensive. *b. Gold was discovered in central California c. The east was preparing for a civil war. d. They wanted to establish religious settlements.

37. Thou shalt • Include only one concept in each statement and only one correct answer. The United States should adopt a foreign policy based on: a. A strong army and control of the North American continent. b. Achieving the best interest of all nations. c. Isolation from international affairs. *d. Naval supremacy and undisputed control of the world’s sea lanes.

38. Thou shalt • NEVER use ‘all of the above’ alternatives, only occasionally use ‘none of the above’ alternatives [to increase item difficulty].

39. Well-designed MC items CAN measure HOTS A common goal of the Salt March in India, the Boxer Rebellion in China, and the Zulu resistance in southern Africa was to: A. overthrow totalitarian leaders B. force upper classes to carry out land reform programs C. remove foreign powers D. establish Communist parties to lead the countries

40. Numbers of Items is Important! Chance of Scoring 70% or Higher by Blind Guessing • Number of 4-Alternative • MC Items • 2 • 5 • 10 • 15 • 20 • 25 • 1 out of 16 • 1 out of 64 • 1 out of 285 • 1 out of 8,670 • 1 out of 33,885 • 1 out of 942,651

41. Pop Quiz • Can you identify the problem with this item? Due to budget cutbacks, the university library now subscribes to fewer than _?_ periodicals. • 25,000 • 20,000 • 15,000 • 10,000

42. Matching Items Most effective when you need to measure the learner’s ability to identify the relationship or association between similar items. They work best when the course content has many parallel concepts, for example: • Terms and Definitions • Objects or Pictures and Labels • Symbols and Proper Names • Causes and Effects • Scenarios and Responses • Principles and Scenarios to which they apply

43. Thou shalt • Use homogeneous lists. Hitler A. Year in which WWII began Mulroney B. A Canadian Prime Minister Tank C. A German dictator during WWII 1939 D. An armored vehicle used originally to break the trench war stalement in WWI

44. Thou shalt • Randomly assign correct answers to alternative positions.

45. Thou shalt not • let the length of alternatives supply unintended clues. Which of the following is the best indication of high morale in a supervisor’s unit? a. The employees are rarely required to work overtime. *b. The employees are willing to give first priority to attaining group objectives, subordinating any personal desires they may have. c. The supervisor enjoys staying late to plan the next day. d. The unit gives expensive birthday presents to each other.

46. Thou shalt • Use relatively brief lists, placing the shorter words or phrases at the right.

47. Thou shalt • Employ more responses than premises.

48. Thou Shalt • Provide good directions! Directions: Match the following. ____1. City dwellers A. Wild animals ____2. Hunter-gatherers B. Farm ____3. Pastoral nomads C. Apartments D. Graze animals

49. Better… Directions: On the line to the left of each definition, write the letter of the term in the right hand column that is defined. Use each term only once. There are more responses than items. Each correct response is worth 2 points. _1. Live in areas of high population density. A. Pastoral nomads __2. Move from place to place in search of wild animals. B. Ranchers __3. Move from one place to another with grazing animals. C. Hunter-gatherers __4. Till land for cash crops. D. City dwellers E. Farmers

50. Thou shalt • Describe the basis for matching and the number of times responses may be used in the instructions, as well as the point values for each item. • Place all premises and responses for an item on a single page. • Tip: Be sure to decide ahead of time whether you will give partial credit.