Assessing for Learning

1. Assessing for Learning Presented by Peggy L. Maki June 7, 2004 Seattle Pacific University PeggyMaki@aol.com Material from Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing and AAHE.

2. How Do You Learn? • _______________________________ • ________________________________ • ________________________________ • ________________________________

3. Topics: • Research on Learning That Informs the Relationship among Teaching, Learning, and Assessment • Curricular and Co-curricular Coherence (Maps and Inventories)

4. Development of Learning Outcome Statements • Alignment of Assessment Methods with Learning Outcome Statements and Collective Educational Practices

5. Terms: • Learning Outcome Statements—sentences that describe what we expect our students to demonstrate, represent or produce as a result of how and what they learn. • Coherence—degree to which the curriculum and co-curriculum intentionally offer multiple and varied opportunities for students to learn what the institution and its programs and services assert they teach

6. Maps and Inventories—strategies that enable us to ascertain how well we intentionally develop what we assert we expect students to learn • Alignment—degree to which learning outcome statements match how and what we teach; degree to which assessment methods methods match our collective educational practices

7. Research on Learning That Informs the Relationship among Teaching, Learning, and Assessment • Learning is a complex process of interpretation--not a linear process • Learners create meaning as opposed to receive meaning 1.

8. Knowledge is socially constructed (importance of peer-to-peer interaction) • People learn differently—prefer certain ways of learning (learning inventories, such as Kolb or Vark)

9. Deep learning occurs over time—transference • Meta-cognitive processes are a significant means of reinforcing learning (thinking about one’s thinking)

10. Learning involves creating relationships between short-term and long-term memory • Transfer of new knowledge into different contexts is important to deepen understanding

11. Practice in various contexts creates expertise • Surface Learning • Deep Learning

12. Curricular and Co-Curricular Coherence • What do you do in your classes or in your programs or services to promote the kinds of learning or development that the institution seeks? • What do you expect your students to know and be able to do by the end of their education at your institution?

13. What do the curricula and other educational experiences “add up to?” • Which students benefit from specific teaching strategies or educational experiences? • What educational processes are responsible for the intended student outcomes the institution seeks?

14. How can you help students make connections between classroom learning and experiences outside of the classroom? • What pedagogies/educational experiences develop knowledge, abilities, habits of mind, ways of knowing/problem solving?

15. How are curricula, pedagogy, and educational experiences designed to develop knowledge, abilities, habits of mind, ways of knowing? • How do you intentionally build upon what each of you teaches or fosters to achieve programmatic and institutional objectives?

16. What methods of assessment capture desired student learning--methods that align with pedagogy, content, and curricular design?

17. When a Student Becomes aBiologist, Psychologist,Engineer…..

18. Maps and Inventories • Help us determine coherence among our educational practices that enable us to design appropriate assessment methods • Provide a visual representation of students’ journey

19. Help students make meaning of the journey • Help students develop their own learning map

20. Developing Learning Outcome Statements

21. What Is an Outcome Statement? A sentence that describes what students should demonstrate, represent, or produce based on how and what they learn. Emerges from what we value and how we teach or students learn; that is, it emerges from our educational practices and is developed through consensus.

22. Relies on active verbs, such as create, compose, calculate, develop, build, evaluate, translate, etc., that target what we expect students to be able to demonstrate • Is written for a course, program, or institution

23. Is aligned with curriculum and co-curriculum --varied and multiple ways to learn that desired outcome • Is based on intentionality (established through dialogue)

24. Helps students understand expectations and holds them accountable for their learning • Can be assessed quantitatively or qualitatively

25. Example from ACRL ONE OUTCOME: Student examines and compares information from various sources in order to evaluate reliability, validity,accuracy, timeliness, and point of view or bias.

26. Write one outcome statement that captures what a graduate from your institution or a program should be able to demonstrate, represent, or produce:

27. How well does your outcome statement meet characteristics of a good statement? • Describes learning desired within a context • Relies on active verbs

28. Emerges from your collective intentions • Can be mapped to curricular and co-curricular practices (multiple and varied opportunities to learn over time) • Can be assessed quantitatively or qualitatively during students’ undergraduate and graduate careers

29. “The tasks to which students are asked to respond on an assessment are not arbitrary. They must be carefully designed to provide evidence that is linked to the cognitive model of learning and to support the kinds of inferences and decisions that will be based on the assessment results.” National Research Council. Knowing what students know: The science and design of educational assessment . Washington, D.C.: National Academy Press, 2001, p. 47.

30. Assumptions Underlying Teaching Actual Practices Assumptions Underlying Assessment Tasks Actual Tasks

31. Development of Assessment Methods • Align methods with learning outcome statements and students’ educational experiences (verified through maps or inventories)

32. When Will You Seek Evidence? • Formative—along the way? For example, to ascertain progress or development • Summative—at the end? For example, to ascertain mastery level of achievement

33. What Tasks Elicit Learning You Desire? • Tasks that require students to select among possible answers (multiple choice test)? • Tasks that require students to construct answers (students’ problem-solving and thinking abilities)?

34. Some Methods That Provide Direct Evidence • Student work samples • Collections of student work (e.g. Portfolios) • Capstone projects

35. Course-embedded assessment (derive examples; develop agreed upon prompt and ask students to respond to it in class or at a designated time) • Observations of student behavior • Internal juried review of student projects

36. External juried review of student projects • Externally reviewed internship • Performance on a case study/problem • Performance on case study accompanied with students’ analysis

37. Performance on national licensure examinations • Locally developed tests • Standardized tests • Pre-and post-tests

38. Response to critical incident • Essay tests blind scored across units • Mapping • Learning Logs or Journals • Writing, Speaking, Visual Representation

39. Methods That Do Not Provide Direct Evidence but May Be Combined with Other Methods-Indirect Methods • Faculty publications (unless students are involved) • Courses selected or elected by students • Faculty/Student ratios

40. Percentage of students who study abroad • Enrollment trends • Percentage of students who graduate within five-six years • Diversity of student body

41. Focus group (representative of the population) • Interviews (representative of the population) • Surveys

42. Other sources of information that contribute to your inference making: NSSE results, grades, participation rates or persistence in support services, course-taking patterns, majors

43. “What and how students learn depends to a major extent on how they think they will be assessed.” John Biggs, Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141.

44. Works Cited Biggs, J. (1999). Teaching for Quality Learning at University: What The Student Does. Society for Research into Higher Education & Open University Press, 1999, p. 141. Maki, P. (forthcoming, 2004., May). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus Publishing, LLC, and the American Association for Higher Education. National Research Council. 2001. Knowing What Students Know: The Science and Design of Educational Assessment. Washington, D.C.: National Academy Press