Norman Jones Director of General Education & Curricular Integration Utah State University

1. Undergraduate Scholarship and other High Impact Practices: Assessing the Outcomes, Not the Content. Norman Jones Director of General Education & Curricular Integration Utah State University norm.jones@usu.edu

2. Assessment of teaching in 2030 will probably not look like it does now. The delivery of “content” will not be our job. It will be to prepare students for the human jobs that can’t be done by machines, jobs requiring personal interrelations and critical thinking.

3. The Future of Higher Ed “Degrees and other postsecondary credentials can’t simply be defined by the amount of time a student spends in classrooms or labs. Rather, degrees must represent well-defined and transparent learning outcomes. In short, students should get credit for what they know and what they can do. And all learning should count ― no matter how, when or where it was obtained.”Jamie Merisotis, President, Lumina Foundation

4. The traditional assessment method is to measure progress incrementally, using quizzes and tests, measuring individual performance against the average of group performance, over a fixed period of time, calibrated in multiple units of seat time.

5. “Current assessment practice, for the most part, rests on faculty-established goals, developed independently at each institution, for what graduates should know and be able to do.” Whether or not graduates attain these goals is then investigated on average by using various methods to examine the performance of representative samples of students.” Peter T. Ewell, “The Lumina Degree Qualifications Profile (DQP): Implications for Assessment.” http://www.learningoutcomesassessment.org/documents/EwellDQPop1.pdf

6. These methods do not ask about developing individual proficiencies, or mastery, or the possibility of learning that occurs outside the classroom.

7. We Must Teach Toward Proficiency: “a set of demonstrations of knowledge, understanding and skill that satisfy the levels of mastery sufficient to justify the award of an academic degree.”Degree Qualification Profile, 2.0 http://www.luminafoundation.org/publications/DQP/DQP2.0-draft.pdf

8. In order to assess for proficiency we have to rethink what we are asking students to learn. Is it content, or the application of content?

9. “Assignments or examination questions designed to determine proficiency in particular DQP competencies, consequently, must require students to generate a product of some kind—a research paper, an oral presentation, a dance performance, a translation of a text from one language to another, an engineering design.” “Merely identifying a “correct” answer from a set of posed alternatives is not a production task. Because the assessments associated with DQP competencies require students to directly demonstrate mastery, the assessment really is the competency from an operational standpoint.” Ewell, “DQP…Asssessment” 12.

10. WE MUST Decrease emphasis on content. They have libraries in their pockets. Create opportunities for practice, linking content to a need to know. Emphasize acquisition, deployment, and communication of knowledge through experience.

11. High Impact Practices develop their proficiencies, more than their book learning.

12. Pedagogical High Impact Practices that Reinforce Proficiencies First-Year Seminars and Experiences Undergraduate Research Writing-Intensive Courses Capstone Courses and Projects

13. Institutionally Organized Experiences that Reinforce Proficiencies: Common Intellectual Experiences Learning Communities Diversity/Global Learning Internships Service Learning

14. HIP Assessments There are “Process” assessments – based on course design There are “Outcomes” assessment – based on demonstrated proficiency

15. In “Process” assessments we identify the target outcomes and design ways for students to prepare and practice for them. In "Outcomes” assessments, students produce evidence of proficiency.

16. At USU, we built Gen Ed template rubrics that identify the proficiencies we expect a course to deliver WITHOUT PRECISELY DEFINING CONENT.

17. Breadth Life Science Rubric

19. These template rubrics require that achieving proficiency in the outcomes be intentionally designed into the course. Using active verbs, we demand demonstration that students know, understand and are able to do generic things.

20. The template rubrics establish the hierarchies of proficiency in the subject, and give form to the grading rubrics used to assess the demonstrations of competence.

21. HOW DO YOU ASSESS AN EXPERIENCE THAT HAS A PROFICIENCY OUTCOME?

22. We have to think about the “take away” from the experience. If we focus on outcomes, there is no precise question like “Who is buried in Grant’s tomb?” There are no points to be lost for incorrect answers.

23. We want to understand the student’s engagement with practice. Has he or she demonstrated proficiencies in the discipline through use of the tools of the discipline?

24. We have to think about the demonstration of proficiencies within the outcomes, both in courses and in degrees. They build together, reaching apogee in the capstone project.

25. Building Pathways to Increasing Proficiency As we refine our templates and enact them in course rubrics, we can begin to map the way student practice builds proficiency over the curriculum. That means we can envision appropriate pathways to degrees.

27. USU, HIST 4990: Senior Capstone Common Rubric Used by all capstones in the History degree to assess proficiencies.

31. Through assessing the 7 degree outcomes, we are asking if students have sufficient mastery to be granted a degree. The senior thesis is the artifact used as a demonstration.

32. We are “preparing students to tackle nonstandard, unscripted problems and questions. Unscripted problems, by definition, are those where ‘right answers’ are not known and where the nature of the problem itself is likely uncertain at best, and often actively contested.” Carol Geary Schneider in Ewell, “DQP…Asssessment,” 25.

33. Our only real assessment audience for improvement is ourselves. It reassures us that we can honestly say we are educating our students. Is a fancy metric needed? No. Proof of proficiency is.

34. Process assessment + Proof of outcomes performance = evidence of proficiency. An Intentional curriculum, taught by intentional faculty to intentional learners. Through it, every students’ proficiency is established.