What Works - Course and Program Assessment in STEM

1. What Works - Course and Program Assessment in STEM PKAL Network – 7 October 2013 Dr. Dianne Raubenheimer, Director of Research, Planning and Assessment, Meredith College raubenhe@meredith.edu

2. Presentation Outline • By the end of this session, participants will be able to: • Identify some purposes of assessment • Explain selected assessment terminology • Articulate ways to align outcomes, teaching, learning and assessment • Consider strategies for linking course and program assessment

3. What is Assessment ? • Ice Breaker: Write down some thoughts • A show of hands: • Gathering information • Measuring goals and outcomes • Using diverse measures and sources • Analyzing and interpreting information • Reporting results • Engaging stakeholders • Making improvements

4. A Definition Assessment is the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development (Palomba & Banta,1999). 

5. Direct and Indirect Methods • Task: • Individually - Write down different types of assessment tasks you use in your courses. Use one small post-it for each type of assessment. • As a group - Sort the post-its into two columns – direct and indirect assessment methods • Direct assessments provide for direct examination or observation of student knowledge of skills against measureable learning outcomes. • Indirect assessments of student learning ascertain the perceived extent of value of learning experiences. They assess opinions or thoughts about student knowledge or skills.

6. Purposes of Assessment • Task: • Looking at the different assessment strategies. In your groups, discuss some thoughts about the purposes of assessment • You may be able to cluster some of the methods with particular purposes

7. Purposes of Assessment Internal / Intrinsic • Assessment of student learning (knowledge, skills and attitudes) • Improvement of student learning • Improvement of teaching • Program development, monitoring and improvement • SoTL, SoA, SoP • Accreditation • Public accountability / Performance scorecard External / Extrinsic

8. Planning & delivering instruction Focus One:Teaching, Learning and Assessment Assessing instruction and learning Student learning Outcomes Reciprocal relationship

9. Aligningoutcomes, Teaching and Assessment • An example from NC State : • Using an observation rubric, observed 15 instructors in different disciplines teaching • All indicated they were using innovative practices with technology • Instructor noted the learning outcomes for the lesson being taught • Observer used the SOLO taxonomy to score the SOLO level of instruction • Observer used the SOLO taxonomy to score the SOLO level at students were engaged in the class • Then gathered student work from the faculty that tested the outcome • Examined the assessment task and student work using the SOLO taxonomy (Raubenheimer, et al, 2007)

10. SOLO Taxonomy

11. Aligning outcomes, Teaching and Assessment

12. Aligning outcomes, Teaching and Assessment

13. Aligning outcomes, Teaching and Assessment

14. Task • In your groups: • Discuss ways in which you align teaching and assessment of student learning outcomes? • Select 3 individuals to each act as chair, scribe and reporter for the group

15. Focus two:Linking Course Assessment to Program Assessment http://www.arapahoe.edu/about-acc/presidents-office/assessment

16. Task • On your own: • Use the “Program Level-Assessment of Student Learning: Self-assessment worksheet” to assess your program at your institution • In your groups: • Discuss ways in which you align teaching and assessment of student learning outcomes in your courses (columns 1-3)? • Select 3 individuals to each act as chair, scribe and reporter for the group

17. Linking Course Assessment to Program Assessment • Ensure the program outcomes reflect what you want students to have learned/attained/ achieved by the time they graduate • Generating appropriate program outcomes • Develop program outcomes - course assessment matrices • Selecting appropriate assessment tasks/methods • Develop a system for keeping track of program data • Gather, review data and use data for ongoing, systematic program review and improvement

18. Linking Course Assessment to Program Assessment • Develop an implementation matrix associated with program outcomes • H= high, M=medium; Low = L

19. Linking Course Assessment to Program Assessment • Develop an implementation and assessment matrix associated with program outcomes • I = implement; A = assess

20. program Outcome – Course ASSESSMENT Matrix • Articulate sub-outcomes on the matrix

21. Multiple Measures & Performance Criteria • Determine assessment methods and expected performance criteria(level of performance )

22. Program Outcome – Course Assessment Matrix • Develop a system for recording and tracking results

23. Conclusion Best practices indicate that assessment is systematic and systemic. It occurs at multiple levels in an institution, vertically and horizontally, involving processes that are connected and integrated

24. Sources • Assessment: Arapahoe Community College http://www.arapahoe.edu/about-acc/presidents-office/assessment • Biggs, J. (2004). Teaching for quality learning at university. Berkshire, UK: Open University Press. • Heer, R. (2009). A Model of Learning Objectives. Center for Excellence in Learning and Teaching, Iowa State University. • Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus. • Mertler, C.A. (2003). Classroom assessment: A practical guide for educators. Los Angeles: Pyrczak. • Ozturk, H. & Raubenheimer, C.D. (2011). PAT: An online program A\assessment tool. Paper presented at 2011 American Society for Engineering Education International Conference, Vancouver, Canada. • Palomba, C.A & Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in higher education. San Francisco, CA: Jossey-Bass. • Raubenheimer, C. D., Spurlin, J., Martin, S., & Mehlenbacher, B. Faculty in Technology-Rich Contexts: Connecting Teaching, Learning, and Assessment in the Classroom. 2007 UNC TLT Conference Proceedings. • Rogers, G. (2004). Self-Assessment: Quality Assurance of Program Level-Assessment of Student Learning. Rose Hulman Institute of Technology. • Spurlin, J.E., Rajala, S.A., & Lavelle, J.P. (2008). Designing better engineering education through assessment: A practical resource for faculty and department chairs on using assessment and ABET criteria to improve student learning. Sterling, VA, Stylus Publishing.

25. Q & A raubenhe@meredith.edu