Assessment of Core Skills/ General Education Outcomes

1. Assessment of Core Skills/General Education Outcomes Angelina Hill, PhD Associate Director, Office of Academic Assessment

2. Levels of Assessment

3. Activity • Identify the level of assessment that applies to your project • Roughly state the core skills that you aim to assess • Be as specific as you think is necessary

4. Activity • Program assessment: • What courses address these outcomes, and at what level (introduce, reinforce, enhance/master)? • Course assessment: • Where in the course is the outcome addressed (e.g., activities, assignments, tests), and at what level?

5. Assessment Instruments • Design assessment instruments around the type of learning emphasized in the course. • What do you want your students to learn?

6. Direct Methods of Assessment: • Course-embedded assessment (e.g., homework assignment; essays, locally developed tests) • Grading with criteria or rubrics • Comprehensive exams • Senior thesis or major project • Portfolio evaluation • Pre and posttests • Reflective journals • Capstone projects • Internal/external juried review of performances and exhibitions • Internship and clinical evaluation • National Major Field Achievement Tests • GRE subject exams • Certification exams, licensure exams

7. Indirect Methods of Assessment: • Survey of perceived outcome attainment by course (survey students or faculty) • Departmental survey • Exit interviews • Alumni survey • Employer survey • Focus groups • Job placement statistics • Graduation and retention rates • Percentage of students who study abroad

8. Assessment Instruments • Some combination of indirect and direct measures is best (in an ideal world) • Allows for converging evidence • Example: • students have low confidence in their ability to formulate a hypothesis • students perform below expectations on an embedded assignment asking them to formulate a hypothesis

9. Assessment Instruments • What measures do you use already? • What additional measure(s) do you think would be useful?

10. Assessing Large Classes • Challenges include: • Giving rich individual feedback • Managing the quantity of grading (yourself, and coordinating assistants) • Avoiding testing that fosters shallow learning • Assessing a diverse mix of students • Avoiding plagiarism

11. Assessing Large Classes • Some strategies: • Scoring rubrics • Complex products or behaviors can be examined efficiently • Identify characteristics of what you are assessing • Describe the best work you could expect based on these (top category), and the worst (lowest category) • Develop descriptions of intermediate-level products that are meaningful to you • 1 to 3 (novice, competent, exemplary), or 1 to 5 (unacceptable, marginal, competent, very competent, outstanding)

12. Assessing Large Classes • Some strategies: • Using samples of student work • Random or performance based • Technology • Clickers, web-campus quizzes & surveys, on-line discussion boards • Automating the analysis process • Setting up excel templates • Structuring exams to easily capture info

13. Assessing Large Classes • Group projects • Peer & self evaluations • Evaluations by grad assistants, faculty committees, etc. • Ensure that grading material are understood by all staff • Run training sessions where they evaluate various levels of student artifacts

14. Assessing Large Classes • Assess background knowledge early in semester • Use cumulative tasks with more formative feedback that guides efforts on next task • Ask students to consider how topics relate to their discipline area

15. Assessing Multiple Sections • Direct Measures • Imbed common questions in exam for all sections • Create common writing assignment for all sections • Create a multiple choice test to give to all students at end of semester

16. Assessing Multiple Sections • Direct Measures cont. • Create a pre and post test to give to all students • Compare different modes of delivery! • Indirect Measures • Survey of student’s perceived learning (SALG) • Survey faculty

17. Assessing Multiple Sections • How to get agreement in your department??

18. Assessing On-line Courses • Formative assessment in on-line classes is key • Assessment processes in on-line classes should: • Enable students to self-monitor their progress • Stop lecture after certain time, ask students to reflect, write down insights, submit feedback as short notes

19. Assessing On-line Courses • Gather regular feedback from students • Pose a question via e-mail about teaching and invite students to respond • Students can respond with personal email • Give regular feedback to students • One sentence summaries, minute papers • Paper or project prospectus (brief structured first draft plan)

20. Developing a Timeline • How frequently can you feasibly measure each outcome? • Or, how frequently can you feasibly use a specific instrument to measure an outcome?

21. Establishing Criteria • Criteria help you make sense of your results. • May not be “right” the first time • Expectations that are very low or very high yield less meaningful results

22. Closing the Loop • Using results to make improvements is the ultimate goal! • Can you do anything about met or unmet expectations? • If not, why see if their met?