Assessment

1. Assessment Is like a dancer’s mirror. It improves one’s ability to see and improve one’s performance. Alexander Astin 1993

2. ASSESSMENT . . . “a rich conversation about student learning informed by data.” -- Ted Marchese -- AAHE

3. Assessment of Individual Student Development • Assessment of basic skills for use in advising • Placement • Counseling • Periodic review of performance with detailed feedback • End-of-program certification of competence • Licensing exams • External examiners • CLAST

4. Key Results of Individual Assessment • Faculty can assign grades • Students learn their own strengths and weaknesses • Students become self-assessors

5. A Second Look • Across students • Across sections • Across courses

6. Where is learning satisfactory? • What needs to be retaught? • Which approaches produce the most learning for which students?

7. Group Assessment Activities • Classroom assignments, tests, projects • Questionnaires for students, graduates, employers • Interviews, focus groups • Program completion and placement • Awards/recognition for graduates • Monitoring of success in graduate school • Monitoring of success on the job

8. Use of Results of Group Assessment • Program improvement • Institutional and / or state peer review • Regional and / or national accreditation

9. Outcomes Assessment The process of providing credible evidence of the processes and outcomes of higher education undertaken for the purpose of improving programs and services within the institution. Banta, T. W.

10. Some Purposes of Assessment 1. Students learn content 2. Students assess own strengths 3. Faculty improve instruction 4. Institutions improve programs/services 5. Institutions demonstrate accountability

11. Assessment of Learning 1. Faculty agreement on outcomes and performance standards 2. Evidence in syllabi and assignments that outcomes are taught 3. Collective faculty review of group performance 4. Use of findings to improve instruction and curriculum

12. Most Faculty Are Not Trained as Teachers Faculty Development Can Help Instructors: • Write clear objectives for student learning in courses and curricula • Individualize instruction using a variety of methods and materials • Ask questions that make students activelearners • Develop assessment tools that test higher order intellectual skills

13. Cognitive domain categories Knowledge Comprehension Application Analysis Synthesis Evaluation Sample verbs for outcomes Identifies, defines, describes Explains, summarizes, classifies Demonstrates, computes, solves Differentiates, diagrams, estimates Creates, formulates, revises Criticizes, compares, concludes Taxonomy of Educational Objectives(Bloom and Others, 1956)

14. Organizing for Assessment

15. Planning for Learning and Assessment

16. Faculty and Staff Development • Focus faculty and student affairs professionals on improving learning in and outside class • Attend conferences together • Study literature on student learning • Provide workshops on teaching and learning • Provide resources (e.g., grants, summer salary, release time)

17. Direct Measures of Learning Assignments, exams, projects, papers Indirect Measures Questionnaires, inventories, interviews - Did the course cover these objectives? - How much did your knowledge increase? - Did the teaching method(s) help you learn? - Did the assignments help you learn?

18. Select or DesignAssessment Methods 1. Match with goals 2. Use multiple methods 3. Combine direct and indirect measures 4. Combine qualitative and quantitative measures 5. Consider pre - post design to assess gains 6. Use built-in points of contact with students

19. Outcomes Assessment Requires Collaboration • In setting expected program outcomes • In developing sequence of learning experiences (curriculum) • In choosing measures • In interpreting assessment findings • In making responsive improvements

20. Barriers to Collaboration in the Academy • Graduate schools prepare specialists • Departments hire specialists • Much of our scholarship is conducted alone • Promotion and tenure favor individual achievements -- interdisciplinary work is harder to evaluate

21. Change the Focus from TEACHING to LEARNING Barr & Tagg 1995

22. Functions of the Work of theProfessoriate • Scholarship of Discovery • Scholarship of Integration • Scholarship of Application • Scholarship of Teaching Ernest Boyer - 1990

23. Good assessment is good research . . . • An important question • An approach to answer the question • Data collection • Analysis • Report -Gary R. Pike (2000)

24. Campus Interest in Assessment WHAT WORKS in…. • increasing student retention? • general education? • use of technology in instruction? • curriculum in the major?

25. Measures of Critical Thinking 1. Academic Profile (ETS) 2. Collegiate Assessment of Academic Proficiency (ACT) 3. California CT Dispositions Inventory (Faciones) 4. California CT Skills Test (P. Facione) 5. Cornell CT Test (Ennis & Millman) 6. Tasks in CT (ETS) 7. Reflective Judgment Inventory (King & Kitchener) 8. Watson Glaser CT Appraisal (Psych Corp)

26. Collegiate Assessment of Academic Proficiency(CAAP from ACT) Individual Modules • Reading • Writing (MC and Essay) • Mathematics • Science Reasoning • Critical Thinking

27. COLLEGE BASE Competences Interpretive Reasoning Strategic Reasoning Adaptive Reasoning English S U B J E C T S English Subject Clusters Interpretive Reasoning Math Writing Skills Science Reading/Literature Skills Reading Critically Reading Analytically Understanding Literature Social Studies

28. Are Standardized Tests the Answer? • Not available in many fields • Do not measure all that is taught • Usually assess knowledge, not performance • May be standardized on unrepresentative norm group • Provide few, if any, subscores • Do not indicate why scores are low

29. Start with Measures You Have • Assignments in courses • Course exams • Work performance • Records of progress through the curriculum

30. Primary Trait Scoring Assigns scores to attributes (traits) of a task STEPS • Identify traits necessary for success in assignment • Compose scale or rubric giving clear definition to each point • Grade using the rubric

31. Narrows and defines topic Produces bibliography Develops outline Produces first draft Produces final draft Presents oral defense Can Develop a Research Paper

32. Bibliography Outstanding – References current, appropriately cited, representative, relevant Acceptable – References mostly current, few citation errors, coverage adequate, mostly relevant Unacceptable – No references or containing many errors in citation format, inadequate coverage or irrelevant

33. Mapping Course Outcomes to Program Outcomes

34. Score 3 2 1 0 Criterion Clear conceptual understanding, consistent notation, logical formulation, complete solution Adequate understanding, careless errors, some logic missing, incomplete solution Inadequate understanding, procedural errors, logical steps missing, poor or no response Problem not attempted or conceptual understanding totally lacking Ball State University Sophomore Competence in Mathematics(Multiple choice responses & supporting work)

35. Entries accurately and vividly record objective observations of site experiences (events, people, actions, setting) Entries convincingly record subjective responses to site experience (thoughts, emotions, values, judgments) Entries effectively analyze/ evaluate your experiences (find insights, patterns, meaning, causes, effects) Journal Evaluation

36. Assessment in Sociology and Anthropology Focus groups of graduating students • Given a scenario appropriate to the discipline, a faculty facilitator asks questions related to outcomes faculty have identified in 3 areas: concepts, theory, methods. • 2 faculty observers use 0-3 scale to rate each student on each question • GROUP scores are discussed by all faculty • Murphy & Goreham North Dakota State University

37. Methods of Assessment • Paper and pencil tests • Individual or group projects • Portfolios • Observation of practice • Observation of simulated practice • Analysis of case studies • Attitude or belief inventories • Interviews and focus groups • Surveys

38. Assessing Student GrowthThe Portfolio - Some Examples of Content • Course assignments • Research papers • Materials from group projects • Artistic productions • Self-reflective essays (self-assessment) • Correspondence • Taped presentations

39. Student Electronic Portfolio • Students take responsibility for demonstrating core skills • Unique individual skills and achievements can be emphasized • Multi-media opportunities extend possibilities • Metacognitive thinking is enhanced through reflection on contents - Sharon J. Hamilton IUPUI

40. PRINCIPLES OF UNDERGRADUATE LEARNING (PULs) • Core communication and quantitative skills • Critical thinking • Integration and application of knowledge • Intellectual depth, breadth, and adaptiveness • Understanding society and culture • Values and ethics Approved by IUPUI Faculty Council May 1998

41. ePort Goals • Promote clearer understanding among faculty and students of how the curriculum supports increasing mastery of the PULs. • Contribute to assessment of student learning of the PULs at multiple levels of aggregation. • Support student engagement with the PULs over their entire undergraduate experience.

42. PULs Levels of Competence • Introductory: What all undergraduate students should know and be able to do within the first 26 credit hours. • Intermediate: What all undergraduate students should know and be able to do within the first 56 credit hours. • Advanced: What all baccalaureate recipients should know and be able to do in their major or profession or academic program. • Experiential: Connecting curricular and co-curricular learning

43. Background Knowledge Probe(Pre-Test – Indirect Measure) • ARCHAEOLOGY A. Have never heard of this B. Have heard of it, but don’t really know what it means C. Have some idea what it means, but not too clear D. Have a clear idea what this means and can explain it - Classroom Assessment Angelo and Cross

44. Fast Feedback(at end of every class) • Most important thing learned • Muddiest point • Helpfulness of advance reading assignments for day’s work in class • Suggestions for improving class / assignments Bateman & Roberts Graduate School of Business University of Chicago

45. Student Suggestions for Improvement • Install a portable microphone • Increase type size on transparencies • Leave lights on when using projector • Don’t cover assigned reading in detail • Provide more examples in class

46. Outcomes of Classroom Assessment 1. Classes more student centered 2. Focus on learning, not teaching 3. Students take ownership for learning 4. Faculty look for data to make improvements continuously - Jann Freed Assessment Update - 1999

47. In a Comprehensive Assessment Program... INVOLVE • Students • Faculty • Student Affairs Staff • Administrators • Graduates • Employers

48. Involve Students 1. Set learning expectations in recruiting 2. Communicate learning outcomes in orientation 3. Involve student leaders in promoting learning 4. Involve students in evaluating courses/curricula 5. Let students know their recommendations are used.

49. Student Advisory Council at MontevalloA way to provide continuous student assessment Student Recommendations • Develop a statement of expected ethical behaviors for students • Add a second research course with lab • Increase comparative psychology • Add terminals for statistics lab • Increase opportunities for research, writing, and speaking

50. Guidance from Alumni • Alumni surveys emphasized that graduates valued skills in writing, speaking, working collaboratively, and information literacy • Now the Faculty Senate’s General Education Committee has developed 5 learning elements, at least 3 of which must be integrated in any course approved for general education -Michael Dooris Penn State University