Course Assessment Basics: Evaluating Your Construction

1. Course Assessment Basics: Evaluating Your Construction 1. Please log into the phone as well as the computer with the same code (185104). 2. Please put phones on mute *6 We will start at 11:00 3. Please type your name, college and your role with SLOs on your campus into the chat line and hit send. Dr. Janet Fulks, Bakersfield College Marcy Alancraig, Cabrillo College

2. To Deal with Student Learning Outcomes You Need: • Sense of Play • Willingness to Experiment • Ability to translate assessment theory and jargon into plain English

3. A Quiz on your SLO Knowledge Type in your answers to the following questions

4. True or False? SLOs (student learning outcomes) and assessment are new fads that began with Dr. Spellings and the Bush administration.

5. True or False? According to both the Academic Senate and WASC, writing SLOs and designing assessment processes is a faculty responsibility.

6. The Accreditation Standards require that SLOs are written and assessed in • Courses • Programs • General Education • Student Services • All of the above

7. Defining SLOs

8. So What’s an SLO? • Knowledge • Skills • Abilities • Attitudes • Beliefs • Opinions • Values that a student can demonstrate after a course of study

9. SLO Exercise • Pick your favorite course • What is the one big thing -- idea, attitude, value, skill, ability or self-realization – that you want students to retain? • Type in your answer

10. SLOs should: • Require HIGHER LEVEL thinking skills • Synthesize many discreet skills • Require students to APPLY what they’ve learned • Result in a product • Product must be evaluated or assessed by faculty

11. Course level SLOs should cover • Discipline knowledge • Discipline skills • Discipline values and beliefs • Answers a & b only • All of the above

12. Objectives vs. Outcomes Objectives Outcomes Requires synthesis of skills Uses higher level thinking skills Results in a product • Describes small, discreet skills • Requires basic thinking skills • Do not necessarily result in a product

13. Difference between Objectives and Outcomes

14. Example: Reading 961 SLOs (two levels below college-level English) : • Utilize vocabulary skills to comprehend assigned readings. • Determine and differentiate main ideas and supporting details in assigned readings. • Make appropriate inferences in assigned readings

15. Reading 961 objectives: • Apply knowledge of vocabulary commonly used in college reading, writing, and speaking. • Identify main idea in assigned readings. • Identify supporting details in assigned readings. • Identify organizational patterns and relationships of ideas in assigned readings. • Utilize graphic organizers (mapping, outlining, summarizing) as a method of organizing ideas in prose reading. • Apply contextual clues as a method of improving comprehension through informing vocabulary in assigned readings. • Apply critical thinking skills including distinguishing fact from opinion, making inferences, and identifying author’s purpose and tone in assigned readings.

16. Assessing SLOs

17. What is Assessment? • An ongoing process aimed at improving student learning. • Faculty making learning expectations explicit and public. • Faculty measure if students have met them. • Faculty make changes based on results- in the classroom, student service, administrative program and the college.

18. Assessment Defining (and Re-assessing) Assessment: A Second TryT. A. Angelo, (1995) AAHE Bulletin no.48, p.7. • "Assessment is an ongoing process aimed at understanding and improving student learning. • It involves making our expectations explicit and public; • setting appropriate criteria and high standards for learning quality; • systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and • using the resulting information to document, explain, and improve performance.

19. Assessment Defining (and Re-assessing) Assessment: A Second TryT. A. Angelo, (1995) AAHE Bulletin no.48, p.7. When it is embedded effectively within larger institutional systems, assessment can help us • focus our collective attention, • examine our assumptions, and • create a shared academic culture dedicated to assuring and improving the quality of higher education. "

20. WYMIWIG • What you measure is what you get • Formative Assessment • Summative Assessment

21. Types of Assessment Formative Summative Done at the end of learning Final evaluation of student learning High stakes Used to evaluate student performance Provides feedback for future teaching and learning (benefits future students) Examples: portfolios, major assignments, final exams • Done while learning is in process • Helps students improve performance • Low stakes • Acts primarily as diagnostic tool • Provides feedback for current teaching and learning (benefits current students) • Examples: CATS, quizzes, homework

22. SLO Assessments should: • Require HIGHER LEVEL thinking skills • Synthesize many discreet skills • Require students to APPLY what they’ve learned • Result in a product • Product must be evaluated or assessed by faculty (sound familiar?)

23. Which of the following are methods for assessing course SLOs? • Major assignment graded by a rubric • Portfolios • Embedded test questions • Projects • All of the above

24. Types of Information Direct Data Indirect Data Ask students or others to evaluate skills Is expressed through opinions; learning is inferred Setting: unconfined and unstructured Examples: surveys, focus groups • Evaluate what students can do • Is expressed through work you can evaluate • Setting: confined and structured • Examples: exams, papers, projects

25. Direct or Indirect Data? 1. What is polling information on who people will vote for in an election? a) direct data b) indirect data 2. What is the actual vote count reported after an election? a) direct data b) indirect data

26. Direct or Indirect Data? 3. People’s opinion about their favorite make of car. a) direct data b) indirect data 4. The number and make of automobiles actually sold. a) direct data b) indirect data

27. Qualitative versus Quantitative? A faculty member is convinced that field trips are the most effective way to teach geology but it is impacting the budget. Which data would be most convincing in a budget discussion? • A narrative on the benefits of field trips (qualitative data) • A collection of student opinions about field trips (indirect data could be qualitative or quantitative) • An example of student grades related to topics covered on the field trip that compares the scores of students who went on the field trip and those who did not (direct, quantitative) • A list indicating the number of the other institutions and geology programs that support geology field trips as an integral part of the pedagogy (indirect, quantitative) • A combination of these data

28. Qualitative versus Quantitative? An ESL instructor has discovered from feedback from her students that the most important outcome they are hoping for is proper pronunciation when they speak. Which would be the most useful type of assessment data both for the individual student and for the course outcomes as a whole? • Direct statistical data from a multiple choice test about the fundamental rules in pronunciation (quantitative). • A national standardized ESL test (quantitative). • A student log book created as a result of listening and analyzing recordings of their own speaking (qualitative). • An interview developed to assess pronunciation and evaluated using a rubric that indicates the major types of errors and a narrative summary of the overall pronunciation expertise (could be qualitative and quantitative). • A classroom speech evaluated by comments from fellow classmates (qualitative).

29. Models for Assessing Classroom SLOs • Embedded test questions • Cabrillo KISS method • Projects – Engineering at BC • English Portfolio at Cabrillo • Culinary Arts Portfolio at BC • Pre and Post testing at Cabrillo • Capstone – Case Study at BC

30. Activity: Assessing Course SLOs • What can you use to assess your SLO? Think about major assignments or activities. Type in something you might use.

31. Closing the Loop

32. What is Closing the Loop? • Final step in SLO Assessment process • A time to reflect on data • A time to dialogue with colleagues about how to improve teaching and learning • Should be part of all levels of SLO assessment

33. Develop or modify Student Learning Outcomes (SLOs) Develop, modify, or review a curriculum, course, program, or service. Determine refinements based on outcomes data.   Closing the Assessment Loop Design & Measure Student Learning as a result of the Curriculum, Course, or Program Collect, discuss, and analyze data.

34. A Few Words of Final Advice • Keep it simple. • Keep it safe. • Focus on just one or two SLOs at a time. • Start small. Complete an entire loop with one course or a series of classes and see what it teaches you. • Keep assessment a faculty responsibility. Don’t give it over to others. • Use the assessment samples posted to create dialogue about the best instrument. 

35. Next Steps Write down one thing you will do in response to the information you received in this webinar. Comments and Questions