Comprehensive Assessment Reports

1. Comprehensive Assessment Reports Fred Trapp, Ph.D. Administrative Dean, Institutional Research/Academic Services (Retired) Long Beach City College Cambridge West Partnership, LLC Robert Pacheco, Ed.D. Director Of Institutional Planning, Research and Resource Development Barstow College

2. The participant will be able to Describe the comprehensive assessment report concept. Locate best practice examples from other colleges through web links. Discuss how the comprehensive report idea can be used as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public. Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes. Please hold questions until the end. Outcomes for the Session

4. Curriculum Map 4

8. ACCJC Institutional Effectiveness Rubric Part III Student Learning Outcomes Proficiency Comprehensive assessment reports exist and are completed on a regular basis.

9. What Did We Look At, With Whom Did We Consult? ACCJC. Institutional Effectiveness Rubric ACCJC. 2002 Standards ACCJC. Themes ACCJC. Guide to Evaluating Institutions Professional literature Efforts of national groups/institutes Institutional web sites, listservs and colleagues

10. Guiding Questions How can the report writing experience: Help faculty explore the student learning process? Determine the extent to which the curriculum is working? Where can time, energy and/or money be allocated for continuous improvement in learning? Exploit the writing process and dialogue about results to gain broader institutional learning experiences? Help meet our quality assurance pledge to the community?

11. Illustration Selections • Council for Higher Education Accreditation (CHEA) • Annual Award for Outstanding Institutional Practice in Student Learning Outcomes • Demonstrated commitment to & developed highly effective practice to use SLO assessment • Willingness to share the practices they developed • Selection committee • Selection criteria • Articulation & evidence of outcomes • Success with regard to outcomes • Information to the public about outcomes • Using outcomes for improvement

12. Illustration selections (continued) • Cited by scholars and peers • Schools with assessment work cited in scholarly books and articles • Schools with assessment work selected for presentation at national conferences • Web presentations publicly available for you to ease drop upon • Prominent national movements/initiatives regarding learning outcomes & assessment documentation (including public quality assurance)

13. What Might Be Included? • Assessment focus- course, program, general ed, etc. • What outcomes were assessed? • How and when were they assessed? • Who was assessed? • What were the results? • Who reviewed the results, made sense of the them and what conclusions were reached? • What are the implications for practice and/or policy or future assessment work?

14. CC of Baltimore County (MD)Course-level Reporting • Middle States Commission on Higher Education • Community College Futures Assembly, Bellwether Award, 2008 • Instructional Programs & Services for High Impact Course Level Assessment • CHEA award winner, 2006 • Institutional Progress in Student Learning Outcomes • National Council on Student Development (NCSD) Exemplary Practice Award Winner

15. CC of Baltimore County (MD)Course-level Reporting Projects are at least three semesters long Individual and high-impact courses (all sections) included Project proposal by a faculty group Measurable objectives External review & approval in selecting methods/instrument & analyzing results. Benchmarking should be included if possible. Controls and sample size considered. Course improvements based on data analysis Reassessment expected Results/report shared across the college and web posted

16. CC of Baltimore County (MD) • Learning Outcomes Assessment Final Report Template • Design & proposal for the LOA project • Implementation of design & data collection • Redesign of the course to improve student learning • Implementation of course revisions & reassessment of student learning • Final analysis and results • eavesdropping • http://www.ccbcmd.edu/loa/CrseAssess.html • Two-page executive summaries available

17. CC of Baltimore County (MD) CHEM 108 An initial “failure” turned to success and collaboration with a four-year school HLTH 101 Addressing an achievement gap with professional development and increased communication with students CRJU 101 and 202 Statewide group assessment development effort and creativity in the interventions used

18. Program-level Reporting • North Central Association of Colleges & Schools, Higher Learning Commission • CHEA award winner, 2008 • Institutional Progress in Student Learning Outcomes

19. Hocking College (OH) Learning outcomes data collected in a student E-portfolio Direct internal and external evidence (1 to 10 measures) Indirect evidence (1 to 4 measures) Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric Eight general education outcomes (student success skills) Discipline-specific exit competencies or outcomes See the Hocking wheel

21. Hocking College (OH)Program-level Report eavesdropping Cloud reference, not college URL as links are broken there Various reports available in each program profile Curriculum matrix Criteria statements (exit competencies) Instructional Program Outcomes (assessment plan) Trend Charts for performance criteria

22. Hocking College (OH)Program-level Report • Example reports and analysis • Culinary Arts Technology • Forestry Management Technology • Nursing Technology

23. Mesa College (AZ)General-Education Reports • North Central Association of Colleges & Schools, Higher Learning Commission • CHEA Award winner, 2007 • Institutional progress in Student Learning Outcomes

24. Mesa CollegeGeneral Education Report Multiple outcomes assessed Annually Annual Report elements Executive Summary Methodology Results & observations (GE & workplace) Indirect measures findings Appendices of past results

25. Mesa Community College (AZ) • General education studies completed 2007-08; 2005-06 • Numeracy • Scientific inquiry • Problem solving/critical thinking (2008-09) • Information literacy • Workplace skills (CTE) (2009-10) • General education studies completed 2006-07; 2004-05 • Arts & humanities • Cultural diversity • Oral communication • Written communication

26. Mesa College (AZ)General Education Reports eavesdropping http://www.mesacc.edu/about/orp/assessment/index.html Annual reports and summaries available Nine years of history and experience 14 years of assessment work

27. New England Association of Schools and Colleges, Commission on Institutions of Higher Education Cited in the Art and Science of Assessing General Education Outcomes: A Practical Guide (AAC&U, 2005) eavesdropping http://www.ccc.commnet.edu/slat/ Annual reports and summaries available Capital CC (CT)General-Education Reports

28. Portland CC (OR)General Education Reports Northwest Accrediting Commission eavesdropping http://www.pcc.edu/resources/academic/learning-assessment/ One general education theme a year Learning Assessment Focus for 2009-10- Critical Thinking & Problem Solving Physical Science, Geology and General Science Bioscience Technology Management and Supervisory Development Culinary Assistant Program

29. Truman State University (MO)Various Reports Southern Association of Schools and Colleges, Commission on Colleges eavesdropping Assessment work began in 1970 http://assessment.truman.edu/ Assessment Almanac- A compilation of results from each year’s assessment work (versions from 1997 to 2009 are posted) General Education outcomes are assessed in the context of the major field of study Portfolio Project- required of all seniors to show best work assessed by faculty for the nature & quality of the liberal arts and sciences learning outcomes (versions from 1997 to 2008 are posted)

30. Authorship • Course-level, program-level & general education • Teaching faculty study team with technical assistance from • institutional research or assessment committee • No “lone ranger” authors • Institutional summary • Academic administrator with assistance from • Learning outcomes coordinator or assessment committee • Compilation of work accomplished in one or two academic years across the institution

31. Putting the Reports to “Work” • To all affected participants • Campus committees • Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership • Campus fairs, brown-bag lunches, poster sessions for information sharing • Faculty professional development programs • Accreditation self-study committee work groups • Local governing board presentation • College web site for the public

32. Reports & a Learning Organization Learning organization Environment that promotes a culture of learning Individual & group learning enriches & enhances the organization as a whole Systematic problem solving using data for decisions Learning from experiences in assessing organizational performance Comparing yourself to others (benchmarking) and borrowing ideas Adriana Kezar ed. Organizational Learning in Higher Education New Directions for Higher Education. No. 131, Fall 2005. Jossey-Bass.

33. Reports as Institutional Learning & Resource Allocation • The assessment data sense-making process = a faculty learning experience • Linking results to future interventions = a learning experience • Using results to inform an intervention, then reassess = a learning experience (accomplished one or more terms later) • Reference for future assessment work and other groups on campus

34. Reports as Institutional Learning & Resource Allocation Hocking College (OH) Annual summary Improvements in the program in the previous year brought on by study of assessment results Expenditures of time, money & materials for the assessment program Requests for assistance in implementing assessment Recommendations for altering the institution’s assessment process Transition from evaluating individual students to assessing groups of students & the curriculum experience

35. Reports as Institutional Learning & Resource Allocation Community College of Baltimore County (MD) Learning Outcomes Assessment Advisory Board Links findings in assessment reports to other college-wide initiatives and professional development opportunities Use of assessment processes and (findings) results Challenged faculty to reexamine prompts used in assessment Clarity of written prompt & extent it supports program goals Common assignment options and common rubric increases faculty understanding and buy-in Builds faculty unity toward common goals Public web page enhances communication and accessibility to information

36. Reports as Institutional Learning& Resource Allocation • CHEA award winner, 2010 • Institutional Progress in Student Learning Outcomes

37. Reports as Institutional Learning& Resource Allocation Feedback & recognition Feedback rubric for annual assessment reports Conversations and action Collection and analysis of evidence Implementation of findings Recognition (achievement & excellence)

38. Reports as Institutional Learning& Resource Allocation Seal of Assessment Achievement Academic programs earning this recognition have demonstrated in their annual report that learning outcomes have been assessed through two or more methods, and findings have been discussed among the faculty.

39. Reports as Institutional Learning& Resource Allocation Seal of Assessment Excellence Academic programs earning this recognition have demonstrated a thorough implementation of assessment plan(s) the reporting of meaningful assessment data the discussion of findings among faculty and perhaps students the use of findings to showcase student achievements and to make curricular adjustments.

40. Reports as Institutional Learning& Resource Allocation Mesa College (AZ) Results Outreach Committee Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate Groups of faculty offer a proposal for summer or academic year work above the course level Resulting report placed on the web and used for campus discussion and action

41. Report as Quality Assurance • NILOA • 2010 Webscan report Exploring the Landscape: What Institutional Websites Reveal About Student learning Outcomes Assessment Activities • 2010 Connecting State Policies on Assessment with Institutional Assessment Activity • 2011 Transparency Framework (online) • eavesdropping • www.learningoutcomeassessment.org

42. Report as Quality Assurance Promising Vehicles for Expanding Information to the Public Brief narrative report from annual assessment reports Simple statistical reports on learning outcomes or surveys Best practices stories supported by assessment Peter Ewell. Accreditation & the Provision of Additional Information to the Public about Institutional and Program Performance, CHEA, May 2004

43. Quality Assurance to the Public Voluntary System of Accountability APLU & AASCU (520 public institutions, award 70% of bachelor’s degrees in the US each year) College Profile (includes learning outcomes & links to campus) Proactive initiative to document learning gains and average institutional scores (choice of 3 national instruments) Proactive initiative to illustrate unique campus learning outcomes assessment work Promoting a learning institution eavesdropping http://www.collegeportraits.org/

44. Quality Assurance to the PublicVSA Example, CSU Monterey Bay http://www.collegeportraits.org/map Cal Poly Pomona http://www.collegeportraits.org/CA/CPP/learning_outcomes Learning Assessment Examples

45. Quality Assurance to the Public National Association of independent Colleges and Universities Assessment programs on campus tied to institution’s mission eavesdropping http://www.naicu.edu/special_initiatives/accountability/Student_Assessment/id.514/default.asp Pepperdine University http://services.pepperdine.edu/oie/learning-outcomes/learning-outcomes-overview.aspx

46. Where Can I Go?Resource Filesanywhere.com http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d

47. Contacts & Questions • Robert Pacheco (Barstow College) • Rpacheco@Barstow.edu • Fred Trapp (Cambridge West Partnership) • fredtrapp@gmail.com • Questions and Comments

48. Session EvaluationOutcomes for the Session The participant will be able to Describe the comprehensive assessment report concept. Locate best practice examples from other colleges through web links. Discuss how the comprehensive report idea can be used as part of the institution’s learning process and as a means by which the institution provides quality assurance to the public. Indicate national trends and efforts of consortia/national organizations to provide quality assurance about student experiences and learning outcomes.