480 likes | 995 Vues
Using Data from the CCSSE to Encourage Change Maureen Pettitt, Ph.D. Leslie Croot, M.S. Session Topics CCSSE Overview CCSSE-Generated Reports SVC-Generated Presentation & Reports Use of Reports CCSSE Overview Emphasis on student engagement and student learning (e.g., Astin, Tinto, Pace)
E N D
Using Data from the CCSSE to Encourage Change Maureen Pettitt, Ph.D. Leslie Croot, M.S.
Session Topics • CCSSE Overview • CCSSE-Generated Reports • SVC-Generated Presentation & Reports • Use of Reports
CCSSE Overview • Emphasis on student engagement and student learning (e.g., Astin, Tinto, Pace) • Adapted from the National Survey of Student Engagement (NSSE) • Funding from Lumina Foundation for Education and Pew Charitable Trusts • Administered by Community College Leadership Program at UTA • Pilot test in 2001; field test in 2002
Student Engagement • Students who get involved with the people and activities of the college (beyond the traditional classroom) have • higher retention rates; • greater personal growth, achievement and satisfaction; and • increased participation in further learning opportunities (Astin, 1985)
CCSSE Contents • Active & Collaborative Learning • Student Effort • Student-Faculty Interaction • College Contribution to Knowledge, Skills & Personal Development • Mental Activities • Student Demographics
2003 CCSSE Participants • National • 93 community colleges; 65,300 usable surveys • Northwest Consortium • Six WA community colleges & one in BC; 3,480 usable surveys • 19 additional questions • SVC • Surveyed 850 students in 60 classes; 765 usable surveys
CCSSE-Generated Reports • College Results • Summary: Successes and Areas for Improvement • Aggregate percents for all colleges and for SVC • Mean Comparisons • SVC, consortium, all colleges • Frequency Distributions • SVC, consortium, all colleges
CCSSE-Generated Reports • Benchmarks -- five benchmarks from 38 engagement items on the CCSSE • Active & Collaborative Learning • Student Effort • Academic Challenge • Student-Faculty Interaction • Support for Learners
Other CCSSE “Tools” • On the website • Interpreting and working with results • Developing storylines • Powerpoint template & talking points • National reports • Highlights
Internal Reports: The Challenge • Creating reports that are both meaningful and can inform change at the institution • Knowing what the issues are and how the data can shed light on those issues (also: what issues can be “fixed” with information?) • Having a network of faculty & staff at the college who know the issues and will ask for specific reports
Internally-Generated Reports & Presentations • Gen Ed Task Force Summer Retreat • Two Fall In-Service Presentations • Article for Fall Teaching & Learning Newsletter • Library & Media Services • Diversity Steering Committee • Information Technology • Student Services Programs • Student Government
General Education Task Force • Gen Ed Task Force Summer Retreat 2003 • Data presented according to existing gen ed principles and outcomes • Generated discussions about what wasn’t in our current outcomes (i.e., Information Literacy) • Provided insight into the efficacy of our pedagogical approaches (i.e., learning communities, infusion of critical thinking and cultural pluralism, etc.)
General Education Outcomes • Communications • Critical Thinking • Quantitative Reasoning • Development & Features of Culture • Putting Knowledge into Action • Overarching Learning & Development Goals
To what extent has your experience at this college contributed to your knowledge, skills, and personal development in the following area: Acquiring a broad general education
In your experience at this college during the current school year, about how often have you done each of the following: Prepared two or more drafts of a paper or assignment before turning it in
In-Service Presentation Fall 03 • Purpose – provide an overview and generate discussions (information was also the basis for T & L Newsletter article) • “Most Frequent” (at least 20% of the respondents reported they did this “very often”) and “Least Frequent” (at least 20% of the students reported they had “never” done this) • General Education-related outcomes • Student satisfaction with services
Most Frequent • Worked on a paper or project that required integrating ideas or information from various sources – 31.1% • Prepared two or more drafts of a paper before turning it in – 29.5% • Asked questions in class or contributed to class discussions – 27.1% • Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment – 24.0%
Most Frequent • Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.) – 23.5% • Worked with other students on projects during class – 22.7% • Had serious conversations with students who differ from you in terms of their religious beliefs, political opinions, or personal values – 20.5%
Least Frequent • Participated in a community-based project as part of a regular course – 67.5% • Worked with instructors on activities other than coursework – 62.0% • Tutored or taught other students (paid or voluntary) – 59.4% • Discussed ideas from your readings or classes with instructors outside of class – 35.5%
Least Frequent • Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment – 23.3% • Come to class without completing readings or assignments – 22.3% • Talked about career plans with an instructor or advisor – 20.9% • Used e-mail to communicate with an instructor – 20.4%
Learning Communities • Always questions on campus about the benefit of collaborative courses • CCSSE question asking whether the student had taken a learning community, had not, but planned to, or had not and did not intend to take.
Mental Activities • Similar in structure to Bloom’s Taxonomy • Prompt: “During the current school year, to what extent has your coursework emphasized the following mental activities…” • Response Options: Range from 1 “Very Little” to 4 “Very Much”
Groupings • SVC 1 = SVC students who had taken a learning community or linked course • SVC 2 = SVC students who had not, but planned to take a learning community or linked course • SVC 3 = SVC students who had not taken and did not plan to take a learning community or linked course • Consortium = all students in the Northwest consortium of colleges, excluding those from Skagit Valley College and Douglas College • All = all students who participated in the CCSSE Survey, based on 93 colleges
”memorizing facts, ideas, or methods from your courses and readings so you can repeat them in pretty much the same form.”
“synthesizing and organizing ideas, information, or experiences in new ways.”
Library/Media Services • Additional questions on library use, library resources, use of Internet, and knowledge/skill acquired to conduct information searches • Only comparison with consortium colleges • SVC used the library more, acquired more skills/knowledge, but were less satisfied with the library resources.
Diversity Steering Committee • Burning question: how did students of color compare with their white counterparts • There were few significant differences, except SOC had better relationships with administrative personnel • Limitations—N for SOC = 78; these were students of color who had transitioned to the “mainstream” (versus ESL students)
Information Technology • IT Dean was then-chair of the State Information Technology Council • Prepared a report he presented at an ITC meeting showing consortium results for nine IT-related CCSSE items • Also prepared an SVC report showing college-consortium comparisons
Student Services • Created a 71-page report just for Student Services: • Summary of Significant Findings • Career Counseling • Admissions • Placement • Advising/Financial Aid Advising • Registration • Financial Aid • Student Life
Counselors/Advisors • Where do you most often get information about the requirements for your educational program? • Academic advising/planning: How often do you use? How satisfied? How important? • How often do you use career counseling? • How often do you use transfer credit assistance?
Students with Risk Factors • Analysis of CCSSE student responses based on risk factors: • Academically Under-prepared • Financially Independent • Single Parent • English Not Native Language • Parent’s Education • Disabled
Breakdown by Risk Category (N=765) • Three categories in analysis: • No Risk -- risk factors = 0 (19%) • Low Risk -- risk factors =1 (34%) • At Risk -- 2 or more risk factors (47%)
Students at Risk Study • Used this data to follow-up with SVC Student Support Services (TRIO Grant) program participants • SSS program goal: provide educational opportunities for disadvantaged students
TRIO Program Evaluation • Created an instrument with CCSSE items; very targeted; no demographic data • Given at end of quarter in College Success Skills course for SSS students • Part of the SSS program’s Outcomes Assessment Plan • Track results of SSS students over time/against CCSSE baseline using students designated as “at risk” (two or more risk factors) for comparison
College Support • SSS participants feel the college helped them cope with non-academic responsibilities more than all other groups (p < .001). • SSS participants feel the college helped them thrive socially more than all other groups (p < .001). • SSS participants feel the college provided the support they needed to succeed more than all other groups (p = .006).
Use of Services • SSS participants sought financial aid advising, and career counseling more than the "Low Risk" and "At Risk" groups. • SSS participants used services for people with disabilities more than all other groups. • SSS participants used academic advising/ planning more than all of the other groups • SSS participants used peer or other tutoring more than the "No Risk" and "Low Risk" groups, but not the “At Risk.”
The "At Risk" group felt they had a poorer relationship with instructors compared to the "No Risk" group (p = .007). However, the "SSS Users" felt they had a better relationship with instructors compared to the "At Risk" group (p < .001). There were no other differences between the groups.
The "At Risk" group felt they had a poorer relationship with administrative personnel compared to the "No Risk" group (p = .028). However, the "SSS participants" felt they had a better relationship with administrative personnel than the "Low Risk" group (p = .001) or the "At Risk" group (p < .001). There were no other differences between the groups.
Student Government • Short report and presentation at ASSVC meeting • Student government reps were interested in general demographics, how students spent their time, and student participation in extra- and co-curricular activities
Using the results • Included data in Board of Trustee Monitoring Reports on Ends Policies • Student Satisfaction & Success • Institutional Diversity • Provided reports and presentations for various groups
Using the Results • Folks at the college who moved to an action phase spent time identifying specific things they wanted to improve, prioritizing them, and designing low-cost, uncomplicated interventions (“quick wins”)
Fifth-Year Accreditation Report • Spring 2004 (1 yr later) • All reports compiled in a big notebook • Included documentation from CCSSE recognizing SVC as a “Best Practice College” and one of four colleges in the nation to get the MetLife Foundation award
Future Plans • Administer CCSSE in Spring 2005 and 2007 • Cross-time comparisons available for accreditation self-study and 10-year visit in 2009