Disability Resource Center

2014 AHEAD Conference Culturally Responsive Evaluation: A Tool for Assessing Faculty Understanding of Serving Students with Disabilities Disability Resource Center

Disability Resource Center 2014 AHEAD Conference Sacramento, CA Susan A. Aase, J.D., M.S.Ed. Donna Johnson, M.A., M.S. Disability Resource Center (DRC) University of Minnesota – Twin Cities July 18, 2014

Agenda • Introductions/Ground Rules • Learning Objectives • Culturally Responsive Evaluation • Application • Discussion • Wrap Up

Cautionary Tale Participating in a 90-minute session on evaluation does not make one an expert but…..

Learning Objectives • Participants will learn the differences between Culturally Responsive Evaluation and traditional Western models. • Participants will learn how Culturally Responsive Evaluation may be a useful approach in evaluating faculty understanding of serving students with disabilities using the lens of strengths of the communities served. • Participants will learn practical strategies for conceptualizing, developing, and implementing a component of Culturally Responsive Evaluation that may be implemented by a small office at minimal cost.

Evaluation in the Context of the Office for Equity and Diversity (OED) • At the University of Minnesota, the following offices report to OED: • Business Community Economic Development • Conflict Resolution • Disability Resource Center (DRC) • Diversity in Graduate Education • Equal Opportunity and Affirmative Action • Gay, Lesbian, Bisexual, Transgender, and Ally Programs • Institute for Diversity, Equity, and Advocacy • Multicultural Center for Academic Excellence • Women’s Center

The OED Philosophy By including disability as an aspect of human diversity, OED views disability through a “cultural lens.” OED selected Culturally Responsive Evaluation as the methodology that worked for all OED units.

Your Program’s Philosophy What is your program’s philosophy? How is disability viewed?

Culturally Responsive Evaluation: What Is It? Culturally Responsive Evaluation is a collection of practical strategies and frameworks that attend to culture and context when preparing for an evaluation, conducting it, and disseminating and using the results of the study. (Samuels & Ryan, 2011)

Culturally Responsive Evaluation Is new paradigm of evaluation Utilizes cultural strengths in a community Aligns assessment perspectives and methods with traditional knowledge, values, worldviews, lived experience and ways of knowing Is strengths-based and culturally meaningful (White Shield, Justilien & Acharya, 2012)

Western Paradigm Methods Quantitative: counting, checklists, surveys, pre-post tests, analysis of statistics Qualitative: case studies, anecdotes, focus groups, observations, analysis of files (White Shield& Acharya, 2012)

Non-Western Paradigm Methods Focus on discovery of knowledge already present within the community Congruent with communities’ worldviews and intrinsic cultural strengths Rely on research/evaluator’s lived experience to help inform the process (White Shield & Acharya, 2012)

Non-Western Paradigm Methods In what other settings might you operate using a Non-Western paradigm?

Created a Logic Model A logic model is a tool used most often by program managers to evaluate the effectiveness of a program. Logic models are usually a graphical depiction of the logical relationships between the resources, activities, outputs and outcomes of a program.

What Is a Logic Model?

Developed A Purpose Statement A purpose of the Disability Resource Center is to facilitate the University of Minnesota’s recognition of disability as a natural part of the human experience.

Purpose Statement What is apurpose of your disability services?

Explored Cultural Strengths of People with Disabilities • Resiliency – Ability to adapt to situation • Flexibility – Ability to do tasks in different ways • Diversity – People with disabilities are not a monolithic group • Ability to move along a spectrum of cooperation to get things done • Ability to tap into unique perspectives on and experiences with defining success and failure

Developed Assumptions (If/Then Statements) • If the Disability Resource Center (DRC) approaches service delivery from a continuous improvement standpoint, then information is sought from its many stakeholders on what it can do to better meet the needs of university students, staff, faculty, and guests with disabilities. • If the DRC considers disability to be an equity issue, then the DRC will collaborate with campus partners to create an accessible campus.

Developed Overarching Outcomes • The expectation is that the Disability Resource Center will: • gather qualitative and quantitative data to modify our decision-making based on evidence. • be responsive to the needs of its users in the university community. • assist qualified people with disabilities to fully participate in and contribute to the university community.

Overarching Outcomes What might your overarching outcomes be?

Selected the Following Output/Activities • Develop Faculty Advisory Committee • 2. The Disability Resource Center (DRC) will gather data to modify its decision-making based on evidence. The first priority addressed was the exploration of faculty understanding of the student role, the faculty role, and the DRC’s role.

Selected the Following Inputs • UM Faculty with disabilities • 2. UM Faculty without disabilities

Data Collection Tool Development Faculty Advisory Committee members developed the questionnaire using an iterative process. Created tool to assess faculty understanding of their role in accommodating students with disabilities. Met in person and then worked together remotely over the course of several weeks.

Data Collection Methods • The Disability Resource Center (DRC) used a community expert (kinship) model in which members of the DRC Faculty Advisory Committee were asked to recommend 1-2 faculty to complete an e-mail questionnaire.

Data Collection Methods • These were sent to thirty-three (33) faculty in the College of Liberal Arts, College of Science and Engineering, Humphrey Center, College of Biological Sciences, Academic Health Center, College of Education and Human Development, and the School of Public Health, Twenty-two (22) faculty completed surveys for a 66 percent response rate.

Sample Faculty E-mail Dear Professor Aase,Professor Fuller, a member of University of Minnesota-Twin Cities Disability Resource Center’s advisory committee, recommended you to participate in a brief questionnaire to assist the Disability Resource Center in assessing how faculty perceives the disability accommodation implementation process. The questionnaire takes about 10 minutes to complete. Your answers do not need to be lengthy. We are looking for key thoughts around each of the questions. Please type your answers in the attached document and return it to Donna Johnson, Director, Disability Resource Center at johns042@umn.edu by Tuesday, January 16, 2013. If you have any questions, please contact me at 612-624-4120. Sincerely, Donna JohnsonDirectorDisability Resource CenterUniversity of Minnesota-Twin Cities

Sample Questions • Do you understand your role in implementing disability-related accommodations in the classroom, labs, testing environment, and/or course related experiences? • If yes, briefly explain your role. • If you are unsure as to how to support students with disabilities, how might the Disability Resource Center assist you in implementing disability-related accommodations in the classroom, labs, testing environment, and/or in course-related field experiences? Please be specific.

Sample Questions • 4. How do you facilitate students with disabilities informing you of their accommodation needs? (Check all that apply): • I include a syllabus statement inviting students to discuss their disability-related needs with me. • I discuss disability-related accommodations as part of my introductory lecture on the first day of class. • I include a statement on my course Moodle site inviting students to discuss their disability-related needs with me.

Sample Questions • 5. On average, how many students with disabilities self-identify as needing accommodations in a given semester? (Check the appropriate response): • 1-2 students per semester • 3-4 students per semester • 5-6 students per semester • 7 or more students per semester • 6. Of the students who identified themselves as needing disability accommodations, what types of accommodations have you implemented?

Sample Questions 7. Do you feel comfortable contacting the Disability Resource Center regarding a question about a disability-related accommodation listed in a student accommodation letter? Why/Why not? 8. Please briefly describe what has worked when when implementing accommodations for students with disabilities at the University of Minnesota? 9. Please briefly describe what has not worked well when implementing accommodations for students with disabilities at the University of Minnesota? 10. What recommendations do you have to improve communication between faculty and the Disability Resource Center?

Sample Questions Based on your overarching outcomes, what questions might you ask?

Analysis Methods A thematic analysis was used. Disability Resource Center (DRC) staff compiled and reviewed the data to identify themes. Peer and member checks were used to validate the process by reviewing the findings with members of the DRC Faculty Advisory Committee.

Key Findings • Faculty are unclear regarding the DRC’s role – access vs. success. • Faculty need multiple methods of communication, regarding providing accommodations to students with disabilities in their courses (from DRC, Dept. Chairs, and Provost). • Faculty consider DRC as a “just in time” service unit – contact with DRC is initiated only when a problem or crisis occurs. • Faculty have observed more students with mental health issues, behavioral issues, and unrealistic expectations.

Key Messages • Faculty want to know how many and what types of disability conditions are present on campus. • Faculty want a flow chart that outlines the faculty role, the student role, and DRC’s role in providing accommodations. • Faculty want additional information on how to approach students who may have not disclosed the disability. • Faculty want a list of frequently asked questions around disability issues that are modified for specific academic audiences.

Key Recommendations • DRC should tailor messaging for faculty based on faculty needs/interests. • DRC outreach efforts should include basic elements of what is a disability, what is the purpose of the accommodation letter, what is the student’s role, the faculty role, and DRC’s role in implementing the accommodation, and what is the process to address issues if the accommodation is not working?

Key Accomplishments • Created Disability Toolkit for Faculty on DRC Website. • FAQs on testing accommodations. • Accommodation process flowchart. • Revised accommodations letter. • Established practice for emailing faculty. • Participate in new faculty orientation.

Key Accomplishments • Participate in Provost’s New Chairs Training. • Developed video on the Interactive Process. • Developing “Who is in Your Classroom video. • Developing On-Line Training for Faculty with Data Tracking Capabilities

The Culturally Responsive Evaluation Process • Shifted the focus on understanding the “lived experience” of faculty – more inclusive than asking faculty to complete the questionnaire. • Community expert (kinship) method involved asking faculty to contribute questions. Faculty reviewed the final questionnaire to validate that the questions were reflective of their lived experience. Faculty took ownership of the data gathering. • Faculty Advisory Committee members completed the questionnaire and then asked 1-2 other faculty colleagues to complete the questionnaire – This was faculty led, using faculty voice, faculty-focused analysis with peer member checks and outcomes.

The Culturally Responsive Evaluation Process 4. Faculty could see how their work lead to meaningful action to improve communication around the role of student, faculty, and DRC staff in the interactive process. 5. Faculty Advisory Committee serves as a Community of Practice (Non-Western concept of shared knowledge). • The process of implementing Culturally Responsive Evaluation strengthened the DRC’s reframing of disability as a aspect of diversity and as an equity issue.

The Culturally Responsive Evaluation Process 7. Helped build trust and buy-in and was a way for faculty to feel heard. 8. Helped the DRC have a better shared understanding of faculty concerns. 9. Faculty advisory committee members had a vested interest in “harvesting” information.

Evaluation Thank you for your participation! Please complete the evaluation.

Thank You! Susan A. Aase, J.D., M.S.Ed. Associate to the Director, Disability Resource Center 612-624-2993 aase0020@umn.edu Donna Johnson, M.A.. M.S. Director, Disability Resource Center 612-624-4120 johns042@umn.edu

Disability Resource Center