Turning Data Into Action Workshop June 26, 2014

1. Turning Data Into Action Workshop June 26, 2014

2. Kentucky prioritized gap closing as a major focus for the upcoming academic year

3. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Associate degrees) Graduation rate gaps: Underrepresented Minorities Gap between the graduation rate of African-American, Latino, American Indian and Hawaiian/Pacific Islander students and the graduation rate of White and Asian students . Data Source:CPE Comprehensive Database (KPEDS)

4. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Associate degrees) Graduation rate gaps: Underprepared Gap between the graduation rate of students who did not meet statewide college Readiness benchmarks and those who did. Data Source:CPE Comprehensive Database (KPEDS)

5. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Associate degrees) Graduation rate gaps: Low Income Gap between the graduation rate of Pell grant recipients and non-recipients . Data Source:CPE Comprehensive Database (KPEDS)

6. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Bachelor degrees) Graduation rate gaps: Underrepresented Minorities Gap between the graduation rate of African-American, Latino, American Indian and Hawaiian/Pacific Islander students and the graduation rate of White and Asian students (using the IPEDS six-year graduation rate). Data Source:CPE Comprehensive Database (KPEDS)

7. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Bachelor degrees) Graduation rate gaps: Underprepared Gap between the graduation rate of students who did not meet statewide college Readiness benchmarks and those who did (using the IPEDS six-year graduation rate). Data Source:CPE Comprehensive Database (KPEDS)

8. Policy Objective 4: Increase high-quality degree production and completion rates and close achievement gaps. (Bachelor degrees) Graduation rate gaps: Low Income Gap between the graduation rate of Pell grant recipients and non-recipients (using the IPEDS six-year graduation rate). Data Source:CPE Comprehensive Database (KPEDS)

9. Today, we will cover… Turning Data Into Action Workshop Agenda 9:00 Welcome & Meeting Overview 9:40 Understanding Our Capacity to Use Data to Make Decisions on Campus 10:45 Break 11:00 Understanding Current Progress and Identifying Institutional Factors 11:45 Lunch 12:30 Prioritizing and Focusing Our Efforts 1:30 Identifying Relevant Metrics 2:15 Break 2:30 Determining Who and How 3:30 Identifying Next Steps 4:00 Meeting Close

10. “delivery”(n.) is a systematic process through which leaders can drive progress and deliver results. What are we trying to do? 1 How are we planning to do it? 2 At any given moment, how will we know whether we are on track? 3 If not, what are we going to do about it? 4 The delivery approach produces results by focusing leaders on four fundamental questions It involves asking the following questions consistently and rigorously:

11. EDI supports implementation of education reform in a number of states and systems Current EDI Partner Systems Our mission is to partner with K-12 and higher education systems with ambitious reform agendas and invest in their leaders' capacity to deliver results.  By employing a proven approach, known as delivery, we help state leaders maintain the necessary focus to plan and drive reform. K-12 Higher Education Both

12. Our Higher Education Delivery Network consists of a growing number of systems across the country EDI focuses on supporting systems in their implementation efforts to: • Increase the percent of students who graduate with a degree on time • Close equity gaps in postsecondary access • Close equity gaps in postsecondary success • Current Higher Education Partnerships • California State University System • Colorado State University System • University of Hawai’i System • Kentucky Council on Postsecondary Education • Minnesota State Colleges and Universities • University of Missouri System • New Jersey Office of Higher Education • Pennsylvania State System of Higher Education • Tennessee Board of Regents • The University of Texas System • University of Wisconsin System

13. We will use this workshop to focus on using data as an action strategy to close gaps Assess campus capacity to use data to drive decision making Understand current progress on campus and in the Council, as it relates to achievement gaps Discuss institutional factors contributing to outcomes Refine key campus initiatives intended to improve outcomes Identify metrics in order to monitor those initiatives Identify who will be involved in monitoring and making mid-course corrections and how monitoring will occur

14. To meet our objectives, we will tackle the gap challenges by answering four questions

15. There are simple steps you can take to make the most of the workshop! • Choose a team leader/facilitator • Suspend disbelief! • Ask yourself, “What can we do now, with what we have?” • Try not to get caught up in the minute details • Leverage the expertise at the table • Leverage the efforts underway at your campuses LET’S GET STARTED!

16. We believe that these behaviors are necessary for campuses to use data effectively in decision making EDI’s tenets for data use • Campuses must: • Have clear targets against which to measure performance • Use timely data (both headline metrics and leading indicators) that are monitor progress on a regular basis • Disaggregate the data in order to isolate outcomes for specific groups of students • Establish routinesthat help teams come to a shared view of progress • Involve IR in relevant conversations and analyses to take advantage of existing expertise • Empower decision makers to seek and use relevant data to support students

17. As we begin to plan how we’ll use data to make decisions, we need to be clear about some terms Term Description A coherent set of initiatives that are designed to maximize impact on your target metrics Strategy A specific change in the way a campus supports and educates its students Initiative An outcome used to measure progress on an initiative, strategy, or both Metric Depending on what your team is prepared to discuss, you can choose to either focus on one discrete initiative or on a strategy comprised of multiple initiatives during this workshop.

18. Poll Everywhere: Data-informed Decisions and Your Campus • How well does your campus currently use data to drive/inform decision making? • We do it, and we do it well • We do it well in a few areas across campus • Analysis Paralysis – We have it, but we rarely move to action • “Who needs data? We go with our gut!” • Data? What data? This is a challenge area for our campus

19. Workshop Agenda Understanding Our Capacity to Use Data to Make Decisions Followed by a break

20. Our goal is to not only understand our status but to discuss why, and how we can ensure that we maximize our impact Our graduation rate for African American students went up 4% this year! That’s probably because the cohort of students that entered when we began the new multicultural office support program graduated this year. Our graduation rate for African American students went up 4% this year! Yep! Early indicators indicated that this program was improving retention and credits earned for African American students. Not sure though…we should find out how many students this program served and what the early indicators said. Huh. Interesting. We should monitor and scale up that support program from now on, so we maximize the impact from this important strategy!

21. This workshop allows campuses to develop a plan to assess progress on key campus strategies by asking 4 questions Teams will review their campus’ data to develop a shared understanding of student success

22. This workshop allows campuses to develop a plan to assess progress on key campus strategies by asking 4 questions Teams will discuss the institutional factors that need to be addressed or leveraged to improve student performance

23. This workshop allows campuses to develop a plan to assess progress on key campus strategies by asking 4 questions Teams will identify, clarify, and refine key campus strategies to address challenges

24. This workshop allows campuses to develop a plan to assess progress on key campus strategies by asking 4 questions Teams will develop a routine to regularly assess progress on selected strategies

25. Before we can answer these questions, however, we must reflect on our current capacity to use data to drive action How much capacity do we have on campus to answer these questions?

26. EDI has created a rubric to help campuses think about their ability to use data to drive decisions

27. The self-assessment tool helps campuses understand current capacity to answer these questions Rubric: Capacity to Use Data for Decision Making on Campus Rubric category This category asks us to think about… Access and efficiency The ease and speed with which data can be accessed or retrieved from campus systems Monitoring Actions taken to understand progress on campus goals and metrics The processes we use to turn what we learn from campus data into action around campus strategies Problem solving Culture The attitudes and reactions of campus faculty and staff toward making decisions using campus data

28. Use the rubric to determine how you would rate your campus in each category Red: Our campus must focus on improving in this area to use data effectively in decision making Orange: Our campus must improve in this area, though in some aspects more than others Yellow: Our campus should continue in the aspects we find strong, but focus on weak points Green: Our campus is successfully using our capacity in this area to use data to make decisions Category Rating Rationale Summary Access and efficiency Capacity to use data to make decision on campus Monitoring Problem solving Culture

29. Example: one campus realized that monitoring and problem solving needed improvement in order to use data effectively Orange: Our campus must improve in this area, though in some aspects more than others Red: Our campus must focus on improving in this area to use data effectively in decision making Example of completed rubric with rationales Yellow: Our campus should continue in the aspects we find strong, but focus on weak points Green: Our campus is successfully using our capacity in this area to use data to make decisions Category Rating Rationale Summary Unified data system with IR support, but faculty and staff generally do not know how to make data requests or what questions to ask Access and efficiency Capacity to use data to make decision on campus Our strategic plan calls for regular monitoring, but meetings get cancelled; when they do happen, we don’t have helpful data available Monitoring Conversations around areas of improvement almost never focus on solutions; we do not have priority owners Problem solving We feel that faculty, staff, and campus leadership are generally confident in our data systems and want to use data to drive the work Culture

30. Exercise: Capacity To Use Data on Campus Self-Assessment • Individually, read your rubric and consider what your “traffic light” rating would be in each of the four areas. Write your thoughts on the rubric. • As a team, discuss your ratings and determine where your campus is strongest, where it needs the most improvement, and why. Questions to ask each other… What prevents us from being green? Do our ratings mean the same thing? Are we being too harsh? Too easy?

31. Your Role in Strengthening Capacity • How can you leverage your role to strengthen your campus’ capacity to use data to make decisions? • Record your answer on a card • Share your card with your team members • Place card on brown paper

32. Workshop Agenda Understanding Current Progress and Identifying Institutional Factors Break Followed by Lunch

33. First, we will determine the level of challenge we face in serving our students and then discuss contributing factors

34. “Good data, while essential, is only a start— you then have to use it!” – Instruction to Deliver

35. There are two steps to understand the drivers of performance and evaluate relevant campus strategies Description Why important 1 • Pinpoint the root causes, or drivers of performance patterns • Identify “usual suspects” when possible • Utilize formal problem-solving tools when drivers are not evident • A campus cannot know where to focus unless it identifies drivers Identify the drivers of your student success patterns 2 • Identify what the campus is currently doing to try and address drivers of underperformance • Evaluate whether current initiatives are sufficient and recommend changes where necessary • Creating new or revamping initiatives without understanding existing strategies wastes resources Identify and evaluate campus strategies related to drivers of performance patterns

36. Traditionally, data show several factors that are likely to impact how groups of students progress and persist Some of those factors include:

37. Florida State University used data to understand student success and which students were lost along the way Six-Year Graduation Rates 2002-2010, Underrepresented Minorities and White Students Source: Florida State University What could Florida State do to make a difference for them? Why were some students successful while others were not?

38. In looking at attrition rates, the team considered a range of time points and student background characteristics Source: Florida State University Yearly Attrition Rates by Cohort: White, Female, First-Time In-State Students 1995-2005 MIN MAX 6.7% 9.8% 2.1% 3.9% 0.9% 3.3% 0.5% 1.6% 10.2% 17.1%

39. Hispanic female Pell recipients exhibited very different attrition rate patterns, suggesting the need for continued support over time to avoid dropouts Source: Florida State University Yearly Attrition Rates by Cohort: Hispanic, Female, Pell Recipient, First-Time, In-State Students 1995-2005 MIN MAX 0% 4.1% -1.4% 3.9% 6.9% 17.7% -1.2% 10.5% 6.3% 17.6%

40. Black male Pell recipients exhibited different attrition rate patterns, suggesting the need for continued support over time to avoid dropouts Source: Florida State University Yearly Attrition Rates by Cohort: Black, Male, Pell Recipient, First-Time In-State Students 1995-2005 MIN MAX 0% 11.0% 0% 11.1% 3.3% 15.3% -0.9% 13.6% 6.0% 14.0%

41. A number of other institutional factors also create unclear pathways to graduation • Insufficient academic guidance/planning • Policies to pose as barriers • Misunderstanding degree requirements • Inappropriate preparation/enrollment in prerequisite courses • General education requirement confusion • Course withdrawal or failure • Switching majors • Inability to enroll in necessary coursesat the appropriate time

42. Strengthening Advisement The Difference Between Satisfying a Requirement and Being on Track Graduation Rate in Major by Introductory Course Grade

43. Let’s look at Kentucky Campuses Success Gap data to help review your campus’s student success patterns Are our gaps shrinking? Growing? What do we know about closing gaps in higher education attainment? Colorado State University System | Kentucky Council of Postsecondary Education | Louisiana Board of Regents | Minnesota State Colleges and Universities | Mississippi Institutions of Higher Learning | Montana University System | New Jersey Higher Education | Pennsylvania State System of Higher Education | Rhode Island Board of Governors for Higher Education | State University of New York | State University System of Florida | Tennessee Board of Regents | The California State University System | University of Hawaii System | University of Louisiana System | University of Missouri System | University of North Carolina System | University of Texas System | University of Wisconsin System | 19 Systems, 300 Campuses, 3.5 Million Students18% of Public Undergrads | 866K URMs | 1 million Pell Recipients

44. A2S Metrics Count All Students • Include allstudents, not just first-time, full-time • Include part-time and transfer students, many of whom are low-income and underrepresented minorities • Report retention and success rates for low-income and transferstudentsat institution and in system

45. Benchmarking Progress on Success • Are we increasing success rates for underrepresented students? • GAP: Are we increasing success rates for underrepresented students relative to peers?

46. Benchmarking Progress on Success The data handouts answer these and other questions about student success on your campus.

47. Are we increasing overall graduation rates? • What is the overall freshmen rate? • What is the rate for transfer students? • How have these rates changed over time?

48. Are we increasing graduation rates for underrepresented students? • What is the grad rate of Pell freshmen? • What is the grad rate of non-Pell freshmen? • How do rates for these student groups differ?

49. Are we increasing graduation rates for underrepresented students? • How have graduation rates for Pell and non-Pell freshmen changed over time?

50. Are we increasing graduation rates for underrepresented students? • How has the Pell freshmen rate changed over time? • How has the non-Pell freshmen rate changed? • Has the graduation rate gap between these groups narrowed or widened over time?