Data for Higher Education Policymaking: Issues of Access, Analysis, and Presentation

1. Data for Higher Education Policymaking: Issues of Access, Analysis, and Presentation ASHE Graduate Student Public Policy Seminar Hans L’Orange and David Wright November 17, 2005

2. Introductions Who/What is SHEEO? • National association of state higher education coordinating and governing boards • Broadly, the mission of SHEEO is to assist its members and the states in developing and sustaining excellent systems of higher education

3. Introductions SHEEO’s mission objectives: • Emphasize the importance of planning and coordination for higher education by promoting strategic planning and statewide coordination. • Promote cooperative relationships in the collection and exchange of data and information, development of standard definitions and practices, and conduct of studies. • Formulate and recommend desirable guidelines for state and federal relationships to institutions of higher education. • Encourage studies and other action to advance statewide planning and coordination.

4. Introductions SHEEO pursues its mission by: • Organizing regular professional development meetings for its members and their senior staff; • Maintaining regular systems of communication among the professional staffs of member agencies; • Serving as a liaison between the states and the federal government; • Studying higher education policy issues and state activities and publishing reports to inform the field; and • Implementing projects to enhance the capacity of the states and SHEEO agencies to improve higher education.

5. Background Data important as the environment has become increasingly… • Competitive • For higher ed, systems, institutions, and faculty • Distributed • Technological advancements have “freed” data • Greater access, expectations, and abilities • Political

6. Background Functions of data within the policy setting • Set the context, boundaries, and conditions for decision-making • Ideally, you need data before the conversation begins (not always the case) • Agree on “the facts” before discussing goals, objectives, strategy • Understand “where we are” • Support tough decisions about where you’re going and how to get there

7. Background Higher ed challenges that require good data • Enhance student enrollment • Support student success • Maintain financial viability • Operate strategically • Plan realistically • Allocate resources appropriately • Support decision making • Assess management outcomes • Renew accreditation • Demonstrate accountability

8. Background Data, information, and knowledge • Outside some context, data are just meaningless points in space and time, without reference to either space or time. • Information relates to description, definition, or perspective (what, who, when, where). • Knowledge comprises strategy, practice, method, or approach (how). • Wisdom embodies principle, insight, moral, or archetype (why). Gene Bellinger (2004). http://www.systems-thinking.org/kmgmt/kmgmt.htm

9. Background Assertions of Truth Beliefs Data to policy Dogma Preconceptions Policy Experience Bias Anecdotes Informed Stakeholders (Knowledge) Stakeholder Needs “Story” Facts Anecdotes Graphs/Tables Needs Analysis Information Analysis Data Requirements Data

10. Background Good data REVEALING USEFUL AVAILABLE RELEVANT COMPREHENSIVE TIMELY INTUITIVE RELIABLE AND VALID APPROPRIATE COMPARISON GROUP USED REPRESENTATIVE JUST ENOUGH Source: Merrill Schwartz, AGB

11. Background The right amount of the right data “There are two equally effective ways of keeping a board in the dark. One is to provide them with too little information. The other, ironically, is to provide them with too much.” - From “Building Better Boards,” by David A. Nadler, Harvard Business Review, May 2004, p. 109

12. Background Common mistakes in the use of data for policymaking at the institution board level • Data overload • Inappropriate level of detail • Lack of governance perspective • Lack of strategic relevance • Insufficient distribution • Inattention to time constraints • Reliance on anecdote • Lack of context

13. Data access Data, data everywhere • General references • Measuring Up • Higheredinfo.org • NCES products • IPEDS • Tools: Executive Peer Tool (ExPT), Peer Analysis System (PAS), and Dataset Cutting Tool (DCT) • Census products (CPS, American Community Survey • Regional sites (SREB, WICHE) • Mortenson’s Postsecondary OPPORTUNITY

14. Data access Data, data everywhere • Students and learning • NCES: IPEDS (GRS), sample surveys • ACT and College Board • NSSE and CCSSE • Faculty and staff • NCES • IPEDS (Fall Staff, Faculty Salaries) • NSOPF • AAUP • CUPA

15. Data access Data, data everywhere • Finance and facilities • NCES: IPEDS, NPSAS • SHEEO SHEF (annual), T&F policies survey (triennial) • Illinois State “Grapevine” survey • APPA (core indicators on facilities) • NASBO (fiscal survey) • NACUBO (endowments, comparative stats) • Tuition surveys - Washington HECB, College Board, NASULGC • NASSGAP (state financial aid) • ACE Pell Grant report

16. Data access Data, data everywhere • Adult/workforce • NCES: Nat’l Assessment of Adult Literacy • www.higheredinfo.org (under “Special Analyses) • ACE GED report • Still other data sources address: • Governance • Policy (WICHE SPIDO) • Emerging policy issues • Technology, distance education • K-12

17. Data access Recent initiatives that could affect data availability and quality • HEA Reauthorization • Major issues are affordability, net price, the impact of student mobility and graduation rates, and “better consumer information” • IPEDS unit record (UR) proposal • Alternatives to a federal UR system • “Huge IPEDS” • Linking state UR systems • Building on Following the Mobile Student

18. Data access 39 states have unit record systems

19. Data access Other alternatives to a federal UR system • Expand NCES sample studies • National Student Clearinghouse • Academy One • Data Quality Campaign (K-12) • Any solution must adequately address issues of… • Burden • Privacy (FERPA) • Relevance • Currency/timeliness • Security • Trust • (translates as “accountability”)

20. Data analysis and presentation The two cultures differ in terms of… • Rhythm and calendar • Language and writing style • R2 vs. ROI • Academic freedom • Nature and timing of “iterativeness” • Where the conversation is carried out • Academic: journals, conferences, listservs • Political: policy transfer, websites, hallways

21. Data analysis and presentation To maximize your effectiveness in the policy world… • Know the policy! • Your data often contain artifacts of policy • Know the basic analytical conventions • FTE; inflation adjustment • Produce an “elevator ride” document • In addition to report and executive summary • “Write backward” • Recommendations first, methodology last

22. Data analysis and presentation To maximize your effectiveness in the policy world… • Tell stories • For illustration; not the same as basing policy on anecdote • Paint pictures

23. Data analysis and presentation “Telling stories” • “Despite increases, tuition is below national average” • by Jennifer Peltz • Fort Lauderdale Sun-Sentinel • June 27, 2005 • Florida makes a point of keeping tuition low at its public universities and community colleges. But tell that to Sean Chapman. • He's a full-time student with a full-time job, working his way through a Florida Atlantic University finance degree with a position at a brokerage. Tuition and fees have risen 19 percent since he started four years ago, and they're likely to jump another 5 percent in the fall. Chapman's got about $7,000 in student loans already and at least a year to go. • "I just try not to think about it," groans Chapman, 21, a finance major who grew up near Pembroke Pines.

24. Data analysis and presentation “Telling stories” • “Despite increases, tuition is below national average” (cont’d) • Together, public universities and community colleges educate eight of every 10 college students in Florida, according to federal statistics. The schools are founded on the premise that the state and its students are sharing costs, with the state picking up the lion's share. • But that share is shrinking. Students' portion of the tab grew from about 23 percent to about 28 percent between 1995-96 and 2003-04, the last year for which figures are available from State Higher Education Executive Officers. The nonprofit group's annual studies reflect what's actually collected from students and their families, not what's paid on their behalf by federal, state or college-sponsored grants. • If that doesn't sound like much of a difference, try this: An FAU or FIU student's tuition and fees grew more than five times as fast as the state's per-student spending on higher education, as measured by the officers group. A local community college student's costs grew more than three times as fast. The student costs are calculated for students like Chapman: full-time undergraduates paying a discounted Florida-resident rate…

25. Data analysis and presentation Pictures worth a thousand words

26. Data analysis and presentation Pictures worth a thousand words U.S. Counties by Educational Needs Index Quartile Source: www.educationalneedsindex.com

27. Analysis and presentation Pictures worth a thousand words U.S. Counties by Educational Needs Index Quartile Percent loss at each stage of transition Source: www.higheredinfo.org, from NCES, ACT Institutional Survey, and IPEDS Graduation Rate Survey

28. Data analysis and presentation Pictures worth a thousand words Highest Attainment Level of HS Dropouts 10 Years Later 13,742 High School Dropouts from 1990-1991 • Dropouts who would have graduated with the class of 1991 attained education credentials, including high school diplomas or equivalencies, at much lower rates than their HS grad counterparts. Masters <1% College Credit Attainment Status Vocational unchanged 68.2%* <1% Bachelors ~1% AA ~1% Vocational GEDs Certificates 20.8% 4.4% Adult High School Diplomas 5% Note: Excludes any credentials earned out of state. Source: Florida Education & Training Placement Information Program

29. Data analysis and presentation Pictures worth a thousand words

30. Data analysis and presentation Pictures worth a thousand words (variation on a theme)

31. Data analysis and presentation Bringing it all together • A few real-world examples: web site demonstrations

32. Parting shots • You’re entering a grand conversation • Being a good conversationalist means… • Having something to say • Listening • Allowing that the other person just might be right • Acknowledge biases • Disciplinary • Institutional • Personal

33. Parting shots On the nature of biases in the policy world Pareidolia - a type of illusion or misperception involving a vague or obscure stimulus being perceived as something clear and distinct.

34. Parting shots A decade-old toasted cheese sandwich said to bear an image of the Virgin Mary sold on eBay for $28,000.

35. Parting shots Candles, flowers and a painting of the Virgin Mary embracing John Paul line the section of the Kennedy Expressway underpass believed to hold an image of the Virgin. 

36. Parting shots In 1996, a cinnamon bun thought to bear the likeness of Mother Teresa was sold in a Nashville bakery.

37. Parting shots

38. Contact Info: Hans L’Orange Director, SHEEO/NCES Network Director, Data and Information Management Hans@sheeo.org David WrightSenior Research Analyst David@sheeo.org 3035 Center Green Drive, Suite 100 Boulder, CO 80301 www.sheeo.org