Getting to Outcomes: Next Steps

Getting to Outcomes: Next Steps

Doug TannerYouth Catalytics978-544-2067dtanner@youthcatalytics.com

Workshop Objectives Know how a data management project can help: • Improve program design • Demonstrate effectiveness • Highlight the best work being done • Compete for funding, and • Mobilize public support.

Workshop Objectives Learn about: • Identifying expected outcomes, and • defining objectives and incremental indicators of success consistent with your mission.

Workshop Objectives • Understand the elements and stages of a data management planning process • Be familiar with common barriers and costs associated with data management

Workshop Objectives • Learn how quality data can influence and inform the strategic planning process. • Explore options for tracking and using data efficiently at reasonable cost.

Examples of Data Compilation: • City of Pittsfield Neighborhood evaluation • Combining & analyzing data from multiple sources • DIAL/SELF (Greenfield, MA) Transitional Living Program housing outcomes • Sorting and interpreting data from a single collection source (Lets go to visit source tables in Excel then come back to PowerPoint to review graphs)

Building permits over 20K by Pittsfield neighborhoods:

Intake by age range

Issues at intake by age

Housing Outcome Data

Project Planning 101: • Bring key people together at each stage of planning process • Administration, program directors and supervisor participation is critical in early stages, but direct care staff can be helpful too (ask questions!)

Project Planning 101: • Initial planning stages require a deep understanding of the resources (funding, technology/equipment, and staff time) required to plan, implement and maintain a data management project/data driven culture.

Project Planning 101: • It may be worthwhile to invest in a consultant or devote substantial administrative time to produce useful estimates of the time and cost involved in implementing and maintaining a data driven culture

Planning Guidelines • Understand the purpose of your project - what will this data do for your organization? • Identify data priorities • Plan to start small and efficiently – you can grow as you learn and achieve - look for the intersection of what data you can easily obtain and what you would want to know in an ideal world! (go to flip chart )

Staff Commitment & Support • As you move into more detailed planning, direct care staff input becomes extremely important. • Involve staff in a formal way and carefully assess what support they will need to succeed! • Design formal systems for Training, Support and Accountability

Gather the Information You Need to Get Started • Reports/data you already need for funders • Identify information for internal evaluation and improvement (even if it isn’t currently required by funders) • Develop a functional draft of outcomes, objectives and indicators (your dataset) prior to shopping for a database or building a data collection system

ImProve OutcomesSM A Brief Summary

ImProve OutcomesSM Language • Objectives = desired participant changes or achievements • Indicators = measurable events • Outcomes = level of achievement

ImProve OutcomesSM& Logic Models Basic Logic Model Inputs resources Outputs actions Outcomes achievements ImProve OutcomesSM Model Objectives expectations Outcomes achievements Inputs resources Outputs actions *Identify tracking method Indicators events

ImProve OutcomesSM is… • Extension of logic models • Based on incremental change • Means of prioritizing information • Method of categorizing information

Bloom’s Taxonomy Levels of Learning Mastery • Knowledge/Comprehension • (learn about it) • Application • (use it, try it out) • Synthesis • (integrate with other knowledge)

Indicators Should be SMART • Specific • Measurable • Achievable • Relevant • Timely

Indicators are Activity or Behavior Based (observable) • Use active verbs to describe indicators • Look for achievement opportunities at levels that are relevant to the services, time frame or intervention level of your program • Indicators reflect participant capacity for positive change and choices that indicate forward movement

Database Options • Web/Cloud Based • Require reliable, high speed internet connection(s) • Each user has own license – can access from anywhere • Easy to monitor data entry • Evaluate capability and cost of compilation, sorting and reporting • Carefully evaluate ownership of data and “worst-case scenarios” (e.g., you or the provider go out of business?) • PC-Based • You own software and data that is on your computer • Speed depends on speed of machine • May require additional software to run the database • Can be difficult to synchronize data from multiple sources. • Ease of data retrieval depends a lot on initial design and software used.

Tools you can use – now! • 1. Surveys: • Useful to capture information from participants • You have to ask the right question(s). That takes planning and some experimentation to gather aggregateable data. • Results can be compiled in Excel – but consider using Survey Monkey where you can get reports and export to excel. • 2. Microsoft Access: • Good for demographic data and tracking objectives and indicator completion – data that changes or needs to be cross-referenced. • Inexpensive, but requires expertise to develop functional applications • Easy to retrieve data through queries

More Tools you can use – now! 3. Daily Logs (paper or software) • Most useful if data is aggregated and entered into a database or spreadsheet regularly (daily, weekly or monthly) • Like surveys, the right questions have to be asked to get useful, accessible information • With proper planning, could be used to track a variety of participant achievements. 4. Exit interviews! • Build some of the questions to have aggregateable answers (e.g., multiple choice, name at least one xxx, etc.)

Implementation Questions? Please Contact: Doug Tanner dtanner@youthcatalytics.org 978-544-2067

Training Questions? Please Contact: Cindy Carraway-Wilson cwilson@youthcatalytics.org 203-561-6099

Getting to Outcomes: Next Steps