Creating Strong Reports

1. Creating Strong Reports New Mexico AmeriCorps April 20, 2006 Sue Hyatt, Project STAR Coach

2. Agenda • Learn some “do’s and don’ts” forperformance measures (PM’s) in the APR. • Reporting Tips • Review of Your Most Recent Report

3. Project STAR Findings:Common PM and ReportingIssues • Outputs versus Outcomes • Intermediate versus End Outcomes • “Creative Writing” - Indicators and Targets • Targets Not Related to Instruments • Weak Community Need Statements

4. REVIEW: Types of Performance Measures • Outputs • Intermediate Outcomes • End Outcomes

5. Are counts of the amount of services delivered, work, or products created that members or volunteers have completed. Usually number of people served Do not answer the question, “what changed as a result of our work?” Do not provide information on benefits or other changes in the lives of beneficiaries. Examples: 30 students tutored in reading 100 tress planted 25 houses built benefiting 25 families Outputs

6. Answer the question, “So what?” Changes that have occurred in the lives of beneficiaries that are significant Examples: Students will not engage in drug or alcohol use Increasing homeownership among clients Students demonstrate better in school behavior End Outcomes

7. Intermediate Outcomes • Answer the question, “what difference did it make”? • Do not represent the final result you hope to achieve for beneficiaries • Likely preconditions for longer-term gains • When positive are usually a sign that your project is on track to achieve the related end outcomes. Examples: Students report feeling safe and not pressured to engage in risky behavior. Decrease in volunteer attrition. Students will complete homework assignments.

8. Possible Beneficiary Changes • Outputs: Number of People Served • Outcomes: How People Changed • Behavior or Skills • Attitudes • Knowledge

9. What is an Indicator? • Information you collect to determineprogress toward achieving results • Concrete and measurable • Answers the question, “What is myevidencethat something has occurred?” • Output indicator usually begins with “Number of...” • Outcome indicator usually begins with “Percent of...”

10. Internal Alignment: End Outcome Members will provide 1 on 1 reading tutoring to third graders at least 3 times a week for 20 minutes/session • End Outcome Result: Students have improved proficiency in reading • Indicator: Percentage of students tutored who have improved proficiency in reading • Target: 75% (263 of 350) students tutored will have improved proficiency in reading • Instrument: Reading Grades

11. Internal Alignment: End Outcome (NOT!) Members will provide 1 on 1 reading tutoring to third graders at least 3 times a week for 20 minutes/session • End Outcome Result: Students have improved proficiency in reading • Indicator: Teachers will report that 75% students have completed more reading homework. • Target: Students tutored will show improved behavior in the classroom and be sent to the Principal’s office less often. • Instrument: logs and surveys

12. APR Do’s and Don’ts:Do create performance measurement plans

13. What is a Performance Measurement Plan? • Written description of steps you will take to obtain data to measure outputs and outcomes • The PM plan specifies: • Who will do each step • When they will do it • Also known as data collection plan • PM plan is instrument-focused • The PM plan is your roadmap during the program year. It helps you obtain reliable, useful information about the effect of your program on beneficiaries.

14. Performance Measurement Plans (continued)

15. Performance Measurement Plans (continued)

16. APR Do’s and Don’ts:Do include strong descriptions of service activities

17. APR Do’s and Don’ts:Do clearly identify instruments

18. APR Do’s and Don’ts:Do talk about change in outcome results

19. APR Do’s and Don’ts:Don’t get carried away with output codes

20. Why Do Progress Reports? • Enhances program accountability • Opportunity for reflection and strategic thinking—see the forest for the trees • Helps you become a “great communicator”—the more youdo it, the better you get at it. • Program staff turnover: Progress report is a permanent document. • Helpful in identifying TA needs.

21. Entire “needy population” Your beneficiaries Beneficiaries you measured “Successful” beneficiaries Calculating Targets: Who Should I Report On?

22. “Successful” Beneficiaries You Measured 287 67% 428 Beneficiaries You Measured Example of Target Calculation Target: Sixty percent of neighborhood residents will report an increase in recycling.

23. Completing theAPR Progress Report

24. Well-written Progress Reports Answer These Questions: • How much work is complete? • What work is currently in progress? • What work hasn’t started yet? • What challenges have arisen? • What steps are being taken to address these challenges? These same questions apply to reporting on performance measurement!

25. Describing Your Work Before Results Are Available “...We don’t have any results yet.”

26. Describing Your Work Before Results Are Available • Report with respect to: • Developing logic models and indicators • Refining data collection plans • Data access/confidentiality (human subjects) • Developing and pilot testing instruments • Training data collectors

27. Describing Your Results • Tell the reader: • How and when data were collected • How data were analyzed • Account for any data that were inaccessible or unusable. • Include numbers, not just percentages. • Don’t delay analyzing and reporting baseline data!

28. Describing Challenges & Solutions • Worst thing to report: “No challenges” • Explain how challenges affect program operation. • Report steps taken to address challenges. • Always have a backup plan (in case your proposed solution doesn’t work) • Bring past challenges to closure

29. Exercise • Review Your Recent Progress Report and Note What You Want to Remember to Upgrade Next Time You Do a Report

30. Additional Resources: • Contact Project STAR • http://nationalserviceresources.org//sites/star/ • 1-800-548-3656 • star@aiweb.com