PLANNING and EVALUATION: Portfolio Assessment for CSREES Strategic Objectives

1. PLANNING and EVALUATION:Portfolio Assessment for CSREES Strategic Objectives Jeffrey Gilmore Director Higher Education Programs

2. Private Industry Collaborating to Solve National Problems in the U.S. Federal Government • Hammons Products • American Forest and • Paper Association • Forest Resources Association • Professional societies (TWS, SRM,SAF) • University organizations (NASULGC, NAPFSC, NAUFWP) • Conservation organizations (TNC, NACD,AFF) NGO’s Research Institutes & Universities • 1862, 1890, 1994 Land Grant Universities • Non Land Grant colleges and universities • Hispanic-Serving Institutions • Alaska Native and Native Hawaiian-Serving Institutions

3. US EPA CSREES and USEPA are collaborating on indoor air quality DOD NSF CSREES and FS are collaborating on the National Agrogorestry Center at Alabama A&M CSREES and NSF are collaborating on Microbial Observatories DOE NASA CSREES, NRCS, FS and DOE are collaborating in biomass & fuel processing waste for energy and products CSREES and NASA are collaborating and funding the Geospatial Extension Program USDA Collaborations with Other Federal Entities USDA

4. USDA and Its Seven Mission Areas USDA Farm and Foreign Agricultural Service Food Safety Rural Development Research, Education & Economics Natural Resources & Environment Marketing and Regulatory Food, Nutrition & Consumer Service

5. The Research, Education and Economics Mission Area Economic Research Service (ERS) Cooperative State Research Edu. & Ext. Service (CSREES) National Agricultural Statistics Service (NASS) Agricultural Research Service (ARS) National Agricultural Library (NAL)

6. CSREES Emphasis Areas Food, Nutrition & Health Agricultural and Food Biosecurity Plants and Plant Products Biotechnology & Genomics Natural Resources & Environment Agricultural Systems Economics & Commerce Technology and Engineering Pest Management Animals and Animal Products Families, Youth, & Communities

7. The Agency • CSREES is a federal assistance agency. • CSREES - <400 people, $1.2 billion. • University System, mainly land grant: • >130 colleges of agriculture/natural resources • 60 agricultural experiment stations • 58 cooperative extension services • 65 schools of forestry • eighteen 1890 institutions • thirty three 1994 institutions

8. Portfolio Review • Significant accomplishments have been achieved. • The work of our partners = the work of CSREES. • Impacts/Outcomes of accomplishments now being realized…short, medium & long term

9. Time Period • The Portfolio timeframe (1999-2003) reflects work that occurred in 1997-2001. • Now 6 months into FY 2005. • FY2006 President’s Budget released on February 7, 2005. • The budget planning process for the FY2007 President’s Budget is underway.

10. Current Priorities not in Review Cycle • Agro-security • Obesity • Nanotechnology • Water security • Air quality • e-Extension • Higher education reporting into CRIS

11. CSREES Goal 5 Portfolio 5.1 Goal 5: Protect & Enhance the Nation’s Natural Resource Base & Environment Portfolio 5.2

12. PA 124: Urban Forestry PA 135: Aquatic & Terrestrial Wildlife PA 125: Agroforestry PA 121: Mgmt of Range Resources PA 123: Mgmt & Sustainability of Forest Resources PA 136: Conservation of Biological Diversity PA 122: Mgmt & Control of Forest & Range Fires Portfolio 5.1 Forests, Range, Fish and Wildlife by Problem Areas Portfolio 5.1

13. Portfolio 5.2: Management of Soil, Air, and Water by Problem Areas PA 605 Natural Resource Economics PA 101 – 104 Soil resources PA 131 Alternate Uses Of Land PA 133 Pollution Prevention & Mitigation PA 141 Air Resource Conservation & Management Portfolio 5.2 PA 111, 112, & 405 Water Resources PA 132 Weather and Climate PA 403 Waste Disposal, Recycling, & Reuse

14. CSREES Inputs to Portfolio • Funding to partners • Formula funds • Competitively awarded funds • Congressionally directed funds

15. CSREES Inputs to Portfolio • Human resources • National Program Leaders • Program Specialists and Analysts • Administrative Support Staff

16. CSREES NPL Functions • Program leadership • Serve as a technical and disciplinary expert. • Network and collaborate to identify mission-relevant goals. • Initiate and facilitate networks, liaisons, and collaborations to identify mission-relevant goals • Promote a shared and integrated agenda. • Mobilize rapid response to immediate issues. • Conceptualize, formulate and execute programs and activities to respond to emerging issues and opportunities.

17. CSREES NPL Functions • Program management • Conceptualize, formulate and direct program policy, technical standards, and guidelines. • Develop and oversee the implementation of evaluation strategies. • Prepare requests for applications. • Remove barriers to program efficiency and effectiveness. • Allocate scarce resources.

18. The Scope of the Portfolio:Partners, Integration, and Programs

21. Other CSREES Governmental Partners DOC EPA State Environ. Protection Dept. DOI State Agriculture Depts. CSREES NRCS County Extension Councils ARS County Conservation Districts Forest Service

22. CSREES Non-government Partners Non-Profit Environ. & natural resources orgs. Professional Societies NASULGC CSREES Commodity Groups Small Business Industry For-Profit

23. Who Benefits From Our Work?Our Stakeholders -- • Historical focus on agricultural & rural landowners, citizens, businesses & communities. • Today, all Americans who utilize or benefit from sound agricultural, natural resources & environmental management. • Members of The Partnership. • Secondary, college & university students. • Socially disadvantaged citizens.

24. CSREES Functions & Integration • Integration of Research, Education & Extension is an overall CSREES goal. • Many funding lines are for specific mission elements. • Portfolio supported largely by formula and competitively awarded funds. • Some units focus on single agency functions (e.g. Competitive Programs). • Other program units (e.g., NRE, PAS, SERD) integrate multiple functions (research, teaching, and extension). • Trend toward greater NPL functional fluidity & inter-Unit cooperation.

25. CSREES Problem Areas (PAs) • 84 R, E & E “programs” described in 5 goals of CSREES 2004-2009 Strategic Plan. • Derived from USDA Current Research Information System (CRIS): 80 numerical Research Problem Areas (RPAs). • System used for data organizational & retrieval purposes. • PA concept is an attempt to link CSREES strategic plan to activities.

26. Problem Areas (cont’d) • PAs do not directly reflect organizational structure or staffing. • CSREES doesn’t staff or plan using PAs. • PAs (and CRIS RPAs) expected to “evolve” over time as issues change. • PAs will stabilize and provide a consistent arrangement and connection to strategic plans and implementation of activities.

27. Logic Model, Accomplishments, and OMB Criteria

28. Logic Model • A program leader’s model for the program theory. • Illustrates the relationships between the components of the strategy for the portfolio. • Specifies potential cause and effect relationships. • Serves as the framework for developing a program evaluation plan. • Links the components with intended outcomes. Ref: E. Norland, US EPA.

29. Generic Logic Model Program Development Planning-Implementation-Evaluation Program Action- Logic Model Inputs Outputs Activities Participation Outcomes-Impact Short term Medium Term Long Term What we invest Staff Volunteers Time Money Research base Materials Equipment Technology Partners Conduct workshops, meetings Deliver services Develop products, curriculum, resources Train Provide counseling Assess Facilitate Partner Work with Media What we do Who we reach Participants Clients Agencies Decision-makers Customers Satisfaction What the short term results are Learning Awareness Knowledge Attitudes Skills Opinions Aspirations Motivations What the medium term results are Action Behavior Practice Decision-making Policies Social Action What the ultimate impact(s) is Conditions Social Economic Civic Environmental Priorities: Consider: Mission Vision Values Mandates Resources Local Dynamics Collaborators Competitors Intended Outcomes Situation Needs and Assets Symptoms versus problems Stakeholder engagement Assumptions External Factors Evaluation Focus - Collect Data – Analyze and Interpret - Report

30. Researchers Practitioners Educators Paraprofessionals Producers • Proposals: • Submitted • Reviewed • Accepted for • funding • Awarded • Implemented Target Audience Financial Resources Situation Inputs Outputs Human Resources Research Logic Model Integrating Research, Extension, and Education Extension Outcomes Outreach Outcomes Education Long Short Medium Change Learning Action Assumptions External Factors Source: Planning and Accountability, 2004

31. Portfolio Review –What It Encompasses All Functions • Research • Education • Extension Across All Budget Lines • Smith-Lever Act • Hatch Act • Evans-Allen • RREA • McIntire-Stennis Cooperative Forestry Research • National Research Initiative • Small Business Innovation Research • Higher Education Programs • Congressionally-Directed Projects • Etc.

32. Portfolio Accomplishments • New knowledge and discoveries from research activities provide new management practices. • Extension programs provide the technology transfer to enable adoption of new practices. • Higher education programs accelerate the development of new curriculum and the training of new scientists and food, agriculture, and natural resources professionals.

33. What does this mean? • The “work” of the agency is the work of the partners. • CSREES facilitates that work through numerous funding mechanisms. • Assessing the accomplishments of the portfolio involves examining the work of the partners and the agency leadership and guidance that accompanies the funding.

34. Assessing Programs for the Office of Management and Budget Relevance (scope – focus – emerging issues – integration – balance): • Are the problems being addressed the relevant issues of the day? • Are they being addressed in a broad, interdisciplinary manner?

35. Assessing Programs for the Office of Management and Budget Quality (outputs and findings – assessment – alignment – methodologies): • Are the outputs and findings significant? • Do stakeholders participate in assessing the work and have opportunities to provide input? • Are appropriate methodologies used in carrying out the research, education and extension work?

36. Assessing Programs for the Office of Management and Budget Performance (productivity – completeness – timeliness – guidelines – accountability) • Is productive work being accomplished: • New discoveries? • Changes in attitudes, knowledge practices? • Competent faculty and workforce ready graduates? • Does work get done in a timely manner? • Does the agency provide sufficient guidance in order for productive accomplishments to be achieved?

37. Budget and Performance Integration

38. New Directions and Expanded Efforts: Program Management • Strategic Planning/Directions - most programs have no strategic plans • Program Accountability: • Budget and performance integration: • Department and agency goals. • Performance criteria – measures, indicators, baseline values and targets for out-years. • Logic model.

39. Researchers Practitioners Educators Paraprofessionals Producers • Proposals: • Submitted • Reviewed • Accepted for • funding • Awarded • Implemented Target Audience Financial Resources Situation Inputs Outputs Human Resources Research Logic Model Integrating Research, Extension and Education Extension Outcomes Outreach Outcomes Education Long Short Medium Change Learning Action Assumptions External Factors Source: Planning and Accountability, 2004

40. Example of Performance Integration

41. Example of an Activity Data Matrix Problem Area 124: Urban Forestry

42. Summary and Discussion

43. Summary • Process is useful and enlightening for National Program Leaders. • What we know. • What we don’t know. • What we thought we knew but had not compiled the data.

44. Summary • Reviews have utility not only for the Office of Management and Budget but also for the Agency and National Program Leaders. • Program direction – RFA’s; strategic planning; joint planning • Program integration – strategies for which, when and how • Planning and reporting – what is reasonable and affordable

45. Summary • Accountability will continue to be emphasized. • Which programs are making a difference and which are not? • Budget decisions are being made based on program performance. • Budget and program integration (BPI).

46. Summary • We need to negotiate and collaborate with our partners to identify performance indicators that are relevant, reportable and meaningful.

47. Thank-You!!! • CSREES extends our thanks and appreciation for the work that you are undertaking and the outcomes that it will produce.

48. Questions???