Develop an Evaluation Plan For Health Promotion

Developing an Evaluation plan for health promotion - using an example for mental health

What is an evaluation plan ? An evaluation plan is similar to a roadmap. It clarifies the steps needed to assess the processes and outcomes of a program. An effective evaluation plan is more than a column of indicators added to your program's work plan.. The evaluation plan clarifies how you will describe the “What,” the “How,” and the “Why It Matters” for your program. The “What” reflects the description of your program and how its activities are linked with the intended effects. It serves to clarify the program’s purpose and anticipated outcomes. The “How” addresses the process for implementing a program and provides information about whether the program is operating with fidelity to the program’s design. Additionally, the “How” (or process evaluation), along with output and/or short-term outcome information, helps clarify if changes should be made during implementation. The “Why It Matters” provides the rationale for your program and the impact it has on public health. This is also sometimes referred to as the “so what” question. Being able to demonstrate that your program has made a difference is critical to program

Need for an evaluation plan ... ● ● A written evaluation plan can— ƒ create a shared understanding of the purpose(s), use, and users of the evaluation results, ƒ foster program transparency to stakeholders and decision makers, ƒ increase buy-in and acceptance of methods, ƒ connect multiple evaluation activities—this is especially useful when a program employs different contractors or contracts, ƒ serve as an advocacy tool for evaluation resources based on negotiated priorities and established stakeholder and decision maker information needs, ƒ help to identify whether there are sufficient program resources and time to accomplish desired evaluation activities and answer prioritized evaluation questions, ƒ assist in facilitating a smoother transition when there is staff turnover, ƒ facilitate evaluation capacity building among partners and stakeholders stakeholders, provide a multi-year comprehensive document that makes explicit everything from stakeholders to dissemination to use of results, and ƒ facilitate good evaluation practice. ● ● ● ● ● ● ● ● ● ● ● ● ●

Writing an evaluation plan …. The elements of an evaluation plan include: ƒ Title page: Contains an easily identifiable program name, dates covered, and basic focus of the evaluation. ƒ Intended use and users: Fosters transparency about the purpose(s) of the evaluation and identifies who will have access to evaluation results. It is important to build a market for evaluation results from the beginning. Clarifying the primary intended users, the members of the stakeholder evaluation workgroup, and the purpose(s) of the evaluation will help to build this market. ƒ Program description: Provides the opportunity for building a shared understanding of the theory of change driving the program. This section often includes a logic model and a description of the stage of development of the program in addition to a narrative description. ƒ

Evaluation focus: Provides the opportunity to document how the evaluation focus will be narrowed and the rationale for the prioritization process. Given that there are never enough resources or time to answer every evaluation question, it is critical to work collaboratively to prioritize the evaluation based on a shared understanding of the theory of change identified in the logic model, the stage of development of the program, the intended uses of the evaluation, as well as feasibility issues. This section should delineate the criteria for evaluation prioritization and include a discussion of feasibility and efficiency. Methods: Identifies evaluation indicators and performance measures, data sources and methods, as well as roles and responsibilities. This section provides a clear description of how the evaluation will be implemented to ensure credibility of evaluation information. ƒ Analysis and interpretation plan: Clarifies how information will be analyzed and describes the process for interpretation of results. This section describes who will get to see interim results, whether there will be a stakeholder interpretation meeting or meetings, and methods that will be used to analyze the data.

ƒ Use, dissemination, and sharing plan: Describes plans for use of evaluation results and dissemination of evaluation findings. Clear, specific plans for evaluation use should be discussed from the beginning. This section should include a broad overview of how findings are to be used as well as more detailed information about the intended modes and methods for sharing results with stakeholders. This is a critical but often neglected section of the evaluation plan. ●

Sample evaluation plan for a Mental Health programme aimed at Indigenous mothers ... The project aims to evaluate the process as well as the impact of mental health awareness and uptake of health programs. The process evaluation ● will include the assessment of the type, quantity and quality of the program activities. The impact evaluation will involve the collection of data around the increased usage of health facilities awareness by indigenous mothers and improved mental health awareness of Indigenous mothers in a small rural village of Australia .The following evaluation questions formulated reflecting the objectives framed in the evaluation plan. What is the impact of the program on the Indigenous mother community ■ What are the perceived barriers in the uptake of the program ■ How many sessions of awareness, fun sessions were conducted in a month ■ How many Indigenous mothers attended the awareness sessions . ■ Whether the services are executed in a planned manner or not ■

Methodology and Data collection… ● The process evaluation can be done by survey and examining the records while impact evaluation can be done through focus group discussion and in-depth interview. Within the outer eastern Melbourne demographic, there is a population of disadvantaged mothers which is selected as the participant's population. Qualitative - Qualitative techniques, such as focus group discussions and in-depth interviews, can help highlight the impact of ● mental health education on Indigenous mothers. Participants' thoughts and opinions concerning the current health care services supplied to the disadvantaged mother population in the outer eastern Melbourne demography were the focus of the discussions. Exploration was carried out around the following thematic areas Obstacles and deficiencies that need additional re-design with community participation to determine mother’s perceptions ● Perceptions of the feasibility of the health service ● Perceptions of options for improving health services response ●

Data collection continued…. Quantitative data collection - This method will employ the use of data collection instruments via survey and routine records. Surveys given to participants can be used to gather data on reach as well as process variables like satisfaction with activity presenters and content. Questions on satisfaction with informative resource dissemination, as well as the informational resources themselves, can be included in surveys. The surveys are essential in determining whether the program is running in a planned manner or not. Further, a Follow-up survey should be conducted with Indigenous mothers to see if they used the referrals they were given (e.g., sought help from a therapist). Surveys can be conducted over the phone, by mail, or online. Information gathered from attendance sheets, logbooks, case records, and other similar documents is analyzed. The activity reach is a significant aspect in the intervention i.e. how extensively the activity was distributed, and which Pape mothers took part in it. Data on reach may be readily available for some types of interventions through administrative records—data frequently gathered for administrative or billing purposes. It may be required to obtain reach data for different types of interventions. For different dimensions of reach, different data collection strategies will be used. Logs can assist you to figure out how many times a session of health education has been delivered. Pre-registration or sign-in sheets are useful for determining the number of intervention activity recipients. If participant’s information cannot be included on registration materials or sign-in sheets, surveys can be used to determine the number of persons.

Outcomes… ● The majority of the data collected for the outcomes will be quantitative and will come from surveys or existing records. The programs aim to improve pape mother's knowledge, attitudes, abilities, and behaviors in general. The assessment program outlined outcomes that might be examined for variables of mental health programs—those targeted at decreasing stigma and discrimination while also promoting mental health. The analysis and interpretation will include the use of atlas TI for the FGDs and the interviews. Also, ● statistical methods will be used to analyse the survey data and the records.

Dissemination of findings The evaluation findings will be used to make changes to the existing health program being given ● out for mental health awareness and screening in Indigenous mothers and improve it in delivering mental health care to the disadvantaged group of population. Intentional communication and dissemination plan should be included in your evaluation plan. As previously stated, the planning stage is the time for the program to address the best way to share the lessons you will learn from the evaluation. The communication-dissemination phase of the evaluation is a two-way process designed to support use of the evaluation results for program improvement and decision making. In order to achieve this outcome, a program must translate evaluation results into practical applications and must systematically distribute the information or knowledge through a variety of audience-specific strategies ●

Justification of the methodology Choosing an appropriate design to address evaluation questions is an ethical, moral, ideological, and political effort, not a technical one. The research design’s interactive model states that the evaluation question is central to the research design. Choosing one data collection method over another, on the other hand, limits the researcher's capacity to gain from the capabilities of a variety of research methodologies. The qualitative data collection method as applied to health research seeks to comprehend behavioral patterns and how these patterns may affect and interact with health and health-seeking behaviors (Austin & Sutton, 2014). They may help develop recommendations that are suited to certain social situations and/or groups of people by identifying priorities and requirements that are relevant to these contexts and/or groups of people. The majority of the methodologies utilized in this study were qualitative, although another approach such as quantitative is also used in the plan to triangulate the results.

● For evaluation questions 1, and 2, a descriptive and interpretive situation analysis was conducted using a range of qualitative methods such FGDs and In-depth interviews. For evaluation questions, 3,4,5 qualitative findings of service uptake were triangulated by collecting quantitative data through a survey and examining the records. ● Mixed methods may be utilized in a way that supports or expands on qualitative data and effectively deepens the description, but qualitative data gathering methods and analysis are mostly used. As a result, the study largely employs qualitative approaches, but it also intended to use quantitative approaches to correlate participant impressions with data that would help us better understand both the potential for gathering data on the magnitude and its constraints. Research on mental instability and mental health variables in Indigenous mother community group members fits in the ● conceptualization of sensitive research. Qualitative data collection instrument is more usual since it is more interactive, personal, contextual, open, and phenomenological. It is also the best way for examining sensitive problems or for researchers who want to capture the complexities of sensitive everyday life to comprehend the viewpoints of individuals who live it.

Develop an Evaluation Plan For Health Promotion