2018 C4CD Plus Pilot for 0-3 Children Impact Evaluation

2018 C4CD Plus Pilot for 0-3 ChildrenImpact Evaluation • 14 December 2018

Overview C4CD Plus 0-3 Components Impact Evaluation

Background • Sustainable Development Goal 4.2 • “By 2030, ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education.” Indicator 4.2.1: Proportion of children under 5 years of age who are developmentally on track in health, learning and psychosocial well-being, by sex Indicator 4.2.2: Participation rate in organized learning (one year before the official primary entry age), by sex https://developingchild.harvard.edu/resources/inbrief-science-of-ecd/ C4CD Plus 0-3 Components Impact Evaluation

Background • “There is therefore an urgent need to increase multi-sectoral coverage of quality programming that incorporates health, nutrition, security and safety, responsive caregiving, and early learning.” • Research calls for multi-sectoral solution to improve child development Source: Lancet Series 2016 C4CD Plus 0-3 Components Impact Evaluation

Methodology C4CD Plus 0-3 Components Impact Evaluation

Methodology • Experimental design for the quantitative research study to understand “what” and“how large” the impacts of the program were. • Qualitative research to help understand “how” and “why” the program worked • Mixed-methods approach C4CD Plus 0-3 Components Impact Evaluation

Methodology • Did the C4CD Plus program have an effect on young children’s development? • Does the C4CD Plus program have an effect on the home learning environment for young children? • Does the C4CD Plus program have an effect on caregivers’ interactions with their children? • Research Questions C4CD Plus 0-3 Components Impact Evaluation

Methodology • Research design: quantitative C4CD Plus 0-3 Components Impact Evaluation

Methodology • Quantitative sample C4CD Plus 0-3 Components Impact Evaluation

Methodology • Caregiver Reported Early Development Instruments • Quantitative assessment tool: CREDI • Motor • E.g., “Can the child walk several steps on his/her own, without holding on or receiving support?” • Cognitive • E.g. “Can the child use a tool (e.g., a stick or spoon) to reach objects that are far away?” • Mental Health • E.g. “Does the child often kick, bite, or hit other children or adults?“ • Language • “Does the child try to repeat sounds or words said by other people? “ • Social-Emotional • E.g. “Does the child smile when others smile at him/her?” C4CD Plus 0-3 Components Impact Evaluation

Methodology • Quantitative assessment tool: CREDI Does the child look at an object when someone says "look!" and points to it? Motor Social Emotional Cognitive Language C4CD Plus 0-3 Components Impact Evaluation

Methodology • Quantitative assessment tool: Caregiver questionnaire C4CD Plus 0-3 Components Impact Evaluation

Methodology • Qualitative C4CD Plus 0-3 Components Impact Evaluation

Methodology • Timeline of data collection C4CD Plus 0-3 Components Impact Evaluation

Results • Child developmental outcomes (CREDI) C4CD Plus 0-3 Components Impact Evaluation

Results • Understanding distributions C4CD Plus 0-3 Components Impact Evaluation

Results • Understanding distributions More children Fewer children Lower score Higher score C4CD Plus 0-3 Components Impact Evaluation

Results • Understanding distributions Many children score in the middle A few children score very low A few children score very high C4CD Plus 0-3 Components Impact Evaluation

The CREDI Z-Score compares children to a “reference group” of children with educated parents and 4+ learning activities at home Results • CREDI Z-scores Zero is the “average” score in the CREDI reference group Negative scores are below the reference group Positive scores are above the reference group C4CD Plus 0-3 Components Impact Evaluation

Results • Distributions of CREDI at baseline and endline C4CD Plus 0-3 Components Impact Evaluation

Results • Distributions of domain scores at endline C4CD Plus 0-3 Components Impact Evaluation

Results Small effect Medium effect Large effect • Effect sizes (Cohen’s d) C4CD Plus 0-3 Components Impact Evaluation

Results • Caregivers and Health Assistants see a change in the way that children behave and communicate • Caregivers recognize that these changes are important for the children’s later pre-primary or primary schooling • Impressions of impact on children • “[The children] become more open, friendly, mingle with other children also talk, respond to what we ask.” • -Health Assistant • “Holding pencil and drawing is useful because later in the future when my child goes to attend school he will know how to write. This is very important.” • -Caregiver • “Our children learn so much and they can be prepared for PP class when they join school.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Home Learning Environment and Caregiver practices C4CD Plus 0-3 Components Impact Evaluation

Caregivers in the treatment group had significantly increased learning and play activities with children. Results • Caregivers in the treatment group reported having significantly more storybooks and homemade toys • Home Learning Environment and Caregiver practices • Caregivers in the treatment group were less likely to report spanking their children C4CD Plus 0-3 Components Impact Evaluation

Results • Home Learning Environment: Reading materials * = p < .05; ** = p < .01; *** = p < .001 C4CD Plus 0-3 Components Impact Evaluation

Results • Caregivers and Health Assistants had changed attitudes regarding the use of reading materials with children • Caregivers recognize that children can love reading and how using books can introduce them to new concepts • Impressions of reading materials • “We feel that if we give books to children they will tear it up and spoil the book but this is not true…if we give [them] book[s] then they will develop the reading habit from an early age. ” • -Heath Assistant • “Children no matter what age love to read and enjoy books. So we have to introduce book reading to children. This will help them to know, pictures, colors, and it also help them later when they start school.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Home Learning Environment: Play objects * = p < .05; ** = p < .01; *** = p < .001 C4CD Plus 0-3 Components Impact Evaluation

Results • Caregivers using lessons from the sessions and playing with new types of toys with their children. • Impressions of play objects • “I play activities such as sorting leaves, pebbles and games with household objects and mostly in the afternoon.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Learning activities in the home * = p < .05; ** = p < .01; *** = p < .001 C4CD Plus 0-3 Components Impact Evaluation

Results • HAs reported observing improved practices in caregivers, especially around reading and book activities • Caregivers report applying many of the lessons in their daily lives, resulting in increased learning interactions, talk, and play between caregivers and children • Learning activities in the home • “We have seen [caregivers] hugging their child, speaking softly, increased interactions, genuinely interested in reading books.” • -Heath Assistant • “I try my best to go through the books and also make sure that I keep talking and also answer when child ask me something.” • -Caregiver • “[I like] showing things to children like ‘This is a stone, this is a flower’ they feel happy and we also enjoy taking children outside and showing things to our children.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Negative and positive discipline * = p < .05; ** = p < .01; *** = p < .001 C4CD Plus 0-3 Components Impact Evaluation

Results • Caregivers voice recognition of the importance of avoiding physical and harsh discipline • Leaning about how and why children cry provides context and helps caregivers cope with frustration. • Despite recognition, caregivers admit that it is difficult to change their behaviors • Caregivers’ impressions of use of discipline • “[I] talk softly, explain to our child but not hit hard” • -Caregiver • “In my opinion if a child is crying and crying we do not know what could be the reason as we are also tired and get frustrated and sometime we even resort to hitting the child which I think is not proper… So [no matter] how much frustration we have we have to take long breath, be cool and address the needs of the child. This is I find very useful.” • -Caregiver • “[My children are] very naughty and restless. I still beat them sometimes.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Strengths and challenges C4CD Plus 0-3 Components Impact Evaluation

Results • The focus of the program on demonstrated concrete activities • Increasing the knowledge of developmental milestones provided a useful reference for caregivers • Strengths of the program • “In other health programs we go to the community we give some health talk. In C4CD plus we introduce the sessions and the four games, practice the activities with the parents and the parents have time to play the games with their children.” • -Health Assistant • “Because of the lack of knowledge about what children can do at age specific time parents do not know so when we started asking age-specific activities about [what] the child can do, then the parents were aware and since we had counselling ideas for parents if their children cannot do age appropriate activity it helped us immensely.” • -Health Assistant C4CD Plus 0-3 Components Impact Evaluation

Results • Monitoring and coaching with the Health Assistant provided a key source of support • Caregivers thought sessions were delivered effectively and enthusiastically and were appreciative of the HA attention • “[M]onitoringis good and important because if we are told to do this but we cannot do well and there is no help or monitoring it is of no use. We will have nobody to ask if we have doubts.” • -Health Assistant • “Monitoring is important. DHO should conduct monitoring and collect reports on time.” • -District Health Officer • “If we did not understand they would explain again.” • -Caregiver • Strengths of the program C4CD Plus 0-3 Components Impact Evaluation

Results • Favorite sessions • Caregivers were most enthusiastic about sessions that involved new ways of playing with young children, especially outside • “I like playing with cups, my child understands how to knock things (cups/blocks) down and build it back. She enjoys when it is knocked down.” • -Caregiver C4CD Plus 0-3 Components Impact Evaluation

Results • Health Assistants reported that the impact of the program was not limited to attendees • Once caregivers learned the lessons, they spread them throughout the community • Caregivers also reported positive effects on their own happiness • “[We] apply the knowledge we got from the training to other children in the community” • -Health Assistant • “[W]hen we see people/mothers handling children in not very friendly method we tried to intervene. Initially few mothers got annoyed… later they realize and appreciate our suggestions. • -Caregiver • “[W]hen we play with them, they are happy and we also feel happy” • -Caregiver • Positive spillovers C4CD Plus 0-3 Components Impact Evaluation

Results • Staffing was a key issue, with many workers overburdened • Lack of awareness of the importance of the program was a barrier some HAs noted • Heterogeneous groups can add difficulties when levels of understanding vary • “[A challenge is a] single district health person in the office also responsible for other health related tasks.” • -District Health Officer • “Attendance was good in the beginning. Later we had to emphasis / awareness on the importance of the program and so they started to come. We had to force them to come also.” • -Health Assistant • “Some business women were educated. So when I try to make simple this group would not show interest. If taught in high profile there is problem in understanding [among the others].” • -Health Assistant • Challenges C4CD Plus 0-3 Components Impact Evaluation

Results • Suggestions and Recommendations C4CD Plus 0-3 Components Impact Evaluation

Results • Strong support for increasing the reach of the program • Lack of equity is a motivation to implement in other areas • Need for scaling • “All children should get equal opportunity.” • -District Health Officer • “[Some community members were] angry with us and asking why we are not doing this for them • -Health Assistant • “Most importantly it should be given to others remote parts of Bhutan because research has shown the difference child development rural and urban areas.” • -Health Assistant C4CD Plus 0-3 Components Impact Evaluation

Results • Integration with other programs might be a possible solution, and potentially the EPI (Expanded Program Immunization) • DHOs suggestions • Suggestions for improvements • “In future I suggest that integrate as component in the Royal Institute of Health Sciences (RIHS) curriculum and train each and every health worker who are young and can deliver this program. • -Health Assistant • Refreshments to encourage/award caregiver attendance • Grouping according to the age to have a more focused discussion • Health workers are better to facilitate, but can also use ECCD workers and teachers in remote areas C4CD Plus 0-3 Components Impact Evaluation

Results • Health Assistant recommendations • Suggestions for improvements • Working to include fathers • Allowing the caregivers in a community to fix the time • Splitting into language-specific groups to avoid language barriers • Limiting the groups to 6 parents each – and extending manpower to reach them • Simplify some of the songs • Making the sharp home cards into a safe and convenient booklet • Consider providing toys to attract participation C4CD Plus 0-3 Components Impact Evaluation

Results • Caregivers’ recommendations • Suggestions for improvements • More consistent HA communication between sessions • Make more time for the sessions, as often it feels in a hurry • Create cards for the sessions on health and nutrition as well as protection and safety • Awareness raising activities with communities about the purpose of the program • Combining sessions with the immunization schedule • Repeat sessions C4CD Plus 0-3 Components Impact Evaluation

Discussion C4CD Plus 0-3 Components Impact Evaluation

Discussion • Summary of results C4CD Plus 0-3 Components Impact Evaluation

2018 C4CD Plus Pilot for 0-3 Children Impact Evaluation