Completing the post KAP survey

1. Completing the post KAP survey

2. Purpose of post campaign KAP survey Assess the impact & success of your campaign and determine next steps

3. How do we achieve that? By measuring SMART objective attainment and exposure to campaign activities

4. What data are we looking for ? • Measure for each step of the results chain for each TA = SMART Objective Questions • Exposure • Comparability of data • Audience segmentation

5. Further objectives? • Collect baseline data • Understand the preferred media sources • Identify trusted sources • Ground truth assumptions • Identify Barriers/Benefits to BC.

6. Objective By the end of the day you will have: finalized your post campaign KAP survey questionnaire, sampling methodology & plan. revised and finalized your SMART objectives & result chains.

8. The steps

9. The steps • Step 1:Revise your sampling methodology and create a survey plan. • Step 2:Check your SMART objectives and mark them in your survey • Step 3:Revise your campaign survey in word • Step 4:Hide the questions in Survey-Pro • Step 5 & 6: Implement your survey and carry out data entry

10. Schedule for the day

11. Step 1:Revise your sampling methodology and create a survey plan. • Decide which groups you will sample for the post campaign KAP survey and revise your sampling methodology accordingly. • Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested survey length).

12. Decide which groups you will sample for the post campaign KAP survey and revise your sampling methodology accordingly • Keep primary audiences (fishers & community) • Other groups are optional • For each TA sampling methodology should be exactly the same as for pre-KAP

13. Sampling methodology template • Define geographic scope of survey and include a map showing area included in survey • Sample size calculation & sampling methodology • Survey plan

14. B. Sample size • Name of TA • Total n of TA (and who it includes with breakdown • Sample size (specify CL and CI) • Final sample size achieved

15. B. Sampling methodology • N of TA in each locality • N of samples per TA per location (stratified sampling) • Method of sampling:- for pre – KAP: amend to reflect reality- for post KAP – same as amended for Pre • If there are any differences between pre-and post please take note of these in the table

16. A. Define geographic scope of survey and include a map showing area included in survey

17. C. Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested survey length).

19. Step 2:Check your SMART objectives and mark them in your survey • CHECK THAT: • Smart objectives are aligned with your results chain • Smart objectives enable you to measure the success of your campaign • There are 1-2 obj. for each section of the ToC, for primary TA - link to your ideas for BC measurement from yesterday • Data in the smart objectives matches your pre-campaign results and the answer options of those results • Smart objectives targets are realistic (22K, 13A, 28IC, 14BC • Question numbering in the monitoring plan matches final survey

20. What’s the issues? Result chain (BR):Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities. Smart Objective: The % of Arbor fishers and gleaners that are able to give at least one benefit of establishing the Arbor/Granada Marine Sanctuaries will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

21. Correction Result chain (BR):Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities . Smart Objective: The % of Arbor fishers and gleaners that are able to give at least one benefit of establishing the Arbor/Granada Marine Sanctuaries Management Committee will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

22. Alternative Results chain (BR) There is a functional 24/7 MPA patrolling and guarding in place at Arbor and Grenada NTZ SMART objectiveThe % of Arbor fishers and gleaners who agree that there Arbor and Granada MPAs have a functional 24/7 guarding and patrolling system in place ("Yes, in Arbor but not in Granada" AND "Yes, in both Arbor and Granada") will increase from 23 measured in February 2011 to 33 measured in June 2012 (an increase of 10pp, Q29 in KAP survey)

23. WHAT’S WRONG HERE? 44% 36%

25. Step 3:Revise your campaign survey in word Make sure the survey is your latest version Make revisions in English version (cross check with vernacular) Use track changes

26. Guiding principles for shortening surveys • All questions should remain exactly the same (even if they had a mistake) • Questions and answer options should remain in the same order • Reduce survey to 30 minutes – max 40

27. Should be retained: • Socio-economic data - Section 2 • Questions you used during analysis for cross tabulations and filters (make sure you check which ones these were. i.e. sampling methodology, which category best describes you) • Compulsory cohort questions, selection to be retained (see provided list in Appendix 2) • Climate change questions • Questions needed for your SMART objective

28. Can be removed : • Trusted Sources of Information & Media Access/Exposure- Section 3 • Flagship sp question - Section 11 • Any questions that you no longer deem useful, including cohort questions, provided they are not part of the categories to be retained

29. To be added: • A question asking respondents whether they have taken this survey before • 3-5 answer options to the exposure question “I am going to ask you about a number of ways in which you may or may not have seen or heard about the XXX Sanctuary. For each method, I would like you to tell me whether you remember seeing or hearing that source in the past 6 months.” - Section 10Answer options should relate specifically to campaign events, activities or materials that you implemented over the last 6 month. Expl: “boat awnings”, “battle of the bands”, or “MPA blessing” retain at least two “dummies” as a measure of “false negative”

30. Finally please make sure Under “survey period” there is a “post campaign survey” answer option – Section 1

32. Step 4:Hide the questions in Survey-Pro • Open survey pro • Go to the “questionnaire” tab • Double click on the question you want to hide • Click on the “advanced options” tab • On the right select “Hide” for all answer options • The question will still appear in the survey but all answer options will be hidden • For grid questions:The process is the same as above but at n4 above click on “rows”.

33. Step 5 : Implement your survey

34. Choosing & Training enumerators • Enumerators “neutral” in the community • Should be committed to survey so there is not a high turnover • Should receive a day training & practice • Have clear instructions in the survey for enumerators

35. Training enumerators agenda • Intro to campaign => Motivation • Ethos / Neutral approachable attitude • Process (one on one; don’t share paper but interview)

36. Training enumerators - agenda • Run through questions and instructions one by one (translated version) • Keep tally of respondents who don’t want to be interviewed • Record any changes in sampling methodology • Have feedback system

37. Training enumerators - agenda • Go over survey plan Make sure enumerators understand sampling methodologyrandom sampling – (map + dice to select street?)stratified sampling – (create simplified version) • Provide additional materials needed (MAP for survey question, survey plan with MPAs of area, Community Based Monitoring System (CBMS) map from MPDO; ID) • Do a practice run & debrief

38. Instructions for enumerators • (55) Please name 3 creatures that live in the sea that you are particularly fond or proud of. (do not read out options but tick box as appropriate and/or fill in the blank for any additional answers) • FLAGSHIP SP QUESTION - 1st OPTION • [ ] grouper "lapu-lapu" [ ] giant clam [ ] giant sea fan [ ] XXX ADD OPTIONS AS APPROPRIATE [ ] XXX ADD OPTIONS AS APPROPRIATE [ ] Other ________________ [ ] N/A

39. Top Tip • Laminate survey questionnaire with hole and ring binder • Create answer sheet => Instructions for enumerators in both documents

40. Training enumerators • Do a pilot survey 3-5 per enumerator(test on people random sample in least important area first) • Do a debrief from pilot survey

41. Training enumerators • Agree on n of surveys per day and target • Have organizer within group • Safety protocol • At a minimum take part in first 2 days of surveying • Do data entry of pilot survey with encoders!

42. Step 6: carry out data entry

43. KAP Survey TOP TIPS • Encoding • Dry run • Change the structure of the open ended questions before encoding • Encoders who are willing to work evening/night shifts. Don’t try and do all the encoding yourself: get some help! • Be strategic about the order in which you encode (and also survey) the target audiences start by encoding fishers and the local community (TA1 &TA2) • Write the corresponding form number in Survey pro on the paper survey

44. Data entry • Bottleneck! Start early • Save every form • System for keeping paper copy => BACK UP every day!

45. Good luck!

46. Diskarte