Sigried Caspar

1. Sigried Caspar • European Commission, DG Employment & SocialAffairs • Moderator

2. Occurrence est certifiée ISO 9001 depuis 2004 Workshop A7: how to evaluate your communication activities 2014-2020 December 10, 2013 – Brussels Céline Mas

3. "Half the money I spend on advertising is wasted;the trouble is I don’t know which half." John WanamakerInventor of mass retailing in the United States Occurrence - Workshop A7 - 10/12/2013

4. Key issue How muchdoes it contribute? Whatdoes it contribute? €  4 • Communications are under pressure: • How much does it cost? • View it as an investment and not as an expense • Provide the resources to prove your effectiveness: 5% to evaluation. • "How much does overlooked inefficiency cost?“ : if you can not assess it, you can not improve it! Occurrence - Workshop A7 - 10/12/2013

5. The missing link in a virtuous circle Communication plan A goals and resources « contract » with institution or company management Most of the time, evaluation is occasional or partial. It rarely shapes dialogue between the communications team and the other decision makers. Actions A goals and resources « contract » with company management Assessment Reporting on the achievement of the targets,or on the progress and effectivenessof the implementation Occurrence - Workshop A7 - 10/12/2013

6. Behaving like any other function Ongoing improvement Quality management system 6 Communications must not exclude itself from Quality and Operating Excellence systems Communications is a job and a skill; it must include ongoing improvement procedures The Deming Wheel Occurrence - Workshop A7 - 10/12/2013

7. The 4 main benefits of an evaluation 1. PROMOTING 2. MANAGING • Allocating resources in accordancewith performance indicators • Identifying the most effective initiativesfor achieving your various objectives • Circulating resultsand performancesto other departments/teams • Achieving investment choices that are based on targets, and not on expenditures on resources 4. SHARING 3. SAVING TIME • Gathering all the activity and effectiveness data • Highlighting best practices • Prioritizing/Sorting initiatives by orderof effectiveness • Concentrating your effortsand budget on the most effective initiatives AND SPECIFIC TO THE PUBLIC SECTOR • Giving evidence of a sound use of public money • Reinforce citizen’s trust 7 Occurrence - Workshop A7 - 10/12/2013

8. 2 main categories of performance (KPI) indicators Activity including Resources (What?) e.g. Number of initiatives/tools, type of initiatives/tools, assessment of the content issued (Press releases, and internal communications), Ressources: Who? How much? How long? Effectiveness including the Audiences (For what purpose?) e.g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customers Audience: How many people attended? How many Likes ? How many readers? … 8 A third, highly practical approach is possible: assessing the satisfaction of (internal) customers Defining performance thresholds for each indicator Occurrence - Workshop A7 - 10/12/2013

9. 60% Gap between results and objectives Current score: 60 % Targeted score :90 % !! ! OK Process : global picture Target audience Communications objectives Communications Dashboard communications plan  Segmentation Opinions/perceptions to share Do you agree with the following assertion ?  1. Define 2. Count 3. Contact details - "ABC respects environment” - … Surveys, research media analysis,observations Yes  Indicators Yes :No :  Don’t know : x 1 [File qualification] -- "ABC respects environment”  x N [Sample survey]  • Management • Allocate • Understand • Adjust • Maintain • Etc… Occurrence - Workshop A7 - 10/12/2013

10. Advice 10 • Efficiency means producing the desired effects on the desired target audiences • Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold • As a starting point: KISS (Keep It Simple, Stupid) • Keep things simple at the beginning • BUT assess them regularly • And don't change the assessment system for each evaluation • Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools • Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative • Share the results and the decisions they help to take in order to enhance the value added of evaluation Occurrence - Workshop A7 - 10/12/2013

11. Thank you • Contact : Céline Mas • Partner & Research Director Occurrence • celine.mas@occurrence.fr Occurrence - Workshop A7 - 10/12/2013

13. Workshop B7: How to evaluate communication activities 2014-2020UK Government perspective • December 9, 2013 – Brussels • Paul Njoku • Cabinet Office, UK

14. Agenda

15. A1 Why evaluate? 16

16. A2 Context Austere times • Need to make every € count • View as an investment not an expense Media landscape & consumption patters • Evidence of what works and what does not • Optimise use of scarce resources The role of communications • How it supports achievement of policy outcomes • Business planning & activity prioritisation

17. Sub-objective Sub-objective Sub-objective A3 Strategic alignment DEPARTMENTAL OBJECTIVETo address specific issue Policy development, policy delivery, reputation management COMMUNICATION OBJECTIVERole that communication will play in achieving departmental objective Overall communication strategy - how communication will achieve its objective Communication plan Specific activities, channels, target audiences Communication plan Specific activities, channels, target audiences Communication plan Specific activities, channels, target audiences

18. A4 Barriers – stopping it happening Lack of SMART or unrealistic policy objectives / targets Culture & entrenched behaviours Insufficient time / resource /budget Difficulty accessing the right data / tools Gaps in evaluation standards & capability 19 19

19. B1 How to go about it Five key principles P • Pragmatic – best available within budget, not best ever • Realistic –prove what you can, acknowledge what you can’t • Open –record and share as much as possible • Objective –be honest & constructive about results, to inform future learning • Fully integrated – integral part of planning & delivery, not an add-on R O O F

20. B2 Evaluation stages –The Big IDIA 1 • Task 1: Define what you need to evaluate by asking: • What activity am I evaluating? • What do I know & what factors could affect the outcome? • What is my evaluation expected to achieve? • Output: Summary of your proposed evaluation approach IdentifyThe scope of your project 2 DevelopYour evaluation plan • Task 2: Define how you’ll measure success: • Set SMART objectives & defining your target audience • Map out how activity will work • Set performance metrics (KPIs) & agreeing targets • Output: Draft evaluation plan 3 ImplementGather data to measure performance • Task 3: Identify and gather evaluation data: • Make most of existing data • Gather additional data (research, feedback & proxies) • Review data gaps (more budget ?) manage expectations • Output: Completed evaluation plan 4 Analyse & reportPerformance against plan • Task 4: Assess the success of your activity: • Analyse effectiveness & provide insights for future • Demonstrating efficiency and value for money • Demonstrating role of communications in supporting the achievement of policy objective (outcome) • Output: Final evaluation report

21. B3 Key performance indicator categories Activity Effectiveness Result

22. Top tips Strategic alignment – Ensure activity objectives are SMART and supports policy delivery. Business impact – Aim to measure true business impact (outcome) rather than for example, the perceived quality of specific channels. PROOF the big IDIA – Try to adopt the suggested guiding principles and follow the big IDIA stages. Continuous improvement – Ensure results drive appropriate actions and any learnings inform future activities. Best practice – Be objective, share results and make evaluation an integral part of your communications planning process.

23. Thank you! Contact: Paul Njoku Email : paul.njoku@cabinet-office.gsi.gov.uk Web link to guide: https://gcn.civilservice.gov.uk/guidance/evaluation/

24. Appendix

25. Products Customer Service Investments Employment Branding Public Relations Marketing SocialResponsibility How reputation is created Direct Experience Results Perceptions & expectations Supportive Behaviour What your department Says/Does What Others Say MEDIA (Traditional, Social) Topic Experts, Leaders,Friends/Family

26. Reputation drivers & dimensions Reputation Strength Reputation Dimensions A measure of the emotional connection. • The seven dimensions specify at a more operational level, which aspects are most important for stakeholders’ perceptions and expectations – i.e. what’s driving a company’s reputation Supportive Behaviour Reputation Attributes The attributes and dimensions have different meanings and importance for different stakeholders. Beneath the 7 dimensions, 32 attributes underpin the individual dimension themes. Different stakeholder groups typically have unique attributes that are found more important than others (reputation drivers). Reputation has a positive/negative impact on support. An increase in reputation = an increase in support. Support (such as buying products and services, saying something positive, giving the benefit of doubt in times of crisis (etc.) leads to increased business results