Update on National Social Care User Experience Survey SSRG Conference, 8-9 March 2010

1. Update on National Social Care User Experience Survey SSRG Conference, 8-9 March 2010 Suzy Powell (DH), Juliette Malley (PSSRU), Paul Niblett (NHS IC)

2. The role of experience information ‘Experience’ is a key element in the cluster approach to outcomes accountability. The most obvious way to find out what outcomes have been achieved for people is to ask them – and this requires survey programmes and perception-based PIs. Activity Results Results + Experience = Outcomes What did we do? How well did we do it? Inputs & outputs Is anyone better off? Result of activity Outcomes + Activity Costs Means Ends = Analysis of service contribution and value for money Costs Experience How much did it cost? Budgets & funding What was it like for the user? Perceptions

3. The current surveys • What’s wrong with the old programme? • No formal rolling programme – survey groups agreed in advance • No consistent repetition to track progress • Data limited to specific client group or service type • Variability in how representative data is of wider service • Outcomes focus is not consistent: not always asking the right questions • Not aligned with the NHS or other survey programmes • So… • Many groups’ experience not covered • Poor value performance information (both nationally for NI127 and NI128, and locally for monitoring own services) • No common basis for local partners • Little support for delivering the outcomes of Putting People First

4. The concept Our work aims to design and develop new national surveys for adult social care users which better reflect the wider agenda: Focusing on whole user populations; Asking questions about experience of key outcomes, in line with policy priorities and including those which go beyond core social care services; Repeated annually to track progress and act as a basis for planning and Local Area Agreements; Providing a basis for national benchmarking, but flexibility to support additional local surveys; Giving further opportunities to link to other NHS and community surveys through common outcomes.

5. Report on Development Work – Brief History • Discussion paper – proposals for method and content of survey PLUS key challenges • Presentations and opportunity for feedback • Response paper – proposed method and development work

6. Proposed method • Cross-sectional SRS all social care clients • Self-completion questionnaire • Address potential non response by • maximising opportunities for sample members to get help to complete questionnaire • using a validated version for PWLD • Questions on social care-related quality of life (SCRQOL) + factors influencing • MANY CHALLENGES + QUESTIONS

7. Development work • Implications of having ‘help’ • Who helps and how, question revisions, additional questions to help interpretation & guidance • Including residents of care homes • Factors influencing RR & appropriateness of q’aire, guidance & revisions/additions to questions • Accessible self completion for PWLD • LD version & factors influencing RR, guidance • Can advocates be used to help people answer where they don’t have anyone else to help? • Guidance

8. Study 1: Having help to complete • Method • Cognitive interviews to test questions • Service user & helper interviewed together • Key findings • Type of help given very varied e.g. hold pen, read questions, translation into FL, proxy • Helper can influence responses given • Dominance in relationship • Type of help influenced by disability of user • Lack mental capacity – proxy response • Questions about ‘feelings’ difficult for proxies • No obvious frame of reference • Produce random answers, inflate variance + SEs

9. Study 2: Care homes • Method • Interviews with care home managers • (Cognitive) interviews with residents • Key findings • All residents could express some opinions about their care • Nearly all need some kind of help • Writing, reading, understand meaning, formulate responses • Survey format difficult for many • Fitting answers to response options • General terms difficult, need personalising

10. Study 2 continued • Key findings (continued) • Staff most appropriate people to help • Many residents lack regular visitors • Helper needs to know day-to-day life of resident well • Advocates patchy + expensive • Relatives/staff/advocates all could bias answers • Gaining informed consent • Approach of care homes not consistent • Level of ‘informed’ consent required • Variety of methods for dealing with mail • Some questions need modification for care home residents e.g. “home”

11. Study 3: LD version • Method • Focus groups with PWLD to design questions • Interviews with PWLD to test questions • Key findings • Most PWLD cannot read • at all/level required for main q’aire • use of illustrations & simplified language helped with understanding • Still many will need help to complete • Help to understand meaning & formulate responses • Survey format difficult for many • Fitting answers to response options • General terms difficult, need personalising • Summing across different aspects also difficult

12. Study 4: Advocates • Methods • Interviews with LA UES managers & advocacy agency managers • Key findings • Very little experience of using advocates in this way • Availability of advocacy patchy • Risks associated with home visits • Familiarity with clients needed, requires several visits • But many advocacy agencies enthusiastic about survey and providing such support

13. Recommendations for pilot • Encourage people to seek help to prevent high rates of non response • Someone who knows user well/see regularly • Care staff in care homes • Not advocates unless existing relationship • Reduce potential for helper to bias results • Directions on front sheet & to care home staff • Include questions on who helped and how in q’aire • Consider survey for carers • Users who lack capacity to consent should be excluded • Responses from proxies to ‘feelings’ questions (i.e. main measure) are unreliable • Use care managers to identify who lacks capacity to consent • Test and cost method for excluding

14. Recommendations continued • Engage care home managers to prevent high rates NR • Send letter to CH manager • Let CH manager know which residents receive survey • Encourage CH staff to help residents • Reassure not using results to assess CH • Adapt questionnaire for CH residents • Use LD version for PWLD

15. Recommendations for analysis • Check sample not biased due to exclusion of people lacking capacity • Requires detailed population data • Stratified sampling may be better method for future • Check for non response across different groups • Particularly from CHs, PWLD, people with high levels of need, people living alone • High rates of NR may indicate method not applicable • Check for unusual results or differences between group with and without help • Large differences may indicate a problem, require further analysis/data collection • Alternative methods may be indicated

16. Recommendations for future work –dependent on pilot results • Test & cost alternative methods for CH and PWLD • E.g. semi-structured interviews, observation • Consider developing proxy questionnaire • Test stratified sampling method • Test & cost telephone helpline with trained operatives to conduct interview • Consider developing proxy questionnaire • Reconsider role of advocates as advocacy becomes more embedded

17. Pilot Survey • Pilot survey – 3 options • All service users • Residential care • Service users with LD • Timescales • Draft guidance and questionnaire available • Awaiting ethical approval before final guidance issued • Fieldwork in April • Return data to NHS IC by 13 June 2010

18. Timetable • Report on pilot to SSUSG in late July • Ethical review for full survey in early September • Guidance and questionnaire issues in October • Fieldwork in February 2011 • Data returned to NHS IC in late May 2011 • Provisional data used for NIs available for regulator and councils (via NASCIS) on a pre-release access basis in mid June • Provisional data for NIs published in mid August 2011 • Provisional full survey data published end September 2011 • Full survey report published December 2011 • Other development work going on throughout this period to improve survey for 2011-12