Goal Attainment Scaling

1. Goal Attainment Scaling Wigan EPS Approach Simon Jenner Based on work by Francis Mallon (Birmingham)

2. The ‘Customer’ • Crucial question not ‘Who is my customer?’ but ‘Who is my customer’s customer?’ – Emphasises that the purpose of service provision is to enable the recipient to meet their customers’ needs.

3. Problem: • How to evaluate IMPACT of differing approaches with one measure, eg outcomes for a 16 year old with Aspergers educated at home vs. a 4 year old with PMLD vs. whole school work on IEPs

4. Implication • Service evaluation should go beyond the views of the school to looking at outcomes for children – trying to get at the ‘co-production of added-value’… (Checkland & Scholes, 1993) • Requires a common approach to monitoring and evaluating pupil progress.

5. Common approach • ‘Among the candidates for adoption as the common approach, Goal Attainment Scaling (GAS) would appear to have much to commend it.’ (Baxter & Frederickson 2005) • ‘… GAS is an approach that could significantly assist EPs in evaluating individual outcomes and … allow collation of information across services.’ (Frederickson 2002).

6. Goal Attainment Scaling (GAS) • Developed by Kiresuk & Sherman (1968) as a method for evaluating mental health services. • Provides an individualised, criterion-referenced approach to describing behaviour change and documenting the outcomes of intervention programmes. • Provides a measure of the progress towards goals, defined on a 5 point scale.

7. Some examples of GAS applications: • Evaluating a behaviour support service (Imich & Roberts, 1990) • Gross motor function in cerebral palsy (Wright, Boschen & Jutai, 2005) • Behavioural interventions (Urwin and Ballinger, 2005) • Student self-monitoring (Weymeyer et al., 2003) • Brain injury rehabilitation (Ponsford et al., 1999) • Pain evaluation programmes (Zaza, Stolee & Prkachin, 1999) • Pupil mentoring evaluation (Balcazar et al., 2006) • In Birmingham (Mallon, 2008)

8. Goal Attainment Scaling - Reliability • Good inter-rater reliability of 0.9 and above have been reported using GAS

9. Trialled in Wigan by EPS last term • Proposal to try across a wide range of work this year • Settings may find it useful for themselves

10. Goal Attainment Scaling procedure • Prioritise up to 3 areas of concern. • Establish up to three goals (can be more than one per problem area). • Select a baseline descriptor for each against which progress can be measured (evidence) • Specify the expected level of outcome after intervention ( 0).

11. GAS Procedure cont. 5. Specify +1, +2, -1, -2 where +1 = more than expected +2 = much more than expected 0 = expected improvements -1 = less than expected -2 = much less than expected 6. Mark baseline as measure (usually in -1 or -2). 7. Repeat for each goal. 8. Decide review date.

12. GOAL: to raise X’s self esteem When asked about self… • -2 Identifies 0 positive things • -1 Identifies 1 –2 positive things • 0 Identifies 3 – 5 positive things • +1 Identifies 6 - 8 positive things • +2 Identifies 9+ positive things

13. Goal: Produce more classwork With fewer than 2 prompts… • -2 Completes 0-30% • -1 Completes 31-60% • 0 Completes 61-75% • +1 Completes 76-90% • +2 Completes 91-100%

14. Example 1 • Concern: H. is isolated on the playground; does not have the skills to approach others, or join in games. (diagnosis of ASD) • Baseline Descriptor H has not shown or is unable to use any of the skills that are part of interacting with peers socially on the playground. He is isolated on the playground.

15. Example 1 cont. +2 He will be observed as being a leader of games or as a child who starts up games. +1He will be playing with others five times a week. 0 He will be involved in playing with others on the playground 3 to 4 times a week. -1 He will be involved in playing sporadically. (average one day a week) -2 He will remain isolated on the playground.

16. Example 2 • Concern: Behaviour of two pupils in Reception setting • Baseline descriptor: Two pupils (one in each Reception class) have been identified as displaying behaviours which are preventing them (and others) from accessing the curriculum

17. Example 2 Cont. The strategies identified will… +2 eliminate the challenging behaviours displayed by the identified pupil and will have a positive impact on the behaviours of the whole class. +1 significantly reduce the frequency of the challenging behaviours displayed by the identified pupil, and will have a positive impact on the behaviours of the whole class

18. Example 2 cont. 0 reduce the frequency of the challenging behaviours displayed by the identified pupil -1 slightly reduce the frequency of the challenging behaviours displayed by the identified pupil -2 have no effect on the frequency of the challenging behaviours displayed by the identified pupil

19. Example 2 cont. - Evidence • the pupil is beginning to be able to focus on his learning as he is no longer displaying behaviours which prevent him accessing the curriculum • peers playing collaboratively with the pupil, and he is no longer socially isolated as previously • communication between the pupil and adults has improved, and the pupil is now able to express his needs effectively due to strategies used by staff to elicit his views

20. Example 3 Concern: Evaluation of a training course Baseline descriptor: Training not begun, preparation stage. Goal: For training to be successful.

21. Example 3 cont. + 2 70% comments positive and attendees report implemented strategies +1 60% comments positive and attendees report implemented strategies 0 50% comments positive attendees report implemented strategies -1 40% comments positive -2 30% comments positive

22. Guard against goals that are too easily accomplished, or too difficult. Realistic expectations for outcomes should be used to evaluate the programme accurately. • The outcomes should be measurable and specific. • While the outcome does not have to quantified, it must be stated so that hypothetically two independent observers could agree on whether it has been obtained

23. Some common problems • Use of jargon or technical language • Vagueness • Overlapping levels • Gaps between levels • Multidimensional scales • Time consuming?

24. Goal Attainment Scaling (GAS) • Provides an explicit framework to specify targets, progress and outcomes of intervention. • Can be used flexibly – targets can be negotiated and adjusted as required. • It encourages collaborative goal setting and pupils, teachers and parents can be involved. • Is becoming increasingly widely used in assessing the adequacy of progress and evaluation of outcomes.

25. Comments &