Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU

1. Measuring change: responsiveness or sensitivity Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU

2. Objective • Discusses ways of establishing if an OM can measure change over time and how this property can be reported using descriptive and statistical methods. • Critically appraise evidence about the development and measurement proprieties of OMs • Discuss the consideration related to the application of evidence base to OMs

3. Introduction Psychometric properties • Reliable • Valid • Responsiveness

4. Responsiveness to change • Responsiveness ability of instrument to detect/measures meaningful or important change in a clinical statewithin patients over time, or contrast between groups which may be indicative of therapeutic effects. N.B 1- Normally considered as an element of validity N.B 2- Change my reflect natural recovery + response to intervention N.B 3- Responsiveness is not considered a generalizable property

5. Study responsiveness Determine clinical outcome assessment

6. Study responsiveness

7. Reporting responsiveness There are different ways reported when the responsiveness of a measurement is being analyzed. They include: • Effect size (standardized mean difference • Minimal detectable change • Minimal clinically important difference • Receiver operator characteristic curve.

8. Effect size (ES) “average changes” The magnitude of the differences between two mean, may be standardized by dividing this differences by the pooled standard deviation to compare effects measured by differences scales Magnitude of effect size scores • Small ≤ 0.4 • Moderate = 0.5 • Large > 0.8 ES is often displayed as a correlation coefficient, r

9. Effect size (ES) “average changes” • Easy to calculate and understand • Used with ratio or interval level of measures • Determine extent of a relationship/difference between variables • Useful for comparing responsiveness of different health measures • Enables sample size calculations. • Facilitates comparison between scientific studies

11. Statistical significance versus Effect size compute an effect size for each study, we will find the following: Formula: d = 2(t)/df Smith d = 0.50 Jones d = 0.50 The amount of change associated with the outcome is the same in both studies.

12. Differences

13. MDC (Minimal Detectable Change) • The MDC was calculated based on standard error of measurement (SEM) according to the following formulae: • MDC=1.96×2×SEM • SEM=SDall testing scores×(1−r) • The √2 is used to account for the underlying extra uncertainty during measurement in two time points. • The value 1.96 is the z score associated with the 95% confidence level, • The ris the coefficient of the test–retest reliability, which was estimated by ICC. • MDC% was calculated as (MDC/mean) × 100%, • An MDC% of <30% was considered as acceptable and <10% as excellent

14. Now Start use and interpret ES, MDC I think I have given input to `u’

15. Minimal important difference and responsiveness of 2-minute walk test performance in people with COPD undergoing pulmonary rehabilitation Int J Chron Obstruct Pulmon Dis. 2017; 12: 2849–2857.

16. Study responsiveness Determine clinical outcome assessment Objective examined responsiveness, minimal important difference (MID), test–retest reliability, and construct validity of the 2MWT in people with stable COPD attending outpatient pulmonary rehabilitation (PR).

17. Study responsiveness • Inclusion • People with stable COPD confirmed by spirometry • Exclusion • severe comorbidities that prevented exercise training (severe cardiac, neurological, or musculoskeletal conditions) • met any of the absolute contraindications to conduct of a 6MWT.

18. Reporting responsiveness Effect sizes for these measures were calculated using the formula d= (mean post − mean pre)/SD pooled. For MID, were calculated using the formula MDC=1.96× √ 2×SEM Based on standard error of measurement (SEM) was calculated according to the formula SEM = σ1√(1 − r) where σ1 is the baseline standard deviation and r is the test–retest reliability

19. Findings

20. Test–retest reliability of the 10-metre fast walk test and 6-minute walk test in ambulatory school-aged children with cerebral palsy Developmental Medicine & Child Neurology 2008, 50: 370–376

21. Study responsiveness Determine clinical outcome assessment Objective to estimate the test–retest reliability of the 10mFWT and 6MWT in ambulatory school-aged children with CP and within Gross Motor Function Classification System (GMFCS)18subgroups (i.e. GMFCS Levels I, II, and III). The secondary objective was to estimate the minimum detectable change (MDC) in the total sample and subgroups

22. Study responsiveness • Inclusion • (1) diagnosis of spastic CP; • (2) between 4 and 18 years of age; • (3) GMFCS Levels I, II, or III as determined by their developmental pediatrician; • (4) ability to walk independently without stopping for 6 minutes, with or without a walking aid; • (5) ability to follow verbal instructions in English; and • (6) ability to cooperate for at least 30 minutes as judged by their treating physiotherapist • The exclusion • orthopaedic surgery within the past 6 months; and • botulinum toxin type A (BoNT-A) injections within the preceding 3 months.

23. Reporting responsiveness For MID, were calculated using the formula MDC95=1.96× √ 2×SEM Based on standard error of measurement (SEM) was calculated according to the formula SEM = SDb √(1−ICC) where σ1 is the baseline standard deviation and r is the test–retest reliability

24. Findings

25. Evidence-Based Approach to Choosing Outcome Measures

26. Evidence-Based Approach to Choosing OMs • An evidence-based approach to selecting outcome measures involves making judgements about the quality of the validity and reliability studies, interpreting the findings and deciding whether they are applicable to one’s own specific practice.

28. Asking a Question

29. Searching for evidence CINAHL PT now

30. Critical appraisal of OMs Research

31. Appraising Evidence of OMS Several checklists for measurement properties exist • Scientific advisory committee (SAC) of the medical outcome trust in 2002 • COSMIN checklist (www.cosmin.). • Christina Jerosch-Herold”s Checklist for critical appraisal of OMs • Oxford center for evidence base No consensus on terminilogy and defintions None is generally accepted and widely used

32. COSMIN Checklist • COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed in an international Delphi study with the aim to provide tools for evidence-based instrument selection • The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Quality of Life Research • May 2010, Volume 19, Issue 4, pp 539–549 • available at http://www.cosmin.nl/).

33. COSMIN Checklist The COSMIN checklist comprises 12 boxes: • Ten boxes evaluate whether a study meets the standard for good methodological quality. • A=Internal consistency, • B=Reliability, • C=Measurement error, • D=Content validity, • E=Structural validity, • F=Hypothesis-testing, • G=Cross-cultural validity, • H=Criterion validity, • I=Responsiveness • J=Interpretability. • Two additional boxes were included to meet studies that use the Item Response Theory method (IRT box) and general requirements for results generalization (Generalizability box).

34. COSMIN Applications COSMIN aims to improve the selection of outcome measurement instruments both in research and in clinical practice by developing methodology and practical tools for selecting the most suitable outcome measurement instrument. • Systematic review of measurement properties • Measure instrument selection • Designing and reporting study on measurement properties • Identification the need for further research on measurement properties

35. COSMIN Checklist Manual

36. COSMIN Checklist Four steps should be taken to complete the COSMIN checklist

37. Application of evidence to patient/client

38. Now Start appraising OMs I think I have given input to `u’