1 / 29

Mapping

Mapping. Workshop on Universal Reporting Parameters for the Speech of Individuals with Cleft Palate Washington DC, April/May 2004 Tara Whitehill, Hong Kong. Proposed Three-Stage Plan. Evaluating : The usual speech evaluation by the local speech specialist in cleft palate.

kirk
Télécharger la présentation

Mapping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Mapping Workshop on Universal Reporting Parameters for the Speech of Individuals with Cleft Palate Washington DC, April/May 2004 Tara Whitehill, Hong Kong

  2. Proposed Three-Stage Plan • Evaluating: The usual speech evaluation by the local speech specialist in cleft palate. • Mapping: Conversion from the local speech evaluation to the universal reporting measures. • Reporting:Universal parameters that describe the cardinal features of speech in individuals with cleft lip/palate and related craniofacial anomalies regardless of the individual’s primary language.

  3. Already covered • Evaluation (range of protocols and procedures; key issues) • Parameters (proposal; key issues) • Focus: mapping

  4. Fanny LAW Sze Kuen International Parameters for Cleft Palate Speech: A Pilot Study Honours dissertation Bachelor of Science (Speech & Hearing Science) University of Hong Kong May 2004 Supervisor: Dr. T. L. Whitehill DRAFT - in progress

  5. 6 sets of client forms from 5 centres • 5 clients each set = 30 client forms • Original forms only (no recordings, no additional data such as transcriptions, unless provided on original form) • Two groups of “mappers” • 5 “experts”- members of WG who supplied forms • 5 students - little/no previous experience cleft

  6. Used “current” version of parameters and guidelines • Students given session to familiarize/ clarify terminology • Mapped from client forms onto parameters form • Identifying information masked (client, examiner, centre) • Provided feedback on ease/difficulty of mapping each parameter • Either for client form or by set (centre) • Weighted scores - not included in this study

  7. “Outcome measures” • Agreement • Within expert group • Within student group • Across all mappers • Feedback (ease/difficulty)

  8. Why use students? • In future, unlikely mappers will be naïve • However, “expert” group was restricted to individuals who had developed and debated the parameters

  9. Agreement • A big problem was “missing data” • OK if client form stated MD (or WNL or NAD), but what if didn’t? - assume OK? tick MD? • Calculated (a) absolute agreement (Condition 1) (b) disregarding “missing data” (Condition 2)

  10. Table 1. Examples to illustrate how agreement was defined

  11. Primary ParametersHypernasality Agreement • Experts Condition 1 - 60% Condition 2 - 63% • Students Condition 1 - 73% Condition 2 - 73% • All Condition 1 - 57% Condition 2 - 60% Not much missing data

  12. Primary ParametersHypernasality Feedback/Issues to consider • Some difficulties when no. scale points differed (e.g., client form - 7 pt. scale; parameters form - 4 pt. scale) • In such cases, mappers reported difficulty, and agreement decreased • Implication: explicit instructions? • Another difficulty: “mild to moderate” or “moderate to severe” (students selected less severe and experts more severe!) • “Set B” - definition of “mild” = “evident but acceptable”. One expert rater mapped onto WNL.

  13. Primary ParametersHyponasality Agreement • Experts Condition 1 - 53% Condition 2 - 96% • Students Condition 1 - 87% Condition 2 - 100% • All Condition 1 - 50% Condition 2 - 97% Excellent, when missing data is accounted for

  14. Primary ParametersHyponasality Feedback/Issues to consider • Missing data • Otherwise, excellent • (binary scale)

  15. Level One (absent, present, MD) Experts Condition 1 - 63% Condition 2 - 87% Students Condition 1 - 77% Condition 2 - 80% All Condition 1 - 57% Condition 2 - 70% Level Two (subcategories) Experts Condition 1 - 23% Condition 2 - 60% Students Condition 1 - 30% Condition 2 - 38% All Condition 1 - 17% Condition 2 - 27% Primary ParametersAudible Nasal Emissionwith/without Nasal Turbulence

  16. Primary ParametersAudible Nasal Emissionwith/without Nasal Turbulence Feedback/Issues to consider • Low agreement for subcategories • “phoneme specific” - e.g. “audible for plosive and /s/” • Confusion about pervasiveness e.g. “mild but consistent”, “phoneme specific and frequent” • Results of mirror test (if not audible in speech) • (NE w/wout nasal turbulence - not raised here)

  17. Primary ParametersWeak Oral Pressures Agreement • Experts Condition 1 - 17% Condition 2 - 75% • Students Condition 1 - 13% Condition 2 - 62% • All Condition 1 - 0% Condition 2 - 46%

  18. Primary Parameters Weak Oral Pressures Feedback/Issues to consider • All 9 mappers found this parameter difficult to map • Primarily because of lack of data on client forms - no form had specific category for this • Mappers needed to derive the information from other information; particularly difficult for students • Also, students unclear about definition

  19. Level One (absent, present, MD) Experts Condition 1 - 57% Condition 2 - 73% Students Condition 1 - 73% Condition 2 - 85% All Condition 1 - 57% Condition 2 - 70% Level Two (subcategories) Experts Condition 1 - 10% Condition 2 - 30% Students Condition 1 - 10% Condition 2 - 15% All Condition 1 - 10% Condition 2 - 20% Primary Parameters Substitution Errors • Agreement not bad for level one, but very poor for subcategories; missing data was not the main problem

  20. Primary Parameters Substitution Errors Feedback/Issues to consider • All forms included level one (present/absent/MD) • Subcategories: most forms had patterns, but few included frequency information • Some patterns which expert listeners considered ‘related to cleft palate’ not included here (e.g. nasal substitution/realization’, ‘double articulation’) • Confusion between this parameter and ‘Developmental Delay or Other Articulation/Phonological Errors’ (even for experts) e.g. omission, backing /t/ -> [k].

  21. Secondary ParametersErrors related to dentition, occlusion, palatal vault configuration Agreement • Experts Condition 1 - 13% Condition 2 - 69% • Students Condition 1 - 30% Condition 2 - 59% • All Condition 1 - 7% Condition 2 - 53% Better when missing data taken into consideration, but still pretty poor

  22. Secondary ParametersErrors related to dentition, occlusion, palatal vault configuration Feedback/Issues to consider • No specific category for this on client forms; information had to be extracted from other information • Some forms (Sets B, E, F) included categories such as ‘lateralization’, ‘palatalization’, ‘dentalization’ - but still not clear whether these due to organic factors or “Other Articulation/Phonological Errors” - even for experts • Errors related to hearing problem? fistula?

  23. Secondary ParametersDevelopmental Delay or Other Articulation/Phonological Errors Agreement • Experts Condition 1 - 16% Condition 2 - 72% • Students Condition 1 - 30% Condition 2 - 50% • All Condition 1 - 10% Condition 2 - 43% Again, better when missing data taken into account, but still pretty poor

  24. Secondary ParametersDevelopmental Delay or Other Articulation/Phonological Errors Feedback/Issues to consider • Mappers needed to make judgement regarding whether errors were related to cleft or not • Confusion with both “Substitution Errors” and “Errors related to dentition..” • Client forms rarely specified cause or explanation of error patterns - mappers needed to judge • Special cases e.g. “no oral consonants”

  25. Secondary ParametersVoice/Laryngeal Disorder Agreement • Experts Condition 1 - 56% Condition 2 - 90% • Students Condition 1 - 63% Condition 2 - 97% • All Condition 1 - 60% Condition 2 - 96% Agreement was good, especially when missing data was taken into account

  26. Secondary ParametersVoice/Laryngeal Disorder Feedback/Issues to consider • Similar to hyponasality - mappers were familiar with term • When data was provided on client form, it was clear • Binary choice; no subcategories

  27. Global RatingSeverity/Intelligibility • Not decided at time of this study • Only 2 of the 6 sets included such data

  28. Conclusions • Reliability lower among students than experts • Experts were those who developed form • Who will be mapping in the future?

More Related