Translating Evidence-Based Research Findings into Effective Intervention Practices for Individuals with Autism Spectrum

1. Translating Evidence-Based Research Findings into Effective Intervention Practices for Individuals with Autism Spectrum Disorders Diane Twachtman-Cullen, Ph.D., CCC-SLP Editor-in-Chief, Autism Spectrum Quarterly www.ASQuarterly.com ADDCON Center, LLC P.O. Box 709 Higganum, CT 06441-0709 Telephone: (860) 345-4590 Fax: (860) 345-4789 E-Mail: addconcenter@snet.net

2. Autism Spectrum Quarterly (ASQ) The Magajournal� www.ASQuarterly.com Much of the information presented in this workshop has been reported on in Autism Spectrum Quarterly

4. The Agenda Science and pseudoscience Factors to consider regarding autism intervention Quality indicators for evidence-based practice Research related to key deficit areas in autism vis-�-vis quality indicators for EBP Evidence-based intervention snapshots The take-home messages

5. Parents Beware: The Downside of Autism Awareness �It became instantly clear to me that families like mine are now a preyed-upon marketing demographic�seemingly easy victims for less than ethical people.� Amber Kane, Autism Connection Autism Spectrum Quarterly, Summer, 2008 pp 24-25

6. What Have You Got to Lose? �I�ve come to see this as the most devastating question . . . Don�t listen! WE HAVE A LOT TO LOSE. . . .Don�t allow yourself to fall for marketing that offers optimism or perpetrates pessimism. It is our responsibility to sit midway on this spectrum of treatment choices and hover in the realm of realism.� Amber Kane, Autism Connection Autism Spectrum Quarterly, Summer, 2008 pp. 24-25

7. Science �A set of methods designed to describe and interpret observed or inferred phenomena, past or present, and aimed at building a testable body of knowledge open to rejection or confirmation.� Michael Shermer Why People Believe Weird Things

8. Parents Beware: Pseudoscience Relies on anecdotal evidence / testimonials often draped in scientific jargon Face-value acceptance May advance the idea that certain things just can�t be tested Often uses emotive words (cure, recovery) Extremely resistant to change (true believer)

9. DISCLAIMER! There is a vast difference between trying something to increase the comfort level of the individual with autism and claiming that it is the �be all, end all� CURE for autism.

10. Facilitated Communication: The Undisputed Darling of Media Hype �Call it a miracle. Call it an awakening.� -Diane Sawyer, Prime Time �At present, there are no scientifically controlled studies that unambiguously support benefits in expressive language function for people with mental retardation or autism by taking part in FC.� Mulick, Jacobson, and Kobe (1993)

11. The �Unlearned� Lesson from FC �The dangers inherent in relegating scientific scrutiny to the scrap heap while elevating subjectivity and hype to the level of an art form may go undetected amid the evangelistic fervor of the moment. Such unrecognized dangers are nevertheless critical. Saying something is or is not so does not make it so. It is only through scientific investigation that one manages to get closer to what actually is.� Diane Twachtman-Cullen, Ph.D., CCC-SLP A Passion to Believe: Autism and the Facilitated Communication Phenomenon

12. Five Important Factors to Consider Regarding Autism Intervention Services

13. 1. Autism is a multi-faceted disorder and as such, it requires a multi-faceted treatment approach.

14. Comprehensive Treatment Communication / Language Issues Social Behavior Sensory Issues Behavioral Issues Executive Function Deficits Theory of Mind Deficits Accompanying Physical Conditions

15. 2. Intervention practices should be rooted in, and informed by high-quality research across several fields.

16. Language Development Child Development Psychology / Neuropsychology Autism Literature Behavioral Literature

17. Attention to only one small aspect of the research literature is myopic and misleading. The mantra that ABA is the ONLY intervention in autism that has research evidence behind it is seriously misleading and the likely result of a very narrow research focus. Triangulating relevant research across several fields is a �best practice� approach to intervention.

18. 3. One size does NOT fit all!

19. If You Know One Child with Autism, You Know How Autism Affects One Child! According to the National Research Council, �Studies have reported substantial changes in large numbers of children receiving a variety of intervention approaches, ranging from behavioral to developmental.� Barry M. Prizant, Ph.D.

20. Characteristics of Effective Interventions According to the National Research Council, six kinds of interventions should have priority: Functional, spontaneous communication Social instruction in various settings 3. Play skills (appropriate use of toys and play with peers) [leisure skills for older individuals]

21. Characteristics of Effective Interventions According to the National Research Council, six kinds of interventions should have priority: 4. Instruction leading to generalization and maintenance in natural contexts 5. Positive approaches regarding problem behavior 6. Functional academic skills when appropriate

22. In order for an intervention method to be considered effective, there must be functional application of skills in real-world settings (generalization). Needs to be a component that involves application of the skill Can be done via parent training, and/or including a functional training component within the therapeutic setting

23. 5. Effective interventions in autism require direct and sustained attention to the core deficits associated with the syndrome.

24. What Are the Core Features of Autism? Deficits in the understanding and use of language Deficits in social understanding and expression and social relatedness

25. ASHA Position Statement �Evidence-based practice refers to an approach in which current, high-quality research evidence is integrated with practitioner expertise and client preferences and values into the process of making clinical decisions.� ASHA, 2005

26. What Does EBP Mean for Individuals with ASD? According to Attorney Wayne Steedman of Wrightslaw, �new language [in IDEA 2004] creates a new requirement that instructional practices or interventions be based on accepted research.� Autism Spectrum Quarterly Summer, 2006

27. Quality Indicators for EBP The 1 : 1 Correspondence �Test� The Concordance �Test� The Goodness of the Fit �Test�

28. Connecting the Research �Dots� to Create Intervention �Snapshots� How Can Research Findings Related to the Role of Experience in Brain Development Inform Intervention Practices in Autism?

29. Research into Experiential Learning Early experiences have the capacity to influence the structure and formation of the brain. W.T. Greenough & J.E. Black (1992). �Induction of Brain Structure by Experience: Substrates for Cognitive Development.� Developmental Behavioral Neuroscience, 24 pp 155-299 Time Magazine, February 3, 1997

30. The Take Home Message �Experience shapes brains, but you need to interact with the experience.� Jane Holmes Bernstein, M.D.

31. �Mere exposure to an enriched environment will not suffice. Direct interaction with it is required to produce cerebral [i.e., cognitive / brain] effects.� Eric Shopler, Founder of Division TEACCH

32. Caution It has to be the right type of experience, delivered at the right time!

33. The Downside Risks of Inappropriate Developmental Practices �Trying to drill higher-level learning into immature brains may force them to perform with lower-level systems and thus impair the skill in question.� Jane M. Healy, Ph.D. Endangered Minds

34. Experiential Research (continued) Studies demonstrate an increase in metabolic activity in those areas of the brain having to do with emotional regulation, interaction, and sequencing at the time when infants are involved in reciprocal interactions, choice-making, and search behavior. M.A. Bell & N.A. Fox (1994). �Brain Development Over the First Year of Life: Relations between EEG Frequency and Coherence and Cognition and Affective Behaviors.� In G. Dawson & K. Fischer, Eds., Human Behavior and the Developing Brain. Guilford Press. pp 314-45

35. Implications for Early Intervention �In the past we�ve shown that young children with autism have atypical brain responses to social and language information which we think could be the result of not having had early stimulation of the brain systems that underlie those areas. . . .by intervening early we may be able to actually alter the course of development, and we want to measure that.� Dr. Geraldine Dawson Chief Science Officer, Autism Speaks Autism Spectrum Quarterly, Summer 2008 If exp. Is playing a role, there would be fusiform activa. when seeing a fam. face. There is (only hily fam face). If exp. Is playing a role, there would be fusiform activa. when seeing a fam. face. There is (only hily fam face).

36. Use It or Lose It!!! �The plasticity of the young brain is based on the overabundance of synapses, which allows only [emphasis supplied] those that are used to become part of enduring circuits that underlie thinking, feeling, responding, and behaving� (p. 129). Schwartz, J.M. & Begley, S. (2002). The mind and the brain: Neuroplasticity and the power of mental force. NY: Regan Books.

37. How Does This Research Measure Up to the Quality Indicators? Studies demonstrate a 1 : 1 correspondence between actual experience and brain function. There is concordance among many research studies and centers regarding the relationship between experience and brain function. 3. Experiential learning is a �good fit� for children with autism, since it helps them to establish meaning and sense-making.

38. 1st Evidence-Based Intervention Snapshot Given the importance of experience in brain development, emotional regulation, and interaction, language development activities should be embedded in experiences that provide a context for learning, sense-making, and language use.

39. Connecting the Research �Dots� to Create Intervention �Snapshots� How can research findings related to joint attention inform intervention practices in autism?

40. Joint attention is �a cluster of behaviors that share the common goal of communicating with another person about a third entity in a nonverbal way, including eye gaze alternation and gesturing.� Bruinsma, Koegel, & Koegel (2004) What is Joint Attention?

41. Initiating JA Behaviors Eye gaze Affect Gesture Vocalization Symbolic communication From Yoder & McDuffie

42. Behavioral Acts Devoid of Joint Attention Leading / staring without a gaze shift Signing under the table Verbally requesting in an empty room The use of unconventional or undesirable behavior to obtain something Bottom Line: These behaviors are Not indicative of intentional communication!

43. The basic IDEA of COMMUNICATION That it requires at least two people � a SENDER and a RECEIVER That BOTH have to be AWARE of the other That BOTH have to KNOW that they SHARE AWARENESS What is Missing in these Requestive Behaviors?

44. Protodeclarative pointing involves using the pointing gesture not only to direct attention, but also to comment on the object of one�s attention as a topic of interest. It is a joint attention behavior. Protoimperative pointing is requestive behavior (i.e., using the index finger as an instrumental strategy to obtain something for ones own sake). Cookie exampleCookie example

45. Numerous studies have shown that children with autism have a specific deficit in joint attention, and this deficit is one of the most robust & predictive behavioral indicators of autism. What Do We Know About Joint Attention & Children with Autism?

46. �These findings appear to support the proposal that children with autism have difficulty in social orienting and indicate that this impairment is related not only to gaze following but also to the ability to initiate acts of joint attention and language ability.� Leekham, Lopez, & Moore (2000) Leekham, Lopez, & Moore Study

47. �These results therefore provide support for focusing on both dyadic orienting and triadic joint attention in the development of early diagnostic and early intervention measures.� Leekham, Lopez, & Moore (2000) Implications for Diagnosis and Intervention

48. Two Important Questions to Ask How is joint attention related to language development? Are research findings regarding language development for neurotypical children also relevant for children with autism?

49. Joint Attention and Language Development Research has demonstrated in study after study that joint attention is a critical social-cognitive precursor to language development in neurotypical children. Cookie exampleCookie example

50. Relevance to Children with Autism Loveland and Landry, 1986; Mundy, Sigman, and Kasari, 1990 found the same trend in autism. Joint attention and vocabulary size are related in children with autism in the same manner in which they are related in typically developing children, suggesting that the same principles governing language acquisition in typically developing children also govern language acquisition in children with autism. Cookie exampleCookie example

51. Furthermore . . . Siller and Sigman (2002) found that for children with autism, joint attention in their early years had predictive value for language development in their adolescent years. Cookie exampleCookie example

52. Can Joint Attention Be �Trained Up� in Children with Autism?

53. Changes in Affect / Joint Attention Language and Social Change in Toddlers with ASD: Early Intervention. R. Landa, K. Holman, M. Sullivan and J. Cleary. Kennedy Krieger Institute. Objectives: To determine whether language and communication development in toddlers would be related to changes in affective and joint attention development during an intervention program.

54. Results The children showed clinically significant change in language and social domains. Conclusions: Intervention was multi-modal and classroom-based. The greatest language gains were observed in toddlers with the strongest pre-treatment joint attention skills. Improvements in language and joint attention were related.

55. Joint Attention & Symbolic Play Growth in Joint Attention and Symbolic Play. C. Kasari. UCLA. Objective: To assess both the generalization and maintenance of newly learned skills in joint attention and play. Design/Methods: 58 children with autism between the ages of 3 and 4 randomized to a joint attention intervention, a symbolic play intervention, or a control group. (Interventions were conducted 30 minutes daily for 5-6 weeks).

56. Results Children who received the joint attention intervention initiated significantly more joint attention interactions with their mothers than did children in the other two groups. Similarly, children in the symbolic play intervention initiated significantly more symbolic play acts with their mothers than did children in the other two groups.

57. One-Year Follow Up Children in both the joint attention and symbolic play groups increased their joint attention and symbolic play skills, respectively, at faster rates than did children in the control group.

58. Conclusion Children with autism can learn to initiate joint attention and symbolic play skills, and these skills can generalize to other people not involved in the intervention and be maintained over a long period of time.

59. How Does This Research Measure Up to the Quality Indicators? There is a 1 : 1 correspondence between the capacity for joint attention (JA) and language ability / treatment gains. There is concordance among research studies and centers regarding the link between JA and language development in children with autism. 3. Joint attention intervention passes the �goodness of the fit� test, because children with autism demonstrate the need for specific attention to it.

60. 2nd Evidence-Based Intervention Snapshot Given that joint attention is a critical social-cognitive precursor to language acquisition and development; and given that its presence in the early years has predictive value for language development in adolescence, specific and sustained attention should be directed toward the development of joint attention.

61. Types of Activities Turn-Taking Games The anticipatory moment Sensory-based activities Play with toys

62. Shared Storybook Reading as a Context for Establishing Joint Attention

63. Advantages Adult-child reading interactions provide an excellent vehicle for establishing joint focus and shared affect. �Storybooks may be used to foster communicative opportunities for the child in highly contextualized and facilitative routines.� Kaderavek & Justice American Journal of Speech- Language Pathology, November, 2002

64. What Could Cause Information Processing Problems for the Student with ASD? Insufficient background knowledge to afford understanding Difficulty understanding idioms, multiple meanings of words, high-level vocabulary, etc. Possibly delayed processing Background noise / Sensory overload Difficulty dealing with the distractions and rapid pace of the inclusive classroom setting

65. Connecting the Research �Dots� to Create Intervention �Snapshots� How can research findings related to language / information processing in autism inform intervention practices?

66. Just, M.A., Cherkassky, V.L., Keller, T.A., & Minshew, N.J. Published in June, 2004 in Brain: Cortical Activation and Synchronization During Sentence Comprehension in High-functioning autism: Evidence of Underconnectivity

67. �The results [of this study] demonstrated that individuals with autism had enhanced reliance on local processing of individual words, and reduced processing of sentences (i.e., complex information processing).� Reported in: Autism Spectrum Quarterly, Summer 2004, The Cutting Edge: From the Decade of the Brain to the 21st Century.

68. Physical Correlates of Findings Subjects with autism showed less activation in the area of the brain that is associated with semantic, syntactic, and working memory processes, all of which serve to integrate the meanings of individual words into a coherent conceptual and syntactic structure (i.e. information processing).

69. �The subjects also had reduced synchronization between multiple cortical language regions indicating functional underconnectivity.� Reported in: Autism Spectrum Quarterly, Summer 2004, The Cutting Edge: From the Decade of the Brain to the 21st Century.

70. So, what�s the big deal if individuals with autism process information using a word-by-word strategy instead of a more holistic sentence-by-sentence processing strategy??? 

71. Where is Meaning Carried?

72. �The embeddedness of speech in a communicative context is something we take for granted; we do not realize that we are NOT relying simply on the meanings of words put together in a grammatical way.� p. 15 Jerome Bruner & Helen Haste Making Sense: The Child�s Construction of the World

73. �Indeed, it is because speech and thought are usually embedded that the young child is able to engage in apparently complex logical exercises�such as negation�which are way beyond her if presented in disembedded language.� p. 15  Jerome Bruner & Helen Haste Making Sense: The Child�s Construction of the World Grace : Who�s not eating eggs?Grace : Who�s not eating eggs?

74. �Based upon the findings of this study the authors proposed a theory of underconnectivity which they describe as an under-functioning in the integrative circuitry necessary for the integration of complex information at both the neural and cognitive levels.� Reported in: Autism Spectrum Quarterly, Summer 2004, The Cutting Edge: From the Decade of the Brain to the 21st Century.

75. How Does This Research Measure Up to the Quality Indicators? There is a 1 : 1 correspondence between the information processing problems in autism and the decreased activation seen in those areas of the brain responsible for the integration of meaning (complex information processing). There is concordance among many research studies regarding the link between information processing problems and brain function in autism.

76. Robust Findings �Underconnectivity theory enriches Minshew�s previous statements of the theory [of complex information processing] with new findings from fMRI, thus linking the information abnormalities to a specific neurobiologic phenomenon, the brain connectivity itself. � Just, M.A., Cherkassky, V.L., Keller, T.A., & Minshew, N.J. (2004, June 23). Cortical activation and synchronization during sentence comprehension in high-functioning autism: Evidence of underconnectivity. Brain, doi: 10.1093/brain/awh199. Retrieved July 17, 2004, from http://brain.oupjournals.org/

77. How Does This Research Measure Up to the Quality Indicators? 3. This research passes the �goodness of the fit� test, because individuals with autism demonstrate the type of language and information processing problems that reflect word-by-word processing.

78. Robust Findings This new view of the basis of autism stands on the shoulders of previous proposals. It makes sense of some of the lack of convergence of many previous findings, makes good contact with the clinical observations, and provides a link between cognition and brain function� (p. 9). Just, M.A., Cherkassky, V.L., Keller, T.A., & Minshew, N.J. (2004, June 23). Cortical activation and synchronization during sentence comprehension in high-functioning autism: Evidence of underconnectivity. Brain, doi: 10.1093/brain/awh199. Retrieved July 17, 2004, from http://brain.oupjournals.org/

79. The Whole is Greater than the Sum of Its Parts The Bottom Line

80. 3rd Evidence-Based Intervention Snapshot 1. Embed �units� of language (not just individual words) in pragmatically-appropriate, meaningful CONTEXTS that help to fuel understanding and promote sense-making.

81. 3rd Evidence-Based Intervention Snapshot 2. To enable students with autism to process information, teachers should supplement verbal information with visual supports; use repetition and re-phrasing; and provide clarification.

82. The Ultimate Clarification Strategy??? By that I mean . . .

83. The findings of Just et al. (2004) �are also consistent with their earlier findings of decoding strengths and comprehension weaknesses.�Reported in: Autism Spectrum Quarterly, Summer 2004, The Cutting Edge: From the Decade of the Brain to the 21st Century.

84. What Does Research into the Cognitive Strengths and Weaknesses of Individuals with ASD Reveal??? Minshew, N.J. Goldstein, G., Taylor, H.G., & Siegel, D.J. (1994). Academic achievement in high functioning autistic individuals. Journal of Clinical and Experimental Neuropsychology, 16(2) ,261-270 Minshew, N.J., Goldstein, G., & Siegel, D.J. (1995). Speech and language in high-functioning autistic individuals. Neuropsychology, 9(2), 255-261.

85. Strengths of Individuals with ASD The following skills require this type of knowledge: rote memorization word recognition spelling reading decoding

86. Decoding Words Vs. Reading Comprehension Decoding words requires procedural competence (sets the stage for literalness) Comprehending what you read requires a different type of knowledge�one that is a weakness in individuals with ASD

87. Linguistic vs. Pragmatic Comprehension I have a dog (i.e., an animal with four legs that barks). (Tied to linguistic comprehension which is underpinned by procedural knowledge) Vs. That movie was a real dog (i.e., worthless). (Tied to pragmatic comprehension which is underpinned by declarative knowledge.)

88. It�s not enough to exchange words. You also have to exchange meaning!

89. The Bottom Line Pragmatic comprehension ability enables you to read between the lines and infer meaning.

90. Implications for Functioning Given their strengths in procedural knowledge, and their weaknesses in declarative knowledge, individuals with ASD are prone to: Concrete Thinking Literalness

91. Implications for Functioning Individuals with ASD exhibit: Competence re: memorization of numbers, colors, & letters A predilection for facts

92. Academic Weaknesses in Students with ASD The following skills require this type of knowledge:

93. How Do These Research Findings Square with What We See in Students with ASD? Students with ASD have excellent rote memories They tend to be better at reading decoding than comprehension They tend to have difficulty with high-order / critical thinking

94. How Does This Research Measure Up to the Quality Indicators? There is a 1 : 1 correspondence between the pattern of cognitive strengths and weaknesses that individuals with ASD demonstrate, and the research findings regarding underconnectivity in those areas of the brain having to do with higher-order thinking. There is concordance among several research studies regarding the strengths and weaknesses in ASD. 3. This research demonstrates a �good fit� for children with autism, given the procedural strengths they evidence, and the difficulty they have with higher-order / critical thinking (underpinned by declarative knowledge).

95. 4th Evidence-Based Intervention Snapshot Give direct and sustained attention to the development of critical thinking skills beginning in the preschool years and continuing on throughout ALL of schooling.

96. What Does Research Tell Us About Conceptual Development in Children with Autism? There are differences between the way in which children with autism and neurotypical children learn concepts.

97. Concept Formation in Neurotypical Children Typical infants innately abstract generalized representations of categories Dog: Four legs Tail that wags Barking sound Fur

98. Concept Formation in Children with ASD Individuals with ASD do not do this. They learn each example as a separate entity.

99. The Bottom Line People with ASD live in a world of details (can�t see the forest for the trees phenomenon) Leads to situation-specific learning style (cognitive inflexibility)

100. Connecting the Research �Dots� to Create Intervention �Snapshots� How can research findings related to concept formation and concept identification in autism inform intervention practices?

101. �There is a dissociation in nonmentally retarded individuals with autism between concept formation and concept identification. The deficit in concept formation results in cognitive inflexibility and in the inability to spontaneously form schemata or paradigms that organize information.� p. 333 Minshew, N.J., Meyer, J. & Goldstein, G. (2002). Abstract Reasoning in Autism: A Dissociation Between Concept Formation and Concept Identification. Neuropsychology, Vol. 16, No. 3, 327-334.

102. Concept Formation vs. Concept Identification: Replication from the M.I.N.D. Institute Concept Formation and Concept Identification in High Functioning Children with Autism Spectrum Disorders. J. Brown, M. Solomon, N. Bauminger and S. Rogers. UC Davis M.I.N.D. Institute. Objective: To investigate abstract reasoning in children with ASD through an examination of concept formation and concept identification in a sample of 35 children with ASD vs. 35 TD subjects.

103. Results Children with autism were less able than TD children to form concepts They were not impaired relative to controls in their ability to identify concepts.

104. How Does This Research Measure Up to the Quality Indicators? There is a 1 : 1 correspondence between the language use and generalization issues we see in autism and the weaknesses that research demonstrates in concept formation. There is concordance among research studies regarding the dissociation between concept identification and concept formation. 3. This research demonstrates a �good fit� for children with autism, given the strengths and weaknesses that they demonstrate in language that stem from conceptual understanding.

105. Concept Development & Word Learning Are Intimately Related

106. TRUE word learning requires an understanding of its associated concept (i.e., meaning)

107. You Can�t Learn Words without Learning Concepts because . . . The concept that is associated with the word is the word�s meaning

108. The Link between Conceptual Understanding and Word Learning True word learning requires an understanding of its associated concept (i.e., meaning) When you understand the meaning, you have formed the concept and can use the word When you fail to understand the meaning, you may still be able to identify the concept (especially if you�ve been trained to do so). USE of the word, however, will be problematic

109. 5th Intervention Snapshot Concepts (and hence, words) are best learned experientially over time in a social-pragmatic context that supports meaning and language use.

110. How Do We Know This? Because �nobody doubts that for children to learn words they have to be exposed to them in contexts in which they can infer their meanings: this is a truism.� Bloom, P., How Children Learn the Meanings of Words

111. The Power of Experiential Learning �Preschoolers don�t learn language and concepts from two-dimensional flash cards but from multidimensional experience.�

112. What Are the Take-Home Messages?

113. Interventions should be rooted in high-quality research evidence, guided by clinical expertise and family values and preferences (i.e., evidence-based practices). Interventions should target the core deficits in autism, as well as the associated challenges that accompany them (comprehensive in focus).

114. Since not all evidence is created equal, quality indicators related to 1:1 correspondence, concordance among studies and centers, and goodness of the fit should be applied to research findings. A �best practice� approach to intervention in autism requires that practices be rooted in, and informed by high-quality research across several different fields.

115. One size does NOT fit all when it comes to intervention! SCERTS Pivotal Response Training (PRT) ABA FCT RDI Floor Time Treatment efficacy requires that skills learned in the treatment setting be functionally applied in the real-world environment.

116. Be aware�and steer clear�of the �please don�t confuse me with the facts, I�ve already made up my mind� phenomenon.

117. Don�t Believe Everything You Read (or Hear) in the Popular Press! �Journalists typically cover the news with the finding that upsets the apple cart rather than the consensus.� Dr. Steven Pinker Harvard University Journalists often use deceptively definitive and sensational language that can be very misleading (e.g., �Call it a miracle. Call it an awakening.� -Diane Sawyer, Prime Time)

118. BEWARE HYPE: �What is popular isn�t always right, and what is right isn�t always popular.� Develop a critical eye and ear when it comes to evaluating research findings, and triangulate the information.

119. Scientific research is NOT static; hence, it is necessary to remain OPEN to hypotheses that may be rejected over time, as well as to NEW findings that may be added to the research literature. Resist the urge to kill the messenger!!! Last, but not least . . .

120. Some Things Simply Cannot Be Measured! �What was educationally significant and hard to measure has been replaced by what is educationally insignificant and easy to measure. So now we measure how well we�ve taught what isn�t worth learning!� Dr. Arthur Costa The School as Home for the Mind