Meredith Larson, Ph.D. National Center for Education Research and Amy Sussman, Ph.D. National Center for Special Educati

1. First Impressions: How to Win Grants and Influence Your Research Career Meredith Larson, Ph.D. National Center for Education Research and Amy Sussman, Ph.D. National Center for Special Education Research

2. Goals Identify two things you should have with you at all times. Describe a theory of change and discuss why it’s essential grant writing. Have the tools necessary for refining your research statement of purpose. Have a sense of what program officers and reviewers want from you and your applications. All of this is to help prepare you to write winning proposals and articulate your research agenda to the broader (research) community.

3. Our Assumptionsof You and What You Want • We assume that you want • to help improve education • to carve out your own research agenda • to get funding to conduct your research • We assume that you have • expertise in a wide range of topic areas and research methods/analyses • varying levels of familiarity with grant writing

4. What You’re Facing Whenever you ask someone to support your research, remember that it all comes down to convincing them that they should care about your topic and that you have the necessary competencies to reach your goals. …so how do you do it?

5. What You’re Facing Audience awareness • Seek common ground. • Highlight core values or concerns that your research addresses. • Look for ways to build off of things that are familiar. • Be helpful and thorough. • Make things easy to understand, remember, and find. • Be engaging and helpful.

6. What You Need to Do Sell your research idea, promote yourself as the best person to do the research, and build goodwill and trust. How? By demonstrating that you know what the problem is and have a way to address it.

7. Explaining Your Research You should have two things ready at all times: • Statement of Purpose (preferably an “elevator speech” or “research pitch”) • Theory of Change

8. Statement of Purpose We will come back to this later, but in general a ready-made Statement of Purpose should • Be short and attention-getting and • Contain the problem statement and your contribution to solving it. Your fellow researchers, friends, and family members should be able to understand it and see its relevance. NOTE: It’s not as easy to do as you may think. You have to know a lot about what it is you want to do before you can describe it succinctly.

9. Theory of Change (ToC) • This is the model underlying your research. • It is a roadmap to your grant narrative. • It is a source for generating research questions. • It is constantly evolving. NOTE: Some fields and scholars use terms like Logic Model or Logical Framework to mean things similar to what we are calling a Theory of Change here.

10. ToC: What is It? Theories of Change help define how and why an intervention (e.g., curriculum, policy, practice) should lead to particular outcomes. • Makes assumptions explicit • Suggests the causal relations

11. ToC: What is it? Some associate them with program evaluation studies, but they are also useful for exploration work, work on cognitive or behavioral processes, etc. They are frequently represented visually, but they can be simply text.

12. ToC: Why You Should Care Reviewers will evaluate your research on how clear your theory and model are. A strong ToC will help make your case for you. Four Criteria Used to Judge ToCs(Connell & Klem, 2000): • Plausible • Doable • Testable • Meaningful

13. ToC: How to Make One • Start with the long-term outcome and work backwards. • What preconditions need to exist to lead to the outcome? • What is occurring in the context that could hinder or support the outcome? • What assumptions are you making? • Draw it out. • Write it up. • Get feedback. Revise. Repeat.

14. Create the General Framework Intermediate Outcomes Long-term Outcomes Strategies/Activities Initial State Context Come up with the broad framework. Generally, you should start with the outcomes. You can then jump all the way back to thinking about the initial state or hop around.

15. The Basic Components Assumptions Outcomes (proximal & distal) Strategies/Activities Initial State (resources) Context INITIAL STATE: What’s there before the intervention including things such as resources or student characteristics, etc. (sometimes called input) STRATEGIES/ACTIVITIES: The things that happen (e.g., activities, events, curriculum) to the people who participate or who are the targets of the intervention (sometimes called output) OUTCOMES: The proximal and distal changes for people involved in or that are the targets of the intervention ASSUMPTIONS: What you believe to be true of the resources, the people involved, the people targeted, and the intervention in general CONTEXT: The environment (internal or external) in which the intervention takes place (e.g., other policies or practices occurring simultaneously)

16. Example Positive attitudes to school Exposed to intervention 1-hour lesson Wrap-around services Positive supports Focus on early literacy PD for providers 4-year-old pre-K children Demographics Eligibility/need Improved pre-literacy skills Increased school readiness Greater gains in K literacy Learn appropriate school behavior Setting/context Personal and family characteristics Prior experience Teacher/provider experience

17. Detailed Components Now that you have the general aspects, you should specify the details. These will lead into your research design, measures, etc. in your narrative. Indicators: Operationalizing the outcomes (e.g., what indicates success?) Populations: Identifying who should show which change (could be multiple groups) Thresholds: Setting your expectations for change for each outcome (e.g., how much is good enough?) Timeline: Determining when we should meet the threshold for each outcome (Connel & Klem, 2000)

18. Example OUTCOME = Improved pre-literacy skills Indicator: Correctly identifies letters and sounds Population: 4-year-olds, especially those with little print exposure Threshold: 1.5 effect greater than comparison Timeline: After full intervention, 16-weeks

19. Another Example (Slightly different layout ) Motivation to Read Decision to engage in reading and task persistence Instructional Context Text and task characteristics Nature of instructional support Expectancies for success Self-efficacy Perceived control Value Achievement goals Intrinsic motivation Usefulness Social motivation Text Comprehension Reader Characteristics Decoding and fluency proficiency Verbal knowledge Attention and behavior Reading Engagement Physical Cognitive: Building and maintaining coherence Adapted from a model presented by Dr. Sharon Vaughn (R305F100013).

20. Don't Do This • Be overly simplistic • Overwhelm the reader • Use color as a key (audience awareness!)

21. Example: Too Simple The Increasing School Success Program Students with Disabilities Increased Academic Achievement

22. [Processes 3-17 repeat twice in Year 2] 18a. submit Annual Report to schools 17. reviews/revise model based on findings 8a1. Coaches assist PLTs in using all data 16. interview coaches, SLTs, and PLTs about PD and WL processes 13-15b. Observe SLTs, document implementation 8a2. Coaches assist PLTs in weekly WLs and Debriefing through mid-year 6-8b. observe PLTs, documents implementation PLT WL Debrief PLT WL Debrief 15a.Coaches assist SLT in implementing PD with faculty PLT WL Debrief 7a. Coaches teach PLT to conduct WL PLT WL Debrief PLT Begins weekly meetings SLT 1 Implement Prof Devt 14a. SLT and Coaches create PD unit PLT WL Debrief 6a. Coaches facilitate PLT identification of annual goal PLT WL Debrief approx. 3 wks after school begins PLT WL Debrief 5. guide Coaches and PLTs in analyzing data collected PLT WL Debrief PLT Profile Analysis Set instructional goals & WL focus 13a. researchers train SLT to research best-practices in PD area PLT WL Debrief PLT WL Debrief [4 weeks at end of prior year or beg of current year] SLT 1 Research 4. Coaches work with PLT PLT WL Debrief 3b. develops electronic tool PLT WL Debrief PLT WL Debrief [PLT appoints SLT1 to address PD Topic 1] PLT WL Debrief 9. interviews coaches and PLTs about WL process [Processes 4-11 repeat to mid-year] = Begin Process =Feedback for next process = Delegation of PD 3a. Coaches collect 3 yrs. stud. ach. & demo. data per school 10. reviews/revises model based on findings 12. Coaches & PLTs choose 2-4 teachers (based on WLs) to become Dynamic Leadership Team 1] 11. Coaches share/ implement revisions with PLTs 1a. PI Recruits and Trains Coaches 2a. PI matches Coaches to schools (n=5) PLT = Primary Leadership team SLT = Secondary Leadership team approx. 9 wks after school begins 2b. trains coaches to use all data tools 1b. develops data collection tools

23. Draw your Theory of change Take a few minutes and write a couple words about the following: • What are the initial states (inputs)? • What are the strategies/activities (outputs)? • What are the outcomes? Proximal, Intermediate, Distal… • What is in the context?

24. So Now What? Your Theory of Change acts like a roadmap that can help lead you to research questions, a proposal, and so much more.

25. ToC & Research Questions (Milanowski& Kimball, 2009) Questions about initial states or inputs • Were resources provided and used? Questions about strategies/activities or outputs • Did the activities occur as planned? Questions about outcomes • Were there changes in the proximal/distal outcome relative to the control/comparison? Questions about context • Did the context suppress or reinforce effects?

26. ToC & Your Application(s) In IES applications, there are four major parts of the application: Significance, Research Plan, Personnel, and Resources. The first two of them can draw heavily from the ToC. Significance – Draws heavily from the outcomes, context, and assumptions Research Plan – Draws heavily from the strategies/activities and the outcomes

27. ToC & Your Research Plan In your research plan, you need to specify exactly what it is you’re exploring, creating, validating, or testing. You also need to specify how you will do these things. Strategies/Activities What are the pieces that you’ll be exploring, creating, testing, etc.? Outcomes Indicators: What will you measure, and how you will measure it? Populations: Who and where (both in treatment and control/comparison)? Thresholds: What effect (size) should you expect? Timeline: When should you be collecting what data?

28. ToC & Your Application(s) Other components of your narrative and your application can also be informed by the ToC: Personnel – For example, you want someone with experience in each of the outcome areas and with knowledge of how to assess the strategies/activities. Budget – For example, how many observations will you need to take (# of researchers and % effort) and when (during what grant year) and what assessments you will need (how many of which one should you buy).

29. ToC & Your Statement of Purpose Your Theory of Change also contains the core elements of a good statement of purpose, mission statement, elevator speech, etc. You have the pieces. Put them in a short, digestible format that conveys the problem’s significance. And now you have your statement. (It actually requires a bit more than that.)

30. An Example Now we’re going to walk through an example of how you might take a ToC and generate an “elevator speech”, a statement of purpose, even an opening paragraph. Note: Sometimes people have their statement and then create their ToC. You need both, and they inform one another. You do not need to have a ToC to write a statement.

31. From ToC to Statement of Purpose Distal Outcome: Increase student achievement in science Proximal Outcome: Young children (preK – K) with greater ability to think scientifically Strategies/Activities: • Professional development curricula • Four 1-hour long workshops for teachers • Training on science content and early childhood pedagogy • Current, practicing educators • PreK students Initial State: Early childhood educators who lack science content and pedagogical knowledge, lack of curricula to help teachers Assumption: Teachers are essential to student outcomes. Teachers need both content and pedagogical knowledge and expertise. Teaching teachers will improve student outcomes.

32. …continued Speech: Being able to think scientifically at a young age predicts long-term academic success in science. However, many children coming into elementary school lack this ability. Even if they are exposed to science in PreK programs, they are still not acquiring the necessary skills. One reason PreK students may be struggling is that early childhood educators often do not have the background knowledge of science content or instruction to help their students. Because teachers are so important to the learning process, we need to help better prepare them for their role. So I am developing an intervention that will increase early childhood educators’ content and pedagogical knowledge so that they can better prepare their students.

33. Components of a Good Statement Story! Characters, goals, drama, resolution. Facts – but not too many. This will vary based on audience and use. Short – shoot for a paragraph or < 90 seconds. Make them care.

34. Now you try What is the problem? The tension? The drama? What evidence do you have that it’s important? How can you help solve it?

35. Uses of Your Statement • Opening paragraph(s) of your applications or your purpose paragraph • Note: This is true for all documents, such as dissertation proposals, not just IES applications. • Job talks • Communicating with the general public (e.g., when you give your NPR interviews)

36. Importance of Opening Paragraph • Opening paragraph sets the scene for readers. • Identifies the significance of the work to be done and what actually will be done • Readers use it to organize information in rest of the application • You can lose your readers right off with an unclear opening

37. NCER/NCSER Abstracts As an example of an opening/purpose paragraph, we’d like to use examples from some of the abstracts we have posted online: http://ies.ed.gov/funding/grantsearch

38. Example of Opening/Purpose Background Problem Goal: How will you address this problem? Purpose: Teaching children how to think scientifically in the preschool years has the potential to address an existing achievement gap in early science and provide children with the skills necessary to continue learning and thinking critically throughout the school years. Unfortunately, early childhood teachers typically lack content and pedagogical knowledge in science and are not prepared to provide developmentally appropriate experiences that support children's early science learning and readiness. In order to address these challenges, the researchers will develop the Cultivating Young Scientists (CYS) intervention,which includes a professional development program for early childhood educators, science curricular content, and a set of formative assessment tools. The intervention is intended to lead to an increase in teachers' use of science instructional content and practices in preschool settings and improvements in young children science content knowledge and scientific thinking skills.

39. Example of Opening/Purpose Background Problem Goal: How will you address this problem? Core diagnostic features of autism include deficits in social-communicative functioning. Two pivotal skills for young children with autism include joint attention and pretend play, which constitute early foundations upon which later social-communicative skills are built. Joint attention (characterized by behaviors such as pointing, showing, and coordinated looking to share attention toward objects or events with another person) and symbolic play (characterized by the ability to pretend), play important roles in language development and social engagement with peers. Children with autism show deficits in these skills, and as a consequence may lag behind their peers academically and socially. Advancing Social-Communication and Play (ASAP) is an intervention that has recently been developed to help preschool children with autism learn and practice these important skills. However, we do not know how effective it is. The purpose of this research is to conduct a cluster randomized trial to evaluate the efficacy of ASAP. The major goals of the project include investigating whether children experiencing the intervention demonstrate greater gains in the proximal child outcomes of social-communication and play skills as well as the more distal outcomes of language development and engagement with classroom objects and peers. The study will also examine whether child-level and teacher-level (i.e., teacher burnout, general classroom quality) characteristics moderate the impact of the intervention.

40. Final Comments: Helpfulness of the Program Officer • Share your framework and statement of purpose with the Program Officer • Ensure you are submitting to the correct competition/topic • Springboard for further discussion

41. Final Comments: Importance of Clarity of Writing • Readers (e.g., application reviewers) often complain about lack of clarity. • Significance too general • Lack of detail regarding intervention, development cycle, or data analysis • Use of jargon and assumptions of knowledge • Poor writing (e.g., grammar), awkward constructions, etc.

42. Final Comments: Importance of Practicing All forms of communication require practice. Things only other people can tell you: • Is your description of your theory too long or complex? • Is your statement of purpose too long or complex? • Is your idea engaging, and does it invite questions that build off of your ideas (rather than those that try to figure out what your idea really is)?

43. Pulling It All Together Recall Why You’re Here: You want to write winning grants and build a solid research career. But How? By getting people interested in your work and the questions you feel are important and by building trust that you have a plan for addressing an important issue.

44. Pulling It All Together • Have a theory of change • Should be as fully explicated as possible • Is always being refined • Should be made with and informed by others • Have a clear statement of purpose at all times • Your “elevator speech” • Simple, elegant (hard to do, actually)

45. Thank you. Meredith.Larson@ed.gov Amy.Sussman@ed.gov

46. More info Remember: Some scholars distinguish between a Theory of Change and a Logic Model. Others do not. We use the term “Theory of Change” for the purposes of this presentation as a way to discuss how people can present the causal relations among elements. Please use the term most appropriate for your field. (Note that the IES RFAs use “Theory of Change.”) This presentation was informed in part by the following resources: Connell, J., & Klem, A. (2000). You can get there from here: Using a theory of change approach to plan urban education reform. Journal of Educational and Psychological Consultation, 11(1), 93-120. Center for Theory of Change: http://www.theoryofchange.org/. Retrieved May 1, 2013. Harris, E. (2005). An Introduction to Theory of Change (Issue Topic: Evaluation Methodology, vol. 11). Retrieved May 1, 2013, from http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/evaluation-methodology/an-introduction-to-theory-of-change. Kimball , S. & Tony Milanowski, T. (2009). Establishing a Theory of Action and Logic Model for Your Project Session 1 [PowerPoint slides]. Retrieved on May 1, 2013 from Center for Educator Compensation and Reform website: http://www.cecr.ed.gov/pdfs/september2009meeting/TheoryofAction.pdf. Reisman, J., Gienapp, A., Langley, K., & Stachowiak, S. (2004). Theory of change: A practical tool for action, results and learning. Organizational Research Services for Annie E. Casey Foundation.