RESUBMITTING A GRANT APPLICATION

1. RESUBMITTING A GRANT APPLICATION David Felson, M.D., M.P.H. Professor of Medicine and Epidemiology Principal Investigator, B.U. Multidisciplinary Clinical Research Center Grant Director of Training, B.U.CTSI Former Member, NIH SBSR and EDC1 Study Sections

2. Outline of this Talk • Review of Options in Submitting a Grant • Rates of Success of Grants at NIH • Deciding about Resubmission • Examples of Approaches • Writing to the Reviewers

3. Why Would You Want to Write a Grant? • To Have the Resources and Time to Pursue a Question that you believe is important • To protect your time and give you freedom • To pay your salary • As a measure of academic accomplishment

4. Good research skills Salesmanship Good communication skills Persistence Ingenuity and flexibility Administrative skills Good human relations What Qualities Characterize a Successful Grant Writer

5. Career Development Grant (K series) Funds you, a junior investigator Goal is to protect your time/optimize career development Scientific proposal not only part of grant Grant to do Research (R, P series) Provides funds to carry out a research project Scientific proposal is central piece of grant, includes prelim data, experience of investigators What Kind of Grant Should You Write?

6. Career Development Grants at NIH

7. The Travels of an N.I.H. Grant Application (R series)* • Based on title and abstract, Center for Scientific Review (CSR) assigns your grant to institute and study section, a review group (SRA heads study section) • Institute appoints a primary contact person for your grant *K grants go to organ-based NIH institute (e.g. NHLBI) but process is otherwise the same

8. Council News| Initiatives, Budget | Grants, Contracts | Clinical Research | Training Programs | Small Business Contact Staff | Glossary Funding > Grants and Contracts > All About Grants > Supplemental Materials Funding Decision Flowchart This flowchart shows the major choices reviewers and program staff make about each application to determine whether it will be funded. For information about when funds are released, see the Funding Timeline. To return to the page you came from, use your browser's "back" button. Or, view the supplemental materials index for "All About Grants". This page last updated September 6, 2002 (jdlc)

9. N.I.H. Study Section • 14-30 N.I.H.-funded scientists representing a wide range of expertise and geographic & gender diversity • Each grant reviewed by  2 members • Members review 6-10 grants each • Study section reviews 80-100 grants • Each study meets three times per year for 2 days each time

10. Once you’ve submitted, you’re not done • Check Commons to determine review group • Examine the names of listed reviewers; check with your mentors/senior colleagues • Are those with right expertise reviewing your grant? • Act early (with institute assignee) to challenge reviewers if needed.

11. Study Section Review: Topics in Write-up of a Grant Review • Significance • Innovation • Approach (general feasibility, power, methodologic concerns) • Other factors (human studies, productivity or promise of the investigator)

12. Priority Scores • 50% of R applications are unscored and not further discussed. Since there is no discussion, there is no summary of concerns. • Priority scores (1 - 9) are assigned usually by consensus after airing of reviews. • Priority scores get assigned a percentile ranking based on current and previous study section reviews. • Summary Statement is prepared by SRA.

14. From Study Section to Institute Council • Institutes have ‘pay lines’: the percentage of competitive applications funded. These serve as general guides. • Grant and summary statements sent to Institute Advisory Council. Council can change ranking of grant based on institute priorities. • Institute staff make ultimate funding decisions.

15. K23 success by year • 2002: 46.6% (the high point) • 2008: 37.6% • 2009: 43.9% • 2010: 37.8% • 2011: 34%

16. K08 Success by Year

17. Research Project Grant investigators Competing applicants, awardees, and funding rates

18. Research Project Grants Success rates, by gender

19. R01-Equivalent grants, New (Type 1) Success rates, by career stage of investigator

20. 2011 N.I.H. Funding Success Rates by Submission No. of Type 1 Grants* *Type I grants are new grants, as opposed to renewals

21. Your future as Grant Applicant • You will be faced with an unfunded grant. • You will have to decide whether to: • Revise and resubmit • Dump idea • Repackage and submit in another form/to another agency • New policy restricting to one resubmission changes strategy.

22. How to deal with the Summary Statement if your score is not good enough to be funded • Read summary statement seriously only after you have put it away for a week. • Show summary statement to senior colleague/mentor who has experience with NIH reviews.

23. Deciphering the Summary Statement if your score is not good enough to be funded • If >= 2 reviewers raise the same or similar issue, you must address it. • Read the summary of discussion (at beginning) carefully. Written by SRA, it contains main discussion points and will often guide you on whether to resubmit.

24. Deciding on Revise/Resubmit vs. Other • Need to decide soon to revise/resubmit and begin to work on revision/prelim data • Lots of work/time went into original submission. A revision is MUCH less work. Bias toward resubmission. • If you decide not to revise/resubmit, the decision on what to do can wait

25. When NOT to resubmit:If you grant is triaged… • No discussion took place (other concerns could have been raised) • No impact scores provided • Only one resubmission allowed • Do not Resubmit Exception: if all reviewers point to one easily rectifiable problem and no other major ones.

26. When you should resubmit… • Initial score in 10’s, 20’s (reviewers often thought you would be funded and did not have major concerns; they will sometimes be upset NIH did not fund when they see it again.) • When concerns expressed by reviewers are addressable.

27. What types of concerns are usually addressable? • Lack of preliminary data (get data) • Lack of innovation (market this better) • For K grant, lack of institutional support • High level of expense (can influence scores)

28. What types of concerns are usually not immediately addressable? • Lack of productivity (get more papers) • Lack of significance (sometimes can market this but if >1 reviewer, this is problem)

29. And if you decide NOT to resubmit… • Explore other funding options • For Career Development: Private foundations (often specialty based) • For Research Grants: part of Center grants, R03, R21, private foundations (e.g. AHA, Arthritis Foundation); local sources (e.g. pilot projects) Much work and thought involved in developing the idea. Can the idea be modified?

30. R03 • Small grant ($50,000 per year X 2 years) • Ideal for collecting preliminary data • Also used to carry out secondary data analyses • Not available at all NIH Institutes • Funding success rates similar to R01 • Resets the R01 clock

31. R21 • Up to 2 years with total of $275,000 (no more than 200K in any year) • Must be novel or high risk/high reward study. Develop new method/technique • Not available at all NIH institutes • Funding success rates similar to R01 • Resets the R01 clock

32. Can you make your failed application into a NEW ONE? • New Application: objectives, aims, hypotheses all changed; or a new PI • The Following would NOT be new applications: • Title change • Change in wording of aims/objectives • New review panel or institute assignment

33. Example of R01APOS Shoe Trial Popular in Europe As Treatment for Knee OA • Placement of balls on the plantar surface can lead to unloading of painful medial compartment of knee. • Unloading medial compartment may be visualized on MRI as shrinkage of bone marrow lesions. • Walking on hard rubber balls can retrain lower extremity muscles, producing coordination/conditioning. • One controlled (not randomized) trial with controls normal shoes showed markedly positive results for reduction of knee pain. No RCTs. NO sham controls

34. An R01 NOT Resubmitted: APOS shoe for knee OA: the Review • Not discussed (triaged) on 1st submission (2 of 3 reviewers with major scores >=4) • Exerpts from Reviewers’ comments: • Although the investigators may be able to demonstrate a change in the adduction moment at the knee with the use of the novel foot wear will not explain how that foot wear accomplishes it. Little or no theoretical explaination was offered for the use of the foot wear nor is there an attempt to explain how it might work. As such, the significance of the study is severely reduced. • The use of “sham” footwear is valuable, but additional control conditions should be included to better indicate the possible effects of footwear on knee OA, moments, etc. The investigators should seriously consider including some type of “standard” footwear and barefoot conditions.

35. APOS Review • No information is provided regarding how the technician for the APOS foot wear will make the needed adjustments. This introduces a possible confounding variable. The Principal Investigator should at least provide information regarding what adjustments are made and upon what factors or findings. Furthermore, what is the reliability of the person who will be making the adjustments for the footwear? Can they be replicated? • Although the WOMAC is the most frequently used functional outcome tool, it does have some problems particularly in the stiffness and physical function subscales. These problems could lead to a difficulty in demonstrating a change as the result of the treatment. In addition, a change of 7.8 seems to be too low to be clinically relevant although statistically significant. • The prevalence of Bone Marrow Lesions has been reported to be less than 78% (stehling, et al, 2010 reported less than 50%). BMLs may also be greater in those who are more physically active and may be progressive over time. These present moderate concerns regarding sensitivity to change in a sham controlled trial

36. A K23 resubmitted successfully • Initial summary statement concerns in resume (score 200 (scale 100-500)) However, the Review Committee identified some weaknesses that reduced the enthusiasm. While the research area is important, there are some issues related to the research design. For example, there is a concern that the research proposal is based on a large VA database which has complete focus on men when glucocorticoids are major drugs used by women with rheumatic diseases. In addition, there are some concerns about the proposal presentation and discussion. Furthermore, the reviewers note that a closer mentors’ involvement in the candidate’s application preparation will be helpful. Overall, the Review Committee is positive about the candidate’s commitment, mentors’ expertise, career development plan and institutional environment and support.

37. RESPONSE TO REVIEWERS • “Because there are few women, the study will include only men. This will certainly limit the generalizability of the conclusions drawn from this research.” • We have done additional work to verify the number of expected women and men in this study. In VA fiscal year 2005, we identified 225,841 patients who received a GC prescription at least once. 14,914 were women. This estimate is for a single year whereas our study will encompass eight years. Though women are certainly less prevalent than men, this is an older and relatively high risk population; therefore, the rate of CV diseases in both sexes is expected to be high. Thus, there should be enough women to be able to perform sex-specific analyses.

38. An R01 resubmitted: structural correlates of knee pain with MRI (from sum statement resume): • …In slight counterbalance, the review group commented on the lack of details about magnetic resonance imaging (MRI) data acquisition strategy and drawbacks in the interpretation of results; need for further validation of the quantitative assessment of synovial proliferation; hypothesis on pain studies are not carefully thought out (categorizing quality of pain into three variables are inappropriate); need to address clinical site requirements and subject issues (the review group commented on MRI Issues for e.g., the clinical sites currently use 1.5 Tesla magnets and research studies use 3.0 Tesla instead of 1Tesla and further, the instrument proposed is outdated). Again, the review group enthusiasm is modestly tempered by the lack of imaging expert and absence of defined innovation. On balance, these issues do not fatally overshadow the positive features of the proposal thus consequently, the review group has expressed a fairly high (outstanding-excellent interface) level of enthusiasm for its support.

39. SUCCESSFUL RESPONSE TO REVIEWERS: R01 • Added two MSK radiologists to study • Defended the MRI’s to be used and provided specific acquisition parameters to be used • Provided preliminary data validating synovitis • Added additional analyses suggested by reviewers (nocturnal pain) • Pointed out conflicting reviewer concerns • Shortened study to 3 years (from 4).

40. Deciding whether to Resubmit: Know the Odds and Maximize Them • Find out success rates for grants like yours. • Submit to agencies which offer the highest likelihood of success. • Search out private charitable organizations interested in funding your kind of research.

41. Who Reviews the Revision? • Do not ask for different review group. They will raise yet other concerns. • Generally, one of the previous reviewers is assigned the grant. Other reviewers will change. • Most rereviewers (but not all) will focus only on your response to reviewers.

42. Writing Response to Reviewers (1) • The reviewer is always right (even when they’re not!). Be respectful. • You do not have to respond to every issue, but must articulate why not. • If at least two reviewers raise an issue or it is raised in summary, you must address. • Try to ‘throw them a bone’---don’t argue with every suggestion. Give in a bit.

43. Writing Response to Reviewers (2) • Respond to reviewer by reviewer in order. • Make it easy for reviewers to track changes in revised grant. (underline or bold new text).

44. Be nice to your reviewers in your revisions • Large Font if possible • Clarity (a messy grant means messy science). • Tables/Figures—whenever possible • Consistency in #’s/labels! • Make revised part look different from original

45. Other tips on Writing Grants http://grants1.nih.gov/grants/grant_tips.htm