Where am I, and who are you? Developing Research Competencies: Charting the Course

1. Where am I, and who are you?Developing Research Competencies: Charting the Course des Anges Cruser, Ph.D., MPA Associate Professor & Administrative Director ORC Keith Watson, DO Associate Dean for Graduate Medical Education OUCOM/Centers for Osteopathic Research and Education Rorie Lee, PhD, MPH, CHES Assistant Family Practice Residency Director UNECOM des Anges Cruser, Ph.D., MPA

2. What’s the purpose? • Share information about research training activities • Stimulate collaborative discussions about the need and vision for training in the conduct of research

3. KEITH WATSON Broad perspective of trends in research training in osteopathic medical education & the relationship between evidence-based medicine and research training NEXT Each of us will present information about what we are doing in providing research training 10 MINUTES KEITH WATSON Lead a discussion involving you in identifying the conditions that constrain and those that promote the provision of research training to medical students, residents, and in post-graduation professional development venues. What should I expect?

4. Research Competency Challenges Promoting wishes and understanding challenges D. Keith Watson, D.O., FACOS

5. What is Research? (What is Research Education?) • Do all the blind men describe the elephant in the same terms? • Do all the players mean the same thing when referencing ‘research’ or research education? • accrediting bodies • specialty colleges • program directors • trainees • osteopathic medical schools • clinical faculty • basic science faculty • etc.)

6. Goals for Research Education • Meet the program requirements for a research product • Meet the AOA Competencies which require research

7. Goals for Research Education • Develop informed consumers of research outcomes • Equip physicians to become active participants in research trials and studies • Prepare Osteopathic Physicians to be Project/Primary Investigators for quality research initiatives

8. Goals Some Project Investigator Most Active Participant in Research Studies Informed Consumer of Research Outcomes All

9. “Research” and “Research Education” • Evidence Based Medicine • Bench projects • Clinical trials • Prospective? • Retrospective? • Randomized/controls? • Case reports • Statistics course • Literature review • Journal Clubs

10. Evidence Based Medicine - Assumptions • Volume of published material is too great to assimilate/evaluate • Much published material is • Out of date • Not based on valid research design • Not applicable to the current patient care need under study • Not likely to make a real difference in patient care outcome • Most clinicians will NOT be trained or able to do in-depth research • Clinicians need a mechanism to strategically manage this information

11. Evidence Based Medicine—Current Trends • Utilize auxiliary information services to screen literature in a reliable way from many perspectives- • Patient Oriented Evidence that Matters (POEMS) = InfoPOEMS (InfoRetriever) • Up-to-Date • “Just in time” information at the point of care • Critically assess literature and data ABOUT: • Diagnosis • Treatment recommendations • Laboratory/radiologic test necessity and accuracy • Pharmaceutical rep/sales information

12. Application of “Evidence Based Medicine” to Practice • Research MAY define “best practice”for population under study • Consideration of Patient Issues which demand priority against the ‘best practice’ because of individual variation/need • Clinical judgment ALL Must be Weighed for Ideal Patient Outcome

13. “Research” and “Research Education” • Evidence Based Medicine • Bench projects • Clinical trials • Prospective? • Retrospective? • Randomized/controls? • Case reports • Statistics course • Literature review • Journal Clubs

14. Reading and classroom work Volume of patients seen “Show me your references……” “See one, do one, teach one” etc = Clinical Competence Educational Model Assumption X

15. Take a statistics course Do a literature review Write a case report Learn to read/critique journal articles Do a research ‘project’ = “Research Competence” Educational Model Assumption X

16. What are the basic research competencies ALL DOs need to practice contemporary medicine? • Be an informed consumer of research publications • Individual study question • Meta-analysis • Population vs. individual application • Know basic statistical terminology • Know basic study design and inherent flaws • Common wisdom: “Don’t need to DO research, just be educated about it” XXXChallenge to the concept XXX

17. Recommended Minimum Standard for Research Competency in OME • End of medical school years 1-2: • History of osteopathic research • Knowledge of research vocabulary • Ability to do a literature search • Knowledge of basic statistics • Understanding of research problems uniquely osteopathic (OMT) • Awareness of support resources available consistent with level of competency expected Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002 Louisa Burns ORC March 2003

18. Recommended Minimum Standard for Research Competency in OME • End of medical school years 3-4: • Ability to review and summarize journal articles • Ability to formulate a research question/hypothesis • Awareness of support resources available consistent with level of competency expected Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002 Louisa Burns ORC March 2003

19. Recommended Minimum Standard for Research Competency in OME • End of post-graduate years 1-3: • Understand the process of design and implementation of a research project • Ability to critique journal articles • Ability to write a manuscript suitable for publication or a grant application • Awareness of support resources available consistent with level of competency expected Ref: OCCTIC III Fall 2001, ECOP Fall 2001, AOA BOR Nov 2002 Louisa Burns ORC March 2003

20. Challenges • Volume of published studies • Emerging methods to deal with it • Osteopathic profession’s urgency to focus on OPP research • Unclear specialty college requirements for resident research • Untrained (uniformed) program inspectors about research expectations • Poor trainee writing skills • Untrained program directors (faculty mentors) • Ill defined IRB functions

21. Challenges • To Identify competency ‘outcomes’ that are realistic to measure • Define the appropriate achievement for each level of the continuum • To start research education in UGME • To distinguish research from research education • Fund sufficient infrastructure

22. Challenges • Funding for research infrastructure • OPTI inherently disadvantaged to provide from member dues/fees • Few sufficiently organized to obtain federal grant funds independent of the COM (grants management issues). • Some may fund startup activity from foundations with little chance for converting to ‘hard dollars’ • COMs not a likely source for GME trainees • Difficulty in providing faculty research resources now • Growth of grant supported infrastructure will be difficult to expand to GME sites/programs • Conversion to hard dollar support not easy

23. Challenges • Funding/Creating Research Education Infrastructure • Where/everywhere in the curriculum continuum? • Definitions and common objective terms for curriculum committees • Aligning agreement of curriculum committees from: • COMS • Specialty Colleges (RESCs) • OPTIs (OGME Committees/Research Committees)

24. Goals Some Project Investigator Most Active Participant in Research Studies Informed Consumer of Research Outcomes All

26. Where we were (2000) • OUCOM – UME • Summer Undergraduate Research Fellowship (for ~ 8 OU undergraduates BEFORE acceptance to OUCOM—Eight week program) • PMPH curriculum included research concepts • Summer Research Experience (10 weeks/~10 students) for select COM students who have finished Year 1

27. Where we were (2000) • CORE GME • 76 programs + 12 internships (~550 FTEs) • Multiple paper requirements from specialty colleges, struggle with identifying uniform resources/access to help • 10/12 hospitals had functional IRBs • Some had Federal Wide Assurance, none had reciprocal agreements with Ohio University IRB • Two RPACs had a resident paper presentation day • No RPAC or program required IRB approval of research BEFORE the project began • CORE Research Committee had SOME seed money but proposals coming forward were not • Written well • Had poor study design (if any) • Had few program directors/faculty who could mentor well

28. Approach to GME Research • Require all hospitals to have an IRB and obtain FWA—and an Inter-Institutional agreement with OU IRB • Apply for grant funding and hire a Director for CORE Research with both statistical experience and project completion experience • Attempt to meet specialty college requirements as written for a meaningful research product for each resident • Provide a forum(s) for well conceived and completed projects to be presented • “Raise the bar” expectations at the RPAC level for program directors and trainees regarding research education and project completion

29. What we learned • IRB concepts are not common knowledge or high priority in community hospitals • Program directors place little emphasis and allow little time for trainee research activities • Specialty college requirements are too non-specific and give little direction for PDs • GME trainees have difficulty with writing skills • Few know the elements of a research proposal • Few could list the usual sections of a journal article • Fewer could write a coherent project argument based on a bona-fide literature search • Few PDs or trainees understood the ‘life-cycle’ of an idea to completed project or the steps to take/or where to turn for help • Virtually ALL TRAINEES want to be involved and do a good job with a research project….and will do so if given the tools.

30. Where we are (2004) OUCOM--UME • SURF Program continues for pre-admission activity • Summer Research Experience continues for select group of Yr 1-Yr 2 students • The PMPH course (old curriculum) has now been replaced in the new Clinical Presentation Continuum (CPC) with 22 contact hours of epidemiology, statistics, Evidence Based Medicine principles, and population impact presentations. • CPC is case based and small group oriented with select lecture/problem presentation sessions.

31. Where we are (2004)CORE GME Research • Full time Director of CORE Research (grant funded) • Support staff = nil • All CORE Hospitals now have a functional IRB and an FWA status. Some have I-I agreements with OU IRB • Extensive faculty development programs for Program Directors have heightened research awareness and sparked support • Wide dissemination of posters/training materials/etc have helped educate and give direction. • Several forums for presentation have been created • RPAC Research Days • OOA Convention • OUCOM Research Day • Research days at CORE hospitals opened to other institution’s resident participation…..

32. Where we are (2004)CORE GME Research • Resident projects are ‘guided’ through a central process of writing, IRB approval and data management • Current work in progress = ~100 active projects in the cue for IRB and completion (up from only 4 in January of 2003) • Four of eight RPACs held Senior Paper presentations (with judged outcomes) this Spring • Case reports were not encouraged • Many well designed retrospective chart reviews were completed • SEVERAL prospective multi-arm studies were completed and reported with small but significant outcomes.

33. What we have learned • The culture must change on many fronts at the same time • One ‘Director’ can’t do it alone • GME trainees must bring research education WITH them to their residencies • Specialty colleges presently do not have realistic expectations about quality research or research education • Our trainees WANT to do quality research work and applaud our efforts…….but remain critical of the lack of infrastructure available to them.

34. Where we would like to go…. • Integrated continuum of research education from UME through CME….

36. Core Research Competencies II Scholarship and Research Training at the Residency Level Rorie Lee, Ph.D, MPH, CHES

37. Elements of Successful Residency Research Environments • Define research broadly • Require projects • Focus on clinically-relevant research/EBM • Research committee • Integrate curriculum DeHavenMJ, Wison GR, O'Connor-Kettlestrings P.Fam Med 1998:30(7): 501-7.

38. Elements of Successful Residency Research Environments • Program director support • Time • Didactics: planned curriculum, journal club • Faculty involvement: mentors, research committee, support with design/statistics • Opportunities for presenting/forums DeHavenMJ, Wison GR, O'Connor-Kettlestrings P.Fam Med 1998:30(7): 501-7.

39. UNECOM GME Programs • Internship • Residencies in FP, NMM, Combined FP/NMM • Most but not all interns linked to our residencies • 3-year federal training grant to develop EBM and research curriculum components for FP & Combined residencies

40. The Research-Oriented Curriculum at UNECOM • Incorporates elements of Evidence-Based Medicine, informatics, and research skills. • Takes broad view of research and scholarly activity, similar to Boyer’s expanded definition of scholarship*: Discovery Integration Application Teaching • Planned links with AOA Core Competencies *Boyer EL Scholarship reconsidered: Priorities of the professoriate. (1990)

41. Curriculum Components • Longitudinal curriculum • Residents work with faculty mentors to develop, propose, complete, and present research project (original or building on previous research) • Research training • Faculty development

42. Expectations of Residents • Competence in using the medical library and evidence-based electronic resources • Awareness of the basic principles of study design, performance, analysis, and reporting. • Participation in scholarly activity including research • Production of scientific paper suitable for publication and acceptable to specialty college

43. Implementation • Projects begin in 2nd year of residency training • Project must be completed in 3rd year of training (4th year for Combined residents) • Residents present projects to peers and as part of Annual OPTI Research Forum • Residents encouraged to present projects at state/regional/national conferences

44. Implementation PGY 2 • Resident chooses initial topic(s) • Program Director approves topic • Presents preliminary proposal (following Outline) to Mentor group • Mentor assigned once proposal approved • Detailed study proposal prepared • Proposal submitted to IRB • Start project

45. Implementation PGY 3 • Data collection & analysis • Draft of paper submitted • Paper revised • Peer presentation • OPTI/regional/professional presentation • Final paper approved by Program Director, Review Committee & submitted to specialty college

46. Mentors • Residency faculty take lead role • Basic science and other clinical faculty also involved • Resident progress is tracked using Checklist • Time is difficult to find

47. What we’ve learned so far • Successive approximation: take small steps • Internal feedback/review helpful • Faculty & residents need accessible support • Integrating EBM & clinically applied research important in residency training • Finding time is difficult but can be done • Define “research” broadly: approach both projects and people inclusively

48. Challenges • Expand core research mentor group & research training • Continue promoting research collaboration • Involve faculty in additional scholarly activities • Sustain support once current funding ends/obtain additional external funding

50. Training clinician researchers: Inside the box looking out or outside the box looking back in? des Anges Cruser, Ph.D., MPA