Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing

1. University of Maryland Dissemination and Implementation Program Webinar 2: Reporting Research: Methods Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing Ann Gruber-Baldini, PhD University of Maryland, School of Medicine

2. Methods Section • Participants- thorough description, including demographic information, injury or disorder characteristics, mean time since onset • Measures- include a detailed description of all measures used, including their reliability and validity information • Procedure- Include all procedures in detail, so that someone else could replicate your study exactly; include recruitment, research design, and types of statistical analyses

3. Methods • There is no specific page limit, but a key concept is to keep this section as concise as you possibly can. • People will want to read this material selectively. • The reader may only be interested in one formula or part of a procedure. • Materials and methods may be reported under separate subheadings within this section or can be incorporated together.

4. Methods • The objective is to document all specialized materials and general procedures, so that another individual may use some or all of the methods in another study or judge the scientific merit of your work. • It is not a detailed recipe. • it is not a long winded story.

5. Style Challenges • It is awkward or impossible to use active voice when documenting methods without using first person. • First person writing would focus the reader's attention on the investigator rather than the work. • Use third person passive voice. • Use normal prose with complete sentences – avoid informal lists.

6. Things to Avoid In Methods • Materials and methods are not a set of instructions. • Omit all explanatory information and background - save it for the discussion. • Omit information that is irrelevant to a third party, such as what color ice bucket you used, or which individual logged in the data. • NO RESULTS should be included in methods

7. Purpose of the Methods Section • The purpose of the section is to make it possible for interested readers to repeat the author’s experiment …to reproduce results. • Explain exactly what was done. • Think of bench research: • What experiments were run and how they were run, what equipment and materials were used and how they were used, how much, how often, what, where, when, and why.

8. Components to include in Methods: • Design • RCT • Quasi exp • Repeated measures • Single group • Randomization process • Who if anyone was blinded? • Double blinded-interventionist doesn’t know (ie provider in drug trials) and participant doesn’t know • In social sciences what methods were used to preserve blinding

9. Components to Include • Sample • Recruitment process-who did the recruitment? • Number available; number contacted; number approached; number consented; number refused; reason for refusal • Eligibility criteria • Determination of sample size • Stopping rules if relevant • Randomization-who performed this?

10. Critical Component: Ethics • Declaration that an institutional review board governing research has determined that the study protocol adheres to ethical principles. • Without such approval, no research project can be conducted nor can it be published in a reputable, peer reviewed journal.

11. Presentation of Sample • Consolidated Standards for Reporting Trials (CONSORT) Diagram

12. Components to include: • Location of the study • For example: If in institutions as relevant a brief description of site is helpful • Inner city • Rural • 1,000 bed or 25 bed

13. Measures and Protocol for Data Collection • Measures • The what aspect of measures! • Brief description of what data was collected • Brief evidence of reliability and validity • Acknowledgement of no evidence of reliability and validity but rational why the measure was used

14. Measures and Protocol for Data Collection • Protocol for data collection • Who: Unit nurses, research assistant, family • When: baseline, 2, 4 and 6 months post implementation of the intervention • Where: location such as primary care office; home setting; nursing home room • How: Face to face interview; paper and pencil test; internet survey etc.

15. Intervention • Describe your intervention in sufficient detail to conceptualize a replication. • Reference prior work or refer to the web for further detail (as appropriate) • Write this in the past tense and third person ….The exercise program was implemented every second Thursday of the month and lasted for 2 hours. • ….The specimen was centrifuged for 10 minutes.

16. Intervention • Gory details can be placed in a table to save room. • What are the gory details? • Intervention dosage • Strength of dose • Time intervals of dosing • Who is implementing the intervention • See Conn article WJNR http://wjn.sagepub.com/content/34/4/427 The online version of this article can be found at: DOI: 10.1177/0193945911434627

17. Intervention • Gory details • Reference if previously developed • Conceptual framework • Intervention components • Timing of delivery • Dose • Mode(s) of delivery (e.g., face to face; internet) • Intervention target and recipient (e.g., patient or family) • Delivery setting • Culturally relevant • Intervention variations: men/women ; tailoring rules • TREATMENT FIDELITY –plan can be presented here

18. Key elements for each aspect • Content: What was the content and how was it delivered • Provider: who delivered it? • Format: What method was used (telephone, individual) • Setting: Where was it done • Recipient: To whom was the intervention delivered? • Intensity: How often/how long for each touch? • Duration: Over what period of time were there intervention contacts?

20. Intervention • Can include a diagram to demonstrate flow thru the protocol • Include any deviations or decisions made at particular points through the process.

21. Data Analysis • How was nesting/clustering addressed? • Randomization by setting or individual? Adjustments done and how done? • Sufficient detail for replication • Reference statistical methods for more detail

22. Data Analysis • State if an intention-to-treat analysis was done • State if any participants were excluded for any reason from the analyses • State if subgroup or additional analyses were performed • State level of significance used and be prepared to address inflated p for multiple tests (often addressed through method of analysis).

23. Data Analysis • Statistical methods should be described in sufficient detail to enable a knowledgeable reader with access to the original data to verify the reported results. • References for statistical methods should be to standard works when possible. • Any computer programs used should be identified. • Statistical terms, abbreviations, and symbols should be defined. • Details about randomization, if used, should be given, as well as concealment of allocation to treatment groups, and the method of masking (blinding). • Losses to observation (such as dropouts from a clinical trial) should be reported. • It is recommended to include the word “considered” in descriptions of statistical significance, such as “a P value of less than 0.05 was considered statistically significant”, since the choice of this cut-off point is arbitrary.

24. Method Challenges • Hitting just the right level of detail is difficult in these sections. • Enough detail for a reader to reconstruct his/her study, but not so much that the relevant points get buried. • Ask yourself at each place: “Would I need to know this to reproduce this experiment?” • This is tooo much!... • “We rolled the patient over and wiped the skin clean prior to applying the dressing. We did this with every dressing change”.…assume they know something.

25. Common Mistakes in a Methods & Materials Section • Not Enough Information is more commonly the problem • Make sure it is replicable!

26. Common Errors • DO NOT include results in the methods • DO NOT include discussion in the methods

27. Treatment Fidelity • If a plan is in place this can be explained in the methods section • Procedures or Measures as relevant

28. A Word About Qualitative Write Up • Same information is needed in methods • Sample • Intervention-not usually relevant • Measures • Data Analysis

29. Sample • It is probably the case that convenience sampling is the most frequently used in qualitative studies. • State the size and type of sample used in the reported study. • If an unusual variant of sampling is used, it is useful to acknowledge the nature of it. • Other comments about the sampling process may be helpful-snowballing • SIZE and whether saturation was achieved

30. Sample • It is important to explain why your data set is the most illustrative and useful to answer the question you are posing. • Be careful to describe how you picked your sample. What criteria did you use? • Can you compare the data set to other alternatives and why did you choose this one? • Describe the important variations within the data set (for instance age and gender distributions) so that the reader gets a good picture of it.

31. Data Collection/Measures • Descriptive data included? • Most qualitative studies (but not all) the data collection method is usually the interview method. • How the interviews were carried out • Location/timing/by whom/ questions asked • Example of write up: • All students were interviewed by the researcher on two occasions, for between 30 and 45 min. All interviews were recorded, with the permission of the students being interviewed. After the interviews, the recordings were transcribed into computer files. Care was taken by the researcher to assure the respondents that they and the place of their work would not be identifiable in any subsequent report. Once the final research report was written, the tapes from the interviews were destroyed.

32. Data Analysis • Describe how the researcher handled the data • ‘The interviews were recorded and transcribed. The researcher then sorted those data into a range of categories and these are reported below’….is a bit too brief! • Care should be taken with very general terms such as ‘content analysis’, when reporting data analysis. The term is probably so broad as to have little meaning.

33. Data Analysis • Describe carefully each step in the analysis to make it possible for the reader to believe that your conclusions are correct -- or argue against them. • A good rule is to present the analysis of one observation/item/response in detail. • Describe your interpretations during the analysis in a systematic way, in small identifiable steps.

34. Data analysis • For example might describe “in vivo” coding which uses the participants own words for the codes.

35. Example Data Analysis Section • Data analysis was done using basic content analysis(Crabtree B & Miller W, 1992) and started with the first interview. The analysis began with “in vivo” coding(Strauss & Corbin, 1998), or “grounded” coding (Glaser& Strauss, 1967), which involves using the informants' own words to capture a particu­lar idea. The following is an example of “in vivo” coding: The code identified was independence…"They feel more independent, you know, because it seems like most of their ability to do things independently goes away when they come here”. The codes identified were grouped based on similarities and differenc­es. For example, a number of codes arose from the data that focused on facilitators of restorative care such as encouragement of the resident, cueing the resident, or asking them to “help you out”. These were combined under the theme of “Facilitators " of restorative care. Coding was completed initially by the principal investigator and a code book established. The second nurse investigator with experience in implementing restorative care programs review the coded data and revised the codes and added new codes based on her review. This two coders then reviewed the data and codes together until consensus was achieved between the two reviewers.

36. Example of Data Analysis Section • All of the interview transcripts were read by the researcher and coded in the style of a grounded theory approach to data analysis (refs). Eight category headings were generated from the data and under these all of the data were accounted for. Two independent researchers were asked to verify the seeming accuracy of the category system and after discussion with them, minor modifications were made to it. In the grounded theory literature, a good category system is said to have ‘emerged’ from the data (refs). Other commentators have noted that, in the end, it is always the researcher who finds and generates that system (refs).

37. Data Analysis • Carefully describe the reliability and validity/Confirmability of your data analysis process • Reliability-recognizability of the findings/transfirmability • Validity-consensus among a group

38. Reliability and Validity: Credibility • Credibility of the data refers to the believability, fit, and applicability of the findings to the phenomena under study (Denzin & Lincoln, 2000; Lincoln & Guba, 1985). The focus groups were done when the 12 month intervention was completed in each of the treatment facilities. Since completion of the intervention by site occurred at different times, this allowed the investigators to use the findings from the first focus groups to confirm or refute codes and emerging themes in the subsequent groups. • Confirmability or auditability of the data refers to the objectivity of the factual aspects of the data (Lincoln & Guba, 1985). Confirmability of the data was considered by having other members of the research team review the findings and provide feedback as to whether these findings logically fit with other settings and experiences. Specifically the findings were reviewed by three different RCNs that had worked in the treatment facilities, as well as the co-investigators on the study including three epidemiologists with experience in long term care research.

39. Resource • This guide for observational studies may be helpful: von Elm E, Altman DC, Egger M, Pocock SJ, et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. PLoS Med 4(10): e296. http://www.strobe-statement.org/

40. Homework • Methods Section • Design • Sample • Intervention as appropriate • Measures • Data analysis

41. Homework • Homework due Thursday, June 13 • Please send to: vigne1@verizon.net

42. Next Webinar… • We’ll see you back here on Thursday, July 25 at 4:30 p.m.