1 / 24

Evaluating Research Reports

Evaluating Research Reports. Chen-Sheng Chen Institute of Rehabilitation Science and Technology National Yang-Ming University September 15, 2003. Analyzing content of an article. Title and abstract Introduction Methods Subjects Design Instrumentation Procedures Data analysis Results

llara
Télécharger la présentation

Evaluating Research Reports

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating Research Reports Chen-Sheng Chen Institute of Rehabilitation Science and Technology National Yang-Ming University September 15, 2003

  2. Analyzing content of an article • Title and abstract • Introduction • Methods • Subjects • Design • Instrumentation • Procedures • Data analysis • Results • Discussion and Conclusion

  3. Consideration of merit of an article • Reputation of Journal • Two experts to review • Meet the criteria of journal • Elements considered by reviewers • Importance of research • Originality • Appropriateness of research design • Adequacy of the method • Relevance of discussion • Clarity of writing

  4. Science Citation Index • 科學情報研究所 • Institute for Scientific Information，簡稱ISI • 約3500種 理、化、農、林、醫、生命科學、天文、地理、環境、材料、工程技術等自然科學各學科的核心期刊影響因數 • 影響因數 • Impact Factor (IF) • 該刊前二年發表的文獻在當年的平均被引用次數

  5. Spine (4/39)

  6. Title and Abstract • Title should be informative • Next to read abstract • Information of abstract • Purpose • Method • Results • Major consideration • Without studying the details of paper, no way to sure that the study was valid

  7. Introduction • Provide the impetus for the study • What is the problem? Why is it important? • How has the author used the literature to form a logical rationale? • What is the theoretical context for the study? • What is specific purpose? • What are hypotheses or guiding questions?

  8. Methods - subjects • Readers need to know who the subjects were, can be applied to clinical situations • Who were the subjects? • What were inclusion and exclusion criteria? • How were subjects selected? • How many subjects were studied? Is the sample size adequate?

  9. Inadequate situation • Apply the normal to predict the patients • Comparison under totally different groups • Numbers of subjects is an important consideration when interpreting the results

  10. Methods - Design • What is the research design? Is it appropriate for answering the question? • How many groups were tested? • How often were treatments and measurements applied?

  11. Three reasons to notice • First • Relate the finding to the ‘real world’ • Alert to the possibility that extraneous variable interfered with the results • Second • Statistical analysis: numbers of groups, numbers of dependent variables, frequency of measurement, etc. • Third • Clinical trials model or single-case design or, readers should know special rules if can follow up the rules or not

  12. Methods - Instrumentation • In instrumentation described insufficient detail? • How have the authors documented the reliability and validity of the instruments?

  13. Commercial device • Model numbers, names, addresses of manufacturers • Calibrated regularly • Standardized Questionnaire • References must be cited • Reliability • Reproducibility and consistency of measurements • Specific populations

  14. Example • Test grip strength • Reliable to test normal subjects ! • Does it reliable to test patients with wrist pain? • Does reliability differ if measurement are taken at home or in the clinic? • Was the status determined by self-report, interview or observation?

  15. Method - Procedures • Are data collection procedures described clearly and insufficient detail to allow replication? • What are operational definition for all variables? • Position, velocity, acceleration

  16. Data collection • Who performed the measurement? • When or how often measurement were taken? • In vitro test: ligament strength • Spine specimen • Conducted by surgeon • Age of specimen • Temperature • Procedure about removing soft tissue

  17. Data analysis • What statistical procedure were used? • Are they appropriate to analyze the data? • Has the author justified the use of any unique or unusual statistical tests? • Has the author addressed each research question in the analysis? • What alpha level was used for the level of significance? • Inferential statistics, α=0.05

  18. Results • Report finding without interpretation • Do the results address the research question? • Are figures and tables presented accurately? • Are the results statistically significant? • Something to notice • Wrong data collection • Missing information • Subjects -> statistical results

  19. Discussion and Conclusion • How does the author interpret results? • Did the author clarify if hypotheses were rejected or accepted? • How are the finding related to prior reports? • What limitations are described? • Are there limitations that are not addressed? • Does the author discuss how the results apply to practice? • Does the author present suggestions for further study? • Do the stated conclusions flow logically from the obtained results?

  20. Study for what? • What do you do in your study? • Why trust your study? • Any evidence observed in your results? • Inference logically • Strictly obey the test procedure • To THINK any possibility (台積電: 張忠謀)

  21. 核子反應爐工程師 : 9年 麥肯錫東京事務所 核子工程 ??=> 企業管理 損益平衡分析? 公牛乳頭? 圖書室 客戶資料個案微卷 每晚28分鐘的電車 10:48 -11:16 PM 思考問題解決方案(1year) 蕃茄醬廣告 如何擴大某一品牌的蕃茄醬市場? 看板是否能反應銷路? 多元思考 蕃茄醬或蕃茄汁的不同 客戶問題需求 相關思考案例結合 管理大師 / 大前研一 顧問費:43萬/1日

  22. 1920年: 十家啤酒廠商 排名第八 啤酒廣告 最純的啤酒 沒人體會什麼是「純」 專業知識與製酒過程 清澈密西根湖 自鑽500呎深自流井 5年進行1623次實驗 開發最好的酵母 裝酒瓶: 600度蒸汽殺菌 最「純」的啤酒 告訴消費者過程 體會「純」的定義 排名第一的啤酒 (6 months) 施麗茲啤酒(schlitz beer)行銷顧問

  23. Strength and Weakness RST • 聯強國際總裁:杜書伍 • 懂80%:常識 • 專業20:知識 • 贏20%做老大

  24. Do exercise other than study hard • Data adopted from • Ch 31. Evaluating Research Report. • 鄭誠功所長 : 科技論文寫作 • 商業週刊 : no. 805 (2003.4.28-2003.5.4) • :贏20%做老大 (page 125-131)

More Related