1 / 69

Meta-analysis

Meta-analysis. The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir. Definition. Meta-analysis: a type of systemic review that uses statistical techniques to quantitatively combine and summarize results of previous research

johana
Télécharger la présentation

Meta-analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir

  2. Definition • Meta-analysis: a type of systemic review that uses statistical techniques to quantitatively combine and summarize results of previous research • A review of literature is a meta-analytic review only if it includes quantitative estimation of the magnitude of the effect and its uncertainty (confidence limits). Meta analysis

  3. Function of Meta-Analysis(1) • 1-Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure • 2-Increase statistical power and precision to detect an effect • 3-Develop ,refine, and test hypothesis • continued Meta analysis

  4. Function of Meta-Analysis(2) • continuation • 4-Reduce the subjectivity of study comparisons by using systematic and explicit comparison procedure • 5-Identify data gap in the knowledge base and suggest direction for future research • 6-Calculate sample size for future studies Meta analysis

  5. Historical background • Ideas behind meta-analysis predate Glass’ work by several decades • R. A. Fisher (1944) • “When a number of quite independent tests of significance have been made, it sometimes happens that although few or none can be claimed individually as significant, yet the aggregate gives an impression that the probabilities are on the whole lower than would often have been obtained by chance” (p. 99). • Source of the idea of cumulating probability values • W. G. Cochran (1953) • Discusses a method of averaging means across independent studies • Laid-out much of the statistical foundation that modern meta-analysis is built upon (e.g., inverse variance weighting and homogeneity testing) Meta analysis

  6. Basic concepts • The main outcome is the overall magnitude of the effect. • It's not a simple average of the magnitude in all the studies. • Meta-analysis gives more weight to studies with more precise estimates. • The weighting factor is 1/(standard error)2. Meta analysis

  7. Main magnitude of effects • Descriptive • Mean • Prevalence • Analytical • Additive • Mean difference • Standardized mean difference • Risk, rate or hazard difference • Correlation coefficient • Multiplicative • Odds ratio, Risk, Rate or Hazard Ratio Meta analysis

  8. Statistical concepts(1) • You can combine effects from different studies only when they are expressed in the same units. • Meta-analysis uses the magnitude of the effect and its precision from each study to produce a weighted mean. Meta analysis

  9. Statistical concepts(2) The impact of fish oil consumption on Cardio-vascular diseases Meta analysis

  10. Forest plot • the graphical display of results from individual studies on a common scale is a “Forest plot”. • In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis. • A logarithmic scale should be used for plotting the Relative Risk. Meta analysis

  11. Forest plot Meta analysis

  12. Statistical concepts(3) • There are two basic approach to Quantitative meta –analysis: • Weighted-sum • Fixed effect model • Random effect model • Meta-regression model Meta analysis

  13. Fixed effect model • General Fixed effect model- the inverse variance – weighted method • Specific methods for combining odds ratio • Mantel- Haenszel method • Peto’s method • Maximum-Likelihood techniques • Exact methods of interval estimation Meta analysis

  14. Fixed effect model • In this model, all of the observed difference between the studies is due to chance • Observed study effect=Fixed effect+ error • Xi= θ + eiei is N (0,δ2 ) • Xi = Observed study effect • θ = Fixed effect common to all studies Meta analysis

  15. General Fixed effect model • Ť=∑ wiTi/ ∑ wi • The weights that minimize the variance ofŤ are inversely proportional to the conditional variance in each study • Wi=1/vi • Var(Ť)=1/ ∑ wi Meta analysis

  16. Mantel- Haenszel method • Each study is considered a strata. • Ť=∑ai di / ni/ ∑bi ci /ni Meta analysis

  17. Random effect model • The “random effect” model, assumes a different underlying effect for each study. • This model leads to relatively more weight being given to smaller studies and to wider confidence intervals than the fixed effects models. • The use of this model has been advocated if there is heterogeneity between study results. Meta analysis

  18. Source of heterogeneity • Results of studies of similar interventions usually differ to some degree. • Differences may be due to: • - inadequate sample size • - different study design • - different treatment protocols • - different patient follow-up • - different statistical analysis • - different reporting • - different patient response Meta analysis

  19. An important controversy has arisen over whether the primary objective a meta-analysis should be the estimation of an overall summary or average effect across studies (a synthetic goal) • or the identification and estimation of differences among study-specific effects (analytic goal) Meta analysis

  20. Test of Homogeneity • This is a test that observed scatter of study outcomes is consistent with all of them estimating the same underlying effect. • Q= X2homo=∑i=1nwi (mi -M)2 • df=n-1 • wi =weight • M=meta analytic estimate of effect • mi =effect measure of each study Meta analysis

  21. Dealing with statistical heterogeneity • The studies must be examined closely to see if the reason for their wide variation in effect. If it’s found the analysis can be stratified by that factor. • Subgroup analysis • Exclusion of study • Choose another scale • Random effect model • Meta-regression Meta analysis

  22. Random effect model • Assume there are two component of variability: • 1)Due to inherent differences of the effect being sought in the studies (e.g. different design, different populations, different treatments, different adjustments ,etc.) (Between study) • 2)Due to sampling error (Within study) Meta analysis

  23. Random effect model • There are two separable effects that can be measured • The effect that each study is estimating • The common effect that all studies are estimating • Observed study effect=study specific (random )effect + error Meta analysis

  24. Random effect model • This model assumes that the study specific effect sizes come from a random distribution of effect sizes with a fixed mean and variance. • There are five approach for this model: • Weighted least squares • Un-weighted least squares • Maximum likelihood • Restricted Maximum likelihood • Exact approach to random effects of binary data. Meta analysis

  25. Random effect • Xi= θi + eiei is N (0,δ2 ) • Xi = Observed study effect • θi = Random effect specific to each study θi =U+di • U=Grand mean (common effect) • di is N (0,ד2 ) – Random term Meta analysis

  26. Weighted least squares for Random Effect • Ŵ=∑wi/k • S2w=1/k-1(∑wi2-k Ŵ2) • U=(k-1)(Ŵ-S2w/kŴ) • ד2=0 if Q<k-1 • ד2=(Q-(k-1))/U if Q>k-1 • wi* = 1/var.+ ד2 var.=within study variances Meta analysis

  27. Weighted least squares for Random Effect (WLS) • Ť.RND=∑ wi* Ti/ ∑ wi* • Var(Ť.RND)=1/ ∑ wi* • Where Ti is an estimate of effect size and θi is the true effect size in the ith study • Ti = θi +ei ei is the error with which Ti estimatesθi • var(Ti)= דθ2 +vi Meta analysis

  28. random versus fixed effect models • Neither fixed nor random effect analysis can be considered ideal. • Random effect models has been criticized on grounds that unrealistic distributional assumption have to be made. • Random effect models are consistent with the specific aims of generalization. Meta analysis

  29. Peto’s advocates • He suggested a critical value .01 instead of usual .05 to decide whether a treatment effect is statistically significant for a fixed effect model. • This more conservative approach has the effect of reducing the differences between fixed and random effect models. Meta analysis

  30. Meta-regression • If more than two groups of studies have been formed and the characteristic used for grouping is ordered, greater power to identify sources of heterogeneity may be obtained by regressing study results on the characteristic . • With meta-regression, it is not necessary or even desirable to groups the studies. • The individual study results can be entered directly in the analysis. Meta analysis

  31. Meta-Regresion • 1- meta-Regression model( extension of fixed effect model) • 2- Mixed model( extension of random effect model) Meta analysis

  32. Fixed-effects regression • Θi=B0+B1xi1+...+Bpxip • It’s the covariate predictor variables that are responsible for the variation not a random effect; the variation is predictable, not random. Meta analysis

  33. Mixed model • Θi=B0+B1xi1+...+Bpxip+ui • This model assumes that part of the variability in true effects is unexplainable by the model. Meta analysis

  34. Between studies variation • You can and should allow for real differences between studies–heterogeneity–in the magnitude of the effect. • The τ2 statistic quantifies % of variation due to real differences. Meta analysis

  35. Fixed effects model and heterogeneity • In fixed-effects meta-analysis, you do so by testing for heterogeneity using the Q statistic. • If p<0.10, you exclude "outlier" studies and re-test, until p>0.10. • When p>0.10, you declare the effect homogeneous. • But the approach is unrealistic, limited, and suffers from all the problems of statistical significance. Meta analysis

  36. Random effects model and heterogeneity • In random-effect meta-analysis, you assume there are real differences between all studies in the magnitude of the effect. • The "random effect" is the standard deviation representing the variation in the true magnitude from study to study. • You need more studies than for traditional meta-analysis. • The analysis is not available in a spreadsheet. Meta analysis

  37. Concept of analysis in random versus fixed effect models • Fixed effects models: within-study variability • "Did the treatment produce benefit on average in the studies at hand?" • Random effects models: between-study and within-study variability • "Will the treatment produce benefit ‘on average’?" Meta analysis

  38. Limitations • It's focused on mean effects and differences between studies. But what really matters is effects on individuals. • (Aggression bias) • A meta-analysis reflects only what's published or searchable. Meta analysis

  39. Aggregation bias • Relation between group rates or and means may not resemble the relation between individual values of exposure and outcome. • This phenomenon is known as aggregation bias or ecologic bias. Meta analysis

  40. Ecological fallacy BP Education Meta analysis

  41. Meta-analysis of neoadjuvant chemotherapy for cervical cancer Hand Searching 14% Word of Mouth 14% Trial Registers Medline/Cancerlit 14% 58% Meta analysis

  42. Type of reporting Ongoing 5% Published in full 47% Unpublished 24% Published as abstract 24% Meta analysis

  43. Selection bias in Meta analysis • English language bias • Database bias • Publication bias • Bias in reporting of data • Citation bias • Multiple publication bias • Sample size Meta analysis

  44. Publication bias • The results of a meta-analysis may be biased if the included studies are a biased sample of studies in general. • The classic form of this problem is publication bias, a tendency of journals to accept preferentially papers reporting an association over papers reporting no association Meta analysis

  45. Publication bias • If such a bias is operating, a meta-analysis based on only published reports will yield results biased away from the null. • Because small studies tend to display more publication bias, some authors attempt to avoid or minimize the problem by excluding studies below a certain size. Meta analysis

  46. Some meta-analysts present the effect magnitude of all the studies as a funnel plot, to address the issue of publication bias. • A plot of 1/(standard error) vs effect magnitude has an inverted funnel shape. • Asymmetry in the plot can indicate non-significant studies that weren’t published. Meta analysis

  47. Funnel plot 1/SE “funnel” ofunbiasedstudies 0 effectmagnitude Meta analysis

  48. Funnel plot Meta analysis

  49. Measures of Funnel Plot Asymmetry • 1- Linear Regression Approach (Egger’s method) • SND=a + b. precision • SND=OR/SE • The intercept “a” provides a measure of asymmetry- the larger its deviation from zero the more pronounced the asymmetry. Meta analysis

  50. Measures of Funnel Plot Asymmetry • 2- A rank correlation test • This method is based on association between the size of effect estimates and their variance. If publication bias is present, a positive correlation between effect size and variance emerges because the variance of the estimates from smaller studies will also be large. Meta analysis

More Related