1 / 24

Statistical and Practical Significance

Statistical and Practical Significance. Advanced Statistics Petr Soukup. Outline. Reminder of statistical significance Limits of statistical significance Misuses of statistical significance Alternatives to statistical significance Practical significance Effect sizes.

jensen
Télécharger la présentation

Statistical and Practical Significance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical and Practical Significance Advanced Statistics Petr Soukup

  2. Outline • Reminder of statistical significance • Limits of statistical significance • Misuses of statistical significance • Alternatives to statistical significance • Practical significance • Effect sizes

  3. REMINDER OF STATISTICAL SIGNIFICANCE (NHST)

  4. Hypotheses and tests • Tested hypothesis in experiments (Fisher, 1925) • Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937) • Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.

  5. Definition of statistical significance Definition: Conditionalprobability, that our sample can be drawn from population in which null hypothesis is valid (α). Statistical significance is P(D/H0) and not P(H0/D)

  6. LIMITSOF NHST

  7. Assumptions for classical NHST • Big big probability samples from infinite or very big finite populations Three assumptions: • Big (infinite) population (at least 100times bigger than the sample) • Probability sampling (all units same probability of selection) • Big sample (> 30-50 units)

  8. LIMITS OF NHST • 1.data from censuses • 2. data from non-probability samples • 3. data from small samples • 4. data based on sample that are big proportion of the basic population • 5. big data samples from merged (internationally or by time) files

  9. Beyond the limits of NHST in CSR* N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research *CSR-Czech sociological review

  10. MISUSES OF NHST

  11. Objections against NHST (Misuses of NHST) a) Insufficient statement about population, b) null hypotheses are unreal (nill null), c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.), d) statistical significant doesn’t mean important, e) publishing only statistical significant results (file drawer problem).

  12. Misuses of NHST in CSR* N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research *CSR-Czech sociological review

  13. Conference examples (P<0,01)

  14. Conference examples (***)

  15. Conference examples (*** and stepwise)

  16. ALTERNATIVESTO NHST

  17. Some alternatives to statistical significance a) Confidence Intervals (Problems for r, formulas, regression etc.) b) Test power (quite good in sociology), c) Estimate of minimum sample size & What if strategy, d) Comparison of models via information criterias (AIC, BIC) e) Bayesian approach

  18. PRACTICAL SIGNIFICANCE

  19. Practical significance - terminology • Practical significance b) Substantive significance c) Logical significance d) Scientific significance sometimes also: e) result importance or f) result meaningfulness

  20. How to measure Practical sig.? History - Absolute and relative approach Example: Income differencies Absolute and relative difference

  21. How to measure Practical sig.? Effect sizes – measures of practical significance Some well known: Cohen d Hayes ω But also R2, r, C, Fisher η2 are effect sizes Problem: Sometimes published but not interpreted

  22. OTHER SIGNIFICANCES

  23. Special significances • Economic significance • Clinical significance • Etc.

  24. CONCLUSION? Statistical significance is: LIMITED MISUSED BUT NOT BAD Substantive significance is: NOT OFTEN USED BUT NECESSARY

More Related