1 / 35

hypothesis testing with special focus on simulation

hypothesis testing with special focus on simulation. Hypothesis Test answers yes/no question with some statistical certainty H 0 = default hypothesis is a statement H a = alternate hypothesis is the precise opposite. X = test statistic (RANDOM!) sufficient (uses all avail. data)

keelie-peck
Télécharger la présentation

hypothesis testing with special focus on simulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. hypothesis testing with special focus on simulation Hypothesis Testing for Simulation

  2. Hypothesis Test answers yes/no question with some statistical certainty • H0 = default hypothesis • is a statement • Ha = alternate hypothesis • is the precise opposite Hypothesis Testing for Simulation

  3. X = test statistic (RANDOM!) • sufficient (uses all avail. data) • often Z, T, N are used as notation • FX = its probability distribution • a = P[reject H0 | H0 true] Hypothesis Testing for Simulation

  4. ca = critical region for a • a = P[X in ca | H0] • a is our (controllable) risk Hypothesis Testing for Simulation

  5. TWISTED LOGIC • We WANT to reject H0 and conclude Ha, so... • We make a very small, so... • If we can reject, we have strong evidence that Ha is true • This construct often leads to inconclusive results • “There is no significant statistical evidence that...” Hypothesis Testing for Simulation

  6. IMPORTANT • Inability to reject <> H0 true Hypothesis Testing for Simulation

  7. POWER OF THE TEST • b = P[X not in ca | Ha] • 1-b = P[correctly rejecting] Hypothesis Testing for Simulation

  8. VENACULAR • a is type I error • Probability of incorrectly rejecting • b is type II error • Probability of incorrectly missing the opportunity to reject Hypothesis Testing for Simulation

  9. UNOFFICIAL VENACULAR • type III error – answered the wrong question • type IV error – perfect answer delivered too late Hypothesis Testing for Simulation

  10. EXAMPLE! • Dial-up ISP has long experience & knows... Hypothesis Testing for Simulation

  11. We get DSL, observe 12 samples Hypothesis Testing for Simulation

  12. IS DSL FASTER? • H0: mDSL = 50 • Ha: mDSL < 50 • test with P[type I] = 0.01 Hypothesis Testing for Simulation

  13. PROBABILITY THEORY • Z ~ tn-1 • Must know the probability distribution of the test statistic IOT construct critical region Hypothesis Testing for Simulation

  14. for n = 12, a = 0.01, ca = -2.718 99% of the probability above -2.718 Hypothesis Testing for Simulation

  15. our test statistic -2.33 Hypothesis Testing for Simulation

  16. -1.796 (0.05) -2.33 (0.021) -2.718 (0.01) • 0.021 called the p-value • Given H0, we expect to see a test statistic as extreme as Z roughly 2% of the time. Hypothesis Testing for Simulation

  17. CONFIDENCE INTERVALS m Based on the sample So they are RANDOM! la ua • For a given a • P[la <= m <= ua] = 1-a Hypothesis Testing for Simulation

  18. GOODNESS-OF-FIT TEST • Discrete, categorized data • Rolls of dice • Miss distances in 5-ft. increments • H0 assumes a fully-specified probability model • Ha: the glove does not fit! Hypothesis Testing for Simulation

  19. TEST STATISTIC “chi-squared distribution with gnu degrees of freedom” Hypothesis Testing for Simulation

  20. n = observations - estimated param • Did you know... if Zi~N(0, 1), then Z12+ Z22+...+ Zn2 ~ cn2 Hypothesis Testing for Simulation

  21. CELLS • H0 always results in a set of category cells with expected frequencies • EXAMPLE • Coin is tossed 100 times • H0: Coin Fair Hypothesis Testing for Simulation

  22. CELLS AND EXPECTED FREQUENCIES Hypothesis Testing for Simulation

  23. EXAMPLE • Cannon places rounds around a target • H0: miss distance ~ expon(0.1m) • Record data in 5m intervals • (0-5), (5-10), ...(25+) Hypothesis Testing for Simulation

  24. EXPONENTIALS E(X)=1/l Hypothesis Testing for Simulation

  25. RESULTS Hypothesis Testing for Simulation

  26. Hypothesis Testing for Simulation

  27. TEST RESULTS • Degrees of Freedom • 6 cells • 0 parameters estimated • n = 6 • For the c62 distribution, the p-value for 14.14 is about p=0.025 • REJECT at any a > 0.025 Hypothesis Testing for Simulation

  28. DIFFERENT H0 • H0: the miss distances are exponentially distributed • Ha: the exponential shape is incorrect • We estimate the parameter, we lose one degree of freedom Hypothesis Testing for Simulation

  29. RESULTS 2 Hypothesis Testing for Simulation

  30. Hypothesis Testing for Simulation

  31. n = 5 • p-value for 7.83 is larger than 0.05 • CANNOT REJECT • CONCLUSION? Hypothesis Testing for Simulation

  32. SIMULATION vs. STATISTICS • Statistics • Sample is fixed and given • Conclusion is unknown • Significance is powerful • Simulation • Sample is arbitrarily large • Conclusion is known • We need another thought about what is meaningful Hypothesis Testing for Simulation

  33. SAMPLE SIZE EFFECT m = 100 s = 10 Hypothesis Testing for Simulation

  34. HOW LARGE IS A DIFFERENCE BEFORE IT IS MEANINGFUL? Hypothesis Testing for Simulation

  35. SUMMARY • You probably knew the mechanics of HT • You might have a new perspective Hypothesis Testing for Simulation

More Related