The COTOR Challenge

1. The COTOR Challenge Committee on the Theory of Risk November 2004 Casualty Actuarial Society Annual Meeting Phil Heckman’s Remarks: - Distribution of Sample Estimator - Fitting Mixtures - Visualization Tools

2. Distribution of Estimator • Good to know: Distribution of sample estimator for 5x5 layer. • Use Stuart’s TIG(.9,2000,.9) distribution to simulate 9,999 samples of 250 events each. • Tabulate Sum[Med(0,X-5M,5M)]/250 • 2.5% to 97.5% => (0, 40,000)

3. Distribution of Estimator 2

4. Distribution of Estimator 3 • Note mass points: 5M/250 = 20,000. • Sample estimator is most naïve approach. Not surprising if para-metric methods do better. • Subjectivity remains in confidence bounds. Question is what do we really know, how can we use that knowledge?

5. Fitting Mixtures

6. Mixture Example • Next slide shows a log/logit plot of WC claim size probabilities. Data are wild, not generated. • Model is mixture of two lognormals. Single LN is shown for comparison.

7. Mixture Fits

8. Mixture Remarks • Mixture Model fits very closely except for lowest points. • Single LN misses badly. • Should have stats for parameter estimates, but don’t. (Sorry.) • Added flexibility plus accord with intuition make this a useful method.

9. Visualization Tools • Nonparametric approaches are available for visualizing distributions. • Kaplan-Meier (Product Moment) estimator developed for survival analysis. Can be adapted for claim emergence, censored losses, etc. • No need to preview, choose intervals, etc. • Some other time.

10. Consequences of Assuming Normality • The frequency of extreme events is underestimated – often by a lot • Example: Long Term Capital • “Theoretically, the odds against a loss such as August’s had been prohibitive, such a debacle was, according to mathematicians, an event so freakish as to be unlikely to occur even once over the entire life of the universe and even over numerous repetitions of the universe” • When Genius Failed by Roger Lowenstein, p. 159

11. Criteria for Judging • New and creative way to solve the problem • Methodology that practicing actuaries can use • Clarity of exposition • Accuracy of known answer • Estimates of confidence interval

12. Table of Results

13. Observations Regarding Results • These estimations are not easy • Nearly 13 to 1 spread between lowest and highest mean • Only 10% of answers came within 10% of right result • All responders recognized tremendous uncertainty in results (range from upper to lower CL went from 8 to infinity) • Our statistical expert could not understand the description of the method of 30% of the respondents

14. Observations • All but 2 of the methods relied on approaches commonly found in the literature on heavy tailed distributions and extreme values • It is clear that it is very difficult to get accurate estimates from a small sample • The real world is even more challenging than this • 250 claims probably don’t follow any known distribution • Trend • Development • Unforeseen changes in environment • Consulting with claims adjusters and underwriters should provide valuable additional insights

15. Observations • The closest answer was 5% below the true mean • Half of the responses below the true mean, Half were above • Average response was 40% higher than the mean • Average response (ex outlier) was within 2% of the mean • Read: “The Wisdom of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economics, Societies and Nations” by: James Surowiecki • Implications for Insurance Companies?

16. Speakers • Meyers • Evans • Flynn • Woolstenhulme • Heckman

17. Announcement of Winners • Louise Francis – COTOR Chair

18. Possible Next Steps • Make the results of the challenge available to the membership • COTOR subcommittee to evaluate how to make techniques readily available • Another round making the challenge more real world • Include trend and development • Give multiple random samples