Bayesian evaluation and selection strategies in portfolio decision analysis

1. Bayesian evaluation and selection strategies in portfolio decision analysis E. Vilkkumaa, J. Liesiö, A. Salo INFORMS Annual meeting, Phoenix, Oct 14th-17th 2012 The document can be stored and made available to the public on the open internet pages of Aalto University. All other rights are reserved.

2. Post-decision disappointment in portfolio selection = Optimal project based on vE = Optimal project based on v Size is proportional to cost • Project portfolio selection is important • Decisions are typically based on uncertain value estimates vE about true value v • If the value of a project is overestimated, this project is more likely to be selected • Disappointments are therefore likely * Brown (1974, Journal of Finance), Harrison and March (1986, Administrative Science Quarterly), Smith and Winkler (2006, Management Science)

3. Underestimation of costs in public work projects (1/2) • Flyvbjerg et al. 2002 found statistically significant escalation (p<0.001) of costs in public infrastructure projects • This escalation was attributed to strategic misrepresentation by project promoters Frequency (%) Average escalation μ=27,6% Cost escalation (%) Source: Flyvbjerg et al. (2002), Underestimating Costs in Public Work Projects – Error or Lie? Journal of the American Planning Association, Vol. 68, pp. 279-295.

4. Underestimation of costs in public work projects (2/2) • If projects with the lowest cost estimates are selected, the realized costs tend to be higher even if cost estimates are unbiased a priori • Cost escalation could therefore be attributed to ʻpost-decision disappointment’ as well

5. Bayesian revision of value estimates (1/2) • Assume that the prior f(v) and the likelihood f(vE|v) are known such that • By Bayes’ rule we have f(v|vE)f(v)·f(vE|v) • Use the Bayes estimates viB for selection • If Vi~N(μi,σi2), ViE=vi+εi, εi~N(0,τi2) , then Vi|viE~N(viB,ρi2), where

6. Bayesian revision of value estimates (2/2) • Portfolio selection based on the revised estimates viB • Eliminates post-decision disappointment • Maximizes the expected portfolio value given the estimates viE • Using f(v|vE), we show how to: • Determine the expected value of acquiring additional estimates viE • Determine the probability that project i belongs to the truly optimal portfolio (= portfolio that would be selected if the true values v were known)

7. Example (1/2) • 10 projects (A,...,J) with costs from $1M to $12M • Budget $25M • Projects’ true values Vi ~ N(10,32) • A,...,D conventional projects • Estimation error εi ~ N(0,12) • Two interdependent projects: B can be selected only if A is selected • E,...,J novel, radical projects • These are more difficult to estimate: εi ~ N(0, 2.82)

8. Example (2/2) = Optimal project based on vE / vB = Optimal project based on v Size proportional to cost * True value = 52$M Estimated value = 62$M True value = 55$M Estimated value = 58$M

9. Value of additional information (1/2) = Optimal project based on current information • Knowing f(v|vE), we can determine • The expected value (EVI) of additional value estimates VE prior to acquiring vE • The probability that project i belongs to the truly optimal portfolio The probability that the project belongs to the truly optimal portfolio is here close to 0 or 1

10. Value of additional information (2/2) • Select 20 out of 100 projects • Re-evaluation strategies • All 100 projects • 30 projects with the highest EVI • ʻShort list’ approach (Best 30) • 30 randomly selected projects • Due to evaluation costs, strategy 2 is likely to outperform strategy 1

11. Conclusions • Uncertainties in cost and value estimates should be explicitly accounted for • Bayesian revision of the uncertain estimates helps • Increase the expected value of the selected portfolio • Alleviate post-decision disappointment • Bayesian modeling of uncertainties guides the cost-efficient acquisition of additional estimates as well