Two Models of Evaluating Probabilistic Planning

1. Two Models of Evaluating Probabilistic Planning • IPPC (Probabilistic Planning Competition) • How often did you reach the goal under the given time constraints • FF-HOP • FF-Replan • Evaluate on the quality of the policy • Converging to optimal policy faster • LRTDP • mGPT • Kolobov’s approach

2. Heuristics for Stochastic Planning • Heuristics come from relaxation • We can relax along two separate dimensions: • Relax –ve interactions • Consider +ve interactions alone using relaxed planning graphs • Relax uncertainty • Consider determinizations • Or a combination of both!

3. Determinizations • Most-likely outcome determinization • Inadmissible • e.g. if only path to goal relies on less likely outcome of an action • All outcomes determinization • Admissible, but not very informed • e.g. Very unlikely action leads you straight to goal

4. Solving Determinizations • If we relax –ve interactions • Then compute relaxed plan • Admissible if optimal relaxed plan is computed • Inadmissible otherwise • If we keep –ve interactions • Then use a deterministic planner (e.g. FF/LPG) • Inadmissible unless the underlying planner is optimal

5. Dimensions of Relaxation 3 4 Negative Interactions Increasing consideration  1 2 Uncertainty Relaxed Plan Heuristic 1 Reducing Uncertainty Bound the number of stochastic outcomes  Stochastic “width” McLUG 2 FF/LPG 3 4 Limited width stochastic planning?

6. Dimensions of Relaxation Uncertainty -ve interactions

7. Expressiveness v. Cost Node Expansions v. Heuristic Computation Cost Limited width stochastic planning FF McLUG Nodes Expanded FF-Replan Computation Cost h = 0 FFR FF

8. Reducing Heuristic Computation Cost • Exploit overlapping structure of heuristics for different states • E.g. SAG idea for McLUG • E.g. Triangle tables idea for plans (c.f. Kolobov)

9. Triangle Table Memoization • Use triangle tables / memoization C B A A B C If the above problem is solved, then we don’t need to call FF again for the below: B A A B