Control Rules in Planning: Enhancing Efficiency through Temporal Logic

1. Control Rules in Planning Stephen M. Lee-Urban March 26, 2007

2. References • Chapter 10, “Control Rules in Planning”, in Automated Planning Theory and Practice • F. Bacchus and F. Kabanza. “Using Temporal Logics to Express Search Control Knowledge for Planning”, in: Artificial Intelligence, vol 116, 2000 • Planner: TLPlan • (For)Warning: Understanding temporal logic will be necessary for encoding control rules in your domains

3. Motive • Classical planning efficiency suffers from combinatorial complexity (intractable) • Most earlier planners fit in Abstract-Search Procedure • ND search in node space of set of solution plans (set of all reachable solutions from n) • Prune function detects and cuts unpromising nodes • Can improve solving: exponential to polynomial • Same idea used in F.E.A.R. • Uses domain specific rules to guide forward chaining algorithm u is a structured collection of actions and constraints

4. Pruning • Often involves domain-specific tests • Identify less desirable solutions below node than solutions below a different node • Motivating Example: • Forward Search + Prune in Container-Stacking • “Consistent”  all containers below c consistent with g • State Space. Long plans worse than shorter • If container c position is consistent with goal, prune what? • Prune states resulting from apply action moving c • If c’s position inconsistent, action a yields state consistent with goal, and action b moves c to position inconsistent w/ g • Prune states resulting from applying action b • Finds near optimal solutions in low-order polynomial time How do we express such relationships between states?

5. Simple Temporal Logic: υ □ ◊ ○ • STL extends FOL to include “modal operators” • Modal Operators express relationships between current state and subsequent states • Syntax: If L is a func-free FOL, then Lt includes all of L plus: • true, false: constant atoms that are always true/false • Modal Ops: What do we need? • υ (until), □ (always), ◊ (eventually), ○ (next), GOAL • If Φ1 and Φ2 are formulas, then so are • Φ1υΦ2, □Φ1, ◊Φ1, ○Φ1 • If Φ1 has no modal ops, then GOAL(Φ1) is a formula

6. Semantics of STL • Interpreting requires triple (S, si, g) • S = <s0, s1, …> is an infinite sequence • si S is the current state • g is a goal formula • If Lt is based on L of planning domain, and S is sequence of states produced by finite plan, isn’t S finite? • Use NOPs. Append infinite final state. • In coming slides, let Φ be an STL formula. We now define whether (S, si, g) |=Φ

7. But First…. Entailment |= • Aka “logical implication” • A |= B • A entails B iff every model that makes A true also makes B true • Example • A := {“All horses are animals”, “All stallions are horses”} • B := {“All stallions are animals”} • A |= B

8. υ - until □ - always ◊ - eventually ○ - next Definitions • Given ground atom Φ, (S, si, g) |= Φ iff si |= Φ • Eg: (S, ((on a b) (on b c) …), g) |= (on a b) • Quantifiers and logical connectives have the same semantic rules as in FOL • Eg (S, si, g) |= Φ1Φ2 iff (S, si, g) |= Φ1 and(S, si, g) |= Φ2 • (S, si, g) |= □Φ iff • (S, sk, g) |= Φ for k i • (S, si, g) |= ○Φ iff • (S, si+1, g) |= Φ • (S, si, g) |= ◊Φ iff • k | k  i, (S, sk, g) |= Φ • (S, si, g) |= Φ1υΦ2 iff • k | k  i, (S, sm, g) |= Φ1 for m = i, …, k-1 and (S, sk, g) |= Φ2 • (S, si, g) |=GOAL(Φ) iff • Φg If Φcontains no GOAL operators, g is irrelevant. Write simply (S, si) |= Φ

9. υ - until □ - always ◊ - eventually ○ - next Simple Examples • Each means the same for s2 • s2 |= on(c1,c2)  on(c2,c3) • (S,s0) |= ○○(on(c1,c2)  on(c2,c3)) • (S,s1) |= ○(on(c1,c2)  on(c2,c3)) • (S,s2) |= on(c1,c2)  on(c2,c3) • (S,si) |= □holding(crane1,c1) • Holding(crane1,c1) is false for all k  i • (S,si) |= x ( on(x,c1)  □on(x,c1) ) • The same container is on c1 in all subsequent states • (S,si) |= x □( on(x,c1)  ○on(x,c1) )

10. υ - until □ - always ◊ - event. ○ - next Encoding our Motivating Ex. • “Don’t move container if position is consistent with goal” • Φ1(c,d,p) = [GOAL(in(c,p))  q GOAL(in(c,q))]  [GOAL(on(c,d)  e GOAL(on(c,e))] • Holds if acceptable when container c is on item d in pile p (no goal requiring c in another pile or on top of something else) • Φ2(c,p) = ok(c,p) [same(p,pallet)  d (Φ1(c,d,p)  ok(d,p))] • ok(c,p) holds iff c is in pile p and c’s position is consistent with the goal • Φ3(c) = p (Φ2(c,p)  ok(c,p)) • holds iff c’s position is consistent with the goal • Φ = c [Φ3(c)  pd □(in(c,p)  in(c,d))] • holds iff for every container c whose position is consistent with g, c will never be moved (c always remains in same pile and on same item in that pile)

11. Break • Yum, dinner 

12. Progression • Computing progression of a control formula Φ is essential for pruning • Formula progress(Φ,si) is true in si+1 iff Φ is true in si. This is called Φ’s progression. • progress(Φ,si) is the formula produced from progr(Φ,si) by performing usual simplifications • Replace true  d with d, true with false, etc. • Both functions can be computed in low-order polynomial time with algorithms directly implementing their definitions (S,si,g) |= Φ iff (S,si+1,g) |= progress(Φ,si)

13. υ - until □ - always ◊ - eventually ○ - next Definition of progr(Φ,si) • If Φ contains no modal operators, then • progr(Φ,si) = true if si |= Φ, false if si | Φ • Logical connectives are as usual • progr(Φ1Φ2,si) = progr(Φ1,si)  progr(Φ2,si) • progr(Φ,si) = progr(Φ,si) • Modal operators are as follows: • progr(○Φ,si) = Φ • progr(Φ1υΦ2,si) = ( (Φ1υΦ2)  progr(Φ1,si) )  progr(Φ2,si) • progr(◊Φ,si) = (◊Φ)  progr(Φ,si) • progr(□Φ,si) = (□Φ)  progr(Φ,si)

14. υ - until □ - always ◊ - eventually ○ - next Example of Progr progr(□Φ,si) = (□Φ)  progr(Φ,si) • Φ = □on(c1,c2) • si = on(c1,c2) • progress(Φ,si) = • = □on(c1,c2)  progress(on(c1,c2),si) • =□on(c1,c2)  true • = Φ • What if si = on(c1,c2)?

15. Using Control Formulas in Planning • Let S = <s0,s1,…> be infinite and Φ be an STL formula. • If (S,s0,g) |= Φ, then for every finite truncation S’ = <s0,s1,…,si> of S, progress(Φ,S’)  false • Let s0 be a state,  be a plan applicable to s0, and S = <s0,…,sn> be the seq. of states produced by applying  to s0. • If Φ is an STL formula and progress(Φ,S) = false, then S has no extension S’ = <s0,…,sn, sn+1,sn+2,…> such that (S’,s0,g) |= Φ Modify Forward-search to prune any partial plan  such that progress(Φ,S) = false

16. υ - until □ - always ◊ - eventually ○ - next Our Handy Example • Let s0 and g be for a container-stacking problem with constant symbols c1, …, ck. Let Φ1, Φ2, Φ3, and Φ be as before. • progress(Φ,s0) = • = progress(c [Φ3(c)  pd □(in(c,p)  in(c,d))], s0) • = progress([Φ3(c1)  pd □(in(c1,p)  in(c1,d))], s0)  …  progress([Φ3(ck)  pd □(in(ck,p)  in(ck,d))], s0) • Suppose in s0, c1 is consistent with g and is on item d1 in pile p1. Then • s0 |= Φ3(c1) • s0 |Φ3(ci) for i = 2, …, k • Which means progress(Φ,s0) = • = progress( pd □(in(c1,p)  in(c1,d)), s0 ) • = progress( □(in(c1,p1)  in(c1,d1)), s0 ) • = □(in(c1,p1)  in(c1,d1)) • If an applicable action a to state s0 moves c1 then • (s0,a) | progress(Φ,s0) • Thus (s0,a) can be pruned.

17. Finally, the Planning Procedure!

18. Planning Procedure - Comments • Sound and complete, if the problem is solvable and STL formula Φ is entailed for at least one solution of the problem • Soundness follows from soundness of Forward-Search • Completeness follows from what? • the condition on Φ Control formulas are like specialized computer programs and must be debugged.

19. Extensions • Function Symbols • Axioms (Horn-clauses) • Restrict axioms to Horn-clauses and use a Horn-clause theorem prover • Attached Procedures • Allow some func./predicate symbols to be evalutated as attached procedures • Time • Actions with time durations and overlapping • Extended Goals • Add control rules like Φ = □at(r1,bad-loc) • Reach goal in 2 actions or fewer Φ = g V (○g)

20. TLPlan • On vega.cc.lehigh.edu: /home/sml3/planning/sys/tlplan • http://www.cs.toronto.edu/~fbacchus/tlplan-manual.html • Requires one input “script” file referring to two others: • (load-file "BlocksWorld.tlp") • (load-file "BlocksProblems.tlp") • (set-statistics-file "BlocksProblems.csv") • (set-goal (goal0)) • (set-initial-world (state0)) • (set-plan-name "Problem0") • (plan) ;; try and solve the problem • (select-final-world) ;; needed for next line • (print-pddl-plan) • (exit) ;;remove this line for interactive mode • Running: from the directory containing your domain… • ../runtlplan script.tlp Domain File Problem File Defined in Problem File Also in tlplan.log or make a symbolic link in your domain directory via: ln –s <path>/runtlplan

21. Sample Domain File (clear-world-symbols) ;Remove old dom symb ;;; WORLD SYMBOLS (declare-described-symbols (predicate on 2) ;…and so on (predicate ontable 1)) (declare-defined-symbols (predicate goodtower 1) (predicate goodtowerabove 1) ;…and so on (function depth 1)) ;;; DEFINED PREDICATES (def-defined-predicate (goodtower ?x) (and (clear ?x) (goodtowerbelow ?x))) ;;; TEMPORAL CONTROL FORMULA (define (bw-control1) (always (forall (?x) (clear ?x) (implies (goodtower ?x) (next (goodtowerabove ?x))) ))) ;;; OPERATORS (def-strips-operator (pickup ?x) (pre (handempty) (clear ?x) (ontable ?x)) (add (holding ?x)) (del (handempty) (clear ?x) (ontable ?x))) ;;; PRINT ROUTINES and FUNCTIONS Problem File (define (state0) (clear a) (clear b) (clear c) (ontable a) (ontable b) (ontable c) (handempty)) (define (goal0) (on a b) (on b c) (ontable c)) …

22. Defining a Domain: Init/Def • (clear-world-symbols) • Must call first in a new domain definition file • Clears the prev. domain's language definition • Resets temporal control formula and the print world command to their defaults • (declare-described-symbols (function|predicate name arity [no-cycle-check|rewritable]) ...) • must be declared prior to any other symbols • (declare-defined-symbols (function|predicate|generator name arity) ...) • must be declared after the described symbols • (def-defined-predicate (name parameters) (local-vars declarations) formula) • Includes a new predicate, defined in terms of a FO formula involving other predicates • Can be recursive • (define name list) • name is an abbreviation for list. Allows macro subst. in domain definition files. • Use to define temporal control formulas • (:= ?variable value) • assigns value to ?variable

23. Formula Syntax Termsconstant – symbol, number, or string?variable – any symbol starting with “?”(?array index …) – dimension must agree with args(function term …) – built-in or declared w/ init. decl. Atomic Formulas (predicate term …) – predicate declared w/ init. decl.(= term1 term2) – predefined equality binary predicate (TRUE) – const. atomic formula; always true (FALSE) – const. atomic formula; always false First-Order Formulas atomic-formula (and formula …) --  (or formula …) –  (xor formula …) –  (not formula …) –  (implies formula1 formula2) --  (if-then-else formula1 formula2 formula3) (forall var-gen … [formula]) --  (exists var-gen … [formula]) –  (exists! var-gen … [formula]) – unique  (exactly one) Temporal Logic Formulas (TF)first-order-formula(next tf) -- ○(eventually tf) – ◊(t-eventually ispec tf) – ispec is an interval of states(always tf) – □(t-always ispec tf) – ispec is again an interval(t-until ispec tf1 tf2) – υ Modalities(goal formula | tf | generator) – eval in goal world(previous formula | tf | generator) – eval in prev. state(current formula | tf | generator) – eval in cur. state

24. Defining Operators STRIPS Operator(def-strips-operator name pre delcost duration priority);; where…(namev …) – Declares op name and params(pre formula) – Precondition list(add p …) – Add list(del p …) – Delete list(cost n) – Cost of action, default is 1(duration n) – Duration of action, default 1(priority n) – Used in search to order successor states, default 0 (def-strips-operator (pickup ?x) (pre (handempty) (clear ?x) (ontable ?x)) (add (holding ?x)) (del (handempty) (clear ?x) (ontable ?x)))

25. Defining TL Control Formulas • (define name list) (define (bw-control1) (always (forall (?x) (clear ?x) (implies (goodtower ?x) (next (goodtowerabove ?x))) )))

26. Contact Me • Start early, defining control rules is tricky, REQUIRES DEBUGGING and tuning • sml3@lehigh.edu