Tree Clustering for Constraint Networks

1. Chris Reeson Advanced Constraint Processing Fall 2009 Tree Clustering for Constraint Networks By RinaDechter & Judea Pearl Artificial Intelligence, Oct 1988

2. Overview • Contributions of the Paper • Context of the Paper • Algorithms • Tree Clustering • Adaptive Consistency • Relative Merits • Conclusion • Related algorithms

3. Contributions of the Paper • Introduces Tree Clustering (T-C) for backtrack-free search • Introduces the Adaptive Consistency (A-C) algorithm • Compares the two algorithms

4. Context • Two ways to restrict CSPs to make finding the minimal CSP efficient • Topology of constraint graph • Types of constraints • T-C & A-C • Target the topology: tree • Are applicable to binary & non-binary CSPs • Two ways to transform a constraint graph into a tree • Remove redundant arcs in dual graphs (join tree) • Form larger clusters of c-variables (simulates a join tree)

5. Definitions • Hyper, dual, and primal graphs 8 1 2 3 h6 h7 h2 h8 Hypergraph Dual graph Primal graph h3 9 11 10 4 12 13 5 8 7 6 h4 h5 3 8 2 1 h1 h5 h3 h2 h7 h6 11 9 4 h4 10 h8 5 12 13 8 6 7 • The discussion focuses on primal graphs for simplicity

6. From Join Graph to Join Tree • Join Graph • Start w/ the dual graph, remove redundant edges while maintaining the connectedness property • Connectedness property: For each two nodes sharing a variable, there is at least one path of labeled arcs containing the shared variable • Join Tree • When the join graph is a tree, the dual CSP can be solved BT free w/ directional arc consistency • What if there isn’t a join tree?  The idea for Tree Clustering ABC A AEF C AE AC E ACE CDE CE ABC AEF AE AC ACE CDE CE

7. Motivating Problem Constraint Graph A solution C < A C ≠ D A ≠ B D < A C < B D < B 3 B 4 A F > B E > B E 4 C 1 D 2 G 5 G > F G > E 3 F Domains: {1, 2, 3, 4, 5}

8. Overview • Contributions of the Paper • Context of the Paper • Algorithms • Tree Clustering (T-C) • Adaptive Consistency (A-C) • Relative Merits • Conclusion • Related Algorithms

9. Tree Clustering (T-C): Idea • A CSP organized as a join tree can be solved efficiently • Tree Clustering Algorithm • Solves a CSP by breaking it into subproblems • Triangulates the primal graph • Solves subproblems & combines the solutions ABCD BED DFE B B A A EFG BD E E C C D D DE G G F F EF

10. Tree Clustering (T-C): Algorithm • Triangulate the primal graph • Identify all the maximal cliques in the primal chordal graph • Form a join tree • Solve the subproblems • Each cluster becomes single variable • Solve the tree problem • Perform DAC from leaves to root • Instantiate BT-free from root to leaves ABCD BED DFE B 3 A 4 4312, 5312, 5432, … EFG BD 342, 351, 221, … E 4 1 C 2 D DE G 5 234, 415, 153, … 3 F EF 435, 123, 112, …

11. Tree Clustering (T-C): Costs • Given a CSP and its primal graph generate a chordal primal graph: O(n2) • Identify all the maximal cliques in the primal chordal graph: O(|E’|) • Form the dual graph: O(n) • Solve the sub problems: O(kr) where k=domain size • Solve the tree problem: O(n ∙ t log t)… ABCD BED DFE B 3 4 A 4312, 5312, 5432, … EFG BD 342, 351, 221, … 4 E C 1 2 D DE G 5 234, 415, 153, … F 3 EF 435, 123, 112, …

12. Tree Clustering (T-C): Total Cost • Dominated by O(n ∙ t log t) • t is the largest number of solutions in a cluster, t ≤ kr • Time: O(n ∙ kr ∙ r log k) = O(nr ∙ kr ) • Space: O(n ∙ kr )

13. Overview • Contributions of the Paper • Context of the Paper • Algorithms • Tree Clustering (T-C) • Adaptive Consistency (A-C) • Relative Merits • Conclusion • Related Algorithms

14. Adaptive Consistency (A-C) • An ordered constraint graph is backtrack-free if the level of directional strong consistency along this order is greater than the width of the ordered graph • Beware • Enforcing i-consistency for i > 2 often requires the addition of constraints which increase the width

15. Adaptive Consistency (A-C): Idea • Given an ordering d, • d-i-consistency is defined recursively • letting i change dynamically from node to node • (A-C later redefined as bucket elimination)

16. Adaptive Consistency: Algorithm • For i=n downto1 do Steps 2-4 • Compute PARENTS(Xi) • Connect all PARENTS(Xi) • Perform Consistency(Xi, PARENTS(Xi))  joining the constraints between Xi & its parents • Build a solution BT-free in the ordering (X1, …, Xn) A A C C tighten A by 2 consist ACB join CB join AC to tighten AC by 3c B B B A D D BE join CB join AC to tighten ACB by 4c E C D G E E tighten D by 2 consist F F F EF join DE to tighten DE by 3 consist GF join GE to tighten EF by 3 consist G G

17. Adaptive Consistency: Cost • Time: O(n ∙ exp(W*(d) + 1)), see Dechter page 109 • Space: O(n ∙ kW*(d))

18. Overview • Contributions of the Paper • Context of the Paper • Algorithms • Tree Clustering (T-C) • Adaptive Consistency (A-C) • Relative Merits • Conclusion • Related Algorithms

19. Relative Merits • Arcs resulting from triangulation match arcs added by adaptive consistency, for the same ordering • Every cluster in T-C is represented in A-C by a series of smaller constraints • Similar bounds • W*(d) + 1 = the size of the largest clique • A-C eliminates the redundancy of generated solutions • T-C enumerates all solutions that A-C represents via constraints. A A C C B B B B A A D D E E C C D D G G E E F F F F G G

20. Conclusions • Tree clustering groups c-nodes into a tree capable of supporting query answering backtrack-free • Useful in systems that need to answer many questions about a dataset and where the environmental conditions undergo local changes • Recently, researchers have started looking at T-C for solving the CSP, see BTD by Jégou & Terrioux (and others in soft CSPs)

21. Note On Triangulation • Find the triangulated graph w/ smallest maximum clique: NP-hard • Heuristics • Operation: when eliminating a node, connect all its neighbors, to form a clique (fill edges) • H1: choose the node w/ smallest degree • H2: choose the node that, after elimination, yields the smallest number of fill edges • H3: Given any ordering (e.g., maximal cardinality ordering), moralize the graph • Elimination order is the reverse of instantiation order • Elimination order of a triangulated graph is called a perfect elimination scheme • In this ordering, every node is simplicial: forms a clique w/ its neighbors • If you follow elimination order, no fill edges need to be added

22. Maximal Cardinality Ordering • An approximation of min. width ordering • Choose a node arbitrarily a simplicial node • Among the remaining nodes, choose the one that is connected to the maximum number of already chosen nodes, break ties arbitrarily • Repeat… • Reverse the final order Tsang 6.2.4 Dechter Fig 4.5

23. Two Additional Algorithms • Maximal Cliques of the triangulated graph • Join Tree of the triangulated graph