Information-Theoretic Approaches to Branching in Search

1. Information-Theoretic Approaches to Branching in Search Andrew Gilpin and Tuomas Sandholm Carnegie Mellon University Computer Science Department

2. Introduction • Deciding which question to branch on is a key element of search algorithms • We present four families of branching strategies • Each shows promising results over existing methods • Each technique is information-theoretically motivated • Start of search: Most uncertainty • End of search: Zero uncertainty

3. 1 e(x) 0 0 1 0.5 Quantifying uncertainty in 0-1 MIP • Main idea: treat relaxed variables as independent probabilities • Entropy is used to measure amount of uncertainty in a variable: e(x) = -x log2x – (1-x) log2 (1-x) • Entropy is additive for independent variables

4. Entropic lookahead for variable selection Entropic Branching (EB) • Let x be the LP solution at the current node • For each fractional xi • Let xl be solution of LP with xil≤ 0 • Let xu be solution of LP with xiu≥ 1 • entropyi = (1-xi) ∙ entropy(xl) + xi∙ entropy(xu) • Return argminientropyi (This algorithm generalizes to general integer programs)

5. Entropic lookahead for variable selection • Entropic branching is similar to strong branching • Require same amount of computation • Can be combined in hybrid heuristics • Entropic branching can be generalized to: • multi-variable branches • more than one-step lookahead

6. Experimental results for lookahead-based branching strategies • On MIPLIB, entropic and strong branching are dominated by branching on most-fractional variable • Entropic and strong branching perform comparably • On real-world procurement optimization problems: • Strong branching outperforms most-fractional variable branching by 27% • Entropic branching outperforms strong branching by 29.5% • Thus, entropic branching performs as well or better than strong branching • Even though entropic branching ignores objective info

7. Combinatorial procurement auction with max winners constraint • Suppliers submit bids on bundles of items • Buyer specifies a maximum number of winning suppliers • Buyer wishes to choose an allocation such that cost is minimized and the max winners constraint is satisfied • This problem is NP-complete (even if bids are on single items only) • The max winners constraint is the main driver of hardness, and this class of problems has been observed to be difficult in practice • The MIP formulation has a binary variable for each bid, and a binary indicator variable for each supplier

8. Lookahead-free branching strategy for procurement optimization Indicator entropic branching (IEB) • Let yj be the value of supplier j’s indicator variable in the current node’s LP solution • For each j where yj is fractional: • Let entropyj be the sum of the entropies of supplier j’s bids • Branch on yk, where k = argminjentropyj

9. Multi-variable branching • We can generalize entropic branching to branching on sums of variables • Given a set X ={x1,…, xn}, let k = floor(x1+ ∙∙∙ + xn) • We can use the following branches: x1 + ∙∙∙ + xn≤k and x1 + ∙∙∙ + xn≥k + 1 • For each branch, we compute entropy using lookahead to select the least uncertain branch • Prop. Using multi-variable branches does not increase the search space • Experimental results indicate that the search tree is reduced, but an efficient heuristic for selecting the candidate set X is required

10. Conclusion • Introduced new paradigm for branch selection based on an information-theoretic approach • Developed four families of search strategies: • Lookahead entropic branching • Hybrid entropic and strong branching • Lookahead-free entropic branching (for problems with indicator variables) • Multi-variable entropic branching • Experimental results show significant improvement over existing branching strategies