Heuristics

1. Heuristics Definition – a heuristic is an inexact algorithm that is based on intuitive and plausible arguments which are “likely” to lead to reasonable solutions for a given problem, but are not guaranteed to do so. Note: These slides adapted from Y. Fathi, North Carolina State University, lecture notes IE589D.

2. Heuristics Common characteristics among heuristic procedures: • Speed: A heuristic procedure usually has a relatively small computational requirement. • Feasible solutions: A heuristic procedure usually finds a feasible solution for the given problem. • Common sense: Many heuristic procedures are based on common sense, or an adaptation of an exact method for simpler problems. • Search: A heuristic procedure may or may no involve a “search” in the set of feasible solutions. • No guarantee of optimality: A heuristic does not guarantee the solution it obtains is optimal for the given problem.

3. Heuristics Why use a heuristic method to solve a discrete optimization problem (integer problem)?: • The problem is difficult (e.g. known to be NP-Hard), and we must solve relatively “large” instances of the problem. • Needed in the branch and bound and implicit enumeration approaches for solving IPs. The better the heuristic, the faster the B&B or implicit enumeration will solve. • When you don’t necessarily need an optimal solution, but need a good solution fast.

4. Heuristics Greedy Algorithms – also called single pass heuristics, constructive methods, myopic algorithms. • Incremental Feature – construct a sequence, or subset, one item at a time. • No backtracking – once item has been sequenced, no re-sequencing occurs. • Greedy selection – most profitable selection made from among available alternatives.

5. Heuristics Greedy Algorithm Example – Knapsack Problem Max s.t. Greedy Method • Sort items by pj/ wj • Load sack until an item won’t fit. • Skip that item and go to the next item that fits. • Continue step 3 till done

6. Heuristics Greedy Algorithm Example – Knapsack Problem Max s.t. Greedy Method - II • Sort items by pj/ wj • Load sack until an item won’t fit. • Once an item does not fit, compare the value of the item that doesn’t fit to the value of all items loaded so far, if the item is more valuable, throw out the previous items and put this item into the knapsack. • Continue step 3 till done

7. Heuristics Greedy Algorithm Example – PDS1 Sequencing Problem Sequence a series of jobs to minimize the maximum lateness (Lmax) for a set of jobs to be processed on a single machine. Each job belongs to a given part family. A setup time occurs when changing between part families. Greedy Method - II • Sequence all jobs within a family by earliest due date. • Then, sequence families by choosing family with earliest due date first. Greedy Method - I • Sequence all jobs by earliest due date.

8. Heuristics Greedy Algorithm Ex. – Traveling Salesman Problem Given a collection of cities, find the shortest/cheapest way to visit all cities, starting from a “home city” and returning home upon completion of the tour. What can be done to strengthen this greedy algorithm? Greedy Method – I Closest Unvisited City Algorithm, or Nearest Neighbor Algorithm • Starting with any city, go to the nearest neighbor that has not been visited. • Continue step 1 until all cities have been visited. • Return to initial city to complete tour.

9. Heuristics Example Closest Unvisited City Algorithm for TSP

10. Heuristics Greedy Algorithm TSP – based on Minimum Spanning Tree • Find the minimum spanning tree (MSP). • Double up each edge of the MSP, this guarantees each node has an even number of edge. Known as a Euler path. • Trace the Eulerean path. • Skip each city already visited on the Eulerean path.

11. Minimal Spanning Tree Objective: Find the minimum distance such that all nodes are visited once (i.e. no cycles). 9 2 6 3 6 2 7 1 3 3 3 1 4 3 4 5

12. Minimal Spanning Tree One possible spanning tree, Z = 26 (note, no cycles). 9 2 6 3 7 1 3 4 3 4 5

13. Minimal Spanning Tree Algorithm: Step 1 – Select an arbitrary node initially. Identify a node that is closest and include the arc connecting these two nodes in the spanning tree. Step 2 – Out of the remaining nodes, determine the one that is closest to a node already selected in the spanning tree. Include the arc connecting these two nodes in the spanning tree. Whenever there is a tie for the closest node, it is broken arbitrarily.

14. Minimal Spanning Tree Start with node 1. Node 2 is closest. 9 2 6 3 6 2 7 1 3 3 3 1 4 3 4 5

15. Minimal Spanning Tree Node 3 is closest to partial spanning tree {(1,2)}. Remove any cycles. 9 2 6 3 6 2 1 3 3 3 1 4 3 4 5

16. Minimal Spanning Tree Node 4 is closest to partial spanning tree {(1,2), (2,3)}. Remove any cycles. 9 2 6 3 6 2 1 3 3 3 1 3 4 5

17. Minimal Spanning Tree Node 5 is closest to partial spanning tree {(1,2), (2,3), (3,4)}. Remove any cycles. 9 2 6 3 6 2 1 3 3 3 1 4 5

18. Minimal Spanning Tree Node 6 is only remaining node. Remove any cycles. 2 6 3 2 1 3 3 3 1 4 5 Minimal spanning tree is {(1,2), (2,3),(3,4),(3,5),(5,6)}. Z = 12.

19. TSP Heuristic 2. Double up each edge of the MSP, this guarantees each node has an even number of edge. Known as a Euler path. 2 6 3 2 1 3 3 3 1 4 5

20. TSP Heuristic 3. Trace the Eulerean path. 4. Skip each city already visited on the Eulerean path. 2 6 3 2 1 3 3 3 1 4 5

21. TSP Heuristic 3. Trace the Eulerean path. 4. Skip each city already visited on the Eulerean path. 2 6 3 2 1 3 3 3 1 4 5