Algorithmic Techniques: Introductory Lecture for Advanced Study

1. Lecture 1Preliminaries COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski Lecture 1: Preliminaries

2. Announcements • Textbook: J. Kleinberg, E. Tardos. Algorithm Design. Addison-Wesley, 2005 • Lectures: Wednesday 9-11, Friday 15-16 in 3.10 • Tutorials: Thursday 11-12 in 1.01 Ashton Bldg • Eleni Akrida (akridel@hotmail com) • Office hours (room 3.11, Ashton Bldg): • Wednesday 12-13 • Friday 14-15 • Info and contact: • http://www.csc.liv.ac.uk/~darek/comp523.html • darek@liv.ac.uk Lecture 1: Preliminaries

3. Assessment and feedback • Assessment: Exam (75%) and 2 tasks (25% total) • Feedback: • Presentation and discussion of solutions after assignment deadline • In case of specific questions, individual appointments Lecture 1: Preliminaries

4. Algorithms - what does it mean? Algorithm - a list/structure of instructions, which are carried out in a fixed order • to find the answer to a question • to calculate Al-Khwarizmi (Muhammed Ibn Musa) - Arabian mathematician who introduced the decimal positional number system to the Western world, the author of Algebra This module: techniques to design algorithms for problems concerning data commonly processed by computers (written in hard discs, propagated in networks, entered by user or external device, etc.) Lecture 1: Preliminaries

5. Types of algorithms • Constructive vs. Non-constructive • Discrete vs. Numerical • Deterministic vs. non-deterministic (e.g., Randomized,Quantum) • Sequential vs. Concurrent vs. DNA vs. … • Exact vs. Approximate • and many others Lecture 1: Preliminaries

6. Example Problem: Find if a given word occurs in a given text Input: Text and word represented as lists Output: Word found or not o n c e u p o n a t o n a Lecture 1: Preliminaries

7. Example - algorithm Naïve solution (exhaustive search): Intuition: • Check letter after letter, starting from the beginning of the lists, if the corresponding letters are equal • If some corresponding letters are not equal, re-start comparing from the second letter of the text (and the first letter of the word) • Etc. from the 3rd letter, 4th letter, until the end of text or successful comparison Implementation: • Initiate three pointers: • blue_pointer and black_pointer at the beginning of blue_list • yellow_pointer at the beginning of yellow_list • Initiate Boolean variable successful into false Lecture 1: Preliminaries

8. Naïve solution cont. • Repeat • stop := false • Move blue_pointer to the next element • Set black_pointer to blue_pointer • Move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true until successful or blue_pointer is at the end of blue_list Output: successful Lecture 1: Preliminaries

9. Example - how it works o n c e u p o n a t o n a • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list Lecture 1: Preliminaries

10. What we expect from algorithms? • Correctness: computes desired output • Termination: stops eventually (or with high probability) • Efficiency: with respect to • Performance measure • Time (or total number of computation steps) • Size of memory used • Number of messages sent • … • Methods of measuring • Worst-case • Average-case • Smoothed (a subset of possible inputs, specific distribution of inputs, etc.) • Competitive (comparing to the best solution for particular input) • Expected • … Analysis depends on Model! Lecture 1: Preliminaries

11. Example - correctness • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list • Successfulis set to true iff after some execution of the internal loop the yellow_pointer is at the end of the yellow_list • yellow_pointer is at the end of the yellow_list iff it came through the whole yellow_list without coming back to the beginning • it happens if the values pointed by black_pointer and yellow_pointer have been the same during all checks of internal loop in the current run (thus a copy of the word exists in the text) Lecture 1: Preliminaries

12. Example - termination • Repeat • stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list • Repeat • Move black_pointer and yellow_pointer to the next elements in corresponding lists • If values pointed by black_pointer and yellow_pointerare different then stop := true • until yellow_pointer or black_pointer is at the end of its list or stop • If yellow_pointer is at the end of yellow_list and not stop then successful := true • until successful or blue_pointer is at the end of blue_list • External Loop Invariant: At the beginning of the external loop, blue_pointer advances, and it has a finite number of advances to take before reaching the end of the text • Internal Loop Invariant: Each run of the internal loop finishes eventually; It happens because yellow_pointer keeps advancing every iteration (first line of the loop), unless stop condition becomes true or it reaches the end of the word (or black_ pointer reaches the end of the text) Lecture 1: Preliminaries

13. Example - efficiency • Complexity measures: time, size of additional memory • Size of additional memory: • 3 single pointers and 2 binary variables • Time complexity: • Worst-case: if the sequence is not included in the text then time is (almost) proportional to the multiplication of sizes of blue and yellow lists, e.g., text: ‘aab’ repeated ntimes word: aaa 1, 2 or 3 internal loop runs, 3n times, gives about 6nruns Lecture 1: Preliminaries

14. Example - efficiency cont. • Time complexity (cont.): • Average case: text or sequence are randomly selected; average short word should be find quickly in the beginning of an average long text, in time proportional to the squared length of the word (exercise) • Smoothed analysis: text must be randomly selected from “reasonable” set of texts (usually complicated analysis, depends on the family of texts - dictionary) Lecture 1: Preliminaries

15. Efficiency - asymptotic notation • n - integer, size of data, f() - function over integers • Complexity O(f(n)): there is a positive constant csuch that for every positive integer ncomplexity is at mostcf(n) • Complexity (f(n)) : there is a positive constant csuch that for every positive integer ncomplexity is at leastcf(n) • Complexity o(f(n)): complexity divided by f(n)comes to0 with ngoing to infinity (strictly smaller than f(n)) • Complexity (f(n)) : complexity divided by f(n) comes towith ngoing to infinity (strictly bigger than f(n)) Lecture 1: Preliminaries

16. Example - asymptotic complexity • Worst case time: • If size of text is n and size of word is n/2: O((n/2)(n/2)) = O(n2) • If size of text is m and size of sequence is k : O((m-k)k) Lecture 1: Preliminaries

17. Examples 5n3+100 = O(n3) , 5n3+100 ≠ O(n2) 5n3+100=(n3) , 5n3+100 ≠ (n4) log n = o(na) for any positive constanta na = o(cn) for any positive constantsa,c log (4n)= log n + log 4 = O(log n) log(n4)= 4 log n = O(log n) (4n)3 = 64n3= O(n3) (n4)3 = n12 = (n4) (3n)4= 81n = (3n) Logarithms are to the base 2 Lecture 1: Preliminaries

18. Symmetric properties and tight bound Iff(n) = O(g(n)) theng(n) = (f(n)) and vice versa Iff(n) = o(g(n)) theng(n) = (f(n)) and vice versa Definition: If f(n) = O(g(n)) and f(n) =(g(n)) then f(n) = (g(n))andg(n) = (f(n)) Example: 9n2 + n +7 = (3n2) = (n2) Lecture 1: Preliminaries

19. Transitive properties (order) Iff(n) = O(g(n)) and g(n) = O(h(n)) then f(n) = O(h(n)) Iff(n) = (g(n)) and g(n) = (h(n)) then f(n) = (h(n)) Iff(n) = (g(n)) and g(n) = (h(n)) then f(n) = (h(n)) Lecture 1: Preliminaries

20. Sum and maximum f1(n) + … + fa(n) = (max(f1(n), … , fa(n))) for any constant positive integer a Example: • 3n2 + n +7 = (3n2) = (n2) If the range of the summation index is not constant: • in i = n(n+1)/2 = (n2)  (max{1, … , n}) = (n) Lecture 1: Preliminaries

21. Logarithmic time O(log n) • Reduce input to any fraction of it in constant time • Example: searching if a given element xis in a sorted array Lecture 1: Preliminaries

22. Linear time O(n) • It is sufficient to look at each element of the input constant number of times • Example: finding maximum in an (unsorted) array Lecture 1: Preliminaries

23. Time O(n log n) • Split the input into two pieces of the similar size and merge both solutions in linear time • Example: sorting an array (split into halves, sort each part separately, merge sorted parts in linear time) Lecture 1: Preliminaries

24. Quadratic time O(n2) • It is enough to consider pairs of elements • Examples: • finding the closest pair of points located in a plane • sorting by comparison of subsequent elements (insertion sort) Lecture 1: Preliminaries

25. Polynomial time O(na) • Algorithm has many nested loops • Example: are there any two disjoint sets among given family of nsets, each of at most nelements (time O(n3)) Lecture 1: Preliminaries

26. Exponential time O(cn) • Algorithm considers many subsets of the input • Example: exhaustive search to find the largest clique contained in a given graph (time O(2n)) Lecture 1: Preliminaries

27. Beyond exponential time: O(n!), O(nn), … • Algorithm searching in a large space • Example: search performed in the set of all permutations (time O(n!)) Lecture 1: Preliminaries

28. Graphs • Set of nodes |V| = n • Set of edges |E| = m • Undirected edges/graph: pairs of nodes {v,w} • Directed edges/graph: pairs of nodes (v,w) • Set of neighbors of v : set of nodes connected by an edge with v (directed: in-neighbors, out-neighbors) • Degree of a node: number of its neighbors • Path: a sequence of (non-repeating) nodes such that every two consecutive nodes constitute an edge • Length of a path: number of nodes minus 1 • Distance between two nodes: the length of the shortest path between these nodes • Diameter: the longest distance in the graph Lecture 1: Preliminaries

29. Lines, cycles, trees, cliques Line Cycle Tree Clique Lecture 1: Preliminaries

30. Conclusions • Algorithm: list/structure of instructions • Algorithmic methods for problems arising from computer and communication applications • Guarantee correctness, termination and efficiency • Model is important! • Asymptotic analysis of efficiency Lecture 1: Preliminaries

31. Textbook and exercises READING: • Chapter 2, up to section 2.4 OBLIGATORY EXERCISES: • Exercises 1,2,3 page 67 • Solved exercises 1,2 pages 65,66 OPTIONAL: • Exercises 3,4,5,6 pages 67,68,69 • Exercises 7,8 pages 69,70 Lecture 1: Preliminaries