Iterative Algorithms & Loop Invariants

Steps in an Iterative Algorithm Typical Loop Invariants Lower Bound for Sort Iterative Algorithms & Loop Invariants (same as in 3101) Jeff Edmonds York University Lecture1 COSC 3101

vsA Sequence of Assertions A Sequence of Actions Max( a,b,c ) “preCond: Input has 3 numbers.” m = a “assert: m is max in {a}” if( b>m ) m = bendif “assert: m is max in {a,b}” if( c>m ) m = cendif It is helpful to have different ways of looking at it. “assert: m is max in {a,b,c}” return(m) “postCond: return max in {a,b,c}”

Purpose of Assertions Useful for • thinking about algorithms • developing • describing • proving correctness

Definition of Assertions An assertion is a statement about the current state of the data structure that is either true or false. eg. the amount in your bank account is notnegative.

Definition of Assertions It is made at some particular point during the execution of an algorithm. It should be true independent ofthe path followed through the code and independent of the input. If it is false, then something has gone wrong in the logic of the algorithm.

Definition of Assertions An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader.

Definition of Assertions • The current state of the computation • Everything that is true at a particular instant in time. • Imagine flying in from Mars,what do you need to know about the present to be able to continue. • It does not say anything about the past,i.e. how the computation got here. • Eg <x=8, y=3, current line of code = line 84>

Definition of Assertions • The current state of the computation • An assertion, A • is a function • that takes as input the current state • and that outputs • EgA = “x is odd” • A(<x=5, y=3, line84>) = True • A(<x=8, y=3, line84> ) = False Trueor False  Computation is on the path  Something has gone wrong!

Algorithm Termination You need to define someMeasure of progressto prove that your algorithmeventually terminates. • An Measure of Progress, M • is a function • that takes as input the current state • and that outputs • EgMeasure = “value of y” • M(<x=5, y=3, line84> ) = 3 • We must prove that this number goes down (up) each iteration. a real number

Designing an Algorithm Is this sufficient? Exit Exit Exit 0 km Exit 79 km 75 km 79 km to school Exit

52 88 14 14,23,25,30,31,52,62,79,88,98 31 98 25 30 23 62 79 Explaining Insertion Sort We maintain a subset of elements sorted within alist. The remaining elements are off to the sidesomewhere. Initially,think of the first element in the array as a sorted list of lengthone. One at a time, we take one of the elements that is off to theside and we insert it into the sorted list where itbelongs. This gives a sorted list that is one element longer than itwas before. When the last element has been inserted, the array iscompletely sorted.

Establishing Loop Invariant <preCond> codeA <loop-invariant> Maintaining Loop Invariant <loop-invariant> ¬<exit Cond> codeB <loop-invariant> Exit Exit Clean up loose ends <loop-invariant> <exit Cond> codeC <postCond> Partial Correctness Proves that IF the program terminates then it works <PreCond>&<code>Þ <PostCond>

Consider an Algorithm Exit 0 km Exit Exit Exit 79 km 75 km

Loop Invariant Maintained Exit

Computation can always proceed

Computation always makes progress Exit 79 km 75 km 79 km 75 km

Computation Terminates 0 km 0 km 0 km Exit Exit 79km 75 km

Computation Terminates Exit Exit

Consider an Algorithm Exit 0 km Exit Exit Exit 79 km 75 km This is sufficient!

Typical Types of Loop Invariants • More of the input • More of the output • Narrowed the search space • GCD Like • Fairy God Mother

Solution Extra More of the Input Loop Invariant The input consists of an array of objects We have read in the first i objects. We will pretend that this prefix is the entire input. We have a solution for this prefix plus some additional information

Loop Invariants andDeterministic Finite Automaton L = {a Î {0,1}* | a has length at mostthree and the number of 1'sis odd }.

Solution Solution Extra Extra More of the Input Loop Invariant The input consists of an array of objects We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have to one for this larger prefix. (plus some additional information)

Solution Solution Extra Extra Exit 79 km 75 km More of the Input Loop Invariant The input consists of an array of objects i to i+1

Solution Exit More of the Input Loop Invariant The input consists of an array of objects In the end, we have read in the entire input. The LI gives us that we have a solution for this entire input.

52,23,88,31,25,30,98,62,14,79 23,31,52,88 Insertion Sort The input consists of an array of integers Solution We have read in the first i objects. We will pretend that this prefix is the entire input. We have a solution for this prefix

23,25,31,52,88 52,23,88,31,25,30,98,62,14,79 23,31,52,88 Insertion Sort The input consists of an array of integers We read in the i+1st object. We will pretend that this larger prefix is the entire input. We extend the solution we have to one for this larger prefix.

Exit Exit 79 km 75 km More of the OutputLoop Invariant The output consists of an array of objects I have produced the first i objects. Produce the i+1st output object. Done when output n objects. i to i+1

Exit 52,23,88,31,25,30,98,62,14,79 Selection Sort Input: The output consists of an array of integers 14,23,25,30,31,52,62,79,88,98 I have produced the first i objects. Produce the i+1st output object. Done when output n objects.

Define Loop Invariant • Maintain a sub-list. • If the key is contained in the original list, then the key is contained in the sub-list. key 25

Typical Types of Loop Invariants • More of the input • More of the output • Narrowed the search space • GCD Like • An instance has been produced that is smaller than the actual instance • These two instances require the same output. Hence, it is suﬃcient for me to forget about the actual instance and simply give the output for the new smaller one. • Fairy God Mother

GCD(a,b) Replace <x,y> with <x-y,y> GCD(a,b) = GCD(x,y) = GCD(x-y,y) = GCD(x’,y’) Exit 79 km 75 km = <21,9> Input: <a,b> Output: GCD(a,b) = 3 Maintain values <x,y> such that GCD(a,b) = GCD(x,y) GCD(a,b) = GCD(a-b,b) GCD(21,9) = GCD(12,9) = 3 smaller Running Time?

GCD(a,b) Input: <a,b> = <9999999999999,2> <x,y> = <9999999999999,2> = <9999999999997,2> = <9999999999995,2> = <9999999999993,2> = <9999999999991,2> O(a) Time = Poly Time?

Size of paper # of bits # of digits Value - n = 2 in2 - n = 17 bits - n = 5 digits - n = 83920 5 83920 1’’ 2’’ Size of Input Instance • Intuitive • Formal • Reasonable • Unreasonable # of bits = log2(Value) Value = 2# of bits

GCD(a,b) Input: <a,b> = <9999999999999,2> <x,y> = <9999999999999,2> = <9999999999997,2> = <9999999999995,2> = <9999999999993,2> = <9999999999991,2> O(a) Time = = 2O(n) = O(log(a)) number of digits n Size =

GCD(a,b) Exit 79 km 75 km = <21,9> Input: <a,b> Output: GCD(a,b) = 3 Maintain values <x,y> such that GCD(a,b) = GCD(x,y) GCD(a,b) = GCD(a-b,b) = GCD(a mod b, b) GCD(21,9) = GCD(12,9) = 3 = GCD(3,9) Much faster We want the first number the biggest

GCD(a,b) = <21,9> Input: <a,b> Output: GCD(a,b) = 3 Maintain values <x,y> such that GCD(a,b) = GCD(x,y) GCD(a,b) = GCD(a-b,b) = GCD(a mod b, b) = GCD(b, a mod b) GCD(21,9) = GCD(12,9) = 3 = GCD(3,9) = GCD(9,3)

GCD(a,b) Lots of progress Little progress Lots of progress Every two iterations: the value x decreases by at least a factor of 2. the size of x decreases by at least one bit Time = n = O(log(a)+log(b)) Size = Input: <a,b> = <10000000000001,9999999999999> <x,y> = <10000000000001,9999999999999> = <9999999999999,2> = <2,1> = <1,0> GCD(a,b) = GCD(x,y) = 1 O(log(a)+log(b)) = O(n)

GCD(a,b)

Typical Types of Loop Invariants • More of the input • More of the output • Narrowed the search space • GCD Like • Fairy God Mother • I have arrived from Mars. • Some amount of progress has been made already. • My Fairy God Mother has set the state of the computation to be just what I want so that I feel comfortable.

Two Graph Problems • Tournament • Euclidian Cycle

A Lower Bound on Sorting Merge, Quick, and Heap Sort can sort N numbers using O(N log N) comparisons between the values. Theorem: No algorithm can sort faster.

" A, $ I, [ A(I) = P(I) or Time(A,I) ³ Tlower(|I|)] A Lower Bound on Sorting I have an algorithm A that I claim works and is fast. Oh yeah, I have an input I for which it does not . • I win if A on input I gives • the wrong output or • runs slow.

Values:Indexes: 23 67 34 21 87 3 13 11 1 2 3 4 5 6 7 8 Values:Indexes: 3 11 13 21 23 34 67 87 6 8 7 4 1 3 2 5 Values:Indexes: 23 67 34 21 87 3 13 11 1 2 3 4 5 6 7 8 Values:Indexes: 11 3 13 21 23 34 67 87 8 6 7 4 1 3 2 5 A Lower Bound on Sorting • Sorting • Input: • Output: • Lower Bound: • Input: An Algorithm for Sorting A • Output: An instance Ion which alg A either • Takes too much time • Or gives the wrong answer

Algorithm Definition Completed Exit Exit Exit 0 km Exit 79 km 75 km 79 km to school Exit

I must output an instance • on which alg A either • Takes too much time • Or gives the wrong answer • It might as well be a permutation of 1..N(indexes not shown) Values:Indexes: 23 67 34 21 87 3 13 11 1 2 3 4 5 6 7 8 4 7 5 2 8 1 3 6 A Lower Bound on Sorting I give you algorithm AI claim it sorts.

A Lower Bound on Sorting I give you algorithm AI claim it sorts. Need to know what the algorithm does before we can know what input to give it. Break this cycle, one iteration at a time. Need to give the algorithm an input before we can know what the algorithm does with the input.

A Lower Bound on Sorting Restrict the search space

Iterative Algorithms & Loop Invariants