Télécharger la présentation
## Chapter 5

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Chapter 5**Numeric Algorithms**Chapter Outline**• Calculating polynomials • Matrix multiplication • Linear equations**Prerequisites**• Before beginning this chapter, you should be able to: • Do simple algebra • Evaluate polynomials • Describe growth rates and order**Goals**• At the end of this chapter you should be able to: • Evaluate a polynomial by Horner’s method • Evaluate a polynomial by preprocessing its coefficients • Explain the analysis of preprocessed coefficients • Explain matrix multiplication**Goals (continued)**• Trace Winograd’s matrix multiplication • Explain the analysis of Winograd’s matrix multiplication • Use Strassen’s matrix multiplication • Explain the Gauss-Jordan method for solving systems of linear equations**Numeric Algorithm Efficiency**• Numeric programs will do a particular calculation a large number of times • Small improvements can result in significant time savings because of how many times the calculation is done**Numeric Analysis Basics**• We count additions and multiplications • Because additions are typically much faster than multiplications, reducing the multiplications by increasing the additions can still result in an improvement • The analysis will be based on the highest power in a polynomial or the size of the matrices we are working with**Calculating Polynomials**• We will use a generic polynomial form of:where the coefficient values are known constants • The value of x will be the input and the result is the value of the polynomial using this x value**Standard Evaluation Algorithm**result = a[0] + a[1]*x xPower = x for i = 2 to n do xPower = xPower * x result = result + a[i]*xPower end for return result**Analysis**• Before the loop, there is • One multiplication • One addition • The for loop is done N-1 times • There are two multiplications in the loop • There is one addition in the loop • There are a total of • 2N – 1 multiplications • N additions**Horner’s Method**• Based on the factorization of a polynomial • The generic equation is factored as: • For example, the equation: • would be factored as:**Horner’s Method Algorithm**result = a[n] for i = n - 1 down to 0 do result = result * x result = result + a[i] end for return result**Analysis**• The for loop is done N times • There is one multiplication in the loop • There is one addition in the loop • There are a total of • N multiplications • N additions • Saves N – 1 multiplications over the standard algorithm**Preprocessed Coefficients**• Uses a factorization of a polynomial based on polynomials of about half the original polynomial’s degree • For example, where the standard algorithm would do 255 multiplications to calculate x256, we could square x and then square the result seven more times to get the same result**Preprocessed Coefficients**• Used with polynomials that are monic (an=1) and have a highest power that is one less than a power of 2 • If the polynomial has highest power that is 2k– 1, we can factor it as:where j = 2k-1**Preprocessed Coefficients**• If we choose b so that it is aj-1 – 1, q(x) and r(x) will both be monic, and so the process can be recursively applied to them as well**Preprocessed Coefficients Example**• For the example equation of:because the highest degree is 3 = 22 –1, j would be 21 = 2, and b would have a value of a1 – 1 = 6 • This makes one factor x2 + 6, and we divide p(x) by this polynomial to find q(x) and r(x)**Preprocessed Coefficients Example**• The division is:which gives**Analysis**• We analyze preprocessed coefficients by developing recurrence relations for the number of multiplications and additions • In the factorization, we break the polynomial up into two smaller polynomials and do one additional multiplication and two additional additions**Analysis**• Because we required that N = 2k– 1, we get: M(1) = 0 A(1) = 0M(k) = 2M(k–1) + 1 A(k) = 2A(k–1) + 2 • Solving these equations gives about N/2 multiplications and (3N – 1)/2 additions • We need to include the k – 1 (or lg N) multiplications needed to calculate the series of values x2, x4, x8, …**Polynomial Algorithm Comparison**• For the example polynomial: • Standard algorithm: • 5 multiplications and 3 additions • Horner’s method • 3 multiplications and 3 additions • Preprocessed coefficients • 2 multiplications and 4 additions**Polynomial Algorithm Comparison**• In general, for a polynomial of degree N: • Standard algorithm: • 2N – 1 multiplications and N additions • Horner’s method • N multiplications and N additions • Preprocessed coefficients • N/2 + lg N multiplications and (3N – 1)/2 additions**Matrix Multiplication**• Two matrices can be multiplied if the number of columns in the first is the same as the number of rows in the second • Each row of the first matrix is multiplied by each column of the second matrix • The value at location i, j of the result is from the addition of the products when row i of the first matrix is multiplied by row j of the second matrix**Matrix Multiplication**• An example of this process is:**Matrix Multiplication Algorithm**for i = 1 to a do for j = 1 to c do R[i,j] = G[i,1] * H[1,j] for k = 2 to b R[i,j] = R[i,j] + G[i,k] * H[k,j] end for k end for j end for i**Analysis**• The “for k” loop runs b – 1 times and does one multiplication and addition each time • The “for j” loop runs c times and does b multiplications and b – 1 additions each time • The “for i” loop runs a times and does b*c multiplications and (b – 1)*c additions each time • Therefore, the algorithm does a*b*c multiplications and a*(b – 1)*c additions**Winograd’s Matrix Multiplication**• We can consider the row and column as vectors V = (v1, v2, v3, v4) and W = (w1, w2, w3, w4) • Each element of the result becomes a dot product of these two vectors:V•W = v1*w1 + v2*w2 + v3*w3 + v4*w4**Winograd’s Matrix Multiplication**• We can factor this dot product into: • Though this looks like more work, the last two lines can be done once for each row of the first matrix, and each column of the second matrix**Winograd’s AlgorithmPreprocessing Step**d = b/2 // calculate rowFactors for G for i = 1 to a do rowFactor[i] = G[i,1] * G[i,2] for j = 2 to d do rowFactor[i] = rowFactor[i] + G[i,2j-1] * G[i,2j] end for end for // calculate columnFactors for H for i = 1 to c do columnFactor[i] = H[1,i] * H[2,i] for j = 2 to d do columnFactor[i] = columnFactor[i] + H[2j-1,i] * H[2j,i] end for end for**Winograd’s AlgorithmCalculation Step**for i = 1 to a do for j = 1 to c do R[i,j] = -rowFactor[i] - columnFactor[j] for k = 1 to d do R[i,j] = R[i,j] + (G[i,2k-1] + H[2k,j]) * (G[i,2k] + H[2k-1,j]) end for end for end for // add in terms for odd shared dimension if (2 * (b / 2) ≠ b) then for i = 1 to a do for j = 1 to c do R[i,j] = R[i,j] + G[i,b] * H[b,j] end for end for end if**Analysis**• Preprocessing step: • The “for j” loop runs d – 1 times and does one multiplication and addition • The “for i” loop runs a (or c) times and does d multiplications and d – 1 additions**Analysis**• Calculation step for an even shared dimension (b) of the matrices: • The “for k” loop runs d times and does one multiplication and three additions • The “for j” loop runs c times and does d multiplications and 3d + 1 additions • The “for i” loop runs a times and does c*d multiplications and c*(3d + 1) additions**Analysis**• The entire algorithm does:a*d + c*d + a*c*d multiplicationsa*(d–1) + c*(d–1) + a*c*(3d+1) additionswhen b is even and d = b/2**Strassen’s Matrix Mutiplication**• Uses a set of seven formulas to multiply two 2 X 2 matrices • Does not rely on elements being commutative under multiplication • It can be applied recursively, in other words, two 4 X 4 matrices can be multiplied by treating each as a 2 X 2 matrix of 2 X 2 matrices**Strassen’s Formulas**• First we calculate a set of temporary values: x1 = (G[1,1] + G[2,2]) * (H[1,1] + H[2,2]) x2 = (G[2,1] + G[2,2]) * H[1,1] x3 = G[1,1] * (H[1,2] – H[2,2]) x4 = G[2,2] * (H[2,1] – H[1,1]) x5 = (G[1,1] + G[1,2]) * H[2,2] x6 = (G[2,1] – G[1,1]) * (H[1,1] + H[1,2]) x7 = (G[1,2] – G[2,2]) * (H[2,1] + H[2,2])**Strassen’s Formulas**• The result is then calculated by: R[1,1] = x1 + x4 – x5 + x7 R[2,1] = x2 + x4 R[1,2] = x3 + x5 R[2,2] = x1 + x3 – x2 + x6**Analysis**• These formulas require 7 multiplications and 18 additions to multiply two 2 X 2 matrices • The real savings occur when this is applied recursively and we do approximately N2.81 multiplications and 6N2.81 – 6N2 additions • Though not used in practice, Strassen’s method is important because it was the first algorithm that is faster than O(N3)**Linear Equations**• A system of linear equations is a set of N equations in N unknowns: a11x1 + a12x2 + a13x3 + … + a1NxN = b1 a21x1 + a22x2 + aa3x3 + … + a2NxN = b2 aNx1 + aN2x2 + aN3x3 + … + aNNxN = bN**Linear Equations**• When these equations are used, the coefficients (a values) are constants and the results (b values) are typically input values • We want to determine the x values that satisfy these equations and produce the indicated results**Linear Equation Example**• An example set of linear equations with three unknowns is:2x1 – 4x2 + 6x3 = 146x1 – 6x2 + 6x3 = 244x1 + 2x2 + 2x3 = 18**Solving Linear Equations**• One method to determine a solution would be to substitute one equation into another • For example, we solve the first equation for x1 and then substitute this into the rest of the equations • This substitution reduces the number of unknowns and equations**Solving Linear Equations**• If we repeat this, we get to one unknown and one equation and can then back up to get the values of the rest • If there are many equations, this process can be time consuming, prone to error, and is not easily computerized**Gauss-Jordan Method**• This method is based on the previous idea • We store the equation constants in a matrix with N rows and N+1 columns • For the example, we would get:**Gauss-Jordan Method**• We perform operations on the rows until we eventually have the identity matrix in the first N columns and then the unknown values will be in the final column:**Gauss-Jordan Method**• On each pass, we pick a new row and divide it by the first element that is not zero • We then subtract multiples of this row from all of the others to create all zeros in a column except in this row • When we have done this N times, each row will have one value of 1 and the last column will have the unknown values**Example**• Consider the example again: • We begin by dividing the first row by 2, and then subtract 6 times it from the second row and 4 times it from the third row**Example**• Now, we divide the second row by 6, and then subtract -2 times it from the first row and 10 times it from the third row**Example**• Now, we divide the third row by 10, and then subtract -1 times it from the first row and -2 times it from the second row**Example**• This gives the result of: • And we have that x1 = 3, x2 = 1, and x3 = 2**Concerns**• In practice, computer round-off error give inaccurate results • If the matrix is singular, one row will be just a multiple of another row and we will get a divide by zero error • There are other algorithms that can also handle these problems but they are more appropriate for a numerical analysis course