1. Linear Least Squares Approximation Jami Durkee

2. Problem to be Solved • Finding Ax=b where there are no solution y=xy=x+2 • Interpolation of graphs where there are numerous points or if it is not possible to find • Examples: interpolation of: {(-20,1),(-15,5/2),(-15/2,-2),(0,0),(1,0),(2,3),(4,4),(9,-1),(10,3/2),(11,0)} OR

3. Definition • Least squares solution- the closest value to x, in this case the closest line to all data points

4. How to solve it • Since Ax=b, find A= band solve for x1 and x2. then the linear least square is y=x1+x2t • Where A is an nx2 matrix A with n=the number of data points • The first column of A is all 1 and the second column is the x values • Example: with points {(0,0),(1,3),(2,-1)} A= • is the transpose of A • B is an nx1 matrix of the y values • For the example above b=

5. How to develop the algorithm • Expressing perpendicularity in terms of matrix multiplication • (b-Ax)=0 • Transpose of a product is the product of the two matrices transposed and switched • (b-Ax)=0 • So it is perpendicular to every vector x in including itself so, • (b-Ax)=0 • The system of equations that defines the least squares solution is: • Ax=b

6. example A= b AX= b=X = So = Solve to get: x1=1/2 and x2=3/2 So the line of best fit is y=1/2+3/2t For the data points{(0,0),(0,1),(1,2)}, find the linear least squares approximation A= = b=

7. Error • Residual of least squares solution r=b-Ax • Euclidean length of a vector: ||r||2= • Squared error SE= • Root mean squared error RMSE=

8. Advantages • It can be done using any data points and for as many data points as wanted • It is only one variable so it is easier to solve for and graph • Several different errors can be found

9. Disadvantages • It is only an approximation, unless the points are in a line the linear least square will not be on any or all of the points • The graph may go through one or more points, but it does not have to so all points could have an error • Deciding which error to use