Download
1 / 35
350 likes | 478 Vues
This resource provides an in-depth exploration of key concepts in numerical algorithms, focusing on inner products, matrix multiplication, LU decomposition, and the simplex method. It emphasizes optimization problems, such as maximizing probabilities through statistical methods, and minimizes errors in quadratic polynomials. Key computational techniques like gradient descent and Newton iterations are discussed. Additionally, it covers matrix representations, slicing, concatenation, and accessing elements via various layouts. Whether you're dealing with linear algebra or statistical optimization, this guide is your go-to for mastering essential numerical methods.
E N D
Numerical Algs Inner products, Matrix Multiplication, LU-Decomposition (factorization) Simplex Method, Iterative methods (sqrt)
Optimization • Many problems in computing are of the form: • Find the values that minimize (or maximizes) some function subject to constraints
Example (unconstrained) • Find mean and standard deviation of a normal distribution that maximizes the probability of sampleswhere the probability is
Example (unconstrained) • Find a quadratic polynomial where we have made measurementsso that the mean squared error in our measurements is minimized.
Example • (Help me out here)
Matrices • This REALLY should be review for you…. • A matrix is a 2D array of numbers (real or complex) • The number of rows is usually called • The number of columns is usually called • Matrices are represented by upper case letters • Matrix elements two subscripts • is the element in row and column of matrix • Indices start at 1, not zero • Sometimes elements are written lowercase, sometime uppercase. (sloppy) • Matrix multiplication is special
Matrix + Matrix • Addition/Subtraction is done element by element
Matrix * Scalar • Done in parallel • Symmetric ()
Matrix Transpose • Use superscript to indicate transpose. • Exchange the order of subscripts (rows become columns)
Slicing • In this class I will use slicing to indicate a sub-array. • My notation is something like Matlab’s. • Concatenation • I will use the | operator to concatenate matrices: , , or
Vectors • A vector is a single-column matrix. • We write them with lowercase boldface letters. • A row-vector is a single-row matrix. We always write them as to indicate they are the transpose of a column matrix.
Inner Product • Defined for two vectors Other notations for inner product: , , Also called the ‘dot’ product.
Types of Matrix • Zero • Identity • Symmetric • Lower Triangular • Strict Lower Triangular • Upper & Strict Upper • Banded
Accessing Matrix Elements If we know the dimensions of the array ahead of time we can use a fixed-size 2D array. But usually the dimensions are decided at runtime. I cannot pass this as a parameter to a function that operates on arrays with a variable size
Writing functions for 2D arrays • We need to do some extra work to be able to pass matrices arount
Accessing Matrix Elements • The layout is ‘contiguous’ in memory. • The is row major order. • The difference between items on the same row and different columns is _______ • The difference between items on the same column and different rows is _______ • The index of the item at row and column is ________
Accessing Matrix Elements • Other API’s use different layout (Fortran, OpenGL, MatLAB, …) • This is column major order. • The difference between items on the same row and different columns is _______ • The difference between items on the same column and different rows is _______ • The index of the item at row and column is ________ • LaPACK is the most commonly used library for math. • It is implemented in Fortran, and called from c / c++.
Representing a slice (MatLAB style) Let denote the numbers So Consider int A[] = Row 1 of has indices 0:1:2. Column 1 of A has indices 0:3:9.
Basic idea of a slice • Keep track of the start index (or pointer) • Keep track of the number of values (or end index) • Keep track of the difference between consecutive values (stepsize)
C++ Slice • std::slice • Start • Size (number of items) • Step (difference between consecutive items) • std::slice(0, 3, 2) => 0, 2, 4 Valarray<double> x; ….. x[slice(0,3,2)] => x[0], x[2], x[4] • I don’t use std::slice often (I write my own matrix class or use another library) • The idea is very important
Matrix Views & Submatrices • What if we want a submatrix that is not a single row or column? How can we represent ? • We need a 2D version of a slice • Keep track of… • Linear index of in this case • Offset between consecutive rows is , in this case 3 • Offset between consecutive columns is • Keep track of the sizes (in this case )
Matrix View • Imagine that library code & functions always operate on views of some other underlying data. • A vector view might need the following: • A pointer to the memory • Offset to the first item • Offset between consecutive items • The number of items in the vector. • A matrix view might need the following: • Pointer to the memory • Offset to the first item • Offset between consecutive rows • Offset between consecutive items • Number of rows • Number of columns In the STL these are called std::slice_array but do not seem to be used often. There are many custom implementations of this idea though. In the STL these are called std::gslice_array but do not seem to be used often. There are many custom implementations of this idea though.
Matrix Multiplication If then Dimensions must agree: The number of columns in A must match the number of rows in B
