Digital Signal Processing 2 Les 2: Inleiding 2

1. Digital Signal Processing 2Les 2: Inleiding 2 Prof. dr. ir. Toon van WaterschootFaculteit Industriële IngenieurswetenschappenESAT – Departement ElektrotechniekKU Leuven, Belgium

2. Digital Signal Processing 2: Vakinhoud • Les 1: Inleiding 1 (Discrete signalen en systemen) • Les 2: Inleiding 2 (Wiskundige concepten) • Les 3: Spectrale analyse • Les 4: Elementair filterontwerp • Les 5: Schattingsproblemen • Les 6: Lineaire predictie • Les 7: Optimale filtering • Les 8: Adaptieve filtering • Les 9: Detectieproblemen • Les 10: Classificatieproblemen • Les 11: Codering • Les 12: Herhalingsles

3. Les 2: Inleiding 2 • Complex number theory complex numbers, complex plane, complex sinusoids, circular motion, sinusoidal motion, … • Signal transforms z-transform, Fourier transform, discrete Fourier transform • Matrix algebra vectors, matrices, linear systems of equations

4. Les 2: Literatuur • Complex number theory J. O. Smith III, Mathematics of the DFT • Ch. 2, “Introduction to Complex Numbers” • Ch. 4, Section 4.2, “Complex Sinusoids” • Signal transforms S. J. Orfanidis, Introduction to Signal Processing • Ch. 5, “z-Transforms” • Ch. 9, “DFT/FFT Algorithms” M. H. Hayes, Statistical Digital Signal Processing and Modeling • [summary] Ch. 2, Sections 2.2.4, 2.2.5, 2.2.8 • Matrix algebra M. H. Hayes, Statistical Digital Signal Processing and Modeling • Ch. 2, Section 2.3, “Linear Algebra”

5. Les 2: Inleiding 2 • Complex number theory complex numbers, complex plane, complex sinusoids, circular motion, sinusoidal motion, … • Signal transforms z-transform, Fourier transform, discrete Fourier transform • Matrix algebra vectors, matrices, linear systems of equations

6. Complex number theory: overview • Complex numbers: • roots of a quadratic polynomial equation • fundamental theorem of algebra • complex numbers • complex plane • Complex sinusoids • complex numbers  complex sinusoids • circular motion • positive and negative frequencies • sinusoidal motion

7. Complex number theory: complex numbers complex numbers? “imaginary” roots of a polynomial equation

8. Complex number theory: complex numbers • roots of a quadratic polynomial equation: • consider a quadratic polynomial, describing a parabola: • the roots of the polynomial correspond to the points where the parabola crosses the horizontal x-axis

9. p(x) p(x) p(x) x x x Complex number theory: complex numbers • roots of a quadratic polynomial equation: • if the polynomial has 2 real roots, and the parabola has 2 distinct intercepts with the x-axis • if the polynomial has 1 real root (with multiplicity 2), and the parabola has 1 intercept (tangent point) with the x-axis • if the polynomial has no real roots, and the parabola has no intercepts with the x-axis

10. p(x) x Complex number theory: complex numbers • roots of a quadratic polynomial equation: • alternatively, if we could say that the polynomial has 2 “imaginary roots”, and the parabola has 2 “imaginary” intercepts with the x-axis • these imaginary roots are represented as complex numbers: with

11. Complex number theory: complex numbers fundamental theorem of algebra: every n-thorder polynomial has exactly n complex roots Toon van Waterschoot & Marc Moonen INTRODUCTION-1

12. Complex number theory: complex numbers • complex numbers: • complex number: • complex conjugate: • modulus: • argument: • the complex numbers form a field, and all algebraic rules for real numbers also apply for complex numbers Toon van Waterschoot & Marc Moonen INTRODUCTION-1

13. Im complex plane Re Complex number theory: complex numbers • complex plane: • the modulus and argument naturally lead to a radial representation in the complex plane Toon van Waterschoot & Marc Moonen INTRODUCTION-1

14. Complex number theory: complex sinusoids • complex variable  complex sinusoid: • from the radial representation we obtain • replacing • using Euler’s identity we get Toon van Waterschoot & Marc Moonen INTRODUCTION-1

15. Complex number theory: complex sinusoids • circular motion: • a complex sinusoid can be seen as a vector which describes a circular trajectory in the z-plane Im z-plane Re Toon van Waterschoot & Marc Moonen INTRODUCTION-1

16. Complex number theory: complex sinusoids • positive and negative frequencies: • for positive frequencies the circular motion is in counterclockwise direction • for negative frequencies the circular motion is in clockwise direction Im Im Re Re Toon van Waterschoot & Marc Moonen INTRODUCTION-1

17. Complex number theory: complex sinusoids • sinusoidal motion: • sinusoidal motion is the projection of circular motion onto any straight line in the z-plane, e.g., • is the projection of onto the Re-axis • is the projection of onto the Im-axis Im Re Toon van Waterschoot & Marc Moonen INTRODUCTION-1

18. Les 2: Inleiding 2 • Complex number theory complex numbers, complex plane, complex sinusoids, circular motion, sinusoidal motion, … • Signal transforms z-transform, Fourier transform, discrete Fourier transform • Matrix algebra vectors, matrices, linear systems of equations

19. Signal transforms: overview • z-transform: • definition & properties • complex variables • region of convergence • Fourier transform: • frequency response • Fourier transform • Discrete Fourier Transform (DFT): • definition • inverse DFT • matrix form • properties • Fast Fourier Transform (FFT) • Digital filtering using the DFT/FFT

20. discrete-time sequence in integer variable z-transform discrete-time series in complex variable Signal transforms: z-transform • definition:

21. z-transform Signal transforms: z-transform • definition: • z-transform of a discrete-time signal:

22. z-transform Signal transforms: z-transform • definition: • z-transform of a discrete-time system impulse response:

23. Signal transforms: z-transform • properties: • linearity property: • time-shift theorem: • convolution theorem:

24. Signal transforms: z-transform • region of convergence: • the z-transform of an infinitely long sequence is a series with an infinite number of terms • for some values of the series may not converge • the z-transform is only defined within the region of convergence (ROC):

25. Signal transforms: Fourier transform • Frequency response: • for an LTI system a sinusoidal input signal produces a sinusoidal output signal at the same frequency • the output can be calculated from the convolution:

26. Signal transforms: Fourier transform • Frequency response: • the sinusoidal I/O relation is • the system’s frequency response is a complex function of the radial frequency : • denotes the magnitude response • denotes the phase response

27. Signal transforms: Fourier transform • Frequency response: • the frequency response is equal to the z-transform of the system’s impulse response, evaluated at • for , is a complex function describing the unit circle in the z-plane Im z-plane Re

28. Signal transforms: Fourier transform • Frequency response & Fourier transform • the frequency response of an LTI system is equal to the Fourier transform of the continuous-time impulse sequence constructed with h[k] : • similarly, the frequency spectrum of a discrete-time signal (=its z-transform evaluated at the unit circle) is equal to the Fourier transform of the continuous-time impulse sequence constructed with u[k], y[k] : • Input/output relation:

29. Signal transforms: DFT • DFT definition: • the Fourier transform of a signal or system is a continuous function of the radial frequency : • the Fourier transform can be discretized by sampling it at discrete frequencies , uniformly spaced between and : = DFT

30. Signal transforms: DFT • Inverse discrete Fourier transform (IDFT): • an -point DFT can be calculated from an -point time sequence: • vice versa, an -point time sequence can be calculated from an -point DFT: = IDFT

31. Signal transforms: DFT • matrix form • using the shorthand notations the DFT and IDFT definitions can be rewritten as: DFT: IDFT:

32. Signal transforms: DFT • matrix form • the DFT coefficients can then be calculated as • an -point DFT requires complex multiplications

33. Signal transforms: DFT • matrix form • the IDFT coefficients can then be calculated as • an -point IDFT requires complex multiplications

34. Signal transforms: DFT • properties: • linearity & time-shift theorem (cf. z-transform) • frequency-shift theorem (modulation theorem): • circular convolution theorem: if and are periodic with period , then (see also ‘Digital filtering using the DFT/FFT’)

35. Signal transforms: DFT • Fast Fourier Transform (FFT) • divide-and-conquer approach: • split up N-point DFT in two N/2-point DFT’s • split up two N/2-point DFT’s in four N/4-point DFT’s • … • split up N/2 2-point DFT’s in N 1-point DFT’s • calculate N 1-point DFT’s • rebuild N/2 2-point DFT’s from N 1-point DFT’s • … • rebuild two N/2-point DFT’s from four N/4-point DFT’s • rebuild N-point DFT from two N/2-point DFT’s • DFT complexity of multiplications is reduced to FFT complexity of multiplications James W. Cooley Carl Friedrich Gauss (1777-1855 John W.Tukey

36. Signal transforms: Digital filtering • Linear and circular convolution: • circular convolution theorem: due to the sampling of the frequency axis, the IDFT of the product of two -point DFT’s corresponds to the circular convolution of two length- periodic signals • LTI system: the output sequence is the linear convolution of the impulse response with the input signal

37. Signal transforms: Digital filtering • Linear and circular convolution: • the linear convolution of a length- impulse response with a length- input signal is equivalent to their -point circular convolution if both sequences are zero-padded to length : zero padding

38. Les 2: Inleiding 2 • Complex number theory complex numbers, complex plane, complex sinusoids, circular motion, sinusoidal motion, … • Signal transforms z-transform, Fourier transform, discrete Fourier transform • Matrix algebra vectors, matrices, linear systems of equations

39. Matrix algebra: overview • Vectors • definition & geometrical interpretation • elementary operations • inner product & angle • norm • outer product • linear (in)dependence • Matrices • Linear systems of equations

40. 1 2 Vectors: definition & geom. interpretation • Definition • array of real- or complex-valued numbers or variables (in DSP: array of signal samples, DFT coefficients, …) • vector transpose: column vector ⟷ row vector • Geometrical interpretation • array of coordinates of a point in N-dimensional space 2-D example:

41. Vectors: elementary operations • Addition • Scalar product • changes vector length but not direction

42. Vectors: inner product & angle • Inner product (= scalar !) • Angle • let ||x|| denote length of vector x • angle between vectors is then related to inner product • orthogonal vectors have zero inner product

43. Vectors: norm • Norm = Euclidian norm = norm • Geometric interpretation • vector length: • distance between vectors: • Widely used in DSP ! • norm → signal RMS value • squared norm → average signal power • Other norms norm: , norm:

44. Vectors: outer product • Outer product (= matrix !)

45. Vectors: linear (in)dependence • Linear (in)dependence • property of a set of vectors • set of P vectors is linearly independent: • size P of set of linearly independent vectors ≤ length N • trivial example: • Related concepts: space, basis, dimension

46. Matrix algebra: overview • Vectors • Matrices • definition & elementary operations • matrix product • matrix-vector product • matrix decomposition & rank • structured matrices • matrix form convolution • Linear systems of equations

47. Matrices: definition & elementary ops. • Definition: M x N matrix • Elementary operations: • transpose: (= N x M matrix) • addition: • scalar product:

48. Matrices: matrix product • Matrix product • dimensions: • definition: • Example

49. Matrices: matrix-vector product • Matrix-vector product: • Two interpretations: • matrix-vector product = stacking inner products • matrix-vector product = linear combination of columns

50. Matrices: matrix decomposition & rank • Matrix decomposition • N x N matrix can be decomposed as sum of R ≤ N outer products of linearly independent vectors • “eigenvalue decomposition”: • eigenvalues • eigenvectors • general M x N matrices: “singular value decomposition” • Rank = R • rank-deficient / singular matrix: • full-rank matrix: