1. Environmental Data Analysis with MatLab Lecture 20: • Coherence; Tapering and Spectral Analysis

2. SYLLABUS Lecture 01 Using MatLabLecture 02 Looking At DataLecture 03Probability and Measurement ErrorLecture 04 Multivariate DistributionsLecture 05Linear ModelsLecture 06 The Principle of Least SquaresLecture 07 Prior InformationLecture 08 Solving Generalized Least Squares ProblemsLecture 09 Fourier SeriesLecture 10 Complex Fourier SeriesLecture 11 Lessons Learned from the Fourier Transform Lecture 12 Power Spectral DensityLecture 13 Filter Theory Lecture 14 Applications of Filters Lecture 15 Factor Analysis Lecture 16 Orthogonal functions Lecture 17 Covariance and AutocorrelationLecture 18 Cross-correlation Lecture 19 Smoothing, Correlation and SpectraLecture 20 Coherence; Tapering and Spectral Analysis Lecture 21 InterpolationLecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-TestsLecture 24 Confidence Limits of Spectra, Bootstraps

3. purpose of the lecture Part 1 Finish up the discussion of correlations between time series Part 2 Examine how the finite observation time affects estimates of the power spectral density of time series

4. Part 1 “Coherence” frequency-dependent correlations between time series

5. Scenario Ain a hypothetical regionwindiness and temperature correlate at periods of a year, because of large scale climate patternsbut they do not correlate at periods of a few days

6. wind speed 1 2 3 time, years temperature 1 2 3 time, years

7. summer hot and windy winters cool and calm wind speed 1 2 3 time, years temperature 1 2 3 time, years

8. heat wave not especially windy cold snap not especially calm wind speed 1 2 3 time, years temperature 1 2 3 time, years

9. in this casetimes series correlated at long periodsbut not at short periods

10. Scenario Bin a hypothetical regionplankton growth rate and precipitation correlate at periods of a few weeksbut they do not correlate seasonally

11. growth rate 1 2 3 time, years precipitation 1 2 3 time, years

12. growth rate has no seasonal signal plant growth rate summer drier than winter 1 2 3 time, years precipitation 1 2 3 time, years

13. growth rate high at times of peak precipitation plant growth rate 1 2 3 time, years precipitation 1 2 3 time, years

14. in this casetimes series correlated at short periodsbut not at long periods

15. Coherencea way to quantifyfrequency-dependent correlation

16. strategyband pass filter the two time series, u(t) and v(t)around frequency, ω0compute their zero-lag cross correlation(large when time series are similar in shape)repeat for many ω0’s to create a function c(ω0)

17. band pass filter f(t) has this p.s.d. 2Δω |f(ω)|2 2Δω ω -ω0 ω0 0

18. evaluate atzero lag t=0 and at many ω0’s

19. Short CutFact 1A function evaluates at time t=0 is equal to the integral of its Fourier Transform Fact 2 the Fourier Transform of a convolution is the product of the transforms

21. integral over frequency

22. integral over frequency assume ideal band pass filter that is either 0 or 1 positive frequencies negative frequencies

23. integral over frequency assume ideal band pass filter that is either 0 or 1 positive frequencies negative frequencies c is real so real part is symmetric, adds imag part is antisymmetric, cancels

24. integral over frequency assume ideal band pass filter that is either 0 or 1 positive frequencies negative frequencies c is real so real part is symmetric, adds imag part is antisymmetric, cancels interpret intergral as an average over frequency band

25. integral over frequency can be viewed as an average over frequency(indicated with the overbar)

26. Two final steps 1. Omit taking of real part in formula (simplifying approximation) 2. Normalize by the amplitude of the two time series and square, so that result varies between 0 and 1

27. the final result is called Coherence

28. A) B) new dataset: Water Quality Reynolds Channel, Coastal Long Island, New York C) D) E) F)

29. precipitation A) air temperature B) new dataset: Water Quality Reynolds Channel, Coastal Long Island, New York water temperature C) D) salinity E) turbidity F) chlorophyl

30. A) periods near 1 year B) periods near 5 days Fig, 9.18. Band-pass filtered water quality measurements from Reynolds Channel (New York) for several years starting January 1, 2006. A) Periods near one year; and B) periods near 5 days. MatLab script eda09_16.

31. A) periods near 1 year B) periods near 5 days Fig, 9.18. Band-pass filtered water quality measurements from Reynolds Channel (New York) for several years starting January 1, 2006. A) Periods near one year; and B) periods near 5 days. MatLab script eda09_16.

32. B) C) A) one year one year one week

33. high coherence at periods of 1 year B) C) A) one year one year one week moderate coherence at periods of about a month very low coherence at periods of months to a few days

34. Part 2 windowing time series before computing power-spectral density

35. scenario: you are studying an indefinitely long phenomenon … but you only observe a short portion of it …

36. how does the power spectral density of the short piecediffer from the p.s.d. of the indefinitely long phenomenon(assuming stationary time series)

37. We might suspect that the difference will be increasingly significant as the window of observation becomes so short that it includes just a few oscillations of the period of interest.

38. starting pointshort pieceisthe indefinitely long time seriesmultiplied by awindow function, W(t)

40. by the convolution theoremFourier Transform of short pieceisFourier Transform of indefinitely long time seriesconvolved withFourier Transform of window function

41. soFourier Transform of short pieceexactlyFourier Transform of indefinitely long time serieswhenFourier Transform of window functionis a spike

42. boxcar window function its Fourier Transform

43. boxcar window function its Fourier Transform sinc() function sort of spiky but has side lobes

45. Effect 1: broadening of spectral peaks narrow spectral peak wide central spike wide spectral peak

46. Effect 2: spurious side lobes only one spectral peak side lobes spurious spectral peaks

47. Q: Can the situation be improved?A: Yes, by choosing a smoother window functionmore like a Normal Function(which has no side lobes)but still zero outside of interval of observation

48. boxcar window function Hamming window function

50. no side lobes but central peak wider than with boxcar