DIY Data Analysis: Unveiling Insights from Time Series Data

1. Data Analysis Do it yourself!

2. What to do with your data? • Report it to professionals (e.g., AAVSO) • Excellent! A real service to science; don’t neglect this • Publish observations (e.g., JAAVSO) • Analyze it – yourself!

3. But … • I’m not a mathematician • Let the computer do the math • I’m not a programmer • Get programs from the net (often free) • I don’t know how to use or interpret them • Neither do the pros! • Practice, practice, practice …

4. Time Series Analysis • A time series is a set of data pairs • t is the time, x is the data value • Usually, times are assumed error-free • Data = Signal + Error • x can be anthing, e.g. brightness of variables star, time of eclipse, eggs/day from a laying hen

5. Actual Mean = = expected value Standard deviation = expected rms difference from mean Estimated Average = estimated Sample standard deviation = estimated Basic properties of data x

6. Average and sample standard deviation • Average • Sample standard deviation

7. Method #1: world’s best • Eye + Brain: Look at the data! • Plot x as a function of t: Explore! • Scientific name: Visual Inspection • World’s best – but not infallible • Programs: • TS http://www.aavso.org • MAGPLOT http://www.aavso.org

8. Method #2: Fourier Analysis • Period analysis and curve-fitting • Powerful, well-understood, popular • Programs • TS http://www.aavso.org • PerAnSo http://www.peranso.com

9. Method #3: Wavelet Analysis • Time-frequency analysis • Old versions bad, new version good • Programs: • WWZ http://www.aavso.org • WinWWZ http://www.aavso.org

10. Visual Inspection

11. Let’s take a look

12. Fourier Analysis

13. Fourier analysis for period search • Match the data to sine/cosine waves • = frequency • Period = • Amplitude = A = size of fluctuation • Obvious choice is period; mathematically sound choice is frequency

14. Null Hypothesis (important!) • Null hypothesis: no time variation at all • So = constant • So, • Quite important! Often neglected. Even the pros often forget this.

15. Is it real? • Fit produces a test statistic under the null hypothesis • Is usually “ /degree of freedom” (d) • Linear: is significant (not just by accident) at 95% confidence • 95% confidence means 5% false-alarm probability

16. Meaning of significance • Significance does not mean the signal is linear, sinusoidal, periodic, etc. • It only means the null hypothesis is incorrect, i.e., the signal is not constant • Important!!!

17. Pre-whitening • If you find a significant fit, then subtract the estimated signal, leaving residuals • Analyze the residuals for more structure • This process is called pre-whitening

18. How to choose frequency? • Test all reasonable values, get a “strength of fit” for each. Common is “chi-square per degree of freedom” (but there are many) • Plot frequency .vs. fit – the Fourier transform (aka periodogram, aka power spectrum)

19. Fourier decomposition Any periodic function of period P (frequency ) can be expressed as a Fourier series:

20. Fundamental + harmonics For a pure sinusoid, expect response at frequency For a general periodic signal at a given frequency, expect a fundamental component at , as well as harmonics at frequencies etc.

21. Lots of Fourier methods • FFT: fast Fourier transform • Not just fast: it’s wicked fast • Requires even time spacing • Requires N=integer power of 2 • Beware! • DFT: discrete Fourier transform • Applies to any time sampling, but incorrect results for highly uneven (as in astronomy!) • Beware!

22. Problems from uneventime sampling • Aliasing: false peaks, often from a periodic data density • Aliases at • Common in astronomy: data density have a period P = 1 yr = 365.2422 d, so • Solution: pre-whitening

23. Aliasing

24. Aliasing: UZ Hya

25. Problems from uneventime sampling • Mis-calculation of frequency (slightly) and amplitude (greatly); sabotages prewhitening

26. Solution: better Fourier methods (for astronomy) • Lomb-Scargle modified periodogram • Improvement over FFT, DFT • CLEAN spectrum • Bigger improvement • DCDFT: date-compensated discrete Fourier transform (this is the one you want) • CLEANEST spectrum: DCDFT-like for multiple frequencies

27. DCDFT • Much better estimates of period, amplitude

28. Let’s take a look • Peranso (uses DCDFT and CLEANEST) • Available from CBA Belgium • http://www.peranso.com

31. Fourier transform (CLEANEST) of TU Cas

32. Wavelet Analysis

33. Wavelets • Fit sine/cosine-like functions of brief duration • Shift them through time • Gives a time-frequency analysis

34. Problems • Same old same old: uneven time spacing, especially variable data density, invalidate the results • But: even worse than Fourier • Essentially useless for most astronomical data

35. Wavelet methods • DWT: discrete wavelet transform • Just not right for unevenly sampled data (astronomy!) • Solution: WWZ = weighted wavelet Z-transform

36. Let’s take a look

37. Data Analysis • Do it yourself • Use your eyes and brain • Healthy skepticism • tamino_9@hotmail.com • Enjoy!