Implications of Model Specification and Temporal Revisit Designs on Trend Detection

1. Implications of Model Specification and Temporal Revisit Designs on Trend Detection Leigh Ann Starcevich (OSU) Kathryn M. Irvine (USGS) Andrea M. Heard (UCR, NPS)

2. Outline • Question of interest • Case study • Trend models • Simulation results

3. Question of interest • Trend estimation and testing • Components of variation impact power to detect trend • Three approaches to trend estimation and testing suggested by: • Urquhart, Birkes, and Overton (1993) • Piepho and Ogutu (2002) • Kincaid, Larsen, and Urquhart (2004) • Which approach has highest power for trend detection for a given Type I error level?

4. Motivation • SIEN lake chemistry monitoring • Trend and status • Annual effort is limited in vast landscape • ~800 lakes in network

5. Sierra Nevada Network Lakes Sequoia-Kings Canyon NP: ~ 860,000 acres Yosemite NP: ~ 760,000 acres

6. Trend model • Linear mixed model used to estimate trend and components of variance • Trend models contain both fixed and random effects • Fixed effects describe the mean • Random effects describe the variance structure

7. Variance components • Site-to-site variation (σa2) • Does not affect the trend estimate or SE when same sites visited annually (Piepho and Ogutu, 2002) • Year-to-year variation (σb2) • Visiting additional sites will not increase power to detect trend when year-to-year variation is high • Site-by-year variation (σc2) • Requires within-year visits to a site (not estimable in SIEN lakes survey) • Random slope variation (σt2) • Might indicate subpopulations with different trends • Residual error variation (σe2)

8. Power to detect trend • Affected by: • Type I error level (α) • Trend magnitude (β1) • Variance composition • Associated with a particular hypothesis test • One-sided vs. two-sided alternative hypothesis? • Reflects monitoring goals • We examine:

9. Revisit designs • Panel designs allow more sites to be visited over time • When all sites are not visited annually, data are purposefully unbalanced • Serially-alternating augmented designs considered • Connected across time for powerful trend tests • Incorporates more sites for status estimates

10. [1-0] • Notation of McDonald (2003) • Urquhart and Kincaid (1999) showed that [1-0] is best for trend estimation

11. [(1-0),(1-3)]

12. [(1-0),(1-8)]

13. Estimation of fixed effects • Generalized least squares (GLS) used to estimate fixed effects where

14. Estimation of RE variance components • ANOVA Type III when data are balanced • REML for unbalanced data • Piepho & Ogutu (2002); Spilke, et al. (2005) • ANOVA Type III and REML provide the same estimates when data are balanced • Satterthwaite degrees of freedom with Geisbrecht-Burns approximation

15. Approach 1: Urquhart, et al. • Variance components obtained from model without fixed trend slope • Construct Φ(θ)=Var(Y) from variance components • Estimate β and SE(β) with GLS • This approach assumed the variance components were known • Did not address estimation • We use REML

16. Approach 2: Piepho and Ogutu (2002) • Extension of VanLeeuwen, et al (1996) • Random slope effect incorporated • Trend and variance component estimation • Trend testing conducted using a synthetic F test • Wald F-test not invariant to location shifts • P&O relaxed assumption of independence between random site effect and random slope for invariant Wald F-test

17. Approach 3: Kincaid, et al. (2004) • Two models used • Trend model omits RE for year • Variance components model omits fixed linear trend • RE’s for site, year, interaction • This paper focused on status estimation • Trend approach mentioned incidentally

18. Desirable properties • Trend test • Powerful • Nominal test size • Trend model • Ability to accurately estimate trend • Nominal CI coverage for trend • Variance component estimation

19. SIEN Lake Chemistry • Pilot data: Seven lakes study & Western lakes study • Three outcomes chosen for study • Ca: high random site variability • Cl: high random slope variability • NO3: high year-to-year and • residual error variation  • Indicator 3: high year-to-year variation • Indicator 4: high residual error variation

20. Monte Carlo power simulation • Simulate population of lakes from estimated fixed effects and variance components obtained from the case study data • 1000 populations generated • 3 independent random samples selected from each population • Generate known trend • Annual decline of 1% or 4% • 10 years • Impose revisit design  1/3 of effort to annual panel • Simulation power is proportion of times that null hypothesis is correctly rejected at the α = 0.10 level

21. Test size

22. Simulation results • Power approximations are too high when test size exceeds nominal rate • Estimates of β1 are generally unbiased • Bias of β1 most sensitive torevisit design, not trend approach • Observed that bias of SE(β1) was less severe as revisit cycle length increased

23. Rel. Bias of SE(β1): σa2 large, p =-1%

24. Rel. Bias of SE(β1): σt2 large, p= -1%

25. Rel. Bias of SE(β1): σb2 large, p= -1%

26. Rel. Bias of SE(β1): σe2 large, p= -1%

27. Discussion • Approach 2 has most stable test size • When σe2 high, Approach 2 overestimates σa2, σt2, σb2, and SE(β1) • Poor indicator for monitoring • Simulation power is almost always lower than power approximations assuming large-sample theory

28. Conclusions • Trend test size should be assessed when examining power to detect trend • Including a random slope effect that is correlated with the random site effect in the mixed model approach provides nearly-nominal trend tests • Examining variance components is useful for choosing monitoring indicators and revisit designs

29. Ongoing work • Determine if bias of variance components estimates may be reduced • Incorporate autocorrelation estimation • Examine relationship between revisit cycle length and SE(β1)

30. Acknowledgements • NPS Vital Signs Monitoring Agreement • Linda Mutch • James Sickman, John Melack, and Dave Clow • Kirk Steinhorst • N. Scott Urquhart • Tom Kincaid

31. Questions?