Introduction to Intensive Longitudinal Methods

1. Introduction to Intensive Longitudinal Methods Larry R. Price, Ph.D. Director Interdisciplinary Initiative for Research Design & Analysis Professor – Psychometrics & Statistics

2. What are intensive longitudinal methods? • Methods used in natural settings using many repeated data captures (measurements) over time within a person. • For example, daily diaries, interaction records, ecological momentary assessments, real-time data capture. • The term intensive longitudinal methods is an umbrella term that includes the above types of data structures.

3. Areas of notable growth in the use of the methods • Interpersonal processes in dyads and families. • For example, an examination of the link between daily pleasant and unpleasant behaviors (over 14 days) and global ratings of marital satisfaction. • The study of dyads and family processes requires unique data-analytic challenges.

4. Why use intensive longitudinal methods? • Can be used to quantitatively study thoughts, feelings, physiology, and behavior in their natural, spontaneous contexts or settings. • The data that result show an unfolding temporal process (a) descriptively and (b) as a causal process. • For example, it is possible to show how an outcome Y changes over time and how this change is contingent on changes in a causal variable X.

5. Difference between traditional repeated measures analysis • Usually limited to a few repeated measurements taken over long time intervals. • We often use dynamic factor analysis models to study complex change over time in this type of scenario. • Intensive longitudinal methods offers a framework for analyzing intrapersonal change (i.e. within-person process) with extensive or rich outcome data.

6. Advantages of use • Measurement may be continuous or discrete based on a response to an experimental or observational condition captured over many time points in a short total duration. • Important for studying intraindividual and interindividual change within a unified model.

7. Measuring a process over time For example, Time 4 is regressed on Time 3, Time 3 is regressed on Time 2, Time 2 is regressed on Time 1. This yields a autoregressive structure or time series vector. In intensive longitudinal methods, these time series vectors are nested within individual persons yielding a hierarchical structure. Thus the need for a random coefficients mixed model approach (e.g., using SPSS mixed, SAS PROC MIXED, or the HLM program).

8. Advantages of use • Depending on when a variable X measured at one point in time has a maximally causal effect on Y at a later time point, • The precise temporal design of a longitudinal study can greatly influence the observed effects. • We want to avoid using a between-subjects analysis to analyze these type of data.

9. Example – fMRI data structure Small Sample Properties of Bayesian Multivariate Autoregressive Time Series Models (Structural Equation Modeling Journal, 2012)

10. Relationship of measurement model to autoregressive model

11. Multivariate vector autoregressive (MAR) time series model • A MAR model predicts the next value in a d – dimensional time series as a linear function of the p previous vector values of the time series. • The MAR model is based on the Wishart distribution, where V = p x p, symmetric, positive definite matrix of random variables.

12. Multivariate Vector Autoregressive Model Blue = contemporaneous Brown = cross-covariances Red = autoregression of time (t) on t-1

13. Goals of the present study • Bayesian MAR model formulation capturing contemporaneous and temporal components. • Evaluation of the effect of variations of the autoregressive and cross-lagged components of a multivariate time series across sample size conditions where N is smaller than T (i.e., N <= 15 and T = 25 – 125).

14. Goals of the present study • To examine the impact of sample size and time vector length within the sampling theory framework for statistical power and parameter estimation bias. • To illustrate an analytic framework that combines Bayesian statistical inference with sampling theory (frequentist) inference.

15. Goals of the present study • Compare and relate Bayesian credible interval estimation based on the Highest Posterior Density (HPD) to frequentist power estimates and Type I error.

16. Research Challenges • Sample size and vector length determination for statistically reliable and valid results – Bayesian and Frequentist considerations. • Modeling the structure of the multivariate time series contemporaneously and temporally.

17. Research Challenges • Examining the impact of autocorrelation, error variance and cross-correlation/covariance on multivariate models in light of variations in sample size and time series length.

18. Introduction to Bayesian probability & inference • Bayesian approach prescribes how learning from data and decision making should be carried out; the classical school does not. • Prior information via distributional assumptions of variables and their measurement is considered through careful conceptualization of the research design, model, and analysis.

19. Introduction to Bayesian probability & inference Stages of model development: 1. Data Model: [data|process, parameters] - specifies the distribution of the data given the process. 2. Process Model: [process|parameters] - describes the process, conditional on other parameters. 3. Parameter Model: [parameters] accounts for uncertainty in the parameters.

20. Introduction to Bayesian probability & inference • Through the likelihood function, the actual observations modify prior probabilities for the parameters. • So, given the prior distribution of the parameters and the likelihood function of the parameters given the observations , the posterior distribution of the parameters given the observations is determined. • The Bayesian posterior distribution is the full inferential solutionto the research problem.

21. Introduction to Bayesian probability & inference The dashed line represents the likelihood, with 𝜃 being at its maximum at approximately .22, given the observed frequency distribution of the data. Applying Bayes theorem involves multiplying the prior density (solid curve) times the likelihood (dashed curve). If either of these two values are near zero, the resulting posterior density will also be negligible (i.e., near zero, for example, for 𝜃 < .2 or 𝜃 > .6). Finally, the posterior density (i.e., the dotted-dashed line) is more informative than either the prior or the likelihood alone.

22. Bayesian vs. Frequentist probability • Frequentist (a.k.a. long-run frequency) • Estimates the probability of the data (D) under a point null hypothesis (H0) or • A large sample theory with adjustments for small and non-normal samples (e.g., t-distribution and χ2 tests/rank tests) • Bayesian (a.k.a. conditional or subjective) • Evaluates the probability of the hypothesis (not always only the point) given the observed data and that a parameter falls within a specific confidence interval. • Used since 1950’s in computer science, artificial intelligence, economics, medicine.

23. Bioinformatics Application of Bayesian modeling and inference Data obtained from the real world may • Be sparse • Exhibit multidimensionality • Include unobservable variables IRT & CAT

24. The 2-Level hierarchical voxel-based fMRI model 1 voxel = 3 x 3 x 3mm

25. Hierarchical Bayesian SEM in functional neuroimaging (fMRI)

26. Bayesian probability & inference • Advantages • Data that are less precise (i.e., less reliable) will have less influence on the subsequent plausibility of a hypothesis. • The impact of initial differences in the perceived plausibility of a hypothesis tend to become less important as results accumulate (e.g., refinement of posterior estimates via MCMC algorithms and Gibbs sampling).

27. Bayesian probability & inference • Advantages • Sample size is not an issue due to MCMC algorithms. • Level of measurement is not problematic • Interval estimates use the cumulative normal density (CDF) function. • Nominal/dichotomous and ordinal measures use the probit, logistic or log-log link function to map to a CDF. • Uses prior probability of each category given little/no information about an item. • Categorization produces a posterior probability distribution over the possible categories given a description of an item. • Bayes theorem plays a critical role in probabilistic learning and classification.

28. Example: parameters & observations • Recall that the goal of parametric statistical inference to make statements about unknown parametersthat are not directly observable, from observable random variables the behavior of which is influenced by these unknown parameters. • The model of the data generating process specifies the relationship between the parameters and the observations. • If x represents the vector of n observations, and 𝜣 represents a vector of k parameters, 𝜃1, 𝜃2,𝜃3… 𝜃kon which the distribution of observation depends…

29. Example: parameters & observations • Then, inserting 𝜣 in place of y yields • So, we are interested in making statements about parameters (𝜣), given a particular set of observations x. • p (x) serves as a constant that allows for p(𝜣|x) to sum or integrate to 1. • Bayes Theorem is also given as = “proportional to”

30. Example: parameters & observations • In the previous slide, p (𝜣) serves as “a distribution of belief” in Bayesian analysis and is the joint distribution of the parameters prior to the observations being applied. • p (x |𝜣) is the joint distribution of the observations conditional on values of the parameters. • p (𝜣|x) is the joint distribution of the parameters posterior to the observations becoming available. • So, once the data are obtained, p (x |𝜣) is viewed as function of 𝜣 and results in the Likelihood functionfor𝜣 given x , i.e., L (𝜣|x).

31. Example: parameters & observations • Finally, it is through the likelihood function that the actual observations modify prior probabilities for the parameters. • So, given the prior distribution of the parameters and the likelihood function of the parameters given the observations , the posterior distribution of the parameters given the observations is determined. • The posterior distribution is the full inferential solutionto the research problem.

32. Univariate unrestricted VAR time series model • A time series process for each variable contained in the vector y is

33. Univariate unrestricted VAR time series model • Where, • y(t) = n X 1 stationary vector of variables observed at time t; • A(L) = n X n matrix of polynomials in the lag or backward shift operator L; • X(t) = n X nk block diagonal matrix of observations on k observed variables; • β = nk X 1 vector of coefficients on the observed variables; • u(t) = n X 1 vector of stochastic disturbances; • Σ = n X n contemporaneous covariance matrix.

34. Univariate unrestricted VAR time series model • Also, • The coefficient on L0 is zero for all elements of A(L) (i.e., only the lagged elements of y appear on the right side of the equation). • y(t) matrix is equal to • where y(t) is the k X 1 vector of observations on the k variables related to each equation and is the matrix Kroenecker product.

35. Multivariate vector autoregressive time series model • A MAR model predicts the next value in a d – dimensional time series as a linear function of the p previous vector values of the time series. • The MAR model is based on the Wishart distribution, where V = p x p, symmetric, positive definite matrix of random variables.

36. Multivariate vector autoregressive time series model • The MAR model divides each time series into two additive components (a) predictable portion of the time series and (b) prediction error (i.e., white noise error sequence). • The error covariance matrix is Gaussian with zero mean and precision (inverse covariance) matrix Λ.

37. Multivariate vector autoregressive time series model • The model with N variables can be expressed in matrix format as

38. Multivariate Vector Autoregressive Time Series Model • The multivariate prediction error filter is expressed as where,

39. Multivariate vector autoregressive time series model • The model can also be written as a standard multivariate linear regression model as • where are the p previous multivariate time series samples and W is a (p x d)-by-d matrix of MAR coefficients (weights). • There are therefore k = p x d x d MAR coefficients.

40. Multivariate vector autoregressive time series model • If the nth rows of Y, X, and E are yn, xn, and en respectively, then for the n = 1,…N samples, the equation is • Where Y is an N-by-d matrix, X is an N-by-(p x d) matrix and E is an N-by-d matrix.

41. Multivariate vector autoregressive time series model • For the multivariate linear regression model, the data set D = {X,Y}, the likelihood of the data is given as where, is the determinant and ( ) the trace, and the error matrix.

42. Multivariate vector autoregressive time series model • To facilitate matrix operations, the notation is given as • denotes the columns being stacked on top of each other. • To recover the matrix , columns are “unstacked” from the vector . • This matrix transformation is a standard method for implicitly defining a probability density over a matrix.

43. Multivariate vector autoregressive time series model The maximum Likelihood (ML) solution for the MAR coefficients is And the ML error covariance , SML is estimated as where, .

44. Multivariate vector autoregressive time series model • Again, to facilitate matrix operations, the notation is given as • denotes the columns of being concatenated. • This matrix transformation is a standard method for implicitly defining a probability density over a matrix.

45. Multivariate vector autoregressive time series model Using the vec notation , we define as Finally, the ML parameter covariance matrix for is given as Where , denotes the Kroenecker product (Anderson, 2003; Box & Tiao, 1983).

46. The Population Model : Bayesian SEM • Unknown quantities (e.g., parameters) are viewed as random. • These quantities are assigned a probability distribution (e.g., Normal, Poisson, Multinomial, Beta, Gamma, etc.) that details a generating process for a particular set of data. • In this study, our unknown population parameters were modeled as being random and then assigned to a joint probability distribution.

47. The Population Model : Bayesian SEM • The sampling based approach to Bayesian estimation provides a solution for the random parameter vector by estimating the posterior density of a parameter. • This posterior distribution is defined as the product of the likelihood function and the prior density updated via Gibbs sampling and evaluated by MCMC methods.

48. The Population Model : Bayesian SEM • The present study includes Bayesian and frequentist sampling theory - a “dual approach” (Mood & Graybill, 1963; Box & Tiao, 1973) approach . • In the dual approach, Bayesian posterior information is used to suggest functions of the data for use as estimators. • Inferences are then made by considering the sampling properties of the Bayes estimators.

49. The Population Model : Bayesian SEM • The goal is to examine the congruency between the inferences obtained from a Monte Carlo study based on the population Bayes estimators and the full Bayesian solution relative to sample size and time vector length.

50. Data generation & methods • Five multivariate autoregressive (AR1) series vectors of varying lengths (5) and sample sizes (5) were generated using SAS v.9.2. • Autoregressive coefficients included coefficients of .80, .70, .65, .50, and .40. • Error variance components of .20, .20, .10, .15, and .15. • Cross-lagged coefficients of .10, .10, .15, .10, and .10.