Time Series Econometrics: Some Basic Concepts

Time Series Econometrics:Some Basic Concepts Course Incharge: Prof. Dr. Himayatullah Khan Reference : Gujarati, Chapters 21

Time Series Data • One of the important and frequent types of data used in empirical analysis. • But it poses several challengesto econometricians/practitioners. E.g. • Empirical work based on time series data assumes that the underlying time series is stationary. 2. Autocorrelation: because the underlying time series data is non-stationary. 3. Spurious/nonsense regression: a very high R2 and significant regression coefficients (though there is no meaningful relationship between the two variables)

Key Concepts • Stochastic Processes • Stationarity Processes • Purely Random Processes • Non-stationary Processes 2. Random Walk Models • Random Walk with Drift • Random Walk without Drift 3. Unit Root Stochostic Processes 4. Deterministic and Stochastic Trends 5. The Phenomenon of Spurious Regression 6. Tests of Stationarity/non-stationarity • Graphical Method • Unit Root Tests

1) Stochastic Processes • Stochastic (Random) Process: collection of random variables ordered in time. • NOTATIONS: Let Y a random variable, Y(t) if continuous (e.g. electrocardiogram), and Yt if discrete (e.g. GDP, PDI, etc.). • Now, If we let Y represent GDP, then we can have Y1, Y2, Y3, ... , Y20 where the subscript 1 denotes the 1st observation (i.e. GDP for the 1st quarter of 1st year) and the subscript 20 denotes the last observation (i.e. GDP for the 4th quarter of 5th year).

(1) Stochastic Processes • Stationary Stochastic Processes: A stochastic process is said to be stationary/ weakly /covariance/2nd-orderstationary if: • Its mean and variance are constant over time, and • The value of the covariance between the two time periods depends only on the distance/lag between the two time periods and not the actual time at which the covariance is computed. • E.g. let’s Yt be a stochastic process, then; • Mean: E(Yt ) = µ ………………………………………….. (1) • Variance: var (Yt ) = E(Yt − µ)2 = σ2……………………………….. (2) • Covariance: γk = E[(Yt − µ)(Yt +k − µ)] ……………..………… (3) • Where γk, the covariance (or auto-covariance) at lag k, • If k = 0, we obtain γ0, which is simply the variance of Y (= σ2); if k = 1, γ1 is the covariance between two adjacent values of Y

Why are Stationary Time Series so Important? • Because if a time series is non-stationary, we can study its behavior only for the time period under consideration, and as a consequence, it is not possible to generalize it to other time periods. • Therefore, for the purpose of forecasting, such (non-stationary) time series may be of little practical value. • Non-stationary Stochastic Processes: Although our interest is in stationary time series, one often encounters non-stationary time series • A non-stationary time series will have a time-varying mean or a time-varyingvariance or both.

White Noise Processes • We call a stochastic process(time series) purely random/white noise process if it has zero mean, constant variance σ2, and is serially uncorrelatedi.e. [ut ∼ IIDN(0, σ2)]. • Note: Here onward, in all equations the assumption of “white noise” will be applicable on ut .

2) Random Walk Model (RWM) • The classic example of non-stationary time series is the Random Walk Model (RWM). • It is often said that asset prices, such as stock prices or exchange rates, follow a random walk (i.e. non-stationary). • Typesof Random Walks: • Random Walk Without Drift: i.e. no constant/intercept term and b) Random Walk With Drift i.e. a constant term is present

a) Random Walk without Drift • The time series Ytis said to be a random walk without drift, if Yt = Yt−1+ut ……. (4) • Here, the value of Y at time (t) is equal to its value at time (t − 1) plus a random shock; thus it is an AR(1) model. • Believers in the Efficient Capital Market Hypothesis argue that stock prices are essentially random and therefore there is no scope for profitable speculation in the stock market: • If one could predict tomorrow’s price on the basis of today’s price, we would all be millionaires.

Further Explanation • Now from Yt = Yt−1 + ut……. (4) we can write: Y1 = Y0 + u1 => Y2 = Y1 + u2 = Y0 + u1 + u2 => Y3 = Y2 + u3 = Y0 + u1 + u2 + u3 and so on… • In general, if the process started at some time 0 with a value of Y0, we have: Yt = Y0 +∑ut …….. ……………………….(5) • Therefore, E(Yt) = E(Y0 + ∑ut) = Y0 (why?) ………… (6) Because, utis “white noise” • In like fashion, it can be shown that: var (Yt ) = tσ2 …… (7) • Thus, the mean of Y is equal to its starting value, which is constant, but as t increases, its variance increases indefinitely (thus violating the condition of stationarity). • In short, the RWM without drift is a non-stationary stochastic process.

Further Explanation (cont.) • Now, if you write Yt = Yt−1 + ut….. (4) as (Yt − Yt−1) = ∆Yt = ut ……. (8) • It shows that, while Yt is non-stationary, its 1st difference is stationary. • In other words, the 1st differences of a random walk time series are stationary.

b) Random Walk with Drift • Let’s modify, Yt = Yt−1 + ut…………….(4) as follows: Yt = δ + Yt−1 + ut….....………….(9) where δ is the drift parameter. • The name drift comes from the fact that if we write the preceding equation as: Yt − Yt−1 = ∆Yt= δ + ut …………. (10) • It shows that Yt drifts upward/downward, depending on δ being positive/negative.

Further Explanation • Note that model Yt = δ + Yt−1 + ut………….. (9) is also an AR(1) model. • Following the procedure discussed for Random Walk Without Drift, it can be shown that for the random walk with drift model (9), E(Yt) = Y0 + t · δ ……….. (11) var (Yt ) = tσ2…………….... (12) • Here, again for RWM with drift the mean as well as the variance increases over time, again violating the conditions of stationarity. • In short, RWM, with or without drift, is a non-stationary stochastic process. • The random walk model is an example of what is known in the literature as a Unit Root Process.

3) Unit Root Stochastic Process • Let’s write the RWM Yt = Yt−1 + ut….. (4) as: Yt = ρYt−1 + ut − 1 -1 ≤ ρ ≤ 1 …….(13) • If ρ = 1, (13) becomes a RWM (without drift). • If ρ is in fact 1, we face what is known as the unit root problem (non-stationarity); as the variance of Yt is not stationary. • The name unit root is due to the fact that ρ = 1. • Thus the terms non-stationarity, random walk, and unit root can be treated as synonymous. • If, however, |ρ| ≤ 1, then the time series Yt is stationary in the sense we have defined it. • Note: Unit Root Stochastic Process will be further explained in Unit Root Test of Stationarity.

Trend Stationary (TS) and Difference Stationary (DS)Stochastic Processes • Deterministic Trend:if the trend in a time series is completely predictable • Stochastic Trend: if it is not predictable • E.g. consider the following model of the time series Yt. Yt= β1 + β2t + β3Yt−1 + ut………………….(14) • Now we have the following possibilities: • Pure Random Walk: If in (14) β1 = 0, β2 = 0, β3 = 1, we get: Yt = Yt−1 + ut…………(15) • Which is nothing but a RWM without drift and is therefore non-stationary. • But note that, if we write (2.2) as ∆Yt = (Yt − Yt−1) = ut ………… (8) • It becomes stationary, as noted before. • Hence, a RWM without drift is a Difference Stationary Process (DSP).

Trend Stationary (TS) and Difference Stationary (DS)Stochastic Processes (cont.) • Random Walk With Drift: If in Yt= β1 + β2t + β3Yt−1 + ut……(14) β1 ≠ 0, β2 = 0, β3 = 1, we get: Yt= β1 + Yt−1 + ut……..(16) • which is a random walk with drift and is therefore, non-stationary. • If we write it as: (Yt− Yt−1) = ∆Yt = β1 + ut……..(16a) • this means Ytwill exhibit a positive (β1 > 0) or negative (β1 < 0) trend. • Such a trend is called a Stochastic Trend. • Equation (16a) is a DSP process because the non-stationarity in Yt can be eliminated by taking first differences of the time series.

Trend Stationary (TS) and Difference Stationary (DS)Stochastic Processes (cont.) • Deterministic Trend: If in Yt= β1 + β2t + β3Yt−1 + ut……(14) β1 ≠ 0, β2 ≠ 0, β3 = 0, we obtain: Yt = β1 + β2t+ ut………..(17) • Which is called a trend stationary process (TSP). • Although the mean of Yt is β1 + β2t, which is not constant, its variance ( = σ2) is. • Once the values of β1 and β2 are known, the mean can be forecast perfectly. • Therefore, if we subtract the mean of Yt from Yt, the resulting series will be stationary, hence the name trend stationary.

Trend Stationary (TS) and Difference Stationary (DS)Stochastic Processes (cont.) 4. Random Walk With Drift and Deterministic Trend: If in Yt= β1 + β2t + β3Yt−1 + ut……(14) β1 ≠ 0, β2 ≠ 0, β3 = 1, we obtain: Yt= β1 + β2t + Yt−1 + ut……….…..(18) • We have a random walk with drift and a deterministic trend, which can be seen if we write this equation as: ∆Yt = β1 + β2t + ut ……….……….(18a) • Which means that Yt is non-stationary. 5. Deterministic Trend With Stationary AR(1) Component: If in Yt= β1 + β2t + β3Yt−1 + ut……(14) β1 ≠ 0, β2 ≠ 0, β3 < 1, then we get: Yt = β1 + β2t + β3Yt−1 + ut ..……….(19) • Which is stationary around the deterministic trend.

The Phenomenon of Spurious Regression • Stationary Time Series are important, consider the following two random walk models: Yt = Yt−1 + ut ………….. (20) Xt = Xt−1 + vt ………….. (21) • Where we generated 500 observations of ut from ut ∼ N(0, 1) and 500 observations of vtfrom vt ∼ N(0, 1) and assumed that the initial values of both Y and X were zero. • We also assumed that utand vtare serially uncorrelated as well as mutually uncorrelated. • Both these time series are non-stationary; i.e. they are I(1) or exhibit stochastic trends.

The Phenomenon of Spurious Regression (cont.) • Suppose we regress Yton Xt. • Since Ytand Xtare uncorrelated I(1) processes, the R2 from the regression of Y on X should tend to zero; that is, there should not be any relationship between the two variables. • But wait till you see the regression results:

The Phenomenon of Spurious Regression (cont.) • As you can see, the coefficient of X is highly statistically significant, and, although the R2 value is low, it is statistically significantly different from zero. • From these results, you may be tempted to conclude that there is a significant statistical relationship between Y and X, whereas a priori there should be none. • This is the phenomenon of spurious/non-sense regression, first discovered by Yule.

The Phenomenon of Spurious Regression (cont.) • Yule showed that (spurious) correlation could persist in non-stationary time series even if the sample is very large. • That there is something wrong in the preceding regression is suggested by the extremely low Durbin–Watson d value, which suggests very strong first-order autocorrelation.

The Phenomenon of Spurious Regression (cont.) • R2 > d is a good rule of thumb to suspect that the estimated regression is spurious, as in the example above. • This example is a strong reminder that one should be extremely cautious of conducting regression analysis based on time series that exhibit stochastic trends.

Tests of Stationarity Presented by QaziYasirAyub

Tests of Stationarity • In practice we face two important questions: • How do we find out if a given time series is stationary or not? • Is there a way that it can be made stationary? • Prominently discussed tests in the literature are: • Graphical Analysis • The Unit Root Test

1. Graphical Analysis • Before pursuing a formal test, it is always advisable to plot the time series under study • E.g. take the GDP time series. • You will see that over the period of study GDP has been increasing (i.e. showing an upward trend) • This perhaps suggests that the GDP series is not stationary (also more or less true of the other economic time series).

2- The Unit Root Test • The widely popular test of stationarity is the unit root test. • We start with: Yt = ρYt−1 + ut − 1 ≤ ρ ≤ 1 (1.1) Where utis a white noise error term. • We know that if ρ = 1 (i.e. in the case of the unit root) (1.1) becomes a Random Walk Model Without Drift, (a non-stationary stochastic process). • Therefore, why not simply regress Yton its lagged value Yt−1and find out if the estimated ρ is statistically equal to 1? If it is, then Yt is non-stationary.

2- The Unit Root Test (cont.) • Procedure: Yt= ρYt−1 + ut (1.1) => Yt − Yt−1 = ρYt−1 − Yt−1 + ut (4.1) => Yt − Yt−1 = Yt−1 (ρ − 1)+ ut or => Yt − Yt−1 = (ρ − 1)Yt−1 + ut which can be alternatively written as: => ∆Yt = δYt−1 + ut (4.2) where δ = (ρ − 1) and ∆= first-difference (Yt − Yt−1 )

2- The Unit Root Test (cont.) • Hence, to estimate (4.2) and test the H0: δ = 0. • If δ = 0, then ρ = 1 (i.e. unit root/time series under consideration is non-stationary) and (21.9.2) will become ∆Yt = (Yt − Yt−1) = ut (4.3) • Since ut is a white noise error term, it is stationary, which means that the first differences of a random walk time series are stationary. • Now let’s turn to the estimation of (4.2).

2- The Unit Root Test (cont.) • We have to take the first differences of Yt and regress them on Yt−1 and see if the estimated slope co-efficient in this regression (δ) is zero or not. • If it is zero, we conclude that Yt is non-stationary. • But if it is negative, we conclude that Yt is stationary. • The only question is which test we use to find out if the estimated co-efficient of Yt−1 in (4.2) is zero or not? • Unfortunately, under the null hypothesis that δ = 0 (i.e., ρ = 1), the t value of the estimated coefficient of Yt−1 does not follow the t distribution even in large samples; i.e. it does not have an asymptotic normal distribution.

What is the alternative? • Dickey and Fuller have shown that under the null hypothesis that δ = 0, the estimated t-value of the coefficient of Yt−1 in (4.2) follows the τ (tau) statistic. • These authors have computed the critical values of the tau statistic on the basis of Monte Carlo simulations. • Interestingly, if the hypothesis that δ = 0 is rejected (i.e. the time series is stationary), we can use the usual t test.

DF Test • The actual procedure of implementing the DF test involves several decisions. • In discussing the nature of the unit root process in Sections 21.4 and 21.5, we noted that a random walk process may have no drift, or it may have drift or it may have both deterministic and stochastic trends. • To allow for the various possibilities, the DF test is estimated in three different forms, i.e. under three different null hypotheses.

Yt is a random walk: Yt = δYt−1 + ut (4.2) • Yt is a random walk with drift: Yt= β1 + δYt−1 + ut (4.4) • Yt is a random walk with drift around a stochastic trend: Yt = β1 + β2t + δYt−1 + ut (4.5) Where t is the time or trend variable.

In each case, the null hypothesis is that δ = 0; i.e. there is a unit root—the time series is non-stationary. • The alternative hypothesis is that δ is less than zero; i.e. the time series is stationary. • If the null hypothesis is rejected, it means that Yt is a stationary time series with zero mean in the case of (4.2), that Yt is stationary with a nonzero mean [= β1/(1 − ρ)] in the case of (4.4), and that Yt is stationary around a deterministic trend in (4.5).

It is extremely important to note that the critical values of the tau test to test the hypothesis that δ = 0, are different for each of the preceding three specifications of the DF test, which can be seen clearly from Appendix D, Table D.7. • Moreover, if, say, specification (4.4) is correct, but we estimate (4.2), we will be committing a specification error, whose consequences we already know from Chapter 13. • The same is true if we estimate (4.4) rather than the true (4.5). • Of course, there is no way of knowing which specification is correct to begin with. • Some trial and error is inevitable, data mining notwithstanding.

The actual estimation procedure is as follows: • Estimate (4.2), or (4.3), or (4.4) by OLS; divide the estimated coefficient of Yt−1 in each case by its standard error to compute the (τ) tau statistic; and refer to the DF tables (or any statistical package). • If the computed absolute value of the tau statistic (|τ |) exceeds the DF or MacKinnon critical tau values, we reject the hypothesis that δ = 0, in which case the time series is stationary. • On the other hand, if the computed |τ | does not exceed the critical tau value, we do not reject the null hypothesis, in which case the time series is non-stationary. • Make sure that you use the appropriate critical τ values.

Let us return to the U.S. GDP time series. For this series, the results of the three regressions (4.2), (4.4), and (4.5) are as follows: • The dependent variable in each case is Yt = GDPt

GDPt = 0.00576GDPt−1…… (4.6) • t = (5.7980) R2 =−0.0152 d = 1.34 • GDPt = 28.2054 − 0.00136GDPt−1…… (4.7) • t = (1.1576) (−0.2191) R2 = 0.00056 d = 1.35 • GDPt = 190.3857 + 1.4776t − 0.0603GDPt−1 • t = (1.8389) (1.6109) (−1.6252)…….. (4.8) • R2 = 0.0305 d = 1.31

Our primary interest here is in the t ( = τ) value of the GDPt−1 coefficient. • The critical 1, 5, and 10 percent τ values for model (4.6) are −2.5897, −1.9439, and −1.6177, respectively, and are −3.5064, −2.8947, and −2.5842 for model (4.7) and −4.0661, −3.4614, and −3.1567 for model (21.3.8). • As noted before, these critical values are different for the three models.

Before we examine the results, we have to decide which of the three models may be appropriate. • We should rule out model (4.6) because the coefficient of GDPt−1, which is equal to δ is positive. • But since δ = (ρ − 1), a positive δ would imply that ρ> 1. • Although a theoretical possibility, we rule this case out because in this case the GDP time series would be explosive.

That leaves us with models (4.7) and (4.8). In both cases the estimated δ coefficient is negative, implying that the estimated ρ is less than 1. • For these two models, the estimated ρ values are 0.9986 and 0.9397, respectively. • The only question now is if these values are statistically significantly below 1 for us to declare that the GDP time series is stationary.

For model (4.7) the estimated τ value is −0.2191, which in absolute value is below even the 10 percent critical value of −2.5842. • Since, in absolute terms, the former is smaller than the latter, our conclusion is that the GDP time series is not stationary. • The story is the same for model (4.8). • The computed τ value of −1.6252 is less than even the 10 percent critical τ value of −3.1567 in absolute terms. • Therefore, on the basis of graphical analysis, the correlogram, and the Dickey–Fuller test, the conclusion is that for the quarterly periods of 1970 to 1991, the U.S. GDP time series was non-stationary; i.e., it contained a unit root.

Time Series Econometrics: Some Basic Concepts