T6. Time Series Analysis

TOC

1. Basic Regression Analysis 1.1 Time Series Data 1.2 Distributed Lag Model 1.3 Classical Assumptions 1.4 Trends and Seasonality 2. Further Issues for OLS on TS data 2.1 Stationary and Weakly Dependent 2.2 Asymptotic Properties of OLS 2.3 Highly Persistent Time Series 3. Serial Correlation and Heteroskedasticity 3.1 Testing for Serial Correlation

1. Basic Regression Analysis

1.1 Time Series Data

Definition

Time series data are data collected on the same observational unit at multiple time periods.

Unlike cross-section data, time series data has a temporal ordering: the past can affect the future, but not vice versa. We need to alter some of our assumptions since we no longer have a random sample of individuals. Instead, we have a stochastic (i.e. random) process, i.e. a sequence of random variables indexed by time. One realization of a stochastic (i.e. random) process is a time series data.

Notations and Terms

Dataset: are observations on the time series variable . We consider only consecutive, evenly-spaced observations.

Lags: The first lag of a time series is ; the j-th lag is .

First difference of a series :

Growth Rates: The percentage change of a time series between periods and is approximately , where the approximation is most accurate when the percentage chang is small.

Autocorrelation (Serial Correlation)

The correlation (covariance) of a series with its own lagged values is called autocorrelation (autocovariance) or serial correlation.

The j-th sample autocorrelation is an estimate of the j-th population autocorrelation

where . is the sample average of computed over observations

1.2 Distributed Lag Model

Dynamic effects necessarily occur over time. The econometric model used to estimate dynamic causal effects needs to incorporate lags. The distributed lag model of order is:

where

= impact effect (impact propensity) of change in = effect of change in on , holding past constant

= 1-period dynamic multiplier = effect of change in on , holding constant

= 2-period dynamic multiplier = effect of change in on , holding constant

For any horizon , we can define the cumulative effect as , which is interpreted as the change in the expected outcome periods after a permanent, one-unit increase in .

LPR is the cumulative effect after all changes have taken place, it is simply the sum of all of the coefficients.

1.3 Classical Assumptions

TS.1: Linear in parameters

The stochastic process follows the linear model

where is the sequence of errors or disturbances. Here, is the number of observations (time periods).

TS.2: No perfect collinearity

In the sample (and therefore in the underlying time series process), no independent variable is constant nor a perfect linear combination of the others.

TS.3: Zero conditional mean

For each , the expected value of the error , given the explanatory variables for all time periods, is zero, that is

: contemporaneouly exogenous

: strictly exogenous

TS.4 Homoskedasticity

Conditional on , the variance of is the same for all :

A sufficient condition is that volatility of the error is independent of the explanatory variables and that it is constant over time. In the time series context, homoskedasticity may also be easily violated, e.g. if the volatitliy of the dependent variable depends on regime changes

TS.5 No serial correlation

We do not assume the errors for different cross-sectional sampling because we already have the random sampling assumption, under which and are independent for any two observations and .

TS.1~TS.5 are the appropriate Gauss-Markov assumptions for time series applications.

OLS Sampling Variances

Under the time series Gauss-Markov Assumptions TS.1~TS.5, the variance of , conditional on is

where is the total sum of squares of and is the R square from the regression of on the other independent variables.

Unbiased Estimation of the error variance

Under TS.1~TS.5,

Gauss-Markov Theorem

Under TS.1~TS.5, the OLS estimators have the minimal variance of all linear unbiased estimators of the regression coefficients. This holds conditional as well as unconditional on the regressors.

TS.6 Normality

independently of

Normal sampling distributions

Under TS.1~TS.6, the OLS estimators have the usual normal distribution (conditional on ). The usual F and t tests are valid.

1.4 Trends and Seasonality

Trends and Detrends

We must recognize that some series contain a time trend in order to draw causal inference using time series data. Ignoring the fact that two sequences are trending in the same or opposite directions can lead us to falsely conclude that changes in on variable are actually caused by changes in another variable.

The phenomenon of finding a relationship between two or more trending variables simply because each is growing over time is an example of a spurious regression problem. We can add a linear trend term to the regression to “detrend” series, that is

can also be obtained by:

regress each on and , denote the residuals as

regress on without intercept

The from a regression on detrended data, which better reflects how well explain , is usually samller than that on trended data.

Seasonality and Deseasonalize

Often time-series data exhibits some periodicity, referred to seasonality. Seasonality can be dealt with by adding a set of seasonal dummies, e.g.

where are dummy variables indicating whether time period corresponds to the appropriate month. Here, January is the base month.

2. Further Issues for OLS on TS data

2.1 Stationary and Weakly Dependent

Stationary Deifinition

A stochastic process is strict stationary, if for every collection of indices , the joint distribution of is the same as that of for all integers .

Stationary implies:

are identically distributed

the adjacent terms may be highly correlated, but the nature of any correlation between adjacent terms is the same across all periods

Nonstationary: e.g. A process with a time trend.

Covariance Stationary Process

A stochastic proce is covariance stationary if its expected value, its variance, and its covariances are constant over time:

If a (strict) stationary process has a finite second moment, then it must be covariance stationary.

If and have stationary distributions,

the coefficients don’t change within the sample

the results can be extrapolated outside the sample

Autocorrelation Function (ACF)

The graph of is called correlogram.

Partial Autocorrelation Function (PACF)

the h-th PACF is the effect of on after controlling for all intermediate lages . Use OLS to estimate it.

Weakly Dependent Definition

A stationary time series is weakly dependent if and are almost independent as increases.

If for a covariance stationary process

we’ll say this covariance stationary process is weakly dependent or asymptotically uncorrelated.

For LLN and CLT to hold, the individual observations must not be too strongly related to each other; in particular their relation must become weaker the farther they are apart. It ensures in large sample there is sufficient randomness in the data.

Time series counterpart of the “independently distributed” part of i.i.d.

For many applications, a stationary process is weakly dependent, but sometimes not.

Stationary and Weakly Dependent Time Series

White noise:

A sequence of random variables is said to be white noise if for all :

⇒ white noise is (srtict) stationary and weakly dependent

Indepnedent W.N.

(extra condition): are independent.

Gaussian W.N.

Moving average process of order one, MA(1)

the process is a short moving average of an i.i.d. series

⇒ MA(1) is covariance stationary

⇒ MA(1) is weakly dependent

thus the LLN and CLT can be applied to .

Autoregressive process of order one, AR(1)

the process carries over a certain extent the value the previous period, plus random shocks from an i.i.d. series

⇒ AR(1) is covariance stationary (proof needs some technical). Then we know that with , this can happen only if .

⇒ AR(1) is weakly dependent

⇒ thus

Covariance between and for

since , multiply this last equation by and take expectation to obtain

Thus

2.2 Asymptotic Properties of OLS

Assumptions

TS.1* Linear parameters

Same as assumption TS.1 but now the dependent and independent variables are assumed to be stationary and weakly dependent. In particular, the LLN and CLT can be applied to sample averages.

TS.2* No perfect collinearity

Same as assumption TS.2

TS.3* Zero conditional mean

Now the explanatory variabels are assumed to be only contemporaneously exogenous rather than strictly exogenous, i.e.

Under TS.1*~TS.3*, the OLS estimators are consistent

Note: For consistency, it would suffice to assume that the explanatory variables are merely contemporaneously uncorrelated with the error term.

TS.4* Homoskedasticity

The errors are contemporaneously homoskedastic

TS.5* No serial correlation

Conditional on the explanatory variables in periods and , the errors are uncorrelated.

Under TS.1*~TS.5*, the OLS estimators are asymptotically normally distributed. Further, the usual OLS standard errors, t-stats, F-stats, are asympottically valid.

Trends

Time series with deterministic time trends are non-stationary

If they are stationary around the trend and in addition weakly dependent, they are called trend-stationary processes

As long as a trend is included, all is well

2.3 Highly Persistent Time Series

Above shows that, provided the time series we use are weakly dependent, usual OLS inference procedures are valid under assumptions weaker than the classical linear model assumptions. Unfortunately, many economic time series cannot be characterized by weak dependence. Time series in these situations violate weak dependence because they are highly persistent (=strongly dependent). In this case, OLS methods are generally invalid. In some cases, transformations to weak dependence are possible.

Random Walk

The value today is the accumulation of all past shocks plus an initial value.

The random walk is not covariance stationary because its variances and its covariance depend on time.

It is also not weakly dependent because the correlation between observations vanished very slowly and this depends on how large is.

Still, random walks with drift are not covariance stationary and not weakly dependent.

Random walk is a special case of a unit root process.

Random walks with drift

In addition to the usual random walk mechanism, there is a deterministic increase/decrease (=drift) in each period.

This leads to a linear time trend around whhich the series follows its random walk behaviour. As there is no clear direction in which the random walk develops, it may also wander away from the trend.

Random walks with drift may be good models for time series that have an obvious trend but are not weakly dependent

Simulation of random walk and random walk with drift

Transformation of highly persistent TS

Order of intergration:

Weakly dependent time series are integreated of order zero .

This means that nothing needs to be done to such series before using them in regression analysis.
averges of such sequences already satisfy the standard limit theorems

Unit root process, such as random walk (or with a drift), are said to be integrated of order one,

This means that the first difference of the process is weakly dependent (and often stationary)
is often said to be a difference-stationary process
differencing is often a way to achieve weak dependence

Deciding whether a time series is

There are statistical tests for testing whether a time series is : unit root tests.

Alternatively, look at the sample first order autocorrelation:

If the sample first order autocorrelation is close to one, this suggests that the time series may be highly persistent (contains a unit root). Alternatively, the series may have a deterministic trend. Both unit root and tend may be eliminated by differencing.

Dynamically Complete Models

A model is said to be dynamically complete if enough lagged variables have been included as explanatory variables so that further lags do not help to explain the dependent variable

Dynamic completeness implies absence of serial correlation. If further lags actually belong in the regression, their omission will cause serial correlation (if the variables are serially correlated)

A set of explanatory variables is said to be sequentially exogenous if “enough” lagged explanatory variables have been included:

Sequential exogeneity is weaker than strict exogeneity. Sequential exogeneity is equivalent to dynamic completeness if the explanatory variables contain a lagged dependent variable.

3. Serial Correlation and Heteroskedasticity

3.1 Testing for Serial Correlation

Properties of OLS with Serially Correlated Errors

OLS is no long BLUE in the presence of serial correlation

the usual OLS standard error in the presence of serial correlation is invalid

t-stats are no longer valid for testing single hypotheses. Since a smaller standard error means a larger t-stats, the usual t-stats will often be too large when

The usual F and LM stat for testing multiple hypotheses are also invalid

Goodness-of-Fit: our usual goodness-of-fit measures, , adjusted still work provided the data are stationary and weakly dependent.

Testing for AR(1) Serial Correlation with Strictly Exogenous Regressors

For model (multiple regressors model)

Run the OLS regression of on and obtain the OLS residuals, , for all

Run the regression of on , for all . obtaining the coefficient on and its t-stat

Use (or p-value) to test against

Note that any serial correlation taht causes can be picked up by this test. While others , e.g. cannot.

The Durbin-Watson Test Under Classical Assumptions

Under assumptions TS.1~TS.5, the DW test is an exact test (whereas previous t test is only valid asymptotically) for AR(1) serial correlation

v.s.

Even with moderate sample sizes, the approximation is often pretty close. Therefore, tests based on DW and the t test based on are conceptually the same.

Reject if DW < , fail to reject if DW > . The DW test works with a lower and an upper bound for the critical value. In the area between the bounds the test result is inconclusive.

Note that if the regressors are not strictly exogenous, then neither the nor DW test will work.

The test can be easily generalized

Run the OLS regression of on and obtain the OLS residuals, for all

Run the regression of on for all to obtain the coefficient on and its t statistic

Use to test v.s.

Testing for Higher Order Serial Correlation

Just include q lags of the residuals in the regression and test for joint significance

Test:

Using F, Wald, or LM test

An alternative to computing the F test is to use the Lagrange multiplier (LM) form of the statistic.

where is the R square of above step 2.

Q Tests for Serial Correlation