STATE 03 · PROCESSES

Time-series models

The autoregressive and moving-average families, and the forecast as a conditional mean.

The autoregressive family

An autoregressive series expresses its current value as a weighted sum of earlier values plus a fresh disturbance. The weights are called coefficients, and their number is the order of the model. A first-order series carries its dependence one step back; a higher-order series reaches further into the past. The series is stationary exactly when the coefficients satisfy a stability condition, namely that the roots of a certain polynomial lie outside the unit circle.

When the series is stationary its mean is constant, its variance is finite and constant, and the dependence between values separated by a fixed number of steps depends only on that separation. These properties make the autoregressive family the workhorse of the subject: simple to state, rich enough to describe genuine persistence, and tractable in closed form.

The moving-average family

A moving-average series expresses its current value as a weighted sum of the current and recent disturbances. Its order is the number of lags retained. Because only finitely many disturbances enter, the dependence between values separated by more steps than the order simply vanishes. The moving-average family therefore models short memory, while the autoregressive family models memory that decays gradually and never strictly ends.

A moving average may also be written as an autoregressive series of infinite order, and, under a stability condition, an autoregressive series may be written as a moving average of infinite order. These dual representations are two views of one structure, and they are linked through inversion of the lag polynomial.

Lag operators and differencing

The lag operator shifts a series back by one step. Its powers shift by several steps, and polynomials in the lag operator compress a weighted sum of past values into a single symbol. The autoregressive and moving-average constructions then read as products of such polynomials applied to white noise, an economical notation that makes the algebra of series almost mechanical.

Differencing subtracts the previous value from the current one and is the inverse of the lag operator applied through the identity one minus L. A series that becomes stationary after one differencing is said to be integrated of order one; a series that needs two passes is integrated of order two. Differencing removes a drifting trend and is the standard remedy for wandering records.

The forecast as a conditional expectation

A forecast is not a guess about a hidden intention; it is a conditional expectation. Given the observed past, the best forecast of a future value, in the sense of smallest average squared error, is the conditional mean of that value given the past. For a Gaussian series the conditional mean is a linear function of the observed values, which is why linear forecasting rules are not merely convenient but optimal in that setting.

The forecast error is the difference between the realised value and the forecast. Its variance grows with the forecast horizon, since the future accumulates more fresh disturbances the further it lies ahead. A stationary series is eventually unpredictable: at long horizons its conditional mean approaches the unconditional mean, and the forecast error variance approaches the unconditional variance of the series.

The spectral view of these matters, taken up in stationarity and spectra, describes the same dependence in the language of frequency rather than lag.