Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

DZone's Guide to

# Linear ‘Prediction’ for AR Time Series

· Big Data Zone ·
Free Resource

Comment (0)

Save
{{ articles[0].views | formatCount}} Views

The Architect’s Guide to Big Data Application Performance. Get the Guide.

In the exercises for the MAT8181 course, there are two Exercises (16 and 17) about prediction and extrapolation based on MA(1) and AR(1) time series. But before discussing those exercises (I had some request for hints), I wanted to recall the definition of the linear prediction,

$EL(Y\vert \boldsymbol{X})=\boldsymbol{\alpha}^{\star \text{\sffamily T}} \boldsymbol{X}$

where

$\boldsymbol{\alpha}^{\star}=\text{argmin}\left\{\mathbb{E}([Y-\boldsymbol{\alpha}^{\text{\sffamily T}} \boldsymbol{X}>^2)\right\}$

As discussed previously on this blog, we consider here on projection not on $\sigma\{\boldsymbol{X}\}$ (this would be the conditional expectancy) but on the linear subset.

The goal of Exercise 2 was to establish an important result, in the context of Gaussian random vectors. If $\boldsymbol{X}$ is a (multivariate) Gaussian vector, $\mathcal{N}(\boldsymbol{0},\boldsymbol{\Sigma})$, then

$\boldsymbol{\alpha}^{\star}=\boldsymbol{\Sigma}^{-1}\boldsymbol{\gamma}$

where $\boldsymbol{\gamma}$ is the vector $\text{cov}(Y,\boldsymbol{X})$.

Keep those results in mind, and let us look at Exercise 17, for instance. Here, $(X_t)$ is an AR(1) process, with innovation $(\varepsilon_t)$,

$X_{t}=\phi X_{t-1}+\varepsilon_t$

One observation (say $X_\tau$) is missing. We have here 3 questions:

• what is the best linear prediction of $X_\tau$ given $X_{\tau-2}$ and $X_{\tau-1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau+2}$ and $X_{\tau+1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau-1}$ and $X_{\tau+1}$

Case 1. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau-2})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{-2}^\star X_{\tau-2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_\tau,X_{\tau-2})=\phi^2$ and $\text{cov}(X_{\tau-1},X_{\tau-2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

which can be written

$\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)=\frac{1}{\phi^2-1}\left(\begin{array}{ll}-1 & \phi\\\phi & -1\end{array}\right)\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

i.e. $(\alpha_{-2}^\star,\alpha_{-1}^\star)=(0,\phi)$. Which makes sense actually: the AR(1) process is Markovian of order one, so

$EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=EL(X_\tau\vert X_{\tau-1})$

And we seen in class that for an AR(1) process

${}_{\tau}\widehat{X}_{\tau+1}=EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=\phi X_{\tau-1}$

So far, so good.

Case 2. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\alpha_{+1}^\star X_{\tau+1}+\alpha_{+2}^\star X_{\tau+2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau+1})=\phi$$\text{cov}(X_\tau,X_{\tau+2})=\phi^2$ and $\text{cov}(X_{\tau+1},X_{\tau+2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{+1}^\star\\ \alpha_{+2}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi^2\end{array}\right)$

i.e. $(\alpha_{+1}^\star,\alpha_{+2}^\star)=(\phi,0)$.

Case 3. Finally, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{+1}^\star X_{\tau+1}$

One more time, since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_{\tau-1},X_{\tau+1})=\phi^2$ and $\text{cov}(X_{\tau},X_{\tau+1})=\phi$. So here, the relationship above becomes

$\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi\end{array}\right)$Here, we can write

$\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi\\\phi\end{array}\right)=\frac{1}{\phi^4-1}\left(\begin{array}{ll}-1 & \phi^2\\\phi^2 & -1\end{array}\right)\left(\begin{array}{l}\phi\\\phi\end{array}\right)$

i.e.

$\alpha_{-1}^\star=\frac{\phi}{\phi^2+1}=\alpha_{+1}^\star$

So finally, what we got here is

$EL(X_{\tau}\vert X_{\tau-2}, X_{\tau-1})=\phi X_{\tau-1}$

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\phi X_{\tau+1}$

and

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\frac{\phi}{\phi^2+1}\left(X_{\tau-1}+X_{\tau+1}\right)$

The mean squared errors for each of those estimates are obtained computing

$\mathbb{E}\left([X_\tau-EL(X_{\tau}\vert\boldsymbol{X})>^2\right)$

I guess I should probably stop here… that’s a detailed hint actually.

Learn how taking a DataOps approach will help you speed up processes and increase data quality by providing streamlined analytics pipelines via automation and testing. Learn More.

Topics:

Comment (0)

Save
{{ articles[0].views | formatCount}} Views

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

### {{ parent.tldr }}

{{ parent.urlSource.name }}