Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

DZone's Guide to

# Linear ‘Prediction’ for AR Time Series

· Big Data Zone ·
Free Resource

Comment (0)

Save
{{ articles[0].views | formatCount}} Views

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

In the exercises for the MAT8181 course, there are two Exercises (16 and 17) about prediction and extrapolation based on MA(1) and AR(1) time series. But before discussing those exercises (I had some request for hints), I wanted to recall the definition of the linear prediction,

$EL(Y\vert \boldsymbol{X})=\boldsymbol{\alpha}^{\star \text{\sffamily T}} \boldsymbol{X}$

where

$\boldsymbol{\alpha}^{\star}=\text{argmin}\left\{\mathbb{E}([Y-\boldsymbol{\alpha}^{\text{\sffamily T}} \boldsymbol{X}>^2)\right\}$

As discussed previously on this blog, we consider here on projection not on $\sigma\{\boldsymbol{X}\}$ (this would be the conditional expectancy) but on the linear subset.

The goal of Exercise 2 was to establish an important result, in the context of Gaussian random vectors. If $\boldsymbol{X}$ is a (multivariate) Gaussian vector, $\mathcal{N}(\boldsymbol{0},\boldsymbol{\Sigma})$, then

$\boldsymbol{\alpha}^{\star}=\boldsymbol{\Sigma}^{-1}\boldsymbol{\gamma}$

where $\boldsymbol{\gamma}$ is the vector $\text{cov}(Y,\boldsymbol{X})$.

Keep those results in mind, and let us look at Exercise 17, for instance. Here, $(X_t)$ is an AR(1) process, with innovation $(\varepsilon_t)$,

$X_{t}=\phi X_{t-1}+\varepsilon_t$

One observation (say $X_\tau$) is missing. We have here 3 questions:

• what is the best linear prediction of $X_\tau$ given $X_{\tau-2}$ and $X_{\tau-1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau+2}$ and $X_{\tau+1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau-1}$ and $X_{\tau+1}$

Case 1. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau-2})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{-2}^\star X_{\tau-2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_\tau,X_{\tau-2})=\phi^2$ and $\text{cov}(X_{\tau-1},X_{\tau-2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

which can be written

$\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)=\frac{1}{\phi^2-1}\left(\begin{array}{ll}-1 & \phi\\\phi & -1\end{array}\right)\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

i.e. $(\alpha_{-2}^\star,\alpha_{-1}^\star)=(0,\phi)$. Which makes sense actually: the AR(1) process is Markovian of order one, so

$EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=EL(X_\tau\vert X_{\tau-1})$

And we seen in class that for an AR(1) process

${}_{\tau}\widehat{X}_{\tau+1}=EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=\phi X_{\tau-1}$

So far, so good.

Case 2. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\alpha_{+1}^\star X_{\tau+1}+\alpha_{+2}^\star X_{\tau+2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau+1})=\phi$$\text{cov}(X_\tau,X_{\tau+2})=\phi^2$ and $\text{cov}(X_{\tau+1},X_{\tau+2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{+1}^\star\\ \alpha_{+2}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi^2\end{array}\right)$

i.e. $(\alpha_{+1}^\star,\alpha_{+2}^\star)=(\phi,0)$.

Case 3. Finally, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{+1}^\star X_{\tau+1}$

One more time, since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_{\tau-1},X_{\tau+1})=\phi^2$ and $\text{cov}(X_{\tau},X_{\tau+1})=\phi$. So here, the relationship above becomes

$\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi\end{array}\right)$Here, we can write

$\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi\\\phi\end{array}\right)=\frac{1}{\phi^4-1}\left(\begin{array}{ll}-1 & \phi^2\\\phi^2 & -1\end{array}\right)\left(\begin{array}{l}\phi\\\phi\end{array}\right)$

i.e.

$\alpha_{-1}^\star=\frac{\phi}{\phi^2+1}=\alpha_{+1}^\star$

So finally, what we got here is

$EL(X_{\tau}\vert X_{\tau-2}, X_{\tau-1})=\phi X_{\tau-1}$

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\phi X_{\tau+1}$

and

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\frac{\phi}{\phi^2+1}\left(X_{\tau-1}+X_{\tau+1}\right)$

The mean squared errors for each of those estimates are obtained computing

$\mathbb{E}\left([X_\tau-EL(X_{\tau}\vert\boldsymbol{X})>^2\right)$

I guess I should probably stop here… that’s a detailed hint actually.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

Topics:

Comment (0)

Save
{{ articles[0].views | formatCount}} Views

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.