## Spurious Regression

Linear regression might indicate a strong relationship between two or more variables, but these variables may be totally unrelated in reality. Predictions fail when it comes to domain knowledge, this scenario is known as spurious regression.

There is a strong relationship between chicken consumption and crude oil exports in the below graph even though they are unrelated. Source: http://www.tylervigen.com/spurious-correlations   Strong trend / nonstationary and higher R square are observed in spurious regression. Spurious regression has to be eliminated while building the model since they are unrelated and have no causal relationship. Multilinear regression makes use of a correlation matrix to check the dependency between all the independent variables. If the correlation coefficient value is high between two variables any one variable is retained and another one is discarded to remove the dependency. In the dataset when time is a confounding factor multilinear regression fails, the correlation coefficient which is used to eliminate the variable is not time-bound instead it just gives the correlation between the two variables. Image Source: https://en.wikipedia.org/wiki/Granger_causality

Consider the above time-series graph, variable X has a direct influence on variable Y but there is a lag of 5 between X and Y in which case we cant use the correlation matrix. For eg. an increase in coronavirus positive cases in the city and an increase in the number of people getting hospitalized. For better forecasting, here we would like to know if there is a causal relationship.

## Granger Causality comes to Rescue

Prof. Clive W.J. Granger, recipient of the 2003 Nobel Prize in Economics developed the concept of causality to improve the performance of forecasting.

It is basically an econometric hypothetical test for verifying the usage of one variable in forecasting another in multivariate time series data with a particular lag.

A prerequisite for performing the Granger Causality test is that the data need to be stationary i.e it should have a constant mean, constant variance, and no seasonal component. Transform the non-stationary data to stationary data by differencing it, either first-order or second-order differencing. Do not proceed with the Granger causality test if the data is not stationary after second-order differencing.

Let us consider three variables Xt , Yt , and Wt preset in time series data.

Case 1: Forecast  Xt+1 based on past values Xt

Case 2: Forecast Xt+1 based on past values Xt and Yt.

Case3 : Forecast Xt+1 based on past values Xt , Yt , and Wt, where variable Yt has direct dependency on variable Wt.

Here Case 1 is univariate time series also known as the autoregressive model in which there is a single variable and forecasting is done based on the same variable lagged by say order p.

The equation for the Auto-regressive model of order p (RESTRICTED MODEL, RM)

Xt = α + 𝛾1 X𝑡−1 + 𝛾2X𝑡−2 + ⋯ + 𝛾𝑝X𝑡−𝑝

where  p parameters (degrees of freedom) to be estimated.

In Case 2 the past values of Y contain information for forecasting Xt+1. Yt is said to “Granger cause” Xt+1 provided Yt occurs before Xt+1 and it contains data for forecasting Xt+1.

Equation using a predictor Yt (UNRESTRICTED MODEL, UM)

Xt = α + 𝛾1 X𝑡−1 + 𝛾2X𝑡−2 + ⋯ + 𝛾𝑝X𝑡−𝑝  + α1Yt-1+  ⋯ + α𝑝 Yt-p

2p parameters (degrees of freedom) to be estimated.

If Yt causes Xt, then Y must precede X which implies:

• Lagged values of Y should be significantly related to X.
• Lagged values of X should not be significantly related to Y.

Case 3 can not be used to find Granger causality since variable Yt is influenced by variable Wt.

## Hypothesis test

Null Hypothesis (H0) : Yt does not “Granger cause” Xt+1 i.e., 𝛼1 = 𝛼2 = ⋯ = 𝛼𝑝 = 0

Alternate Hypothesis(HA): Yt does “Granger cause” Xt+1, i.e., at least one of the lags of Y is significant.

### Calculate the f-statistic

Fp,n-2𝑝−1 = (𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 𝑜𝑓 𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒) / (𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 𝑜𝑓 𝑈𝑛𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒)

Fp,n-2𝑝−1 =  ( (𝑆𝑆𝐸𝑅𝑀−𝑆𝑆𝐸𝑈𝑀) /𝑝) /(𝑆𝑆𝐸𝑈𝑀 /𝑛−2𝑝−1)

where n is the number of observations and
SSE is Sum of Squared Errors.

If the p-values are less than a significance level (0.05) for at least one of the lags then reject the null hypothesis.

Perform test for both the direction Xt->Yt and Yt->Xt.

Try different lags (p). The optimal lag can be determined using AIC.

## Limitation

• Granger causality does not provide any insight on the relationship between the variable hence it is not true causality unlike ’cause and effect’ analysis.
• Granger causality fails to forecast when there is an interdependency between two or more variables (as stated in Case 3).
• Granger causality test can’t be performed on non-stationary data.

### Resolving Chicken and Egg problem

Let us apply Granger causality to check whether the egg came first or chicken came first.

#### Importing libraries

```import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import pandas as pd```

Data is from the U.S. Department of Agriculture. It consists of two-time series variables from 1930 to 1983, one of U.S. egg production and the other the estimated U.S. chicken population.

`df = pd.read_csv('chickegg.csv')`

#### Exploring the Dataset

`df.head()`
`df.dtypes`
`df.shape`

(53, 3)

`df.describe()`

Check if the data is stationary, if not make it stationary to proceed.

```# Draw Plot
def plot_df(df, x, y, title="", xlabel="Date", ylabel="Value", dpi=100):
plt.figure(figsize=(16,5), dpi=dpi)
plt.plot(x, y, color="tab:red")
plt.gca().set(title=title, xlabel=xlabel, ylabel=ylabel)
plt.show()
plot_df(df, x=df.Year, y=df.chicken, title="Polulation of the chicken across US")
plot_df(df, x=df.Year, y=df.egg, title="Egg Produciton")```

By visual inspection, both the chicken and egg data are not stationary. Let us confirm this by running Augmented Test (ADF Test).

ADF test is a popular statistical test for checking whether the Time Series is stationary or not which works based on the unit root test. The number of unit roots present in the series indicates the number of differencing operations that are required to make it stationary

Consider the hypothesis test where:

Null Hypothesis (H0): Series has a unit root and is non-stationary.

Alternative Hypothesis (HA): Series has no unit root and is stationary.

```from statsmodels.tsa.stattools import adfuller
print(f'Test Statistics: {result}')
print(f'p-value: {result}')
print(f'critical_values: {result}')```
```if result > 0.05:
print("Series is not stationary")
else:
print("Series is stationary")```
`result = adfuller(df['egg'])`
```print(f'Test Statistics: {result}')
print(f'p-value: {result}')
print(f'critical_values: {result}')
if result > 0.05:
print("Series is not stationary")
else:
print("Series is stationary")```

p-values of both the egg and chicken variables are greater than the significant value (0.05), the Null hypothesis is valid and the series is not stationary.

## Data Transformation

Granger causality test is carried out only on stationary data hence we need to transform the data by differencing it to make it stationary. Let us perform the first-order differencing on chicken and egg data.

```df_transformed = df.diff().dropna()
df = df.iloc[1:]
print(df.shape)
df_transformed.shape```
`df_transformed.head()`
```plot_df(df_transformed, x=df.Year, y=df_transformed.chicken, title="Polulation of the chicken across US")
plot_df(df_transformed, x=df.Year, y=df_transformed.egg, title="Egg Produciton")```

Repeat the ADF test again on differenced data to check for stationarity.

```result = adfuller(df_transformed['chicken'])
print(f'Test Statistics: {result}')
print(f'p-value: {result}')
print(f'critical_values: {result}')
if result > 0.05:
print("Series is not stationary")
else:
print("Series is stationary")```

```result = adfuller(df_transformed['egg'])
print(f'Test Statistics: {result}')
print(f'p-value: {result}')
print(f'critical_values: {result}')
if result > 0.05:
print("Series is not stationary")
else:
print("Series is stationary")```

Transformed chicken and egg data are stationary, hence there is no need to go for second-order differencing.

## Test the Granger Causality

There are several ways to find the optimal lag but for simplicity let’s consider 4th lag as of now.

Do eggs granger cause chickens?

Null Hypothesis (H0) : eggs do not granger cause chicken.

Alternative Hypothesis (HA) : eggs granger cause chicken.

`from statsmodels.tsa.stattools import grangercausalitytests`
`grangercausalitytests(df_transformed[['chicken', 'egg']], maxlag=4)`

p-value is very low, Null hypothesis is rejected hence eggs are granger causing chicken.

That implies eggs came first.

Now repeat the Granger causality test in the opposite direction.

Do chickens granger cause eggs at lag 4?

Null Hypothesis (H0) : chicken does not granger cause eggs.

Alternative Hypothesis (HA) : chicken granger causes eggs.

`grangercausalitytests(df_transformed[['egg', 'chicken']], maxlag=4)`

The p-value is considerably high thus chickens do not granger cause eggs.

The above analysis concludes that the egg came first and not the chicken.

Once the analysis is done the next step is to begin forecasting using time series forecasting models.

## EndNote

Whenever you come across time-bound data having multiple variables be suspicious about high R2 and possible spurious regression. Make use of the time series forecasting for better performance. Check for bidirectional Granger causality between each variable and eliminate the variable based on test results before proceeding with forecasting techniques.  I hope you enjoyed the article and gained insight into the Granger causality concept. Please drop your suggestions or queries in the comment section.