Homoscedasticity and Heteroscedasticity

In statistics, -scedasticity means the distribution of error terms. A linear model in the form of \(y_{i} = X_{i}\beta + \epsilon_{i}\) is homoscedasticity if the variance of \(\epsilon_{i}\) is the same across all samples. Otherwise, it is heteroscedasticity.

Figure 1: \(y_{i} = X_{i}\beta + \epsilon_{i}\) where \(X_{i}\beta = 0\) a) Left: Example of homoscedasticity; b) Right: Example of heterscedasticity

Figure 1a is an example of homoscedasticity in which \(\epsilon_{i}\) is independent of \(x_{i}\). Meanwhile, Figure 1b is in heteroscedasticity which the variance of \(\epsilon_{i}\) is a function of \(X_{i} : \epsilon_{i}(x_{i}) = {x_{i}}^2\). In general, we want the residual to be homoscedasticity (\(\epsilon_{i}\) to be an independent variable of \(x_{i}\)) so that the model \(y_{i} = X_{i}\beta + \epsilon_{i}\) describes all the relationships between \(y_{i}\) and \(x_{i}\).

To test the heteroscedasticity of a linear model, we can use Breusch–Pagan test. The idea is simple. If \(\epsilon_{i}\) is dependent on \(x_{i}\), it is heteroscedasticity. So we fit a linear model on \(\epsilon_{i}\) such as \(\epsilon_{i} = X_{i}\alpha + \mu_{i}\) and try to test the goodness of fit of the model. If the model is a good model \((\alpha \neq 0)\), it is in heteroscedasticity.

⁣1. Fit a linear model on \(\epsilon_{i} : \epsilon_{i} = X_{i}\alpha + \mu_{i}\)

⁣2. Calculate the coefficient of determination of the model :

\begin{equation} R^2 = 1 - \frac{\sum_{i}{\mu_{i}}^2}{\sum_{i}(\epsilon_{i}-\bar{\epsilon_{i}})^2} \end{equation}

where \(\bar{\epsilon_{i}} = \frac{1}{n}\sum_{i}{\epsilon_{i}}\)

⁣3. Apply the F test: \(F = nR^2\) and calculate the p-value. \(H_{0}\) : The variances of \(\mu_{i}\) are equal.

If the p-value is smaller than the significance level (such as 0.05), we can reject the null hypothesis and assume it is heteroscedasticity. Otherwise, we can say that it does not provide strong evidence that it is heteroscedasticity.