Jump to content

Econometric Theory/Normal Equations Proof

From Wikibooks, open books for an open world

Below is the proof of the Normal Equations for OLS.

The goal of OLS is to minimize the sum of squared error terms to find the best fit, also called the Residual Sum of Squares (RSS). This is denoted by .

Defining the RSS

[edit | edit source]

Known:

RSS = =

Differentiate the RSS (so that we can then minimise it)

[edit | edit source]

=


=

So we have two equations:

and

(The two(2) here is divided from both sides)

setting them both equal to

We get

(This is the first OLS Normal Equation)

and

(This is the second OLS Normal Equation)

Solve the Normal Equations

[edit | edit source]

Divide the first equation by n

Leaves us with

Now we know how to get α(hat), we can work on β(hat)

We can move β(hat) to one side

- n \bar{X} \bar{Y}

And now we have our Normal equations for OLS.

Since we have two equations and two unknowns, we are able to solve for them ().