Least Squares Method

[Goldberg Variations]

Hey! I'm working on a question where you are supposed to use the Least Squares Method to be used on a line.

I don't have the correct answer to this question but would appreciate it if anyone here could double check my answer or tell me if I've done something wrong.

Translated to English the question says the following:

"Adjust the line y = a + bx according to the Least Squares Method with the following data: "

\(\displaystyle x|1\hspace{1 mm}2\hspace{1 mm}3\hspace{1 mm}4\)
\(\displaystyle \overline{y|1\hspace{1 mm}1\hspace{1 mm}0\hspace{1 mm}-1}\)


My solution:

\(\displaystyle y=a+bx\implies y=bx+a\)

\(\displaystyle \begin{pmatrix}
1& 1\\
2& 1\\
3& 1\\
4& 1
\end{pmatrix}
\begin{pmatrix}
a\\
b
\end{pmatrix}
=
\begin{pmatrix}
1\\
1\\
0\\
-1
\end{pmatrix}
\hspace{10 mm} A\bar{x}=\bar{y}
\)

Normal Equation gives:
\(\displaystyle A^TA\bar{x}=A^T\bar{y}\)

So:

\(\displaystyle \begin{pmatrix}
1 & 2& 3 & 4\\
1 & 1 &1 & 1
\end{pmatrix}
\begin{pmatrix}
1 & 1\\
2 & 1\\
3 & 1\\
4 & 1
\end{pmatrix}
\begin{pmatrix}
a\\
b
\end{pmatrix}
=
\begin{pmatrix}
1 & 2& 3 & 4\\
1 & 1 &1 & 1
\end{pmatrix}
\begin{pmatrix}
1\\
1\\
0\\
-1
\end{pmatrix}\implies\)

\(\displaystyle \implies\begin{pmatrix}
30 & 10\\
10 & 4
\end{pmatrix}
\begin{pmatrix}
a\\
b
\end{pmatrix}
=
\begin{pmatrix}
-1\\
1
\end{pmatrix}\)

Gauss Elimination:

\(\displaystyle 30a + 10b = -1\)
\(\displaystyle 10a + 4b = 1\)

Reduce Second row three times to first row

\(\displaystyle -2b = -4\)
\(\displaystyle 10a + 4b = 1\)

b = 2, now solving for a

\(\displaystyle 10a = 1 - 4*2=1-8=-7\implies a=\dfrac{-7}{10}\)

Which gives me my answer:

\(\displaystyle y=2+\dfrac{-7}{10}x\)

Am I on the right path? Thanks in advance for any input.

 

[tkhunny]

No good. Your very first matrix is incorrect. Oddly, the transpose of it is correct. Can't really say how you managed that.

Did you plot your line against the data?

 

[Goldberg Variations]

How can the first matrix be wrong and the Transpose be correct as the transpose is based on the first matrix. How else would it look like? The first matrix I mean. Elaborate.


EDIT: I doubled checked my matrix (the first one) in matlab and took the transpose of it. It is correct.

EDIT 2: My answer is correct! Input the field values with the following online calculator for "Least Squares" http://www.physics.csbsju.edu/stats/QF_NROW_form.html gives me my answer.

Neat!

 

[Goldberg Variations]

Hey JeffM!

No I never implied that the book had a different answer. I meant that I did not have the answer to the question, its just random values I made up for the line.

I was looking at my old notes for these kinds of questions and it would seem my solution is correct. The way you've sovled it JeffM is an unknown method to me. But I always appreciate input.

In my notes my method is called "Ordinary Least Squares", what the difference is I am not sure. I reckon my old math teacher maybe wanted to make it as simple as possible?

Anyway, thanks for the input everyone.

 

[tkhunny]

Not sure why I thought the transpose was correct. It's also incorrect as it is written.

I think you are thinking straight, just not writing straight.


You went to the trouble to change your linear model from y = a + bx to y = bx + a,

but you failed to change your matrix model from (a,b)^T to (b,a)^T

 

[Goldberg Variations]

Yep you are right about that tkhunny. I switched variables but I kept in the back of my head which one was which for the original equation. Sorry about that! So when I solved for my a it was actually b I solved, as you can see near the end of my solution.

Reason for that is because I find it easier for b to be the constant and a with the unknown x. When I changed the equation to y = bx + a, I saw it as y = ax + b (just changing variable names). I failed to point that out. Apologies!