Linear Algebra/Orthogonal Projection Onto a Line/Solutions

From Wikibooks, open books for an open world
< Linear Algebra‎ | Orthogonal Projection Onto a Line
Jump to: navigation, search

Solutions[edit]

This exercise is recommended for all readers.
Problem 1

Project the first vector orthogonally onto the line spanned by the second vector.

  1.  \begin{pmatrix} 2 \\ 1 \end{pmatrix} ,  \begin{pmatrix} 3 \\ -2 \end{pmatrix}
  2.  \begin{pmatrix} 2 \\ 1 \end{pmatrix} , 
\begin{pmatrix} 3 \\ 0 \end{pmatrix}
  3.  \begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix} , 
\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}
  4.  \begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix} , 
\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}
Answer

Each is a straightforward application of the formula from Definition 1.1.

  1. \displaystyle \frac{\begin{pmatrix} 2 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ -2 \end{pmatrix}}{
\begin{pmatrix} 3 \\ -2 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ -2 \end{pmatrix}}
\cdot\begin{pmatrix} 3 \\ -2 \end{pmatrix}
=\frac{4}{13}
\cdot\begin{pmatrix} 3 \\ -2 \end{pmatrix}
=\begin{pmatrix} 12/13 \\ -8/13 \end{pmatrix}
  2. \displaystyle \frac{\begin{pmatrix} 2 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 0 \end{pmatrix}}{
\begin{pmatrix} 3 \\ 0 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 0 \end{pmatrix}}
\cdot\begin{pmatrix} 3 \\ 0 \end{pmatrix}
=\frac{2}{3}
\cdot\begin{pmatrix} 3 \\ 0 \end{pmatrix}
=\begin{pmatrix} 2 \\ 0 \end{pmatrix}
  3. \displaystyle
\frac{\begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}}{
\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}}
\cdot\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}
=\frac{-1}{6}
\cdot\begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}
=\begin{pmatrix} -1/6 \\ -1/3 \\ 1/6 \end{pmatrix}
  4. \displaystyle
\frac{\begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}}{
\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}}
\cdot\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}
=\frac{1}{3}
\cdot\begin{pmatrix} 3 \\ 3 \\ 12 \end{pmatrix}
=\begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix}
This exercise is recommended for all readers.
Problem 2

Project the vector orthogonally onto the line.

  1. \begin{pmatrix} 2 \\ -1 \\ 4 \end{pmatrix}, \{c\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}\,\big|\, c\in\mathbb{R}\}
  2. \begin{pmatrix} -1 \\ -1 \end{pmatrix}, the line y=3x
Answer
  1. \displaystyle
\frac{\begin{pmatrix} 2 \\ -1 \\ 4 \end{pmatrix}\cdot\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}}{
\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}\cdot\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}}
\cdot\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}
=\frac{-19}{19}\cdot\begin{pmatrix} -3 \\ 1 \\ -3 \end{pmatrix}
=\begin{pmatrix} 3 \\ -1 \\ 3 \end{pmatrix}
  2. Writing the line as \{c\cdot \begin{pmatrix} 1 \\ 3 \end{pmatrix}\,\big|\, c\in\mathbb{R}\} gives this projection.
    
\frac{\begin{pmatrix} -1 \\ -1 \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 3 \end{pmatrix}}{
\begin{pmatrix} 1 \\ 3 \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 3 \end{pmatrix}}
\cdot\begin{pmatrix} 1 \\ 3 \end{pmatrix}
=\frac{-4}{10}\cdot\begin{pmatrix} 1 \\ 3 \end{pmatrix}
=\begin{pmatrix} -2/5 \\ -6/5 \end{pmatrix}
Problem 3

Although the development of Definition 1.1 is guided by the pictures, we are not restricted to spaces that we can draw. In \mathbb{R}^4 project this vector onto this line.


\vec{v}=\begin{pmatrix} 1 \\ 2 \\ 1 \\ 3 \end{pmatrix}
\qquad
\ell=\{c\cdot\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}\,\big|\, c\in\mathbb{R}\}
Answer

\displaystyle
\frac{\begin{pmatrix} 1 \\ 2 \\ 1 \\ 3 \end{pmatrix}\cdot\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}}{
\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}}
\cdot\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}
=\frac{3}{4}
\cdot\begin{pmatrix} -1 \\ 1 \\ -1 \\ 1 \end{pmatrix}
=\begin{pmatrix} -3/4 \\ 3/4 \\ -3/4 \\ 3/4 \end{pmatrix}

This exercise is recommended for all readers.
Problem 4

Definition 1.1 uses two vectors \vec{s} and \vec{v}. Consider the transformation of \mathbb{R}^2 resulting from fixing


\vec{s}=\begin{pmatrix} 3 \\ 1 \end{pmatrix}

and projecting \vec{v} onto the line that is the span of \vec{s}. Apply it to these vectors.

  1. \begin{pmatrix} 1 \\ 2 \end{pmatrix}
  2. \begin{pmatrix} 0 \\ 4 \end{pmatrix}

Show that in general the projection tranformation is this.


\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}
\mapsto
\begin{pmatrix} (x_1+3x_2)/10 \\ (3x_1+9x_2)/10 \end{pmatrix}

Express the action of this transformation with a matrix.

Answer
  1. \displaystyle
\frac{\begin{pmatrix} 1 \\ 2 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}{
\begin{pmatrix} 3 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}
\cdot \begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\frac{1}{2}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\begin{pmatrix} 3/2 \\ 1/2 \end{pmatrix}
  2. \displaystyle
\frac{\begin{pmatrix} 0 \\ 4 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}{
\begin{pmatrix} 3 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}
\cdot \begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\frac{2}{5}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\begin{pmatrix} 6/5 \\ 2/5 \end{pmatrix}

In general the projection is this.


\frac{\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}{
\begin{pmatrix} 3 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}}
\cdot \begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\frac{3x_1+x_2}{10}\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}
=\begin{pmatrix} (9x_1+3x_2)/10 \\ (3x_1+x_2)/10 \end{pmatrix}

The appropriate matrix is this.


\begin{pmatrix}
9/10  &3/10  \\
3/10  &1/10
\end{pmatrix}
Problem 5

Example 1.5 suggests that projection breaks \vec{v} into two parts, \mbox{proj}_{[\vec{s}\,]}({\vec{v}\,}) and \vec{v}-\mbox{proj}_{[\vec{s}\,]}({\vec{v}\,}), that are "not interacting". Recall that the two are orthogonal. Show that any two nonzero orthogonal vectors make up a linearly independent set.

Answer

Suppose that \vec{v}_1 and \vec{v}_2 are nonzero and orthogonal. Consider the linear relationship c_1\vec{v}_1+c_2\vec{v}_2=\vec{0}. Take the dot product of both sides of the equation with \vec{v}_1 to get that


\vec{v}_1\cdot(c_1\vec{v}_1+c_2\vec{v}_2)
=c_1\cdot(\vec{v}_1\cdot\vec{v}_1)
+c_2\cdot(\vec{v}_1\cdot\vec{v}_2)
=c_1\cdot (\vec{v}_1\cdot\vec{v}_1)+c_2\cdot 0
=c_1\cdot (\vec{v}_1\cdot\vec{v}_1)

is equal to \vec{v}_1\cdot\vec{0}=\vec{0}. With the assumption that \vec{v}_1 is nonzero, this gives that c_1 is zero. Showing that c_2 is zero is similar.

Problem 6
  1. What is the orthogonal projection of  \vec{v} onto a line if  \vec{v} is a member of that line?
  2. Show that if \vec{v} is not a member of the line then the set \{\vec{v},\vec{v}-\mbox{proj}_{[\vec{s}\,]}({\vec{v}\,})\} is linearly independent.
Answer
  1. If the vector \vec{v}\, is in the line then the orthogonal projection is  \vec{v} . To verify this by calculation, note that since \vec{v}\, is in the line we have that  \vec{v}=c_{\vec{v}}\cdot\vec{s}\, for some scalar c_{\vec{v}}.
    
\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}\cdot\vec{s}
=\frac{c_{\vec{v}}\cdot \vec{s}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}}\cdot\vec{s}
=c_{\vec{v}}\cdot\frac{\vec{s}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}}\cdot\vec{s}
=c_{\vec{v}}\cdot 1 \cdot\vec{s}
=\vec{v}
    (Remark. If we assume that \vec{v}\, is nonzero then the above is simplified on taking \vec{s}\, to be \vec{v}.)
  2. Write c_{\vec{p}}\vec{s}\, for the projection \mbox{proj}_{[\vec{s}\,]}({\vec{v}}). Note that, by the assumption that \vec{v} is not in the line, both \vec{v} and \vec{v}-c_{\vec{p}}\vec{s}\, are nonzero. Note also that if c_{\vec{p}}\, is zero then we are actually considering the one-element set \{\vec{v}\,\}, and with \vec{v} nonzero, this set is necessarily linearly independent. Therefore, we are left considering the case that c_{\vec{p}} is nonzero. Setting up a linear relationship
    
a_1(\vec{v})+a_2(\vec{v}-c_{\vec{p}}\vec{s})=\vec{0}
    leads to the equation (a_1+a_2)\cdot\vec{v}=a_2c_{\vec{p}}\cdot\vec{s}. Because \vec{v}\, isn't in the line, the scalars a_1+a_2 and a_2 c_{\vec{p}} must both be zero. The c_{\vec{p}}=0 case is handled above, so the remaining case is that a_2=0, and this gives that a_1=0 also. Hence the set is linearly independent.
Problem 7

Definition 1.1 requires that \vec{s}\, be nonzero. Why? What is the right definition of the orthogonal projection of a vector onto the (degenerate) line spanned by the zero vector?

Answer

If \vec{s}\, is the zero vector then the expression


\mbox{proj}_{[\vec{s}\,]}({\vec{v}})=
\frac{ \vec{v}\cdot\vec{s} }{
\vec{s}\cdot\vec{s} }\cdot\vec{s}

contains a division by zero, and so is undefined. As for the right definition, for the projection to lie in the span of the zero vector, it must be defined to be  \vec{0} .

Problem 8

Are all vectors the projection of some other vector onto some line?

Answer

Any vector in  \mathbb{R}^n is the projection of some other onto a line, provided that the dimension  n is greater than one. (Clearly, any vector is the projection of itself into a line containing itself; the question is to produce some vector other than \vec{v} that projects to  \vec{v} .)

Suppose that  \vec{v}\in\mathbb{R}^n with  n>1 . If  \vec{v}\neq\vec{0} then we consider the line  \ell=\{c\vec{v}\,\big|\, c\in\mathbb{R}\} and if  \vec{v}=\vec{0} we take  \ell to be any (nondegenerate) line at all (actually, we needn't distinguish between these two cases— see the prior exercise). Let  v_1,\dots,v_n be the components of  \vec{v} ; since  n>1 , there are at least two. If some  v_i is zero then the vector  \vec{w}=\vec{e}_i is perpendicular to  \vec{v} . If none of the components is zero then the vector  \vec{w}\, whose components are  v_2,-v_1,0,\dots ,0 is perpendicular to  \vec{v} . In either case, observe that  \vec{v}+\vec{w} does not equal  \vec{v} , and that  \vec{v} is the projection of  \vec{v}+\vec{w} onto  \ell .


\frac{(\vec{v}+\vec{w})\cdot\vec{v}}{\vec{v}\cdot\vec{v}}\cdot\vec{v}=\bigl(\frac{\vec{v}\cdot\vec{v}}{\vec{v}\cdot\vec{v}}
+\frac{\vec{w}\cdot\vec{v}}{\vec{v}\cdot\vec{v}}\bigr)\cdot\vec{v}=\frac{\vec{v}\cdot\vec{v}}{\vec{v}\cdot\vec{v}}\cdot\vec{v}
=\vec{v}

We can dispose of the remaining n=0 and n=1 cases. The dimension  n=0 case is the trivial vector space, here there is only one vector and so it cannot be expressed as the projection of a different vector. In the dimension n=1 case there is only one (nondegenerate) line, and every vector is in it, hence every vector is the projection only of itself.

This exercise is recommended for all readers.
Problem 9

Show that the projection of \vec{v} onto the line spanned by \vec{s} has length equal to the absolute value of the number \vec{v}\cdot\vec{s} divided by the length of the vector \vec{s}.

Answer

The proof is simply a calculation.


|\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}
\cdot\vec{s}\, |
=
|\frac{\vec{v}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}}|\cdot |\vec{s}\, |
=
\frac{|\vec{v}\cdot\vec{s}\,|}{
|\vec{s}\,|^2}\cdot|\vec{s}\,|
=
\frac{|\vec{v}\cdot\vec{s}\,|}{|\vec{s}\,|}
Problem 10

Find the formula for the distance from a point to a line.

Answer

Because the projection of  \vec{v} onto the line spanned by  \vec{s} is


\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}\cdot\vec{s}

the distance squared from the point to the line is this (a vector dotted with itself \vec{w}\cdot\vec{w} is written \vec{w}^2).

\begin{array}{rl}
|\vec{v}-
\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}
\cdot\vec{s}\, |^2
&=\vec{v}\cdot\vec{v}
-\vec{v}\cdot(
\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}\cdot\vec{s}
)
-(
\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}
\cdot\vec{s}\,
)\cdot\vec{v}
+(\frac{\vec{v}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}}\cdot\vec{s}\,)^2     \\
&=\vec{v}\cdot\vec{v}
-2\cdot(\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}})
\cdot\vec{v}\cdot\vec{s}
+(\frac{\vec{v}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}})\cdot\vec{s}\cdot\vec{s}  \\
&=\frac{(\vec{v}\cdot\vec{v}\,)\cdot(\vec{s}\cdot\vec{s}\,)
-2\cdot(\vec{v}\cdot\vec{s}\,)^2
+(\vec{v}\cdot\vec{s}\,)^2}{
\vec{s}\cdot\vec{s}}                               \\
&=\frac{(\vec{v}\cdot\vec{v}\,)(\vec{s}\cdot\vec{s}\,)
-(\vec{v}\cdot\vec{s}\,)^2}{\vec{s}\cdot\vec{s}}
\end{array}
Problem 11

Find the scalar  c such that  (cs_1,cs_2) is a minimum distance from the point  (v_1,v_2) by using calculus (i.e., consider the distance function, set the first derivative equal to zero, and solve). Generalize to  \mathbb{R}^n .

Answer

Because square root is a strictly increasing function, we can minimize  d(c)=(cs_1-v_1)^2+(cs_2-v_2)^2 instead of the square root of  d . The derivative is dd/dc=2(cs_1-v_1)\cdot s_1 +2(cs_2-v_2)\cdot s_2. Setting it equal to zero 2(cs_1-v_1)\cdot s_1 +2(cs_2-v_2)\cdot s_2
=c\cdot(2s_1^2+2s_2^2)-(v_1s_1+v_2s_2)=0 gives the only critical point.


c=\frac{v_1s_1+v_2s_2}{{s_1}^2+{s_2}^2}
=\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}

Now the second derivative with respect to c


\frac{d^2\,d}{dc^2}=2{s_1}^2+2{s_2}^2

is strictly positive (as long as neither  s_1 nor  s_2 is zero, in which case the question is trivial) and so the critical point is a minimum.

The generalization to \mathbb{R}^n is straightforward. Consider d_n(c)=(cs_1-v_1)^2+\dots+(cs_n-v_n)^2, take the derivative, etc.

This exercise is recommended for all readers.
Problem 12

Prove that the orthogonal projection of a vector onto a line is shorter than the vector.

Answer

The Cauchy-Schwarz inequality |\vec{v}\cdot\vec{s}\,|
\leq|\vec{v}\,|\cdot|\vec{s}\,| gives that this fraction


|\frac{\vec{v}\cdot\vec{s}}{\vec{s}\cdot\vec{s}}
\cdot\vec{s}\, |
=|\frac{\vec{v}\cdot\vec{s}}{
\vec{s}\cdot\vec{s}}|
\cdot|\vec{s}\, |
=
\frac{|\vec{v}\cdot\vec{s}\,|}{
|\vec{s}\,|^2}\cdot|\vec{s}\,|
=
\frac{|\vec{v}\cdot\vec{s}\,|}{|\vec{s}\,|}

when divided by  |\vec{v}\,| is less than or equal to one. That is,  |\vec{v}\,| is larger than or equal to the fraction.

This exercise is recommended for all readers.
Problem 13

Show that the definition of orthogonal projection onto a line does not depend on the spanning vector: if  \vec{s} is a nonzero multiple of  \vec{q} then  (\vec{v}\cdot\vec{s}/\vec{s}\cdot\vec{s}\,)\cdot\vec{s} equals  (\vec{v}\cdot\vec{q}/\vec{q}\cdot\vec{q}\,)\cdot\vec{q} .

Answer

Write  c\vec{s} for  \vec{q} , and calculate:  (\vec{v}\cdot c\vec{s}/c\vec{s}\cdot c\vec{s}\,)\cdot c\vec{s}=(\vec{v}\cdot \vec{s}/\vec{s}\cdot \vec{s}\,)\cdot \vec{s} .

This exercise is recommended for all readers.
Problem 14

Consider the function mapping to plane to itself that takes a vector to its projection onto the line  y=x . These two each show that the map is linear, the first one in a way that is bound to the coordinates (that is, it fixes a basis and then computes) and the second in a way that is more conceptual.

  1. Produce a matrix that describes the function's action.
  2. Show also that this map can be obtained by first rotating everything in the plane  \pi/4 radians clockwise, then projecting onto the  x -axis, and then rotating  \pi/4 radians counterclockwise.
Answer
  1. Fixing
    
\vec{s}=\begin{pmatrix} 1 \\ 1 \end{pmatrix}
    as the vector whose span is the line, the formula gives this action,
    
\begin{pmatrix} x \\ y \end{pmatrix}
\mapsto
\frac{\begin{pmatrix} x \\ y \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 1 \end{pmatrix}}{
\begin{pmatrix} 1 \\ 1 \end{pmatrix}\cdot\begin{pmatrix} 1 \\ 1 \end{pmatrix}}\cdot\begin{pmatrix} 1 \\ 1 \end{pmatrix}
=\frac{x+y}{2}\cdot\begin{pmatrix} 1 \\ 1 \end{pmatrix}
=\begin{pmatrix} (x+y)/2 \\ (x+y)/2 \end{pmatrix}
    which is the effect of this matrix.
    
\begin{pmatrix}
1/2  &1/2  \\
1/2  &1/2
\end{pmatrix}
  2. Rotating the entire plane \pi/4 radians clockwise brings the y=x line to lie on the x-axis. Now projecting and then rotating back has the desired effect.
Problem 15

For  \vec{a},\vec{b}\in\mathbb{R}^n let  \vec{v}_1 be the projection of  \vec{a} onto the line spanned by  \vec{b} , let  \vec{v}_2 be the projection of  \vec{v}_1 onto the line spanned by  \vec{a} , let  \vec{v}_3 be the projection of  \vec{v}_2 onto the line spanned by  \vec{b} , etc., back and forth between the spans of \vec{a} and \vec{b}. That is, \vec{v}_{i+1} is the projection of \vec{v}_i onto the span of \vec{a} if i+1 is even, and onto the span of \vec{b} if i+1 is odd. Must that sequence of vectors eventually settle down— must there be a sufficiently large i such that  \vec{v}_{i+2} equals  \vec{v}_{i} and  \vec{v}_{i+3} equals  \vec{v}_{i+1} ? If so, what is the earliest such i?

Answer

The sequence need not settle down. With


\vec{a}=\begin{pmatrix} 1 \\ 0 \end{pmatrix}
\qquad
\vec{b}=\begin{pmatrix} 1 \\ 1 \end{pmatrix}

the projections are these.


\vec{v}_1=\begin{pmatrix} 1/2 \\ 1/2 \end{pmatrix},
\quad
\vec{v}_2=\begin{pmatrix} 1/2 \\ 0 \end{pmatrix},
\quad
\vec{v}_3=\begin{pmatrix} 1/4 \\ 1/4 \end{pmatrix},
\quad
\ldots

This sequence doesn't repeat.

Linalg projection sequence.png