Linear Algebra/Inverses

From Wikibooks, the open-content textbooks collection

Current revision (unreviewed)

We now consider how to represent the inverse of a linear map.

We start by recalling some facts about function inverses.^[1] Some functions have no inverse, or have an inverse on the left side or right side only.

Example 4.1

Where $\pi:\mathbb{R}^3\to \mathbb{R}^2$ is the projection map

$\begin{pmatrix} x \\ y \\ z \end{pmatrix} \mapsto \begin{pmatrix} x \\ y \end{pmatrix}$

and $\eta:\mathbb{R}^2\to \mathbb{R}^3$ is the embedding

$\begin{pmatrix} x \\ y \end{pmatrix} \mapsto \begin{pmatrix} x \\ y \\ 0 \end{pmatrix}$

the composition $\pi\circ \eta$ is the identity map on $\mathbb{R}^2$ .

$\begin{pmatrix} x \\ y \end{pmatrix} \stackrel{\eta}{\longmapsto} \begin{pmatrix} x \\ y \\ 0 \end{pmatrix} \stackrel{\pi}{\longmapsto} \begin{pmatrix} x \\ y \end{pmatrix}$

We say $π$ is a left inverse map of $η$ or, what is the same thing, that $η$ is a right inverse map of $π$ . However, composition in the other order $\eta\circ \pi$ doesn't give the identity map— here is a vector that is not sent to itself under $\eta\circ \pi$ .

$\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \stackrel{\pi}{\longmapsto} \begin{pmatrix} 0 \\ 0 \end{pmatrix} \stackrel{\eta}{\longmapsto} \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}$

In fact, the projection $π$ has no left inverse at all. For, if $f$ were to be a left inverse of $π$ then we would have

$\begin{pmatrix} x \\ y \\ z \end{pmatrix} \stackrel{\pi}{\longmapsto} \begin{pmatrix} x \\ y \end{pmatrix} \stackrel{f}{\longmapsto} \begin{pmatrix} x \\ y \\ z \end{pmatrix}$

for all of the infinitely many $z$ 's. But no function $f$ can send a single argument to more than one value.

(An example of a function with no inverse on either side is the zero transformation on $\mathbb{R}^2$ .) Some functions have a two-sided inverse map, another function that is the inverse of the first, both from the left and from the right. For instance, the map given by $\vec{v}\mapsto 2\cdot \vec{v}$ has the two-sided inverse $\vec{v}\mapsto (1/2)\cdot\vec{v}$ . In this subsection we will focus on two-sided inverses. The appendix shows that a function has a two-sided inverse if and only if it is both one-to-one and onto. The appendix also shows that if a function $f$ has a two-sided inverse then it is unique, and so it is called "the" inverse, and is denoted $f - 1$ . So our purpose in this subsection is, where a linear map $h$ has an inverse, to find the relationship between $Rep B, D (h)$ and $Rep D, B (h - 1)$ (recall that we have shown, in Theorem II.2.21 of Section II of this chapter, that if a linear map has an inverse then the inverse is a linear map also).

Definition 4.2

A matrix $G$ is a left inverse matrix of the matrix $H$ if $G H$ is the identity matrix. It is a right inverse matrix if $H G$ is the identity. A matrix $H$ with a two-sided inverse is an invertible matrix. That two-sided inverse is called the inverse matrix and is denoted $H - 1$ .

Because of the correspondence between linear maps and matrices, statements about map inverses translate into statements about matrix inverses.

Lemma 4.3

If a matrix has both a left inverse and a right inverse then the two are equal.

Theorem 4.4

A matrix is invertible if and only if it is nonsingular.

Proof

(For both results.) Given a matrix $H$ , fix spaces of appropriate dimension for the domain and codomain. Fix bases for these spaces. With respect to these bases, $H$ represents a map $h$ . The statements are true about the map and therefore they are true about the matrix.

Lemma 4.5

A product of invertible matrices is invertible— if $G$ and $H$ are invertible and if $G H$ is defined then $G H$ is invertible and $(G H) - 1 = H - 1 G - 1$ .

Proof

(This is just like the prior proof except that it requires two maps.) Fix appropriate spaces and bases and consider the represented maps $h$ and $g$ . Note that $h - 1 g - 1$ is a two-sided map inverse of $g h$ since $(h - 1 g - 1)(g h) = h - 1 (id) h = h - 1 h = id$ and $(g h)(h - 1 g - 1) = g (id) g - 1 = g g - 1 = id$ . This equality is reflected in the matrices representing the maps, as required.

Here is the arrow diagram giving the relationship between map inverses and matrix inverses. It is a special case of the diagram for function composition and matrix multiplication.

Beyond its place in our general program of seeing how to represent map operations, another reason for our interest in inverses comes from solving linear systems. A linear system is equivalent to a matrix equation, as here.

$\begin{array}{*{2}{rc}r} x_1 &+ &x_2 &= &3 \\ 2x_1 &- &x_2 &= &2 \end{array} \quad\Longleftrightarrow\quad \begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 3 \\ 2 \end{pmatrix} \qquad\qquad (*)$

By fixing spaces and bases (e.g., $\mathbb{R}^2,\mathbb{R}^2$ and $\mathcal{E}_2,\mathcal{E}_2$ ), we take the matrix $H$ to represent some map $h$ . Then solving the system is the same as asking: what domain vector $\vec{x}$ is mapped by $h$ to the result $\vec{d}\,$ ? If we could invert $h$ then we could solve the system by multiplying ${\rm Rep}_{D,B}(h^{-1})\cdot{\rm Rep}_{D}(\vec{d})$ to get ${\rm Rep}_{B}(\vec{x})$ .

Example 4.6

We can find a left inverse for the matrix just given

$\begin{pmatrix} m &n \\ p &q \end{pmatrix} \begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix} = \begin{pmatrix} 1 &0 \\ 0 &1 \end{pmatrix}$

by using Gauss' method to solve the resulting linear system.

$\begin{array}{*{4}{rc}r} m &+ &2n & & & & &= &1 \\ m &- &n & & & & &= &0 \\ & & & &p &+ &2q &= &0 \\ & & & &p &- &q &= &1 \end{array}$

Answer: $m = 1 / 3$ , $n = 1 / 3$ , $p = 2 / 3$ , and $q = - 1 / 3$ . This matrix is actually the two-sided inverse of $H$ , as can easily be checked. With it we can solve the system ( $*$ ) above by applying the inverse.

$\begin{pmatrix} x \\ y \end{pmatrix} =\begin{pmatrix} 1/3 &1/3 \\ 2/3 &-1/3 \end{pmatrix} \begin{pmatrix} 3 \\ 2 \end{pmatrix} =\begin{pmatrix} 5/3 \\ 4/3 \end{pmatrix}$

Remark 4.7

Why solve systems this way, when Gauss' method takes less arithmetic (this assertion can be made precise by counting the number of arithmetic operations, as computer algorithm designers do)? Beyond its conceptual appeal of fitting into our program of discovering how to represent the various map operations, solving linear systems by using the matrix inverse has at least two advantages.

First, once the work of finding an inverse has been done, solving a system with the same coefficients but different constants is easy and fast: if we change the entries on the right of the system ( $*$ ) then we get a related problem

$\begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 5 \\ 1 \end{pmatrix}$

with a related solution method.

$\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 1/3 &1/3 \\ 2/3 &-1/3 \end{pmatrix} \begin{pmatrix} 5 \\ 1 \end{pmatrix} = \begin{pmatrix} 2 \\ 3 \end{pmatrix}$

In applications, solving many systems having the same matrix of coefficients is common.

Another advantage of inverses is that we can explore a system's sensitivity to changes in the constants. For example, tweaking the $3$ on the right of the system ( $*$ ) to

$\begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 3.01 \\ 2 \end{pmatrix}$

can be solved with the inverse.

$\begin{pmatrix} 1/3 &1/3 \\ 2/3 &-1/3 \end{pmatrix} \begin{pmatrix} 3.01 \\ 2 \end{pmatrix} = \begin{pmatrix} (1/3)(3.01)+(1/3)(2) \\ (2/3)(3.01)-(1/3)(2) \end{pmatrix}$

to show that $x 1$ changes by $1 / 3$ of the tweak while $x 2$ moves by $2 / 3$ of that tweak. This sort of analysis is used, for example, to decide how accurately data must be specified in a linear model to ensure that the solution has a desired accuracy.

We finish by describing the computational procedure usually used to find the inverse matrix.

Lemma 4.8

A matrix is invertible if and only if it can be written as the product of elementary reduction matrices. The inverse can be computed by applying to the identity matrix the same row steps, in the same order, as are used to Gauss-Jordan reduce the invertible matrix.

Proof

A matrix $H$ is invertible if and only if it is nonsingular and thus Gauss-Jordan reduces to the identity. By Corollary 3.22 this reduction can be done with elementary matrices $R_r\cdot R_{r-1}\dots R_1\cdot H=I$ . This equation gives the two halves of the result.

First, elementary matrices are invertible and their inverses are also elementary. Applying $R_r^{-1}$ to the left of both sides of that equation, then $R_{r-1}^{-1}$ , etc., gives $H$ as the product of elementary matrices $H=R_1^{-1}\cdots R_r^{-1}\cdot I$ (the $I$ is here to cover the trivial $r = 0$ case).

Second, matrix inverses are unique and so comparison of the above equation with $H - 1 H = I$ shows that $H^{-1}=R_r\cdot R_{r-1}\dots R_1\cdot I$ . Therefore, applying $R 1$ to the identity, followed by $R 2$ , etc., yields the inverse of $H$ .

Example 4.9

To find the inverse of

$\begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix}$

we do Gauss-Jordan reduction, meanwhile performing the same operations on the identity. For clerical convenience we write the matrix and the identity side-by-side, and do the reduction steps together.

$\begin{array}{rcl} \left(\begin{array}{cc|cc} 1 &1 &1 &0 \\ 2 &-1 &0 &1 \end{array}\right) &\xrightarrow[]{-2\rho_1+\rho_2} &\left(\begin{array}{cc|cc} 1 &1 &1 &0 \\ 0 &-3 &-2 &1 \end{array}\right) \\ &\xrightarrow[]{-1/3\rho_2} &\left(\begin{array}{cc|cc} 1 &1 &1 &0 \\ 0 &1 &2/3 &-1/3 \end{array}\right) \\ &\xrightarrow[]{-\rho_2+\rho_1} &\left(\begin{array}{cc|cc} 1 &0 &1/3 &1/3 \\ 0 &1 &2/3 &-1/3 \end{array}\right) \end{array}$

This calculation has found the inverse.

$\begin{pmatrix} 1 &1 \\ 2 &-1 \end{pmatrix}^{-1} = \begin{pmatrix} 1/3 &1/3 \\ 2/3 &-1/3 \end{pmatrix}$

Example 4.10

This one happens to start with a row swap.

$\begin{array}{rcl} \left(\begin{array}{ccc|ccc} 0 &3 &-1 &1 &0 &0 \\ 1 &0 &1 &0 &1 &0 \\ 1 &-1 &0 &0 &0 &1 \end{array}\right) &\xrightarrow[]{\rho_1\leftrightarrow\rho_2} & \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &3 &-1 &1 &0 &0 \\ 1 &-1 &0 &0 &0 &1 \end{array}\right) \\ &\xrightarrow[]{-\rho_1+\rho_3} & \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &3 &-1 &1 &0 &0 \\ 0 &-1 &-1 &0 &-1 &1 \end{array}\right) \\ &\vdots \\ &\xrightarrow[]{} & \left(\begin{array}{ccc|ccc} 1 &0 &0 &1/4 &1/4 &3/4 \\ 0 &1 &0 &1/4 &1/4 &-1/4 \\ 0 &0 &1 &-1/4 &3/4 &-3/4 \end{array}\right) \end{array}$

Example 4.11

A non-invertible matrix is detected by the fact that the left half won't reduce to the identity.

$\left(\begin{array}{cc|cc} 1 &1 &1 &0 \\ 2 &2 &0 &1 \end{array}\right) \xrightarrow[]{-2\rho_1+\rho_2} \left(\begin{array}{cc|cc} 1 &1 &1 &0 \\ 0 &0 &-2 &1 \end{array}\right)$

This procedure will find the inverse of a general $n \! \times \! n$ matrix. The $2 \! \times \! 2$ case is handy.

Corollary 4.12

The inverse for a $2 \! \times \! 2$ matrix exists and equals

$\begin{pmatrix} a &b \\ c &d \end{pmatrix}^{-1} = \frac{1}{ad-bc} \begin{pmatrix} d &-b \\ -c &a \end{pmatrix}$

if and only if $ad-bc\neq 0$ .

Proof

This computation is Problem 10.

We have seen here, as in the Mechanics of Matrix Multiplication subsection, that we can exploit the correspondence between linear maps and matrices. So we can fruitfully study both maps and matrices, translating back and forth to whichever helps us the most.

Over the entire four subsections of this section we have developed an algebra system for matrices. We can compare it with the familiar algebra system for the real numbers. Here we are working not with numbers but with matrices. We have matrix addition and subtraction operations, and they work in much the same way as the real number operations, except that they only combine same-sized matrices. We also have a matrix multiplication operation and an operation inverse to multiplication. These are somewhat like the familiar real number operations (associativity, and distributivity over addition, for example), but there are differences (failure of commutativity, for example). And, we have scalar multiplication, which is in some ways another extension of real number multiplication. This matrix system provides an example that algebra systems other than the elementary one can be interesting and useful.

[edit] Exercises

Problem 1

Supply the intermediate steps in Example 4.10.

Answer

Here is one way to proceed.

$\xrightarrow[]{\rho_1\leftrightarrow\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &3 &-1 &1 &0 &0 \\ 1 &-1 &0 &0 &0 &1 \end{array}\right) \;\xrightarrow[]{-\rho_1+\rho_3}\; \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &3 &-1 &1 &0 &0 \\ 0 &-1 &-1 &0 &-1 &1 \end{array}\right)$

$\begin{align} &\xrightarrow[]{(1/3)\rho_2+\rho_3}\; \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &3 &-1 &1 &0 &0 \\ 0 &0 &-4/3 &1/3 &-1 &1 \end{array}\right) \;\xrightarrow[-(3/4)\rho_3]{(1/3)\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &0 &1 &0 &1 &0 \\ 0 &1 &-1/3 &1/3 &0 &0 \\ 0 &0 &1 &-1/4 &3/4 &-3/4 \end{array}\right) \\ &\xrightarrow[-\rho_3+\rho_1]{(1/3)\rho_3+\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &0 &0 &1/4 &1/4 &3/4 \\ 0 &1 &0 &1/4 &1/4 &-1/4 \\ 0 &0 &1 &-1/4 &3/4 &-3/4 \end{array}\right) \end{align}$

This exercise is recommended for all readers.

Problem 2

Use Corollary 4.12 to decide if each matrix has an inverse.

$\begin{pmatrix} 2 &1 \\ -1 &1 \end{pmatrix}$
$\begin{pmatrix} 0 &4 \\ 1 &-3 \end{pmatrix}$
$\begin{pmatrix} 2 &-3 \\ -4 &6 \end{pmatrix}$

Answer

Yes, it has an inverse: $ad-bc=2\cdot 1-1\cdot(-1)\neq 0$ .
Yes.
No.

This exercise is recommended for all readers.

Problem 3

For each invertible matrix in the prior problem, use Corollary 4.12 to find its inverse.

Answer

$\displaystyle \frac{1}{2\cdot 1-1\cdot (-1)} \cdot\begin{pmatrix} 1 &-1 \\ 1 &2 \end{pmatrix} =\frac{\displaystyle 1}{\displaystyle 3}\cdot\begin{pmatrix} 1 &-1 \\ 1 &2 \end{pmatrix} =\begin{pmatrix} 1/3 &-1/3 \\ 1/3 &2/3 \end{pmatrix}$
$\displaystyle \frac{1}{0\cdot (-3)-4\cdot 1} \cdot\begin{pmatrix} -3 &-4 \\ -1 &0 \end{pmatrix} =\begin{pmatrix} 3/4 &1 \\ 1/4 &0 \end{pmatrix}$
The prior question shows that no inverse exists.

This exercise is recommended for all readers.

Problem 4

Find the inverse, if it exists, by using the Gauss-Jordan method. Check the answers for the $2 \! \times \! 2$ matrices with Corollary 4.12.

$\begin{pmatrix} 3 &1 \\ 0 &2 \end{pmatrix}$
$\begin{pmatrix} 2 &1/2 \\ 3 &1 \end{pmatrix}$
$\begin{pmatrix} 2 &-4 \\ -1 &2 \end{pmatrix}$
$\begin{pmatrix} 1 &1 &3 \\ 0 &2 &4 \\ -1 &1 &0 \end{pmatrix}$
$\begin{pmatrix} 0 &1 &5 \\ 0 &-2 &4 \\ 2 &3 &-2 \end{pmatrix}$
$\begin{pmatrix} 2 &2 &3 \\ 1 &-2 &-3 \\ 4 &-2 &-3 \end{pmatrix}$

Answer

The reduction is routine.

$\left(\begin{array}{cc|cc} 3 &1 &1 &0 \\ 0 &2 &0 &1 \end{array}\right) \;\xrightarrow[(1/2)\rho_2]{(1/3)\rho_1}\; \left(\begin{array}{cc|cc} 1 &1/3 &1/3 &0 \\ 0 &1 &0 &1/2 \end{array}\right) \;\xrightarrow[]{-(1/3)\rho_2+\rho_1}\; \left(\begin{array}{cc|cc} 1 &0 &1/3 &-1/6 \\ 0 &1 &0 &1/2 \end{array}\right)$

This answer agrees with the answer from the check.

$\begin{pmatrix} 3 &1 \\ 0 &2 \end{pmatrix}^{-1} =\frac{1}{3\cdot 2-0\cdot 1}\cdot \begin{pmatrix} 2 &-1 \\ 0 &3 \end{pmatrix} =\frac{1}{6}\cdot \begin{pmatrix} 2 &-1 \\ 0 &3 \end{pmatrix}$
This reduction is easy. $\left(\begin{array}{cc|cc} 2 &1/2 &1 &0 \\ 3 &1 &0 &1 \end{array}\right) \;\xrightarrow[]{-(3/2)\rho_1+\rho_2}\; \left(\begin{array}{cc|cc} 2 &1/2 &1 &0 \\ 0 &1/4 &-3/2 &1 \end{array}\right)$

$\;\xrightarrow[4\rho_2]{(1/2)\rho_1}\; \left(\begin{array}{cc|cc} 1 &1/4 &1/2 &0 \\ 0 &1 &-6 &4 \end{array}\right) \;\xrightarrow[]{-(1/4)\rho_2+\rho_1}\; \left(\begin{array}{cc|cc} 1 &0 &2 &-1 \\ 0 &1 &-6 &4 \end{array}\right)$

The check agrees.

$\frac{1}{2\cdot 1-3\cdot (1/2)}\cdot \begin{pmatrix} 1 &-1/2 \\ -3 &2 \end{pmatrix} =2\cdot \begin{pmatrix} 1 &-1/2 \\ -3 &2 \end{pmatrix}$
Trying the Gauss-Jordan reduction

$\left(\begin{array}{cc|cc} 2 &-4 &1 &0 \\ -1 &2 &0 &1 \end{array}\right) \;\xrightarrow[]{(1/2)\rho_1+\rho_2}\; \left(\begin{array}{cc|cc} 2 &-4 &1 &0 \\ 0 &0 &1/2 &1 \end{array}\right)$

shows that the left side won't reduce to the identity, so no inverse exists. The check $ad-bc=2\cdot 2-(-4)\cdot (-1)=0$ agrees.
This produces an inverse.

$\left(\begin{array}{ccc|ccc} 1 &1 &3 &1 &0 &0 \\ 0 &2 &4 &0 &1 &0 \\ -1 &1 &0 &0 &0 &1 \end{array}\right) \;\xrightarrow[]{\rho_1+\rho_3}\; \left(\begin{array}{ccc|ccc} 1 &1 &3 &1 &0 &0 \\ 0 &2 &4 &0 &1 &0 \\ 0 &2 &3 &1 &0 &1 \end{array}\right) \;\xrightarrow[]{-\rho_2+\rho_3}\; \left(\begin{array}{ccc|ccc} 1 &1 &3 &1 &0 &0 \\ 0 &2 &4 &0 &1 &0 \\ 0 &0 &-1 &1 &-1 &1 \end{array}\right)$

$\begin{align} &\xrightarrow[-\rho_3]{(1/2)\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &1 &3 &1 &0 &0 \\ 0 &1 &2 &0 &1/2 &0 \\ 0 &0 &1 &-1 &1 &-1 \end{array}\right) \;\xrightarrow[-3\rho_3+\rho_1]{-2\rho_3+\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &1 &0 &4 &-3 &3 \\ 0 &1 &0 &2 &-3/2 &2 \\ 0 &0 &1 &-1 &1 &-1 \end{array}\right) \\ &\xrightarrow[]{-\rho_2+\rho_1}\; \left(\begin{array}{ccc|ccc} 1 &0 &0 &2 &-3/2 &1 \\ 0 &1 &0 &2 &-3/2 &2 \\ 0 &0 &1 &-1 &1 &-1 \end{array}\right) \end{align}$
This is one way to do the reduction.

$\left(\begin{array}{ccc|ccc} 0 &1 &5 &1 &0 &0 \\ 0 &-2 &4 &0 &1 &0 \\ 2 &3 &-2 &0 &0 &1 \end{array}\right) \;\xrightarrow[]{\rho_3\leftrightarrow\rho_1}\; \left(\begin{array}{ccc|ccc} 2 &3 &-2 &0 &0 &1 \\ 0 &-2 &4 &0 &1 &0 \\ 0 &1 &5 &1 &0 &0 \end{array}\right)$

$\begin{align} &\;\xrightarrow[]{(1/2)\rho_2+\rho_3}\; \left(\begin{array}{ccc|ccc} 2 &3 &-2 &0 &0 &1 \\ 0 &-2 &4 &0 &1 &0 \\ 0 &0 &7 &1 &1/2 &0 \end{array}\right) \xrightarrow[\begin{array}{c}\\[-19pt]\scriptstyle -(1/2)\rho_2 \\[-5pt] \scriptstyle (1/7)\rho_3\end{array}]{(1/2)\rho_1}\; \left(\begin{array}{ccc|ccc} 1 &3/2 &-1 &0 &0 &1/2 \\ 0 &1 &-2 &0 &-1/2 &0 \\ 0 &0 &1 &1/7 &1/14 &0 \end{array}\right) \\ &\;\xrightarrow[\rho_3+\rho_1]{2\rho_3+\rho_2}\; \left(\begin{array}{ccc|ccc} 1 &3/2 &0 &1/7 &1/14 &1/2 \\ 0 &1 &0 &2/7 &-5/14 &0 \\ 0 &0 &1 &1/7 &1/14 &0 \end{array}\right) \xrightarrow[]{-(3/2)\rho_2+\rho_1}\; \left(\begin{array}{ccc|ccc} 1 &0 &0 &-2/7 &17/28 &1/2 \\ 0 &1 &0 &2/7 &-5/14 &0 \\ 0 &0 &1 &1/7 &1/14 &0 \end{array}\right) \end{align}$
There is no inverse.

$\begin{array}{rl} \left(\begin{array}{ccc|ccc} 2 &2 &3 &1 &0 &0 \\ 1 &-2 &-3 &0 &1 &0 \\ 4 &-2 &-3 &0 &0 &1 \end{array}\right) &\;\xrightarrow[-2\rho_1+\rho_3]{-(1/2)\rho_1+\rho_2}\; \left(\begin{array}{ccc|ccc} 2 &2 &3 &1 &0 &0 \\ 0 &-3 &-9/2 &-1/2 &1 &0 \\ 0 &-6 &-9 &-2 &0 &1 \end{array}\right) \\ &\xrightarrow[]{-2\rho_2+\rho_3}\; \left(\begin{array}{ccc|ccc} 2 &2 &3 &1 &0 &0 \\ 0 &-3 &-9/2 &-1/2 &1 &0 \\ 0 &0 &0 &-1 &-2 &1 \end{array}\right) \end{array}$

As a check, note that the third column of the starting matrix is $3 / 2$ times the second, and so it is indeed singular and therefore has no inverse.

This exercise is recommended for all readers.

Problem 5

What matrix has this one for its inverse?

$\begin{pmatrix} 1 &3 \\ 2 &5 \end{pmatrix}$

Answer

We can use Corollary 4.12.

$\frac{1}{1\cdot 5-2\cdot 3}\cdot \begin{pmatrix} 5 &-3 \\ -2 &1 \end{pmatrix} =\begin{pmatrix} -5 &3 \\ 2 &-1 \end{pmatrix}$

Problem 6

How does the inverse operation interact with scalar multiplication and addition of matrices?

What is the inverse of $r H$ ?
Is $(H + G) - 1 = H - 1 + G - 1$ ?

Answer

The proof that the inverse is $r^{-1}H^{-1}=(1/r)\cdot H^{-1}$ (provided, of course, that the matrix is invertible) is easy.
No. For one thing, the fact that $H + G$ has an inverse doesn't imply that $H$ has an inverse or that $G$ has an inverse. Neither of these matrices is invertible but their sum is.

$\begin{pmatrix} 1 &0 \\ 0 &0 \end{pmatrix} \qquad \begin{pmatrix} 0 &0 \\ 0 &1 \end{pmatrix}$

Another point is that just because $H$ and $G$ each has an inverse doesn't mean $H + G$ has an inverse; here is an example.

$\begin{pmatrix} 1 &0 \\ 0 &1 \end{pmatrix} \qquad \begin{pmatrix} -1 &0 \\ 0 &-1 \end{pmatrix}$

Still a third point is that, even if the two matrices have inverses, and the sum has an inverse, doesn't imply that the equation holds:

$\begin{pmatrix} 2 &0 \\ 0 &2 \end{pmatrix}^{-1} =\begin{pmatrix} 1/2 &0 \\ 0 &1/2 \end{pmatrix}^{-1} \qquad \begin{pmatrix} 3 &0 \\ 0 &3 \end{pmatrix}^{-1} =\begin{pmatrix} 1/3 &0 \\ 0 &1/3 \end{pmatrix}^{-1}$

but

$\begin{pmatrix} 5 &0 \\ 0 &5 \end{pmatrix}^{-1} =\begin{pmatrix} 1/5 &0 \\ 0 &1/5 \end{pmatrix}^{-1}$

and $(1 / 2) + (1 / 3)$ does not equal $1 / 5$ .

This exercise is recommended for all readers.

Problem 7

Is $(T k) - 1 = (T - 1) k$ ?

Answer

Yes: $T^k(T^{-1})^k=(TT\cdots T)\cdot (T^{-1}T^{-1}\cdots T^{-1}) =T^{k-1}(TT^{-1})(T^{-1})^{k-1}=\dots=I$ .

Problem 8

Is $H - 1$ invertible?

Answer

Yes, the inverse of $H - 1$ is $H$ .

Problem 9

For each real number $θ$ let $t_\theta:\mathbb{R}^2\to \mathbb{R}^2$ be represented with respect to the standard bases by this matrix.

$\begin{pmatrix} \cos\theta &-\sin\theta \\ \sin\theta &\cos\theta \end{pmatrix}$

Show that $t_{\theta_1+\theta_2}=t_{\theta_1}\cdot t_{\theta_2}$ . Show also that ${t_{\theta}}^{-1}=t_{-\theta}$ .

Answer

One way to check that the first is true is with the angle sum formulas from trigonometry.

$\begin{array}{rl} \begin{pmatrix} \cos(\theta_1+\theta_2) &-\sin(\theta_1+\theta_2) \\ \sin(\theta_1+\theta_2) &\cos(\theta_1+\theta_2) \end{pmatrix} &=\begin{pmatrix} \cos\theta_1\cos\theta_2-\sin\theta_1\sin\theta_2 &-\sin\theta_1\cos\theta_2-\cos\theta_1\sin\theta_2 \\ \sin\theta_1\cos\theta_2+\cos\theta_1\sin\theta_2 &\cos\theta_1\cos\theta_2-\sin\theta_1\sin\theta_2 \end{pmatrix} \\ &=\begin{pmatrix} \cos\theta_1 &-\sin\theta_1 \\ \sin\theta_1 &\cos\theta_1 \end{pmatrix} \begin{pmatrix} \cos\theta_2 &-\sin\theta_2 \\ \sin\theta_2 &\cos\theta_2 \end{pmatrix} \end{array}$

Checking the second equation in this way is similar.

Of course, the equations can be not just checked but also understood by recalling that $t θ$ is the map that rotates vectors about the origin through an angle of $θ$ radians.

Problem 10

Do the calculations for the proof of Corollary 4.12.

Answer

There are two cases. For the first case we assume that $a$ is nonzero. Then

$\xrightarrow[]{-(c/a)\rho_1+\rho_2} \left(\begin{array}{cc|cc} a &b &1 &0 \\ 0 &-(bc/a)+d &-c/a &1 \end{array}\right) =\left(\begin{array}{cc|cc} a &b &1 &0 \\ 0 &(ad-bc)/a &-c/a &1 \end{array}\right)$

shows that the matrix is invertible (in this $a\neq 0$ case) if and only if $ad-bc\neq 0$ . To find the inverse, we finish with the Jordan half of the reduction.

$\xrightarrow[(a/ad-bc)\rho_2]{(1/a)\rho_1} \left(\begin{array}{cc|cc} 1 &b/a &1/a &0 \\ 0 &1 &-c/(ad-bc) &a/(ad-bc) \end{array}\right) \xrightarrow[]{-(b/a)\rho_2+\rho_1} \left(\begin{array}{cc|cc} 1 &0 &d/(ad-bc) &-b/(ad-bc) \\ 0 &1 &-c/(ad-bc) &a/(ad-bc) \end{array}\right)$

The other case is the $a = 0$ case. We swap to get $c$ into the $1,1$ position.

$\xrightarrow[]{\rho_1\leftrightarrow\rho_2}\; \left(\begin{array}{cc|cc} c &d &0 &1 \\ 0 &b &1 &0 \end{array}\right)$

This matrix is nonsingular if and only if both $b$ and $c$ are nonzero (which, under the case assumption that $a = 0$ , holds if and only if $ad-bc\neq 0$ ). To find the inverse we do the Jordan half.

$\xrightarrow[(1/b)\rho_2]{(1/c)\rho_1}\; \left(\begin{array}{cc|cc} 1 &d/c &0 &1/c \\ 0 &1 &1/b &0 \end{array}\right) \;\xrightarrow[]{-(d/c)\rho_2+\rho_1}\; \left(\begin{array}{cc|cc} 1 &0 &-d/bc &1/c \\ 0 &1 &1/b &0 \end{array}\right)$

(Note that this is what is required, since $a = 0$ gives that $a d - b c = - b c$ ).

Problem 11

Show that this matrix

$H=\begin{pmatrix} 1 &0 &1 \\ 0 &1 &0 \end{pmatrix}$

has infinitely many right inverses. Show also that it has no left inverse.

Answer

With $H$ a $2 \! \times \! 3$ matrix, in looking for a matrix $G$ such that the combination $H G$ acts as the $2 \! \times \! 2$ identity we need $G$ to be $3 \! \times \! 2$ . Setting up the equation

$\begin{pmatrix} 1 &0 &1 \\ 0 &1 &0 \end{pmatrix} \begin{pmatrix} m &n \\ p &q \\ r &s \end{pmatrix} = \begin{pmatrix} 1 &0 \\ 0 &1 \end{pmatrix}$

and solving the resulting linear system

$\begin{array}{*{6}{rc}r} m & & & &+r & &= &1 \\ &n & & & &+s &= &0 \\ & &p & & & &= &0 \\ & & &q & & &= &1 \end{array}$

gives infinitely many solutions.

$\{\begin{pmatrix} m \\ n \\ p \\ q \\ r \\ s \end{pmatrix} =\begin{pmatrix} 1 \\ 0 \\ 0 \\ 1 \\ 0 \\ 0 \end{pmatrix} +r\cdot \begin{pmatrix} -1 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0 \end{pmatrix} +s\cdot \begin{pmatrix} 0 \\ -1 \\ 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} \,\big|\, r,s\in\mathbb{R}\}$

Thus $H$ has infinitely many right inverses.

As for left inverses, the equation

$\begin{pmatrix} a &b \\ c &d \end{pmatrix} \begin{pmatrix} 1 &0 &1 \\ 0 &1 &0 \end{pmatrix} =\begin{pmatrix} 1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{pmatrix}$

gives rise to a linear system with nine equations and four unknowns.

$\begin{array}{*{6}{rc}r} a & & & & & & & & & & &= &1 \\ & &b& & & & & & & & &= &0 \\ a & & & & & & & & & & &= &0 \\ & & & &c& & & & & & &= &0 \\ & & & & & &d& & & & &= &1 \\ & & & &c& & & & & & &= &0 \\ & & & & & & & &e& & &= &0 \\ & & & & & & & & & &f&= &0 \\ & & & & & & & &e& & &= &1 \end{array}$

This system is inconsistent (the first equation conflicts with the third, as do the seventh and ninth) and so there is no left inverse.

Problem 12

In Example 4.1, how many left inverses has $η$ ?

Answer

With respect to the standard bases we have

${\rm Rep}_{\mathcal{E}_2,\mathcal{E}_3}(\eta) =\begin{pmatrix} 1 &0 \\ 0 &1 \\ 0 &0 \end{pmatrix}$

and setting up the equation to find the matrix inverse

$\begin{pmatrix} a &b &c \\ d &e &f \end{pmatrix} \begin{pmatrix} 1 &0 \\ 0 &1 \\ 0 &0 \end{pmatrix} =\begin{pmatrix} 1 &0 \\ 0 &1 \end{pmatrix} ={\rm Rep}_{\mathcal{E}_2,\mathcal{E}_2}(\mbox{id})$

gives rise to a linear system.

$\begin{array}{*{6}{rc}r} a & & & & & & & & & & &= &1 \\ & &b& & & & & & & & &= &0 \\ & & & & & &d& & & & &= &0 \\ & & & & & & & &e& & &= &1 \end{array}$

There are infinitely many solutions in $a,\ldots,f$ to this system because two of these variables are entirely unrestricted

$\{\begin{pmatrix} a \\ b \\ c \\ d \\ e \\ f \end{pmatrix} =\begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0 \end{pmatrix} +c\cdot \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \\ 0 \\ 0 \end{pmatrix} +f\cdot \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} \,\big|\, c,f\in\mathbb{R}\}$

and so there are infinitely many solutions to the matrix equation.

$\{\begin{pmatrix} 1 &0 &c \\ 0 &1 &f \end{pmatrix} \,\big|\, c,f\in\mathbb{R}\}$

With the bases still fixed at $\mathcal{E}_2,\mathcal{E}_2$ , for instance taking $c = 2$ and $f = 3$ gives a matrix representing this map.

$\begin{pmatrix} x \\ y \\ z \end{pmatrix} \;\stackrel{f_{2,3}}{\longmapsto}\; \begin{pmatrix} x+2z \\ y+3z \end{pmatrix}$

The check that $f_{2,3}\circ \eta$ is the identity map on $\mathbb{R}^2$ is easy.

Problem 13

If a matrix has infinitely many right-inverses, can it have infinitely many left-inverses? Must it have?

Answer

By Lemma 4.3 it cannot have infinitely many left inverses, because a matrix with both left and right inverses has only one of each (and that one of each is one of both— the left and right inverse matrices are equal).

This exercise is recommended for all readers.

Problem 14

Assume that $H$ is invertible and that $H G$ is the zero matrix. Show that $G$ is a zero matrix.

Answer

The associativity of matrix multiplication gives on the one hand $H - 1 (H G) = H - 1 Z = Z$ , and on the other that $H - 1 (H G) = (H - 1 H) G = I G = G$ .

Problem 15

Prove that if $H$ is invertible then the inverse commutes with a matrix $G H - 1 = H - 1 G$ if and only if $H$ itself commutes with that matrix $G H = H G$ .

Answer

Multiply both sides of the first equation by $H$ .

This exercise is recommended for all readers.

Problem 16

Show that if $T$ is square and if $T 4$ is the zero matrix then $(I - T) - 1 = I + T + T 2 + T 3$ . Generalize.

Answer

Checking that when $I - T$ is multiplied on both sides by that expression (assuming that $T 4$ is the zero matrix) then the result is the identity matrix is easy. The obvious generalization is that if $T n$ is the zero matrix then $(I-T)^{-1}=I+T+T^2+\cdots+T^{n-1}$ ; the check again is easy.

This exercise is recommended for all readers.

Problem 17

Let $D$ be diagonal. Describe $D 2$ , $D 3$ , ... , etc. Describe $D - 1$ , $D - 2$ , ... , etc. Define $D 0$ appropriately.

Answer

The powers of the matrix are formed by taking the powers of the diagonal entries. That is, $D 2$ is all zeros except for diagonal entries of ${d_{1,1}}^2$ , ${d_{2,2}}^2$ , etc. This suggests defining $D 0$ to be the identity matrix.

Problem 18

Prove that any matrix row-equivalent to an invertible matrix is also invertible.

Answer

Assume that $B$ is row equivalent to $A$ and that $A$ is invertible. Because they are row-equivalent, there is a sequence of row steps to reduce one to the other. That reduction can be done with matrices, for instance, $A$ can be changed by row operations to $B$ as $B=R_n\cdots R_1A$ . This equation gives $B$ as a product of invertible matrices and by Lemma 4.5 then, $B$ is also invertible.

Problem 19

The first question below appeared as Problem 15 in the Matrix Multiplication subsection.

Show that the rank of the product of two matrices is less than or equal to the minimum of the rank of each.
Show that if $T$ and $S$ are square then $T S = I$ if and only if $S T = I$ .

Answer

See the answer to Problem 15 in the Matrix Multiplication subsection.
We will show that both conditions are equivalent to the condition that the two matrices be nonsingular. As $T$ and $S$ are square and their product is defined, they are equal-sized, say $n \! \times \! n$ . Consider the $T S = I$ half. By the prior item the rank of $I$ is less than or equal to the minimum of the rank of $T$ and the rank of $S$ . But the rank of $I$ is $n$ , so the rank of $T$ and the rank of $S$ must each be $n$ . Hence each is nonsingular. The same argument shows that $S T = I$ implies that each is nonsingular.

Problem 20

Show that the inverse of a permutation matrix is its transpose.

Answer

Inverses are unique, so we need only show that it works. The check appears above as Problem 9 of the Mechanics of Matrix Multiplication subsection.

Problem 21

The first two parts of this question appeared as Problem 12. of the Matrix Multiplication subsection

Show that $(G H) trans = H trans G trans$ .
A square matrix is symmetric if each $i, j$ entry equals the $j, i$ entry (that is, if the matrix equals its transpose). Show that the matrices $H H trans$ and $H trans H$ are symmetric.
Show that the inverse of the transpose is the transpose of the inverse.
Show that the inverse of a symmetric matrix is symmetric.

Answer

See the answer for Problem 12 of the Matrix Multiplication subsection.
See the answer for Problem 12 of the Matrix Multiplication subsection.
Apply the first part to $I = A A - 1$ to get $I={{I}^{\rm trans}}={{(AA^{-1})}^{\rm trans}}={{(A^{-1})}^{\rm trans}}{{A}^{\rm trans}}$ .
Apply the prior item with $A trans = A$ , as $A$ is symmetric.

This exercise is recommended for all readers.

Problem 22

The items starting this question appeared as Problem 17 of the Matrix Multiplication subsection.

Prove that the composition of the projections $\pi_x,\pi_y:\mathbb{R}^3\to \mathbb{R}^3$ is the zero map despite that neither is the zero map.
Prove that the composition of the derivatives $d^2/dx^2,\,d^3/dx^3:\mathcal{P}_4\to \mathcal{P}_4$ is the zero map despite that neither map is the zero map.
Give matrix equations representing each of the prior two items.

When two things multiply to give zero despite that neither is zero, each is said to be a zero divisor. Prove that no zero divisor is invertible.

Answer

For the answer to the items making up the first half, see Problem 17 of the Matrix Multiplication subsection. For the proof in the second half, assume that $A$ is a zero divisor so there is a nonzero matrix $B$ with $A B = Z$ (or else $B A = Z$ ; this case is similar), If $A$ is invertible then $A - 1 (A B) = (A - 1 A) B = I B = B$ but also $A - 1 (A B) = A - 1 Z = Z$ , contradicting that $B$ is nonzero.

Problem 23

In real number algebra, there are exactly two numbers, $1$ and $- 1$ , that are their own multiplicative inverse. Does $H 2 = I$ have exactly two solutions for $2 \! \times \! 2$ matrices?

Answer

No, there are at least four.

$\begin{pmatrix} \pm 1 &0 \\ 0 &\pm 1 \end{pmatrix}$

Problem 24

Is the relation "is a two-sided inverse of" transitive? Reflexive? Symmetric?

Answer

It is not reflexive since, for instance,

$H=\begin{pmatrix} 1 &0 \\ 0 &2 \end{pmatrix}$

is not a two-sided inverse of itself. The same example shows that it is not transitive. That matrix has this two-sided inverse

$G=\begin{pmatrix} 1 &0 \\ 0 &1/2 \end{pmatrix}$

and while $H$ is a two-sided inverse of $G$ and $G$ is a two-sided inverse of $H$ , we know that $H$ is not a two-sided inverse of $H$ . However, the relation is symmetric: if $G$ is a two-sided inverse of $H$ then $G H = I = H G$ and therefore $H$ is also a two-sided inverse of $G$ .

Problem 25

Prove: if the sum of the elements of a square matrix is $k$ , then the sum of the elements in each row of the inverse matrix is $1 / k$ . (Wilansky 1951)

Answer

This is how the answer was given in the cited source.

Let $A$ be $m \! \times \! m$ , non-singular, with the stated property. Let $B$ be its inverse. Then for $n\leq m$ ,

$1 =\sum_{r=1}^{m}\delta_{nr} =\sum_{r=1}^{m}\sum_{s=1}^{m}b_{ns}a_{sr} =\sum_{s=1}^{m}\sum_{r=1}^{m}b_{ns}a_{sr} =k\sum_{s=1}^{m}b_{ns}$

( $A$ is singular if $k = 0$ ).

[edit] Footnotes

↑ More information on function inverses is in the appendix.

[edit] References

Wilansky, Albert (Nov. 1951), "The Row-Sum of the Inverse Matrix", American Mathematical Monthly (American Mathematical Society) 58 (9): 614 .

[0] More information on function inverses is in the appendix.

[1]

Linear Algebra/Inverses

From Wikibooks, the open-content textbooks collection

[edit] Exercises

[edit] Footnotes

[edit] References

Views

Personal tools

Navigation

Search

community

Print/export

Toolbox