Vector space structure on matrices – "Math for Non-Geeks"

Derivation

Let $m,n\in \mathbb{N}$ and let $V$ be an $n$-dimensional and $W$ an $m$-dimensional $K$-vector space. We have already seen that, after choosing ordered bases, we can represent linear maps from $V$ to $W$ as matrices. So let $B$ be an ordered basis of $V$ and $C$ be an ordered basis of $W$.

The space ${\mathrm{Hom}}_K(V,W)$ of linear maps from $V$ to $W$ is also a $K$-vector space. The representing matrix of a linear map $f\in {\mathrm{Hom}}_K(V,W)$ with respect to the bases $B$ and $C$ is an $(m\times n)$-matrix $M_C^B(f)\in K^{m\times n}$. We will try now transfer the vector space structure of ${\mathrm{Hom}}_K(V,W)$ to the space $K^{m\times n}$ of $(m\times n)$-matrices over $K$.

So we ask the question: Can we find addition and scalar multiplication on $K^{m\times n}$, such that $M_C^B(f+g)=M_C^B(f)+M_C^B(g)$ and $M_C^B(\lambda f)=\lambda M_C^B(f)$ for all linear maps $f,g:V\to W$ and all $\lambda \in K$?

On $K^{m\times n}$, is there perhaps even a vector space structure, such that for all finite dimensional vector spaces $V$ and $W$ and all ordered bases $B$ of $V$ and $C$ of $W$, the mapping ${\mathrm{Hom}}_K(V,W)\to K^{m\times n};f\mapsto M_C^B(f)$ is linear?

It is best to think about these questions yourself. There is an exercise for matrix addition and one for scalar multiplication that can help you with this.

A first step is to answer this question is the following theorem:

Theorem (Bijective maps induce vector space structures)

Let $V$ be a vector space with addition $+$ and scalar multiplication $\cdot$ and $W$ be a set. Let $f:V\to W$ be a bijective mapping. Then there exists exactly one vector space structure $(\oplus ,\odot )$, on $W$, such that $f$ is linear.

Proof (Bijective maps induce vector space structures)

Proof step: Existence

For $w,w^{\prime }\in W$ and $\lambda \in K$ we define $w\oplus w^{\prime }:=f(f^{-1}(w)+f^{-1}(w^{\prime }))$, $\lambda \odot w:=f(\lambda \cdot f^{-1}(w))$.

$W$ is closed under these operations, since $f$ always returns us to $W$. That $W$ forms a vector space with these operations follows directly from the vector space structure of $V$. One can view $f$ simply as a renaming of the elements of $V$.

For example, commutativity of the addition on $W$ follows from commutativity of the addition on $V$ as follows: $w\oplus w^{\prime }=f(f^{-1}(w)+f^{-1}(w^{\prime }))=f(f^{-1}(w^{\prime })+f^{-1}(w))=w^{\prime }\oplus w$.

Associativity of the addition on $W$ also follows from associativity of the addition on $V$:

$\begin{array}{cc}& w_1\oplus (w_2\oplus w_3)\\ & \downarrow \text{definition of addition on }W\\ & =f(f^{-1}(w_1)+f^{-1}(w_2\oplus w_3))\\ & \downarrow \text{definition of addition on }W\\ & =f(f^{-1}(w_1)+f^{-1}(f(f^{-1}(w_2)+f^{-1}(w_3))))\\ & \downarrow f^{-1}\circ f\text{ is the identity}\\ & =f(f^{-1}(w_1)+(f^{-1}(w_2)+f^{-1}(w_3)))\\ & \downarrow \text{associativity of the addition on }V\\ & =f((f^{-1}(w_1)+f^{-1}(w_2))+f^{-1}(w_3))\\ & \downarrow f^{-1}\circ f\text{ is the identity}\\ & =f(f^{-1}(f((f^{-1}(w_1)+f^{-1}(w_2))+f^{-1}(w_3))))\\ & \downarrow \text{definition of addition on }W\\ & =f(f^{-1}(w_1\oplus w_2)+f^{-1}(w_3))\\ & \downarrow \text{definition of addition on }W\\ & =(w_1\oplus w_2)\oplus w_3.\end{array}$

The establishment of the other vector space axioms work analogously. Thus, we have found a vector space structure on $W$. Let us now show that $f$ is linear with respect to $(\oplus ,\odot )$. Since $f$ is bijective, it suffices to show that the inverse map with respect to $f$ is linear (see isomorphism ). We have $f^{-1}(w\oplus w^{\prime })=f^{-1}(f(f^{-1}(w)+f^{-1}(w^{\prime })))=f^{-1}(w)+f^{-1}(w^{\prime })$ and $f^{-1}(\lambda \odot w)=f^1(f(\lambda \cdot f^{-1}(w^{\prime })))=\lambda \cdot f^{-1}(w^{\prime })$. Thus $f^{-1}:(W,\oplus ,\odot )\to (V,+,\cdot )$ is linear and hence $f:(V,+,\cdot )\to (W,\oplus ,\odot )$ is also linear.

Proof step: Uniqueness

Uniqueness: Suppose we have a vector space structure $(\oplus ,\odot )$ such that $f:(V,+,\cdot )\to (W,\oplus ,\odot )$ is linear. Then $f^{-1}:(W,\oplus ,\odot )\to (V,+,\cdot )$ is the inverse function of a bijective linear function and hence also linear. Therefore we have that

$w\oplus w^{\prime }=f(f^{-1}(w\oplus w^{\prime }))=f(f^{-1}(w)+f^{-1}(w^{\prime }))$,

$\lambda \odot w=f(f^{-1}(\lambda \odot w))=f(\lambda \cdot f^{-1}(w))$.

That is, any vector space structure on $W$ with respect to which $f$ is linear must be our previously defined vector space structure.

We would now like to explicitly determine the vector space structure of $K^{m\times n}$. Let $B=\{v_1,\dots ,v_n\}$ be a basis of $V$, and $C=\{w_1,\dots ,w_m\}$ a basis of $W$. We define the addition induced by $M_C^B$ on the space of matrices as in the last theorem: $A+A^{\prime }=M_C^B((M_C^B{)}^{-1}(A)+(M_C^B{)}^{-1}(A^{\prime }))$. Now let $A=(a_{ij}),A^{\prime }=(a{\text{'}}_{ij})\in K^{m\times n}$ be arbitrary and $f,g:V\to W$ be the linear maps associated with $A$ and $A^{\prime }$ with $M_C^B(f)=A,M_C^B(g)=A^{\prime }$. Then

$(b_{ij})=A+A^{\prime }=M_C^B((M_C^B{)}^{-1}(A)+(M_C^B{)}^{-1}(A^{\prime }))=M_C^B(f+g).$

We now calculate this $b_{ij}$: In the $j$-th column, $(f+g)(v_j)=\sum _{i=1}^m{b}_{ij}{w}_i$ must hold. However, by definition of $f+g$,

$(f+g)(v_j)=f(v_j)+g(v_j)=\sum _{i=1}^m{a}_{ij}{w}_i+\sum _{i=1}^{m}a{\text{'}}_{ij}{w}_i=\sum _{i=1}^m(a_{ij}+a{\text{'}}_{ij})w_j.$

Since the representation of $f(v_j)$ is unique with respect to $C$, it follows that $b_{ij}=a_{ij}+a{\text{'}}_{ij}$. That is, the addition induced by $M_C^B$ on $K^{m\times n}$ is a component-wise addition.

Let us now examine the scalar multiplication $\lambda \cdot A=M_C^B(\lambda (M_C^B{)}^{-1}(A))$ induced by $M_C^B$. Let again $f=(M_C^B{)}^{-1}(A)$ and consider $(a{\text{'}}_{ij})=A^{\prime }=\lambda \cdot A$. We have that

$(\lambda \cdot f)(v_j)=\sum _{i=1}^{m}a{\text{'}}_{ij}{w}_i.$

Furthermore, we have

$\lambda f(v_j)=\lambda \sum _{i=1}^m{a}_{ij}{w}_i=\sum _{i=1}^m\lambda a_{ij}{w}_i.$

Since $(\lambda \cdot f)(v_j)=\lambda f(v_j)$ we obtain

$\sum _{i=1}^{m}a{\text{'}}_{ij}{w}_i=\sum _{i=1}^m\lambda a_{ij}{w}_i.$

Thus, from the uniqueness of the representation it follows that $a{\text{'}}_{ij}=\lambda a_{ij}$. We see, the scalar multiplication induced from ${\mathrm{Hom}}_K(V,W)$ by $M_C^B$ on $K^{m\times n}$ is the component-wise scalar multiplication.

We also see here that the induced vector space structure is independent of our choice of $V,W,B$ and $C$.

Definition

We have just seen: To define a meaningful vector space structure on the matrices, we need to perform the operations component-wise. So we define addition and scalar multiplication as follows:

Definition (Addition of matrices)

Let $K$ be a field and let $A=(a_{ij}{)}_{i=1,\mathrm{...},m;j=1,\mathrm{...},n}$ and $B=(b_{ij}{)}_{i=1,\mathrm{...},m;j=1,\mathrm{...},n}$ be matrices of the same type $(m\times n)$ over $K$. Then

$A+B:=(a_{ij}+b_{ij}{)}_{i=1,\mathrm{...},m;j=1,\mathrm{...},n}$

Written out explicitly in terms of matrices, this definition looks as follows:

$A+B=\left(\begin{array}{ccc}{a}_{11}& \dots & a_{1n}\\ ⋮& \ddots & ⋮\\ a_{m1}& \dots & a_{mn}\end{array}\right)+\left(\begin{array}{ccc}{b}_{11}& \dots & b_{1n}\\ ⋮& \ddots & ⋮\\ b_{m1}& \dots & b_{mn}\end{array}\right)=\left(\begin{array}{ccc}{a}_{11}+b_{11}& \dots & a_{1n}+b_{1n}\\ ⋮& \ddots & ⋮\\ a_{m1}+b_{m1}& \dots & a_{mn}+b_{mn}\end{array}\right)$

Definition (Scalar multiplication of matrices)

Let $K$be a field and let $A=(a_{ij})$ be a matrix over $K$. Then, for $\lambda \in K$ we have

$\lambda \cdot A=\lambda \cdot (a_{ij}):=(\lambda a_{ij})$

Written out explicitly in terms of matrices, this definition looks as follows:

$\lambda \cdot A=\lambda \cdot \left(\begin{array}{ccc}{a}_{11}& \dots & a_{1n}\\ ⋮& \ddots & ⋮\\ a_{m1}& \dots & a_{mn}\end{array}\right)=\left(\begin{array}{ccc}\lambda a_{11}& \dots & \lambda a_{1n}\\ ⋮& \ddots & ⋮\\ \lambda a_{m1}& \dots & \lambda a_{mn}\end{array}\right)$

Example (Addition of matrices)

We are in ${ℝ}^{2\times 3}$.

$\left(\begin{array}{ccc}2& 4& 6\\ 1& 3& 5\end{array}\right)+\left(\begin{array}{ccc}-1& 2& -2\\ 3& 2& 1\end{array}\right)=\left(\begin{array}{ccc}2+(-1)& 4+2& 6+(-2)\\ 1+3& 3+2& 5+1\end{array}\right)=\left(\begin{array}{ccc}1& 6& 4\\ 4& 5& 6\end{array}\right)$

Example (Multiplication by a field element)

As an example we take the matrix $\left(\begin{array}{ccc}1& -2& 0\\ 2& -3& -1\end{array}\right)\in {ℝ}^{2\times 3}$ and as field element the real number $(-3)$. Then

$(-3)\cdot \left(\begin{array}{ccc}1& -2& 0\\ 2& -3& -1\end{array}\right)=\left(\begin{array}{ccc}(-3)\cdot 1& (-3)\cdot (-2)& (-3)\cdot 0\\ (-3)\cdot 2& (-3)\cdot (-3)& (-3)\cdot (-1)\end{array}\right)=\left(\begin{array}{ccc}-3& 6& 0\\ -6& 9& 3\end{array}\right)$

Theorem (Matrices form a vector space)

The set of $(m\times n)$-matrices $K^{m\times n}$ forms a $K$-vector space with the addition and scalar multiplication defined above. The neutral element of addition of this vector space is the zero matrix $(0)\in K^{m\times n}$ and the additive inverse of a matrix $A=(a_{ij})\in K^{m\times n}$ is $-A:=(-a_{ij})$.

Proof (Matrices form a vector space)

Proof step: Component-wise addition and scalar multiplication form a vector space structure on $K^{m\times n}$

Let $B$ be a basis of $K^n$ and $C$ a basis of $K^m$. For example, we can choose the standard bases. Using the above theorem, we see that the bijective mapping $M_C^B:{\mathrm{Hom}}_K(K^n,K^m)\to K^{m\times n}$ induces a vector space structure on the space of matrices. We have already considered at the end of the derivation that in this vector space structure is given by component-wise addition and scalar multiplication. So, component-wise addition and scalar multiplication generate a vector space structure on $K^{m\times n}$.

Proof step: $(0)\in K^{m\times n}$ is the neutral element of the addition

We need to show that $A+(0)=A$ holds for any matrix $A\in K^{m\times n}$. So let $A=(a_{ij})\in K^{m\times n}$ be arbitrary. By definition of addition of matrices, $(a_{ij})+(0)=(a_{ij}+0)=(a_{ij})$ holds, where we have exploited that $0\in K$ is the neutral element of addition in $K$.

Proof step: Every matrix $A=(a_{ij})\in K^{m\times n}$ has additive inverse $-A:=(-a_{ij})$

We have to show that $A+(-A)=(0)$ holds for any matrix $A\in K^{m\times n}$. So let $A=(a_{ij})\in K^{m\times n}$ be arbitrary. Then by definition of $-A=(-a_{ij})$ and by the definition of addition of matrices, we have $A+(-A)=(a_{ij})+(-a_{ij})=(a_{ij}-a_{ij})=(0)$. In the last equality we used that $-a_{ij}$ is the additive inverse of $a_{ij}$ in $K$.

If we consider matrices just as tables of numbers (without considering them as mapping matrices), we see the following: Matrices are nothing more than a special way of writing elements of $K^{m\cdot n}$, since matrices have $m\cdot n$ entries. Just as in $K^{m\cdot n}$, the vector space structure for matrices is defined component-wise. So we get alternatively the following significantly shorter proof:

Alternative proof (Matrices form a vector space)

We can simply use the proof that coordinate spaces are vector spaces, since $K^{m\times n}$ is a certain coordinate space. As an example, we show the associativity of addition: If $A=(a_{ij}),B=(b_{ij}),C=(c_{ij})$ are three $(n\times m)$-matrices, then $(A+B)+C=((a_{ij}+b_{ij})+c_{ij})=(a_{ij}+(b_{ij}+c_{ij}))=A+(B+C)$. In the second step we used associativity in the field $K$.

Dimension of $K^{m\times n}$

By the above identification of $K^{m\times n}$ with $K^{m\cdot n}$ we obtain a canonical basis of $K^{m\times n}$: Let $B_{ij}$ be for $i\in \{1,\mathrm{...},m\},j\in \{1,\mathrm{...},n\}$ the matrix $B_{ij}=(b_{kl})$ with

$b_{kl}=\{\begin{array}{cc}1& \text{if }k=i\text{ and }l=j\\ 0& \text{else.}\end{array}$

Example

In $K^{2\times 3}$, the basis elements are given by

$\begin{array}{c}{B}_{11}=\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\end{array}\right),B_{12}=\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 0\end{array}\right),B_{13}=\left(\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\end{array}\right),\\ B_{21}=\left(\begin{array}{ccc}0& 0& 0\\ 1& 0& 0\end{array}\right),B_{22}=\left(\begin{array}{ccc}0& 0& 0\\ 0& 1& 0\end{array}\right),B_{23}=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\end{array}\right)\end{array}$

Thus, $K^{m\times n}$ is a $(m\cdot n)$-dimensional $K$-vector space. We constructed the vector space structure on $K^{m\cdot n}$ such that for $n$- and $m$-dimensional vector spaces $V$ and $W$ with bases $B$ and $C$, respectively, we have that the map

$M_C^B:{\mathrm{Hom}}_K(V,W)\to K^{m\times n},f\mapsto M_C^B(f)$

is a linear isomorphism. So ${\mathrm{Hom}}_K(V,W)$ is a $(m\cdot n)$-dimensional $K$-vector space. This result can also be found in the article vector space of a linear map.