Linear independence – "Math for Non-Geeks"

Motivation

Basic motivation

Maybe, you learned about vectors in school, where they were drawn as arrows in the plane or in space. Both a plane and space are vector spaces. But how do they differ?

A spontaneous answer could be: "The plane is two-dimensional, and the space is three-dimensional". But this brings us immediately to further questions:

• What is the dimension of a vector space?
• How can we define it?

In the definition of the vector space the term "dimension" does not occur...

Intuition of a dimension

A sphere is a three-dimensional object

The term "dimension" describes in how many independent directions geometric objects can be extended in a space. The objects can also move in just as many independent directions in space ("degrees of freedom of motion").

The plane has two dimensions - the width and the length. It is flat, no object of the plane can reach out of it "into height". A sphere as a three-dimensional object cannot be part of the plane. In contrast, the space with length, width and height has three dimensions. A sphere can thus be part of space.

We summarize: The dimension intuitively corresponds to the number of independent directions into which a geometric object can expand or move. So, for the definition of dimension, we need to answer the following questions:

• What is a direction in a vector space?
• When are two directions independent?
• How can the number of independent directions be determined?

Derivation of the Definition

What is a direction within a vector space?

Let's take the vector space of the plane as an example. We can represent a direction with an arrow:

Arrow that highlights one direction in the plane

Now, an arrow is nothing but a vector. So with the help of vectors, directions can be represented. Here we must not use the zero vector. As an arrow of length zero, it has no direction. We can generalize this to arbitrary vector spaces:

Important!

Every vector not equal to the zero vector represents a direction in a vector space.

The direction in which the vector points is $\alpha \cdot v:\alpha \in ℝ\}$, that is, the span $\mathrm{span}(\{v\})$ of the vector $v$. To this span belong all extensions $\alpha v$ of the direction vector $v$ and thus describes the straight line, which is spanned by $v$:

A straight line described by the vector v

From the line to the plane

To get from the straight line to the plane, we need not only one vector but several, more precisely at least two vectors ($v,w$). This is intuitively obvious, because a plane can be spanned unambiguously only with two vectors. Therefore, we need a further, linearly independent vector. What does "independent" mean in this case? First, we notice that the new vector must not be the zero vector. This vector does not give any direction. Furthermore, the new vector must also not be a multiple of the original vector, i.e. $w\ne \alpha v$ . This also holds for reflections of straight line vectors, represented by multiplication with a negative factor.

• 
  The vector $w$ is a stretched version of the vector $v$ with a positive stretching factor. This vector $w$ is not pointing in a direction independent of $v$ .
• 
  The vector $w$ is a stretched version of the vector $v$ with a positive stretching factor (which includes a reflection). This vector $w$ is not pointing in a direction independent of $v$.
• 
  The direction of $w$ is independent of $v$. Both vectors together span a plane.

We conclude: The new vector $w$ is independent of the direction vector $v$ exactly when the latter is not on the straight line. So we need $w\ne \alpha v$ for all real numbers $\alpha$. Hence, the new vector must not be in the span of the other one. The two spans have only the zero point as intersection.

From the plane to space

We have just seen that we can characterize a plane by two independent vectors. Now we want to go from the plane to space. Here, we also have to add an independent direction. But what is a direction independent of the plane?

The new vector must not be the zero vector, because this vector does not indicate a direction. The new vector must also not lie in the plane, because in that case, no new direction would be described. Only if the new vector does not lie in the plane, it will point in a new, independent direction:

• 
  The vector $c$ lies in the plane spanned by $a$ and $b$. Hence, $c$ does not point into a direction spanned by $a$ and $b$.
• 
  The vector $u$ does not lie in the plane spanned by $v$ and $w$ . All three vectors span the entire space, which means that $u$ points into a direction independent of $v$ and $w$ .

How can we formulate this insight mathematically? Let $v$ and $w$ be the two direction vectors spanning the plane. This plane is then equal to the set $\{\alpha v+\beta w:\alpha ,\beta \in ℝ\}$. Hence, the plane is the set of all sums $\alpha v+\beta w$ for real numbers $\alpha ,\beta \in ℝ$. In order for the new vector $u$ not to be in the plane, it must be $u\ne \alpha v+\beta w$ for all $\alpha ,\beta \in ℝ$. Thus, $u$ in independent of <ma

Question: We had first required that the new vector $u$ must not be the zero vector. Why is it sufficient that $u\ne \alpha v+\beta w$ for all $\alpha ,\beta \in ℝ$? And why does this imply $u\ne 0$ ?

For $\alpha =\beta =0$ we have $\alpha v+\beta w=0$. Since also for $\alpha =\beta =0$ the new vector $u$ shall not be equal to $\alpha v+\beta w$ , we have $u\ne 0$.

Question: Is it sufficient that $u$ is no multiple of $v$ or $w$ ?

No, take for example $u=\frac{1}{2}v+\frac{1}{2}w$. If $v$ is independent of $w$ , then $u$ is neither a stretched version of $v$ , nor one of $w$. However, this vector lies in the plane spanned by $v$ and $w$ , so it does not point into a direction independent of $v$ and $w$.

A first criterion for linear independence

Let's summarize: To describe a straight line we needed a vector $v$ not being the zero vector. In the transition from the straight line to the plane, we had to add a vector $v$ independent of $w$. Independence of $w$ from $v$ means that $w$ does not lie in the line described by $v$ . So we need to have $w\ne \alpha v$ for all $\alpha \in ℝ$.

In the second step, we added a new direction $u$ to the plane, which is independent of the two vectors $v$ and $w$. Here independence manifests itself in the fact that $u$ is not in the plane spanned by $v$ and $w$ . Hence, we need $u\ne \alpha v+\beta w$ for all real numbers $\alpha$ and $\beta$. We can generalise this to an arbitrary number of vectors (but it is not so easy to visualize anymore):

Important!

The vector $w$ is independent of the vectors $v_1,v_2\dots v_n$, if $w\ne {\lambda }_1{v}_1+{\lambda }_2{v}_2+\dots +{\lambda }_n{v}_n$ for all ${\lambda }_1,{\lambda }_2,\dots {\lambda }_n\in ℝ$.

In the above description, the sum ${\lambda }_1{v}_1+{\lambda }_2{v}_2+\dots +{\lambda }_n{v}_n$ appears. Such a sum is called linear combination of the vectors $v_1$ to $v_n$ . We may also say that $w$ is linearly independent if $w\notin \mathrm{span}\{v_1,\mathrm{...},v_n\}$. The description can be changed to:

Important!

The vector $w$ is independent of the vectors $v_1,v_2\dots v_n$, if $w$ cannot be written as a linear combination of the vectors $v_1$ to $v_n$.

Here we have clarified when a vector is independent of other vectors. Is this sufficient to describe the independence of vectors?! Take the following three vectors $a$, $b$ and $c$ as an example:

Three vectors lying in one plane

Since no vector is a multiple of another vector, the three vectors, seen in pairs, point in independent directions. For example $a$ is independent of $b$ and $c$is independent of $a$. So the three vectors are not independent of each other because they all lie in one plane. We have $c=a+b$ and so $c$ is independent of $a$ and $b$. Accordingly, we have to impose linear independence between $a$, $b$ and $c$:

• $a$ is independent of $b$ and $c$: We have $a\ne \beta b+\gamma c$ for all $\beta ,\gamma \in ℝ$.
• $b$ is independent of $a$ and $c$: We have $b\ne \alpha a+\gamma c$ for all $\alpha ,\gamma \in ℝ$.
• $c$ is independent of $a$ and $b$: We have $c\ne \alpha a+\beta b$ for all $\alpha ,\beta \in ℝ$.

It should be emphasised at this point that it is necessary to require all three conditions. If we were to waive the last two conditions, the first requirement would guarantee that the vector $a$ is linearly independent of the vectors $b$ and $c$, but it is not clear from this requirement that $b$ and $c$ are linearly independent of each other. This does not have to be fulfilled, which would mean that the three vectors would again not be linearly independent of each other.

Therefore, none of the three vectors must be able to be represented as a linear combination of the other two vectors. Otherwise, at least one of the vectors is dependent on the other vectors. We can generalise this to any number of vectors:

Definition (Second criterion for linear independence)

Some vectors $v_1$ to $v_n$ are linearly independent if none of the vectors can be written as a linear combination of the other vectors. This means that the following must apply:

• $v_1\ne {\lambda }_2{v}_2+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n$ for all ${\lambda }_2,{\lambda }_3,\dots {\lambda }_n\in ℝ$.
• $v_2\ne {\lambda }_1{v}_1+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n$ for all ${\lambda }_1,{\lambda }_3,\dots {\lambda }_n\in ℝ$.
• ...
• $v_n\ne {\lambda }_1{v}_1+{\lambda }_2{v}_2+\dots +{\lambda }_{n-1}{v}_{n-1}$ for all ${\lambda }_1,{\lambda }_2,\dots {\lambda }_{n-1}\in ℝ$.

So $v_1$ to $v_n$ are linearly independent, if $v_i\ne \sum _{j\ne i}{\lambda }_j{v}_j$ for all $i\in \{1,2,\dots ,n\}$ and ${\lambda }_j\in ℝ$.

From the first criterion to the formal definition

With our first criterion, which we found above, we have already found a suitable definition for linear independence of vectors. In the following, we will try to find a more concise equivalent criterion, with which we can examine the linear independence of vectors more easily.

Vectors are independent if no vector can be represented as a linear combination of the other vectors. From this we will derive another criterion for linear independence, which is less computationally demanding. Let us take vectors $v_1$, $v_2$ to $v_n$ from a vector space $V$ that are not independent. So there is one vector that can be represented by the others. Let $v_1$ be this vector. There are thus stretching factors (scalars) ${\lambda }_2$ to ${\lambda }_n$, such that $v_1={\lambda }_2{v}_2+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n$ We can transform this equation by computing $-v_1$ on both sides ($0_V$ is the zero vector of the vector space $V$): $0_V=-v_1+{\lambda }_2{v}_2+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n$ This is a so-called nontrivial linear combination of the zero vector. A nontrivial linear combination of the zero vector is a linear combination with the result $0_V$ where at least one coefficient is not equal to $0$. For ${\lambda }_1={\lambda }_2=\dots ={\lambda }_n=0$ we would trivially ${\lambda }_1{v}_1+{\lambda }_2{v}_2+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n=0_V$. This is the so-called trivial linear combination of the zero vector, where all coefficients are equal to $0$. You can always form this trivial linear combination, no matter which vectors $v_1$ to $v_n$ you choose. So it does not carry information. If $v_1$ to $v_n$ are dependent, there is at least one non-trivial linear combination of the zero vector (as we saw above) in addition to the trivial linear combination. So:

Important!

If $v_1$ to $v_n$ are linearly dependent, then the zero vector can be represented by at least one non-trivial linear combination of $v_1$ to $v_n$.

In other words:

Important!

$v_1,\dots ,v_n$ linearly dependent $⟹$ There exists a non-trivial linear combination of $0_V$ using $v_1,\dots ,v_n$

Now we can apply the principle of contraposition: $A⟹B$ holds if and only if $\mathrm{\neg }B⟹\mathrm{\neg }A$. So:

Important!

There is no non-trivial linear combination of $0_V$ using $v_1,\dots ,v_n⟹v_1,\dots ,v_n$ are linearly independent

With this we have found a criterion for linear independence. If the zero vector can only be represented trivially by a linear combination of $v_1$ to $v_n$, then these vectors are linearly independent. However, this criterion can also be used as a definition of linear independence. To do this, we need to show the converse direction of the above implication. If there is a non-trivial linear combination of the zero vector, then the vectors under consideration are linearly dependent.

So let $v_1$ to $v_n$ be vectors for which there exists a non-trivial linear combination of the zero vector. This means, there are coefficients (scalars) ${\lambda }_1$ to ${\lambda }_n$, such that ${\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n=0_V$ where at least one of the coefficients ${\lambda }_1$ to ${\lambda }_n$ is not $0$. Let ${\lambda }_i$ be this coefficient. Then

$0_V={\lambda }_1{v}_1+{\lambda }_2{v}_2+\dots +{\lambda }_i{v}_i+\dots +{\lambda }_n{v}_n$

Since ${\lambda }_i\ne 0$ we can multiply both sides by $-{\lambda }_i^{-1}=-\frac{1}{{\lambda }_i}$ . Then,

$0_V=-\frac{{\lambda }_1}{{\lambda }_i}{v}_1-\frac{{\lambda }_2}{{\lambda }_i}{v}_2-\dots -v_i-\dots -\frac{{\lambda }_n}{{\lambda }_i}{v}_n$

On both sides we can now add $v_i$:

$v_i=-\frac{{\lambda }_1}{{\lambda }_i}{v}_1-\frac{{\lambda }_2}{{\lambda }_i}{v}_2-\dots -\frac{{\lambda }_{i-1}}{{\lambda }_i}{v}_{i-1}-\frac{{\lambda }_{i+1}}{{\lambda }_i}{v}_{i+1}-\dots -\frac{{\lambda }_n}{{\lambda }_i}{v}_n$

Thus $v_i$ can be represented as a linear combination of the other vectors and hence the vectors $v_1$ to $v_n$ are linearly dependent. This proves that the following definition of linear independence is equivalent to the first one:

Definition (Second criterion for linear independence)

The vectors $v_1,\dots ,v_n$ are linearly independent if the only linear combination of them resulting in the zero vector is the trivial linear combination, i.e. if we have ${\alpha }_1,\dots ,{\alpha }_n\in K$ with $0_V=\sum _{i=1}^n{\alpha }_i{v}_i$, the ${\alpha }_i=0$ must hold for all $i\in \{1,\dots ,n\}$.

If there is at least one non-trivial linear combination of the zero vector, the considered vectors are linearly dependent.

Definition of a family

We have talked above about several vectors $v_1,\dots ,v_n$ being linearly independent. But what is this "collection" of vectors $v_1,\dots ,v_n$ from a mathematical point of view? We already know the notion of a set. So it is obvious to understand $M=\{v_1,\dots ,v_n\}$ also as a set. Does this view intuitively fit linear independence? Actually, it turns out problematic, if we have two equal vectors $v,v$ with $v\ne 0$. Both point in the same direction and span no two independent directions. Thus, they are intuitively linearly dependent. And indeed, one can be written as a linear combination of the other as $v=1\cdot v$. Thus, the vectors $v,v$ are also strictly mathematically linearly dependent. However, a set may only contain different elements. That is, the set containing $v$ and $v$ is $M=\{v,v\}=\{v\}$. So the set $M$ contains only one element and does not capture duplications of vectors.

So we need a new mathematical term that also captures duplications. This is the concept of family:

Definition (family)

A family $(a_i{)}_{i\in I}$ of elements from a set $A$ consists of an index set $I$, such that every index $i\in I$ gets assigned an element $a_i\in A$.

If $I$ a finite set, we call it a finite family.

If $I\subseteq J$, then one calls $(a_i{)}_{i\in I}$ a sub-family of $(a_i{)}_{i\in J}$. Conversely, $(a_i{)}_{i\in J}$ is then called a super-family of $(a_i{)}_{i\in I}$.

Formally, a family can be seen as a mapping of the index set $I$ into the set $A$. In contrast to sets, elements may occur more than once in families, namely if they belong to different indices.

If the set $I$ is countable, the elements of the family can be numbered: $(a_1,a_2,\dots )$. However, the index set $I$ may also be uncountable, e.g. $I=ℝ$. In this case $(a_i{)}_{i\in ℝ}$ cannot be written as a sequence $(a_1,a_2,\dots )$. The term family thus contains all sequences, and includes even larger "collections" of mathematical objects.

So when we say the vectors $v$ and $v$ are linearly dependent we can express it by saying that the family $(v_i{)}_{i\in \{1,2\}}$ with $v_1=v_2=v$ is linearly dependent.

Often one writes (with slight abuse of notation) $(a_i)\subseteq A$ if the $a_i$ are elements of $A$ and it is clear from the context what the index set $I$ looks like. Similarly, $a\in (a_i)$ means that there is an $i\in I$ with $a_i=a$.

With this we can rewrite the second definition of linear independence:

Definition (Second criterion for linear independence, new version)

The family $(v_1,\dots ,v_n)$ of vectors is linearly independent if the only linear combination representing the zero vector is the trivial linear combination, i.e. if ${\alpha }_1,\dots ,{\alpha }_n\in K$ with $0_V=\sum _{i=1}^n{\alpha }_i{v}_i$, then ${\alpha }_i=0$ for all $i\in \{1,\dots ,n\}$.

General definition of linear independence

Motivation

We have learned above two definitions for the fact that finitely many vectors $v_1,\dots ,v_n\in V$ are linearly independent:

1. A somewhat unwieldy: vectors are independent if no vector $v_k$ can be written as a linear combination of the others. So $v_k={\lambda }_1{v}_1+\dots +{\lambda }_{k-1}{v}_{k-1}+{\lambda }_{k+1}{v}_{k+1}+\dots +{\lambda }_n{v}_n$ must not occur.
2. A somewhat more compact one: The zero vector $0_V$ can only be represented as a trivial linear combination. So $0_V={\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n$ implies ${\lambda }_1=\dots ={\lambda }_n=0$.

So far we have only considered finitely many vectors. What happens with infinitely many vectors? Can there even be an infinite number of linearly independent vectors? We would need a vector space that has infinitely many linearly independent directions. We know intuitively that the vector space ${ℝ}^2$ has at most two and the ${ℝ}^3$ at most three independent directions. So we need a much "bigger" vector space to get infinitely many independent directions. So we consider a vector space $V$ where every vector has infinitely many coordinates: $v=(x_1,x_2,\dots )$ with $x_1,x_2,\dots \in ℝ$. Accordingly, $v$ corresponds to a real sequence $(x_i{)}_{i\in \mathbb{N}}$ and $V$ is the sequences vector space, or sequence space.

In ${ℝ}^d$ we have the linearly independent unit vectors $(1,0,\dots ,0),(0,1,\dots ,0),\dots ,(0,\dots ,0,1)$. We can continue this construction and obtain for $i\in \mathbb{N}$ the vectors $e_i=(0,\dots ,0,1,0,\dots )$ with the $1$ at the $i$-th place and otherwise $0$.

The infinitely many vectors $e_1,e_2,\dots$ form a family $(e_i{)}_{i\in \mathbb{N}}$. This family intuitively represents "infinitely many different directions" in $V$ and is thus intuitively linearly independent. So it makes sense to define linear independence for infinitely many vectors in such a way that $(e_i{)}_{i\in \mathbb{N}}$ is a linearly independent family. The "somewhat unwieldy definition 1." above would be suitable for this in principle: We could simply copy it and say "a family of vectors $(v_i{)}_{i\in \mathbb{N}}$ is linearly independent if no $v_i$ can be written as a linear combination of the others". In fact, in $(e_i{)}_{i\in \mathbb{N}}$ none of the $e_i$ can be written as a linear combination of the other vectors. Therefore, the definition already makes sense at this point. However, there are infinitely many $e_i$ and thus infinitely many conditions!

We prefer to consider the "somewhat more compact definition 2.": "Vectors $(v_i{)}_{i\in \mathbb{N}}$ are linearly independent if $0_V$ can only be represented by the trivial linear combination." What does this formulation mean explicitly in this example? We are given a linear combination of $0_V\in V$. Linear combinations are finite, that is, we have finitely many vectors $e_{i_1},\dots ,e_{i_n}$ and ${\lambda }_1,\dots ,{\lambda }_n\in ℝ$ such that

$0_V={\lambda }_1{e}_{i_1}+\dots +{\lambda }_n{e}_{i_n}.$

We now have to show that all ${\lambda }_i=0$, since then the linear combination of $0_V\in V$ above is trivial. This works in exactly the same way as in ${ℝ}^d$, except that here we have to compare infinitely many entries.

What do we have to do now to get a general definition for general families and general vector spaces? The "somewhat more compact definition 2." carries over almost literally: "A family $(v_i{)}_{i\in I}$ of vectors is linearly independent if $0_V$ can only be represented by the trivial linear combination." For the written out implication, we can make use of our language of families: We replace the double indices by the word "sub-family".

Definition

Definition (Linear dependence and independence of vectors)

Let $K$ be a field, $V$ be a $K$-vector space and $(v_i{)}_{i\in I}\subseteq V$ a family of vectors from $V$.

$(v_i{)}_{i\in I}$ is called linearly independent, if for every finite sub-family $(v_j{)}_{j\in J}$ and all ${\lambda }_j\in K$ with $j\in J$ the following holds: $\sum _{j\in J}{\lambda }_j{v}_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$ A family $(v_i{)}_{i\in I}\subseteq V$ is called linearly dependent, if it is not linearly independent.

Warning

Linear combinations of elements of a set always consist of finitely many summands, even if the set is infinite.

E.g. the family $(e^x,1,x,x^2,\dots )$ is linearly independent in the vector space $\mathrm{Fun}(ℝ,ℝ)$, although $e^x=\sum _{k=0}^{\mathrm{\infty }}\frac{x^k}{k!}$. This is because the exponential function is not a finite linear combination of the monomials.

Hint

You may often find the term "linearly independent set" instead of "linearly independent family". We have already considered above that it makes more sense to use families here, because unlike sets, families also cover duplications of elements. From every set $M$ one can construct the family $(m{)}_{m\in M}$. Thus, the term "linearly (in)dependent" carries over to sets. Linearly independent families do not contain a vector twice. Families without double elements correspond to sets via the above construction. If we want to have a family of linearly independent vectors (e.g. in the preconditions of a set), we can also ask for a set of linearly independent vectors. If we want to test whether a family of vectors is linearly independent, we cannot first convert it into a set. Because there, doublings disappear and cause linear dependence.

Hint

The definition of linear (in)dependence refers to subfamilies of a vector space. These may contain vectors several times and may even be uncountable.

For finite families, we alternatively talk about the elements, i.e. the statement "The family $(v_1,v_2,\dots v_n)$ is linearly (in)dependent" becomes "$v_1,v_2,\dots v_n$ are linearly (in)dependent".

Implications of the definition

Re-formulating the definition for finite sub-families

We have a definition of linear independence for arbitrary subfamilies of a vector space $V$. Does this agree with our old definition for finite subfamilies? Intuitively, they should agree for finite subfamilies, since we derived the general definition from our old definition. The following theorem actually proves this:

Theorem (Linear independence for finitely many vectors)

1. The vectors $v_1,\dots ,v_n\in V$ are linearly independent if and only if ${\lambda }_1{v}_1+\cdots +{\lambda }_n{v}_n=0_V$ with ${\lambda }_1,\dots ,{\lambda }_n\in K$ implies ${\lambda }_1=\cdots ={\lambda }_n=0$.
2. The vectors $v_1,\dots ,v_n\in V$ are linearly dependent if and only if there are ${\lambda }_1$ to ${\lambda }_n$ not all equal to $0$ , such that ${\lambda }_1{v}_1+\cdots {\lambda }_n{v}_n=0_V$.

Proof (Linear independence for finitely many vectors)

We first prove the first statement. We have to establish an equivalence.

Let $(v_1,\dots ,v_n)$ be linearly independent. By the definition of linear independence we obtain that for every finite sub-family $(v_j{)}_{j\in J}$ of $(v_1,\dots ,v_n)$ and for all scalars ${\lambda }_j\in K$ with $j\in J$ we have:

$\sum _{j\in J}{\lambda }_j\cdot v_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$

$(v_1,\dots ,v_n)$ is a finite sub-family of itself. Therefore, for all ${\lambda }_1,\dots ,{\lambda }_n\in K$ from ${\lambda }_1{v}_1+\cdots +{\lambda }_n{v}_n=0_V$, we get that ${\lambda }_i=0$ for all $i\in \{1,\dots ,n\}$.

Conversely, assume that for all ${\lambda }_1,\dots ,{\lambda }_n\in K$ from ${\lambda }_1{v}_1+\cdots +{\lambda }_n{v}_n=0_V$ it follows that ${\lambda }_1=\cdots ={\lambda }_n=0$. We would like to show that $(v_1,\dots ,v_n)$ is linearly independent. So let $(v_j{)}_{j\in J}$ be a finite sub-family of $(v_1,\dots ,v_n)$. That means $J\subseteq \{1,\dots ,n\}$. Let ${\lambda }_j\in K$ with $j\in J$ be scalars with

$\sum _{j\in J}{\lambda }_j\cdot v_j=0_V.$

We extend this sum so that it covers all $i\in \{1,\dots ,n\}$. This is done by defining ${\lambda }_i:=0$ for all $i\in J\setminus \{1,\dots ,n\}$. Then

$\begin{array}{cc}{0}_V& =\sum _{j\in J}{\lambda }_j\cdot v_j\\ & \downarrow \text{add }{0}_V\\ & =\sum _{j\in J}{\lambda }_j\cdot v_j+0_V\\ & \downarrow 0_V=\sum _{i\in J\setminus \{1,\dots ,n\}}0\cdot v_i\\ & =\sum _{j\in J}{\lambda }_j\cdot v_j+\sum _{i\in J\setminus \{1,\dots ,n\}}0\cdot v_i\\ & \downarrow 0={\lambda }_i\\ & =\sum _{j\in J}{\lambda }_j\cdot v_j+\sum _{i\in J\setminus \{1,\dots ,n\}}{\lambda }_i\cdot v_i\\ & =\sum _{k\in \{1,\dots ,n\}}{\lambda }_k\cdot v_k.\end{array}$

It follows from our premise that ${\lambda }_1=\dots ={\lambda }_n=0$ and hence ${\lambda }_j=0$ for all $j\in J$. So $(v_1,\dots ,v_n)$ is linearly independent.

The second statement is exactly the logical contraposition of the first. For we have shown $A⟺B$ with the two statements

$A=$"$(v_1,\dots ,v_n)$ is linearly independent"

$B=$ "$\mathrm{\forall }{\lambda }_1,\dots ,{\lambda }_n\in K:{\lambda }_1{v}_1+\cdots +{\lambda }_n{v}_n=0_V⟹{\lambda }_1=\cdots ={\lambda }_n=0$"

The second point is the statement $\mathrm{\neg }A⟺\mathrm{\neg }B$. But this is equivalent to $A⟺B$ and thus equivalent to the first statement.

Reducing the definition to finite sub-families

We have defined linear independence for any family $(v_i{)}_{i\in I}$ of vectors, so also for infinitely many vectors. But in the definition we only need to show a statement for finite subfamilies $(v_j{)}_{j\in J}$: For all ${\lambda }_j\in K$ with $j\in J$ we need the following:

$\sum _{j\in J}{\lambda }_j{v}_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$

In the previous theorem we have seen that this statement is exactly linear independence of $(v_j{)}_{j\in J}$.

Theorem (Criterion with finitely many sub-families)

1. A family $(v_i{)}_{i\in I}\subseteq V$ is linearly independent if and only if every finite sub-family $(v_j{)}_{j\in J}$ is linearly independent.
2. A family $(v_i{)}_{i\in I}\subseteq V$ is linearly dependent if and only if it contains a finite linearly dependent sub-family $(v_j{)}_{j\in J}$.

Proof (Criterion with finitely many sub-families)

First we prove the first statement. We have to establish an equivalence. Let $(v_i{)}_{i\in I}$ be a linearly independent family of vectors from $V$. We show that every finite sub-family of $(v_i{)}_{i\in I}$ is linearly independent.

Let for this $(v_j{)}_{j\in J}$ be a finite sub-family of $(v_i{)}_{i\in I}$. From our definition of linear independence it follows that for all scalars ${\lambda }_j\in K$ with $j\in J$ the following holds: $\sum _{j\in J}{\lambda }_j\cdot v_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$ Using the previous theorem , we get that $(v_j{)}_{j\in J}$ is linearly independent.

Conversely, let every finite subfamily of $(v_i{)}_{i\in I}$ be linearly independent. We show that $(v_i{)}_{i\in I}$. Let for this $(v_j{)}_{j\in J}$ a finite subfamily of $(v_i{)}_{i\in I}$. We want to show that for all scalars ${\lambda }_j\in K$ with $j\in J$ the following holds: $\sum _{j\in J}{\lambda }_j\cdot v_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$ According to our premise, $(v_j{)}_{j\in J}$ is linearly independent. So it follows again with the previous theorem that for all scalars ${\lambda }_j\in K$ with $j\in J$

$\sum _{j\in J}{\lambda }_j\cdot v_j=0_V⟹{\lambda }_j=0\text{ for all }j\in J$

holds.

The second statement is exactly the logical contraposition of the first. For we have shown $A⟺B$ with the two statements

$A=$"$(v_i{)}_{i\in I}$ is linearly independent"

$B=$ "every finite sub-family of $(v_i{)}_{i\in I}$ is linearly independent"

The second point is the statement $\mathrm{\neg }A⟺\mathrm{\neg }B$. But this is equivalent to $A⟺B$ and thus equivalent to the first statement.

Overview

The following properties can be derived from the definition of linear independence with a few proof steps. Let $K$ be a field and $V$ a $K$-vector space:

1. Every sub-family of a family of linearly independent vectors is linearly independent. Conversely, every super-family of a family of linearly dependent vectors is again linearly dependent.
2. Let $v\in V$ be a single vector. Then $v$ is linearly independent if and only if $v\ne 0_V$. So "almost always". Conversely, every family (no matter how large) is linearly dependent as soon as it contains the zero vector.
3. Let $v,w\in V$. The vectors $v$ and $w$ are linearly dependent if and only if there is a $\lambda \in K$ with the property $w=\lambda \cdot v$ or $v=\lambda \cdot w$.
4. If a family of vectors is linearly dependent, one of them can be represented as a linear combination of the others.

Sub-families of linear independent vectors are linearly independent

A linearly independent family remains linearly independent if you take away vectors. Linear dependence, on the other hand, is preserved if you add more vectors. Intuitively, the addition of vectors tends to "destroy" linear independence and cannot be restored by adding more vectors.

Theorem

1. Every sub-family of a family of linearly independent vectors is again linearly independent.
2. Every super-family of a family of linearly dependent vectors is again linearly dependent.

Proof

We start with the first statement. Let $B\subseteq V$ be a family of linearly independent vectors from $V$ and $A\subseteq B$ any sub-family of $B$. Let $v_1,\dots ,v_k\in A$ and ${\lambda }_1,\dots ,{\lambda }_k\in K$ with

$0_V={\lambda }_1{v}_1+\dots +{\lambda }_k{v}_k.$

Since $A\subseteq B$ , the vectors $v_1,\dots ,v_k$ are also in $B$. And as $B$ is linearly independent, we have that ${\lambda }_1={\lambda }_2=\dots ={\lambda }_k=0$. So $A$ is linearly independent.

From this we deduce the second statement. Let $A\subseteq V$ a family of linearly dependent vectors from $V$ and $B$ any super-family of $A$. Assume that $B$ is linearly independent. Then, we have with the previous statement that also $A$, as a sub-family of $B$, is linearly independent. But this is a contradiction because $A$ is linearly dependent.

Families including the zero vector are linearly independent

When is a family with exactly one vector linearly independent? This question is easy to answer: whenever this vector is not the zero vector. Conversely, every family with the zero vector is linearly dependent. Including the one that contains only the zero vector itself.

Theorem (Families including the zero vector are linearly independent)

1. The zero vector is linearly dependent.
2. If $v\in V$ is linearly dependent, then $v=0_V$.
3. A family of vectors containing the zero vector is always linearly dependent.

Proof (Families including the zero vector are linearly independent)

1. We have that $1_K\cdot 0_V=0_V$. There is therefore a non-trivial linear combination of the vector $0_V$, which has $0_V$ as a result. Hence, $0_V$ is linearly independent.
2. If $v\in V$ is linearly dependent, then there are $\lambda \ne 0\in K$ with $\lambda \cdot v=0_V$. Since $\lambda \ne 0$, there is a multiplicative inverse ${\lambda }^{-1}\in K$. Multiplying the equation $\lambda \cdot v=0_V$ by ${\lambda }^{-1}$ we get $v={\lambda }^{-1}\lambda v={\lambda }^{-1}{0}_V=0_V$. So $v$ must be the zero vector $0_V$.
3. This assertion follows simply from 1. and the theorem about the linear dependence of superfamilies of linearly dependent families.

Two vectors are linearly dependent if one is a stretched version of the other

When is a family with two vectors linearly independent? We can answer the question by saying when the opposite is the case. So when are two vectors linearly dependent? Linear dependence of two vectors holds if and only if both "lie on a straight line", i.e. one vector is a stretched version of the other.

Theorem

Let $v,w\in V$. The vectors $v$ and $w$ are linearly dependent if and only if there is a $\lambda \in K$ with $w=\lambda \cdot v$ or $v=\lambda \cdot w$.

Proof

We need to prove the following two implications:

1. $\mathrm{\exists }\lambda \in K:w=\lambda \cdot v⟹v$ and $w$ are linearly dependent
2. $v$ and $w$ are linearly dependent $⟹\mathrm{\exists }\lambda \in K:w=\lambda \cdot v$

Proof step: First implication

If one of the two vectors is the zero vector, then according to the previous theorem, $v$ and $w$ are linearly dependent. So let $v\ne 0_V$ and $w\ne 0_V$. Further, let $\lambda \in K$ be chosen such that $w=\lambda \cdot v$. This is w.l.o.g. possible, because if it cannot be done, we swap the labels of the two vectors. So we use $v$ instead of $w$ and $w$ instead of $v$. According to the premise, there must exist a $\lambda \in K$, such that the equation with the new labels holds.

Now we have that $1\cdot w+(-\lambda )\cdot v=0_V$. So we have the zero vector represented as a non-trivial linear combination. This means that $v$ and $w$ are linearly dependent.

Proof step: Second implication

Let $v$ and $w$ be linearly dependent. Then, by definition, there is a non-trivial linear combination of the zero vector. Thus, there exist $\alpha ,\beta \in K$ such that $\alpha$ and $\beta$ are both non-zero and the equation $\alpha \cdot v+\beta \cdot w=0_V$ holds. We consider the case where $\beta \ne 0_K$. Then, from the equation $\beta \cdot w=-\alpha \cdot v$ we conclude

$\begin{array}{cccc}\beta \cdot w& =-\alpha \cdot v& & \\ & \downarrow \text{multiply both sides with }{\beta }^{-1}\\ w& ={\beta }^{-1}\cdot (-\alpha \cdot v)& & \\ & \downarrow \text{associative law of scalar multiplication}\\ & =-({\beta }^{-1}\alpha )\cdot v& & \\ & \downarrow \text{set }\lambda :=-({\beta }^{-1}\alpha )\\ & =\lambda \cdot v\end{array}$

However, if $\beta =0_K$ , then we need to have $\alpha \ne 0_K$. Analogously to the calculation above you can then get $v=\lambda \cdot w$ with $\lambda =-({\alpha }^{-1}\beta )$.

With linear dependence, one vector is a linear combination of the others

For finitely many vectors, we started with the definition that vectors are linearly dependent if one of the vectors can be written as a linear combination of the others (first definition). We have already seen that this definition is equivalent to the null vector being able to be written as a linear combination of the vectors (second definition). For the general definition with possibly infinitely many vectors, we have used the version with the zero vector (the second) as our definition. And one can indeed show that even in the general case the first definition is equivalent to it:

Theorem

Let $V$ be a $K$-vector space and let $v_1,v_2,\dots ,v_n,w\in V$ be linearly dependent vectors, but $(v_1,v_2,\dots ,v_n)$ being linearly independent. Then, there exist ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n\in K$ such that $w=\sum _{i=1}^n{\lambda }_i{v}_i$.

How to get to the proof?

Because of the linear dependence of $v_1,v_2,\dots ,v_n,w$ there exist ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n,\mu \in K$ not equal to $0$ , such that $\sum _{i=1}^n{\alpha }_i{v}_i+\mu w=0$. We want to write $w$ as a linear combination of the $v_i$ . That means, we want to solve the equation $\sum _{i=1}^n{\alpha }_i{v}_i+\mu w=0$ for $w$ . We can transform this equation into

$\mu w=-\left(\sum _{i=1}^n{\alpha }_i{v}_i\right).$

Now we want to divide by $\mu$. This only works if $\mu \ne 0$. So we show that the case $\mu =0$ cannot occur. Suppose $\mu =0$. Then, we have

$0_V=\sum _{i=1}^n{\alpha }_i{v}_i.$

We know that $v_1,v_2,\dots ,v_n$ are linearly independent. So all ${\alpha }_i$ are equal to $0$. Hence, all ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n,\mu$ are equal to $0$. That is a contradiction. Therefore $\mu =0$ cannot occur.

Proof

Since $v_1,v_2,\dots ,v_n,w\in V$ are linearly dependent, there exist ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n,\mu \in K$ with $\sum _{i=1}^n{\alpha }_i{v}_i+\mu w=0$, where not all ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n$ are equal $0$ . Hence,

$\mu w=-\left(\sum _{i=1}^n{\alpha }_i{v}_i\right)$

We first show $\mu \ne 0$. Assume that $\mu =0$. Then, we would have

$0_V=\sum _{i=1}^n{\alpha }_i{v}_i+0\cdot w=\sum _{i=1}^n{\alpha }_i{v}_i.$

By linear independence of $v_i$ we have ${\alpha }_i=0_K$ for all $i\in \{1,\dots ,n\}$. But this is not possible, since ${\alpha }_i,\mu$ are not all equal to 0. So we have $\mu \ne 0$. We can hence divide by $\mu$ and the linear combination we are looking for is

$w=-\left(\sum _{i=1}^n\frac{{\alpha }_i}{\mu }{v}_i\right)=\sum _{i=1}^n(-\frac{{\alpha }_i}{\mu })v_i$

now set ${\lambda }_i:=(-\frac{{\alpha }_i}{\mu })$. Then, $w=\sum _{i=1}^n{\lambda }_i{v}_i$ , which is what we wanted to show.

Linear independence and unique linear combinations

In this section, we will take a closer look at the connection between linear independence and linear combinations. To do this, we recall what it means that the vectors $v_1,\dots ,v_n$ are linearly dependent or independent. Suppose the vectors $v_1,\dots ,v_n$ are linearly dependent. From our definition of linear independence, we know that there must then be a non-trivial zero representation, since at least one scalar ${\lambda }_i\ne 0$ for some $1\le i\le n$. We illustrate this with the following example

Example (Linear independence and non-trivial representation of 0)

Let us consider the vectors $(1,0,0{)}^T,{(1,1,\frac{1}{2})}^T,(0,2,1{)}^T\in {ℝ}^3$. These are linearly dependent, since

$\left(\begin{array}{c}1\\ 1\\ \frac{1}{2}\end{array}\right)=1\cdot \left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)+\frac{1}{2}\cdot \left(\begin{array}{c}0\\ 2\\ 1\end{array}\right)$

By transforming this equation we obtain a representation of the zero vector:

$\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)=1\cdot \left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)-1\cdot \left(\begin{array}{c}1\\ 1\\ \frac{1}{2}\end{array}\right)+\frac{1}{2}\cdot \left(\begin{array}{c}0\\ 2\\ 1\end{array}\right)$

In addition to this representation, there is also the so-called trivial representation of the zero vector, in which every pre-factor is equal to zero:

$\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)=0\cdot \left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)+0\cdot \left(\begin{array}{c}1\\ 1\\ \frac{1}{2}\end{array}\right)+0\cdot \left(\begin{array}{c}0\\ 2\\ 1\end{array}\right)$

Because of the linear dependence, the zero vector can be represented in two ways via a linear combination.

Regardless of whether the considered family of vectors is linearly independent or not, there is always the trivial zero representation, in which all scalars ${\lambda }_1,\mathrm{...},{\lambda }_n$ have the value $0$:

$0=0\cdot v_1+\mathrm{...}+0\cdot v_n$

In case of linear dependence of the vectors, the representation of the zero is no longer unambiguous. We can summarise our results so far in a theorem and generalise them:

Theorem (Linear independence and unique linear combination)

Let $V$ be a vector space and $M\subseteq V$.

All linear combinations of vectors from $M$ are unique $⟺M$ linearly independent.

Proof (Linear independence and unique linear combination)

We show the contraposition:

There is a linear combination of vectors from $M$ that is not unique $⟺M$ is linearly dependent.

"$⟹$" We assume there was a $v\in V$, such that at least two different representations of $v$ are possible using vectors from $M$:

Let $v={\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n$ with ${\lambda }_1,\mathrm{...},{\lambda }_n\in K$ and $v={\mu }_1{v}_1+\dots +{\mu }_n{v}_m$ with ${\mu }_1,\mathrm{...},{\mu }_v\in K$ and $v_1,\dots ,v_n\in M$. Subtraction of the two equations gives

$0_V=({\lambda }_1-{\mu }_1)\cdot v_1+\dots +({\lambda }_n-{\mu }_n)\cdot v_n$

Since the representations of $v$ are different, there is at least one factor ${\lambda }_i-{\mu }_i\ne 0$ for $1\le i\le n$. Hence, the vectors $v_1,\mathrm{...},v_n$ are linearly dependent by definition and thus $M$ is also linearly dependent.

"$⟸$" If $M$ is linearly independent, then $M$ contains a linearly independent subset $v_1\dots ,v_n$.

Then, apart from the trivial representation of zero $0_V$, there is at least one more: because of the linear dependence, there are factors ${\lambda }_i$ that are not all zero, with

${\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n=0_V$

So we have shown that there are then two representations of $0_V$ as linear combinations of these vectors. Thus, linear combinations are not unique.

Exercises

Exercise 1

Exercise (Linear independence)

Show that the vectors $(1,1,0{)}^T,(0,1,0{)}^T,(0,1,1{)}^T\in {ℝ}^3$ are linearly independent.

Solution (Linear independence)

We have to show that the zero vector can only be represented trivially by the given vectors. This means that the following equation with the real numbers ${\rho }_1,{\rho }_2,{\rho }_3\in ℝ$ only has the solutions ${\rho }_1={\rho }_2={\rho }_3=0$:

${\rho }_1\cdot \left(\begin{array}{c}1\\ 1\\ 0\end{array}\right)+{\rho }_2\cdot \left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)+{\rho }_3\cdot \left(\begin{array}{c}0\\ 1\\ 1\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)$

This implies:

$\begin{array}{cc}& {\rho }_1\cdot \left(\begin{array}{c}1\\ 1\\ 0\end{array}\right)+{\rho }_2\cdot \left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)+{\rho }_3\cdot \left(\begin{array}{c}0\\ 1\\ 1\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)\\ & \downarrow \text{scalar multiplication}\\ ⟹& \left(\begin{array}{c}{\rho }_1\\ {\rho }_1\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ {\rho }_2\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ {\rho }_3\\ {\rho }_3\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)\\ & \downarrow \text{vector addition}\\ ⟹& \left(\begin{array}{c}{\rho }_1\\ {\rho }_1+{\rho }_2+{\rho }_3\\ {\rho }_3\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)\end{array}$

Now, two column vectors are equal if every component is equal. So the following equations must hold:

$\begin{array}{cc}{\rho }_1& =0\\ {\rho }_1+{\rho }_2+{\rho }_3& =0\\ {\rho }_3& =0\end{array}$

We hence have ${\rho }_1={\rho }_3=0$. Plugging this into ${\rho }_1+{\rho }_2+{\rho }_3=0$ , we obtain ${\rho }_2=0$. With this we have shown that from the equation ${\rho }_1\cdot (1,1,0{)}^T+{\rho }_2\cdot (0,1,0{)}^T+{\rho }_3\cdot (0,1,1{)}^T=(0,0,0{)}^T$ we get that all coefficients ${\rho }_1$, ${\rho }_2$ and ${\rho }_3$ are equal to $0$. Thus, the three vectors are linearly independent.

Exercise 2

Exercise (Linear dependence)

Show that the following set $M$ of four vectors is linearly dependent:

$M=\{\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right),\left(\begin{array}{c}0\\ 1\\ 0\end{array}\right),\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right),\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)\}$

Solution (Linear dependence)

By definition, the vectors $(1,0,0{)}^T,(0,1,0{)}^T,(0,0,1{)}^T$ and $(1,1,1{)}^T$ are linearly dependent if and only if we can find a nontrivial linear combination of zero. Such a combination is for example given by

$\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)-\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)=0$

Therefore, the vectors are linearly dependent.

Solution (Lineare dependence, alternative)

Vectors are linearly dependent if one of them can be represented as a linear combination of the other ones. Now the vector $(1,1,1{)}^T$ can be represented as a linear combination of the others:

$\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)=\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)$

Thus, the vectors are linearly dependent.

Exercise 3

Exercise (Trigonometrical polynomials)

Let $f:ℝ\to ℝ$ with $f(x)=f(x+2\pi )$ for all $x\in ℝ$. That means, $f$ is a $2\pi$-periodic function. We consider the set ${\mathrm{Abb}}_{2\pi }(ℝ,ℝ)$ of $2\pi$-periodic functions. These form an $ℝ$-vector space.

Are the functions $\cos (0\cdot x),\cos (1\cdot x),\cos (2\cdot x)$ linearly independent?

How to get to the proof? (Trigonometrical polynomials)

We investigate how to write the zero function as a linear combination of the three functions. To do this, we determine the values of ${\lambda }_1,{\lambda }_2,{\lambda }_3$ in the equation ${\lambda }_1\cos (0)+{\lambda }_2\cos (x)+{\lambda }_3\cos (2x)=0$. We can do this by inserting three different values for $x$ and then solving the resulting system of equations.

Values for which we explicitly know the exact values of the cosine are suitable for this - for example $0,\frac{\pi }{2}$ and $\pi$. For those, we know $\cos (0)=\cos (2\pi )=1,\cos \left(\frac{\pi }{2}\right)=0$ and $\cos (\pi )=-1$.

Solution (Trigonometrical polynomials)

Let ${\lambda }_1,{\lambda }_2,{\lambda }_3\in ℝ$, such that

${\lambda }_1\cos (0\cdot x)+{\lambda }_2\cos (1\cdot x)+{\lambda }_3\cos (2\cdot (x))=0$

for all $x\in ℝ$ . We would like to establish ${\lambda }_1={\lambda }_2={\lambda }_3=0$ . Plugging in for $x$ the values $0,\frac{\pi }{2}$ and $\pi$ , we obtain the following system of equations for ${\lambda }_i$:

$\begin{array}{cccc}x=\frac{\pi }{2}& :& {\lambda }_1-{\lambda }_3& =0\\ x=\pi & :& {\lambda }_1-{\lambda }_2+{\lambda }_3& =0\\ x=0& :& {\lambda }_1+{\lambda }_2+{\lambda }_3& =0\end{array}$

The system of equations can now be solved in different ways. We transform the first equation and get ${\lambda }_1={\lambda }_3$. We can substitute this into the second equation and get ${\lambda }_1-{\lambda }_2+{\lambda }_1=0$, so $2{\lambda }_1={\lambda }_2$. If we now substitute our results into the third equation, we have ${\lambda }_1+2{\lambda }_1+{\lambda }_1=0$. This is equivalent to ${\lambda }_1=0$ . From the other equations, we conclude ${\lambda }_2={\lambda }_3=0$.

Thus, we have uniquely determined the coefficients ${\lambda }_1,{\lambda }_2$ and ${\lambda }_3$. That is, there is no non-trivial linear combination of the $0$. The functions are therefore linearly independent.

Exercise 4

Exercise (Linear (in-)dependence?)

Prove or disprove the following statement:

Let $u,v,w\in {ℝ}^3$. The set $\{u,v,w\}$ is linearly dependent if and only if each of the vectors is a linear combination of the other two.

How to get to the proof? (Linear (in-)dependence?)

For the set to be linearly dependent, it is sufficient if two of the three vectors are multiples of each other, while the third can be linearly independent of the two. With this consideration we can construct a counterexample.

Solution (Linear (in-)dependence?)

The statement is not correct. We consider the set

$\{\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right),\left(\begin{array}{c}2\\ 0\\ 0\end{array}\right),\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)\}$

Then we can represent the zero vector as a non-trivial linear combination of the three vectors:

$\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)=-2\cdot \left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)+\left(\begin{array}{c}2\\ 0\\ 0\end{array}\right)+0\cdot \left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)$

Thus, the set is linearly dependent. However, the vector ${(0,0,1)}^T$ is not a linear combination of the other two.

Exercise 5

Exercise (Linearly independent vectors in $K^n$)

Prove: Within the vector space $K^n$ of $n$-tuples over the field $K$ , the vectors $e_1=(1,0,0,\dots ,0{)}^T$, $e_2=(0,1,0,\dots ,0{)}^T$ up to $e_n:=(0,0,0,\dots ,1{)}^T$ are linearly independent.

Solution (Linearly independent vectors in $K^n$)

We have to show that we can uniquely represent the zero vector as a linear combination of the vectors $e_1,\mathrm{...},e_n$. So let us consider the linear combination of the vectors with ${\lambda }_i\in K$ for $1\le i\le n$. We need to have

$\left(\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right)\stackrel{!}{=}{\lambda }_1\cdot \left(\begin{array}{c}1\\ 0\\ ⋮\\ 0\end{array}\right)+{\lambda }_2\cdot \left(\begin{array}{c}0\\ 1\\ ⋮\\ 0\end{array}\right)+\mathrm{...}+{\lambda }_n\cdot \left(\begin{array}{c}0\\ 0\\ ⋮\\ 1\end{array}\right)$

We can interpret this as a linear system of equations as

$\begin{array}{cc}0& ={\lambda }_1\\ 0& ={\lambda }_2\\ & ⋮\\ 0& ={\lambda }_n\end{array}$

Thus, we have shown that the ${\lambda }_i$ are uniquely determined and $0$. After defining the linear independence, we have shown that the vectors are linearly independent.

Exercise 6

Exercise (Linearly independent vectors and endomorphisms)

Let $K$ be a field and $F:V\to V$ an endomorphism of the $K$-vector space $V$. Let $v\in V$, such that for a fixed natural number $n$, we have: $F^i(v)\ne 0$ for $i=1,\mathrm{...},n$ and $F^{n+1}(v)=0$. Here, $F^i(v)=F(F(\mathrm{...}(F(v))\mathrm{...}))$ means that the $i$-fold application of $F$ onto the vector $v$. Prove that then the vectors $v,F(v),\mathrm{...},F^n(v)$ are linearly independent.

How to get to the proof? (Linearly independent vectors and endomorphisms)

We need to show that for ${\lambda }_0,\dots ,{\lambda }_n\in K$ with

${\lambda }_{0}v+{\lambda }_{1}F(v)+\dots +{\lambda }_n{F}^n(v)=0$

we already have ${\lambda }_0=\dots ={\lambda }_n=0$ . We can try to get the individual ${\lambda }_i$ from this equation: We know that $F^{n+1}(v)=0$ . If we now apply $F$ to this equation, we get

$\begin{array}{cc}0=F(0)& =F({\lambda }_{0}v+{\lambda }_{1}F(v)+\dots +{\lambda }_n{F}^n(v))\\ & ={\lambda }_{0}F(v)+{\lambda }_1{F}^2(v)+\dots +{\lambda }_n{F}^{n+1}(v)\\ & ={\lambda }_{0}F(v)+{\lambda }_1{F}^2(v)+\dots +{\lambda }_{n-1}{F}^n(v).\end{array}$

We have thus eliminated one summand. With this we have reduced our problem to a case with $n$ summands. That is, by proceeding with induction, we can now infer the statement.

Solution (Linearly independent vectors and endomorphisms)

We perform an induction over $n$ to iterate the idea of reducing the number of vectors one-by-one.

Theorem whose validity shall be proven for $n\in \mathbb{N}$ bewiesen werden soll:

If $v\in V$ with $F^i(v)\ne 0$ for all $i\le n$ and $F^{n+1}(v)=0$, then $v,F(v),\dots ,F^n(v)$ is linearly independent.

1. Base case:

We need to show that $v$ and $F(v)$ are linearly independent, if $F^2(v)=0$ and $F(v)\ne 0$ hold. That means we have to show that for $\lambda ,\mu \in K$ with $\lambda \cdot v+\mu \cdot F(v)=0$ we already have $\lambda =\mu =0$ . Now,

$\begin{array}{cc}0=F(0)& =F(\lambda \cdot v+\mu \cdot F(v))\\ & =\lambda \cdot F(v)+\mu \cdot F^2(v)\\ & =\lambda \cdot F(v)\\ \end{array}$

Since $F(v)\ne 0$ , we have $\lambda =0$. So by choice of $\lambda$ and $\mu$, we also have $\mu \cdot F(v)=0$. With the same argument, we now have $\mu =0$.

2. Inductive step:

2a. Inductive hypothesis:

If $v\in V$ with $F^i(v)\ne 0$ for all $i\le n$ and $F^{n+1}(v)=0$, then $v,F(v),\dots ,F^n(v)$ is linearly independent.

2b. Induction theorem:

If $v\in V$ with $F^i(v)\ne 0$ for all $i\le n+1$ and $F^{n+2}(v)=0$, then $v,F(v),\dots ,F^{n+1}(v)$ is linearly independent.

2c. Proof of induction step:

Let ${\lambda }_0,\dots ,{\lambda }_{n+1}\in K$, such that ${\lambda }_0\cdot v+{\lambda }_1\cdot F(v)+\dots +{\lambda }_{n+1}\cdot F^{n+1}(v)=0$ . Then,

$\begin{array}{cc}0=F(0)& =F({\lambda }_0\cdot v+{\lambda }_1\cdot F(v)+\dots +{\lambda }_{n+1}\cdot F^{n+1}(v))\\ & ={\lambda }_0\cdot F(v)+{\lambda }_1\cdot F^2(v)+\dots +{\lambda }_{n+1}\cdot F^{n+2}(v)\\ & ={\lambda }_0\cdot F(v)+\dots +{\lambda }_n\cdot F^{n+1}(v)\end{array}$

Applying the induction assumption to $F(v)$ , we get that ${\lambda }_0=\dots ={\lambda }_n=0$ . Hence, ${\lambda }_{n+1}{F}^{n+1}(v)=0$. Since $F^{n+1}(v)\ne 0$ , we also have ${\lambda }_{n+1}=0$. So $v,F(v),\dots ,F^{n+1}(v)$ is linearly independent.