Transwiki:Introduction to metrics, norms and inner products

The following article is an introductory tutorial to inner products, norms and metrics. It is geared towards giving more insight on these topics by the use of common examples that can easily be related to the topics. At first glance, the topics will seem intricate, but after careful derivations and analysis, they will seem less so.

In addition to the analysis of the topics, this article will focus on the applications and various relationships between the inner product, norm and metric. The validation of the examples of inner products, norms and metrics will be done by using the various properties that will be described in detail in the article.

Metrics

Definition

A metric, also known as a distance function, is a function on a set X that returns a value which is an element of the real numbers. Consider a metric d on X; it can be defined as a function acting on X whose result is in the reals as follows:

$d:\,X\times X\rightarrow \mathbb {R}$

Properties

Metrics have three properties that they must all satisfy. They are as follows:

1. $d(x,y)\geq 0\quad \forall x,y\in X$. The metric function will be equal to 0 if and only if x = y

2. $d(x,y)=d(y,x)\quad \forall x,y\in X$

3. $d(x,y)+d(y,z)\geq d(x,z)\quad \forall x,y,z\in X$

Application

Consider a metric d in a discrete metric space defined as follows:

$d(x,y)={\begin{cases}1&if\,x\neq y\\0&if\,x=y\end{cases}}$

The proof that d is a metric will be done as follows:

Property 1 (Metric) and Property 2 (Metric):

It is clear to see that d(x,y) is always greater than or equal to zero and equal to zero when x = y. Property 2 is satisfied because d(x,y) = d(y,x) at all times.

Property 3 (Metric):

Given x, y, and z in the discrete metric space, and supposing that d(x,z) = 0, the following will hold:

$d(x,y)+d(y,z)\geq d(x,z)=0$

Now, suppose d(x,z) = 1, y must be equal to either x or z or neither x nor z. Since x and z are not equal to each other, y cannot be equal to both x and z because that will mean that x and z are equal; which contradicts d(x,z) = 1. From these reasons, the following truth table can be derived:

${\begin{matrix}d(x,y)&d(y,z)&d(x,z)&d(x,y)+d(y,z)\\0&1&1&1\\1&0&1&1\\1&1&1&2\end{matrix}}$

Norms

Definition

A norm on a vector space V over the real numbers or the complex numbers is a real-valued function, denoted as:

$\Vert \cdot \Vert :V\rightarrow \mathbb {R} .$

Properties

A norm must have the following properties satisfied:

1. $\Vert v\Vert \geq 0\quad \forall v\in V$

2. $\Vert sv\Vert =\vert s\vert \Vert v\Vert \quad \forall v\in V,\ s\in F$

3. $\Vert v+w\Vert \leq \Vert v\Vert +\Vert w\Vert \quad \forall v,w\in V$

Norms are often associated with assigning a length to vectors in a vector space.

There is a relation between the norm and the metric in that a metric can be defined in terms of a norm. If there is a function d such that $d(v,w)=\Vert v-w\Vert$ , then d is a valid metric. The reverse of the statement does not hold meaning that a metric is not always a norm. The statement reads similar to: “The distance between v and w is equal to the length of the difference of v and w”.

Application

Suppose that the Euclidean metric d(x,y) is defined as $\Vert x-y\Vert$ , then it can be shown that Property 3 of the metric, $d(x,y)+d(y,z)\geq d(x,z)$, will be satisfied. The proof of the property is as follows:

$d(x,z)=\Vert x-z\Vert =\Vert x-y+y-z\Vert \leq \Vert x-y\Vert +\Vert y-z\Vert =d(x,y)+d(y,z)\geq d(x,z).$

Inner products

Definition

An inner product on a vector space V over the field of real numbers or complex numbers is a function $\langle \cdot \vert \cdot \rangle :V\rightarrow F$.

Properties

The following properties for an inner product must hold for $u,v,w\in Vand\,s\in F$:

1. $\langle u+v\vert w\rangle =\langle u\vert w\rangle +\langle v\vert w\rangle$

2. $\langle sv\vert w\rangle =s\langle v\vert w\rangle$

3. $\langle v\vert w\rangle ={\overline {\langle w\vert v\rangle }}$

4. $\langle v\vert v\rangle >0\ for\ v\neq 0$

Application

A common inner product is the dot product or the standard inner product. A dot product over $F^{n}$ on $v=(v_{1},v_{2},\dots ,v_{n})$ and $w=(w_{1},w_{2},\dots ,w_{n})$ is defined as follows:

$\langle v\vert w\rangle =\sum _{i=1}^{n}v_{i}{\bar {w}}_{i}$

The proof that the dot product is an inner product will be done as follows using $u=(u_{1},u_{2},\dots ,u_{n})$, $v=(v_{1},v_{2},\cdots ,v_{n})$, $w=(w_{1},w_{2},\dots ,w_{n})$ over $F^{n}$ and $s\in F$.

Property 1

${\begin{aligned}\langle u+v\vert w\rangle &{}=\sum _{i=1}^{n}(u_{i}+v_{i}){\bar {w}}_{i}\\&{}=\sum _{i=1}^{n}(u_{i}){\bar {w}}_{i}+\sum _{i=1}^{n}(v_{i}){\bar {w}}_{i}\\&{}=\langle u\vert w\rangle +\langle v\vert w\rangle \end{aligned}}$

Property 2

${\begin{aligned}\langle sv\vert w\rangle &{}=\sum _{i=1}^{n}(sv_{i}{\bar {w}}_{i})\\&{}=s\sum _{i=1}^{n}(v_{i}{\bar {w}}_{i})\\&{}=s\langle v\vert w\rangle \end{aligned}}$

Property 3

The proof of this property will introduce the term, Hermitian, which means the transpose-conjugate. This means $v^{H}=({\bar {v}})^{T}$ for a vector v.

${\begin{aligned}\langle v\vert w\rangle &{}=\sum _{i=1}^{n}(v_{i}{\bar {w}}_{i})\\&{}=w^{H}v\\&{}=(v^{H}w)^{H}\\&{}=\operatorname {conjugate} (\sum _{i=1}^{n}w_{i}{\bar {v}}_{i})\\&{}=\operatorname {conjugate} (s\langle w\vert v\rangle )\end{aligned}}$

Property 4

$\langle v\vert v\rangle =\sum _{i=1}^{n}v_{i}{\bar {v}}_{i}$

For $F=\mathbb {C}$, $v=a+\imath b$, and $a,b\in \mathbb {R}$, $v_{i}{\bar {v}}_{i}=a^{2}+b^{2}$ which is a real number greater than or equal to 0. For $F=\mathbb {C}$, $v_{i}{\bar {v}}_{i}=v_{i}^{2}$ which is also real number greater than or equal to 0. The dot product will equal zero if $v_{i}$.

Since all four properties are satisfied for the inner product, the dot product is thus a valid inner product.