Engineering Analysis/Random Vectors

From Wikibooks, open books for an open world
< Engineering Analysis
Jump to: navigation, search

Many of the concepts that we have learned so far have been dealing with random variables. However, these concepts can all be translated to deal with vectors of random numbers. A random vector X contains N elements, Xi, each of which is a distinct random variable. The individual elements in a random vector may or may not be correlated or dependent on one another.

X = \begin{bmatrix}X_1\\X_2\\\vdots\\X_N\end{bmatrix}

Expectation[edit]

The expectation of a random vector is a vector of the expectation values of each element of the vector. For instance:

E[X] = \begin{bmatrix}E[X_1]\\E[X_2]\\\vdots\\E[X_N]\end{bmatrix}

Using this definition, the mean vector of random vector X, denoted μX is the vector composed of the means of all the individual elements of X:

\mu_X = \begin{bmatrix}\mu_{X_1}\\\mu_{X_2}\\\vdots\\\mu_{X_N}\end{bmatrix}

Correlation Matrix[edit]

The correlation matrix of a random vector X is defined as:

R_X = E[XX^T]

Where each element of the correlation matrix corresponds to the correlation between the row element of X, and the column element of XT. The correlation matrix is a real-symmetric matrix. If the off-diagonal elements of the correlation matrix are all zero, the random vector is said to be uncorrelated. If the R matrix is an identity matrix, the random vector is said to be "white". For instance, "white noise" is uncorrelated, and each element of the vector has an equal correlation value.

Matrix Diagonalization[edit]

As discussed earlier, we can diagonalize a matrix by constructing the V matrix from the eigenvectors of that matrix. If X is our non-diagonal matrix, we can create a diagonal matrix D by:

D = V^{-1}XV


If the X matrix is real symmetric (as is always the case with the correlation matrix), we can simplify this to be:

D = V^TXV

Whitening[edit]

A matrix can be whitened by constructing a matrix W that contains the inverse squareroots of the eigenvalues of X on the diagonal:

W = \begin{bmatrix}\frac{1}{\sqrt{\lambda_1}} & \cdots \\ 0 & \frac{1}{\sqrt{\lambda_2}} \cdots \\ \vdots & \vdots & \ddots \end{bmatrix}

Using this W matrix, we can convert X into the identity matrix:

I = W^TV^TXVW

Simultaneous Diagonalization[edit]

If we have two matrices, X and Y, we can construct a matrix A that will satisfy the following relationships:

A^TXA = I
A^TYA = D

Where I is an identity matrix, and D is a diagonal matrix. This process is known as simultaneous diagonalization. If we have the V and W matrices described above such that

I = W^TV^TXVW,

We can then construct the B matrix by applying this same transformation to the Y matrix:

W^TV^TYVW = B

We can combine the eigenvalues of B into a transformation matrix Z such that:

Z^TBZ = D

We can then define our A matrix as:

A = VWZ
A^T = Z^TW^TV^T

This A matrix will satisfy the simultaneous diagonalization procedure, outlined above.

Covariance Matrix[edit]

The Covariance Matrix of two random vectors, X and Y, is defined as:

Q_X = E[(X-\mu_X)(Y-\mu_Y)^T] = E[(Y-\mu_Y)(X-\mu_X)^T]

Where each element of the covariance matrix expresses the variance relationship between the row element of X, and the column element of Y. The covariance matrix is real symmetric.

We can relate the correlation matrix and the covariance matrix through the following formula:

R = Q + \mu_X\mu_X^T

Cumulative Distribution Function[edit]

An N-vector X has a cumulative distribution function Fx of N variables that is defined as:

F_X(X) = P[X \le x] = P[X_1 \le x_1, X_2 \le x_2, \cdots, X_N \le x_N]

Probability Density Function[edit]

The probability density function of a random vector can be defined in terms of the Nth partial derivative of the cumulative distribution function:

f_X(X) = \frac{\partial^N F_X(X)}{\partial X_1 \partial X_2 \cdots \partial X_N}

If we know the density function, we can find the mean of the ith element of X using N-1 integrations:

\mu_{X_i} = \int_{-\infty}^\infty \int_{-\infty}^\infty \cdots \int_{-\infty}^\infty x_if_X(x) dx_1 dx_2 \cdots dx_n