Random Processes in Communication and Control/M-Sep14

From Wikibooks, open books for an open world
< Random Processes in Communication and Control
Jump to: navigation, search

Last Time[edit]

PMF P[X=x]=P[s:X(s)=x] \!\,


a) P_X(x) \geq 0 \forall x \in S_x \!\,


b) \sum_{x \in S_X} P_X(x)=1 \!\,


c) event B \subset S_X \!\,


P[B]= \sum_{x\in B} P_X(x) \!\,

Some Useful Random Variables[edit]

Bernoulli R.V[edit]

 
P_X(x)= \begin{cases}
1-p & x=0 \\
p   & x=1 \\
0   & o.w.
\end{cases}

\!\,


0 \leq p \leq 1 \!\, success probability

Example[edit]

1) Flip a coin X= \!\,# of H


2) Manufacture a Chip X=\!\,# of acceptable chips


3) Bits you transmit successfully by a modem

Geometric Random Variable[edit]

Number of trials until (and including) a success for an underlying Bernoulli


P_X(x)=p(1-p)^{x-1} \quad x=1,2,3,\ldots \!\,

Example[edit]

1) Repeated coin flips X= \!\,# of tosses until H


2) Manufacture chips X=  \!\,3 of chips produced until an acceptable time

Binomial R.V[edit]

"# of successes in n trials"


P_x(x)= {n \choose x} p^x (1-p)^{n-x} \quad x=0,1,\ldots n \!\,


Example[edit]

1) Flip a coin n times. X= \!\, # of heads.


2) Manufacture n chips. X=  \!\, # of acceptable chips.


Note: Binomial X=Y_1+ Y_2+Y_3 + \ldots+Y_n \!\, where Y_1+Y_2+Y_3 \ldots+ Y_n \!\, are independent Bernoulli trials


Note: n=1; Binomial=Bernoulli; X=Y_1 \!\,

Pascal R.V[edit]

"number of trials until (and including) the kth success with an underlying Bernoulli"


P_X(x)= {x-1 \choose k-1} p^k (1-p)^{x-k} \quad x=k,k+1,k+2,\ldots \!\,


where {x-1 \choose k-1}  \!\, is k-1 \!\, successes in  x-1\!\, trials


Note: Pascal X=Y_1+Y_2+\ldots+Y_k \!\, where Y_1,Y_2,\ldots Y_k \!\, are geometric R.V.


Note: K=1 Pascal=Geometric

Example[edit]

X=\!\,# of flips until the kth H

Discrete Uniform R.V.[edit]

 P_X(x)=
\begin{cases}
\frac{1}{b-a+1} & x=a,a+1,\ldots,b \\
0               & \mbox{otherwise}

\end{cases}
\!\,


Example[edit]

1) Rolling a die. a=1, b=6 \!\,


P_X(x)=
\begin{cases}
\frac16 & x=1,\ldots,6 \\
0       & \mbox{otherwise}
\end{cases}
 \!\,


2) Flip a fair coin. X \!\,=# of H


 
P_X(x)=
\begin{cases}
\frac12 & x=0 \\
\frac12 & x=1 \\
0       & \mbox{ otherwise }
\end{cases}
\!\,


Poisson R.V.[edit]

P_X(x)= e^{-\alpha} \frac{\alpha^x}{x!} \quad x=0,1,2\ldots \!\,


(Exercise) limiting case of binomial with n \rightarrow \infty, p \rightarrow 0, np=\alpha \!\,


PMF is a complete model for a random variable

Cumulative Distribution Function[edit]

F_X(x)= P[X \leq x] = \sum_{x' \leq x} P_X(X=x') \!\,


Like PMF, CDF is a complete description of random variable.

Example[edit]

Flip the coins X= \!\,# of H


P_X(x)=
\begin{cases}
\frac14 & x=0 \\
\frac12 & x=1 \\
\frac12 & x=2 \\
0       & \mbox{otherwise}

\end{cases}
 \!\,



\begin{align}
P[X \leq 0]&=P[X=0]\\
P[X \leq 0.5] &= P[X=0] \\
P[X \leq 1 ]  &= \underbrace{P[X=0]}_{\frac14} + \underbrace{P[X=1]}_{\frac12} \\

\end{align}
 \!\,

Properties of CDF[edit]

  • a) F_X(-\infty)=0 \Leftarrow F_X(-\infty)=P[X \leq -\infty] = 0 \!\,


F_X(\infty)=1 \Leftarrow F_X(\infty)=P[X \leq \infty]=1 \!\,


"starts at 0 and ends at 1"


  • b) For all x' \geq x \!\,, F_X(x') \geq F_X(x) \!\,


"non-decreasing in x"


 
\begin{align}
F_X(x') &\quad F_X(x) \\
P[X \leq x'] &\quad P[X \leq x ] \\
P[s: X(s) \leq x'] &\quad P[s:X(s) \leq x] \\
\{s: X(s) \leq x \} &\subset \{s: X(s) \leq x' \} \\
P[X \leq x] &\leq P[X \leq x']\\
F_X(x) &\leq F_X(x')

\end{align}
\!\,


  • c) For all x,x' \!\,


P[x \leq X \leq x'] = F_X(x')-F_X(x) \!\,


"probabilities can be found by difference of the CDF"


\{ s: X(x) \leq x' \} = \{ s: X(x) \leq x \} \cup \{s: x \leq X(s) \leq x' \} \!\,


 P[X \leq x' ] = P[X \leq x ] + P [x \leq X \leq x']\!\,


  • d) For all x \!\,,


\lim_{\epsilon \rightarrow 0} F_X(x+\epsilon)=F_X(x) \!\,


"CDF is right continuous"


  • e) For x_i \in S_X \!\,


F_X(x_i)- F_X(x_i - \epsilon) = P_X(x_i) \!\,


"For a discrete random variable, there is a jump (discontinuity) in the CDF at each value x_i \in S_X \!\,. This jump equals P_X(x_i) \!\,


P[x_i \leq x] - P[x_i \leq x_i - \epsilon]=P[x_i- \epsilon < x < x_i] = P[X=x_i] \!\,


  • f) F_X(x)=F_X(x_i) \!\, for all x_i \leq x \leq x_{i+1} \!\,


"Between two jumps the CDF is constant"



\begin{align}
P[X \leq x] &= P[X \leq x_i \cup x_i < X \leq x ] \\
            &= P[X \leq x_i + \underbrace{P[x_i < X \leq x ]}_0 \\
            &= F_X(x_i)
\end{align}


  • g) P[X > x] = 1- \underbrace{F_X(x)}_{P[X \leq x]} \!\,

Continuous Random Variables[edit]

outcomes uncountable many


Example[edit]

T: arrival of a partical


S_T= \{t : 0 \leq t < \infty \} \!\,


V: voltage


S_V=\{v : -\infty < v < \infty \} \!\,


\theta \!\,: angle


S_{\theta}=\{ \theta: 0 \leq \theta \leq 2 \pi \} \!\,


X \!\,: distance


S_X= \{ X: 0 \leq x \leq 1 \} \!\,


P[x \in A] = \frac{1}{n } \rightarrow 0\!\,


No PMF, P[X=x]=0 \forall x \!\,

Theorem[edit]

For any random variable (continuous or discrete)


  • a) F_X(-\infty)=0 \quad F_X(\infty)=1 \!\,


  • b) F_X(x) \!\, is nondecreasing in X \!\,


  • c) P[x < X \leq x'] = F_X(x')-F_X(x) \!\,


  • d) F_X(x) \!\, is right continuous


Example[edit]

S_X=[0,1] \!\,


P_[x \in A]= P[ x\ in B]\!\, where A, B are intervals of the same length contained in [0,1]


P[X \leq 1] =1 \Leftrightarrow F_X(1)=1 \!\,


P[X \leq 0] =0 \Leftrightarrow F_X(0)=0 \!\,


P[x_1 < x < x_2]=P[0 < x < x_2-x_1] \!\,


(exercise)F_X(x_2)-F_X(x_1)=F_X(x_2-x_1) \rightarrow F_x(x)=x \!\,


Probability Density Function (PDF)[edit]

f_X(x)= \frac{dF_x(x)}{dx} \!\,


discrete: PMF <--> CDF (sum/difference)


continuous <---> (derivative/integral)


Theorem: Properties of PDF[edit]

  • a) f_X(x) \geq 0 \!\, (F_X(x) \!\, is nondecreasing)


  • b) F_X(x) = \int_{\infty}^x f_X(x) dx \quad (F_X(\infty)=0)\!\,


  • c) \int_{-\infty}^\infty f_X(x)dx=1 \quad (F_x(\infty)=1) \!\,

Theorem[edit]

 
\begin{align}
P[x_1 \leq X \leq x_2] &= F_X(x_2)-F_X(x_1) \\
                       &= \int_{-\infty}^{x_2} f_X(x)dx - \int_{-\infty}^{x_1} f_X(x) dx \\
                       &= \int_{x_1}^{x_2} f_X(x)
\end{align}
\!\,

Some useful continuous Random Variables[edit]

Uniform R.V[edit]

f_X(x)=
\begin{cases}
\frac{1}{b-a} & a \leq x \leq b \\
0             & \mbox{otherwise}
\end{cases}

 \!\,


Exponential R.V[edit]

f_X(x)=ae^{-ax} \quad x \geq 0\!\,


F_x(x)=1-e^{-ax} \!\,


Gaussian (Normal) R.V.[edit]

\mathcal{N} (\mu, \sigma^2) \quad \infty < x< \infty\!\,


f_X(x)=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \!\,


F_X(x) = \int_{-\infty^1}\frac{1}{\sqrt{2 \pi \sigma^2}}e^{-\frac{x-\mu}{2\sigma^2}} \!\,