UMD Probability Qualifying Exams/Aug2010Probability
Problem 1[edit]
Two persons, A and B, are playing a game. If A winsa a round, he gets $4 from B and wins the next round with probability 0.7. If A loses the round, he pays $5 to B and wins the next round with probability 0.5. (i) Write downt he transition matrix of the Markov chain with two states, {A won the current round, B won the current round} and find the stationary probabilities of the states (ii) Find . 
Solution[edit]
(i) The Markov transition matrix will be the 2x2 matrix where corresponds to a win for Player A and corresponds to a loss for Player A. For example, is the probability that Player A wins after winning in the previous hand; is the probability that Player A wins after losing in the previous hand; etc. This will give
The stationary distribution will be the tuple such that . We can calculate this explicitly:
yields the following system of equations: Using the fact that must be a probability (i.e. ) we get .
(ii) Since is positive, and hence ergodic, then any initial probability distribution will converge to the stationary distribution just calculated, . Thus as Player A will win with probability . Can Player A expect to have more money though? For sufficiently large we can compute Player A's expected winnings in one round:
Thus Player A should expect to have more money than before the game with probability 1.
Problem 2[edit]
(i) Let be a random variable with zero mean and finite variance . Show that for any . (ii) Let be a squareintegrable martingale with . Show that for any . 
Solution[edit]
(i) were the secondtolast inequality is the standard Chebyshev's inequality.
Problem 3[edit]

Solution[edit]
Problem 4[edit]
Let be random variables with finite expectations. (i) Show that implies . (ii) Show that if is identically distributed with , then

Solution[edit]
(i) Let . Easy to see that is convex.
Then by Jensen's Inequality we have
. Taking the expectation on both sides gives
.