Let {Xn}n∈N0 be a Markov chain with the transition matrix
Find all stationary distributions.
The chain starts from the state i = 1. What is the expected number of steps before it returns to 1?
How many times, on average, does the chain visit 2 between two consecutive visits to 1?
Each time the chain visits the state 1, $1 is added to an account, $2 for the state 2, and nothing in the state 3.
Estimate the amount of money on the account after 10000 transitions?
You may assume that the law of large numbers for the Markov chains provides an adequate approximation