Question: Markov Chains with Reward. You might consider your automobile and its random failures to be a Markov chain. It makes monthly transitions between the states "running well" and "not running well." When it is not running well it must be taken to a garage to be repaved, at a cost of $50. It is possible to improve the Iikelihood that the automobile will continue to run well by having monthly service at a cost of $10. Depending on your policy your automobile transitions will be governed by either of two Markov chains, defined by
(a) For each of the two policies, what is the equilibrium distribution of states?
(b) In equilibrium, what is the average monthly cost of each of the two policies?