A person goes to his office everyday. He has a single umbrella. When he is home and the umbrella is home, he can choose to bring it. He will certainly do so if it is raining, but must decide what to do if is not raining, as it may rain in the afternoon. A similar decision must be made in the afternoon when he is heading home. Suppose that if it rained yesterday afternoon (this morning), but not this morning (this afternoon), the probability that rains this afternoon (tomorrow morning) is q. On the other hand, if it did not rain yesterday afternoon (this morning) nor today (this afternoon), the chance it rains this afternoon (tomorrow morning) drops to r. If it is raining on a given afternoon (morning), it will rain the following morning (same evening) with probability p, irrespective of what happened before.
(a) Model the policy bring the umbrella only when it rains" as a Markov chain by defining the state space and transition probabilities.
(b) What percentage of the time is the employee going to be wet?