Let qi,i+1 = 2i-1 for all i ≥ 0 and let qi,i-1 = 2i-1 for all i ≥ 1. All other transition rates are 0.
(a) Solve the steady-state equations and show that pi = 2-i-1 for all i ≥ 0.
(b) Find the transition probabilities for the embedded Markov chain and show that the chain is null recurrent.
(c) For any state i, consider the renewal process for which the Markov process starts in state i and renewals occur on each transition to state i. Show that, for each i ≥ 1, the expected inter-renewal interval is equal to 2. Hint: Use renewal-reward theory.
(d) Show that the expected number of transitions between each entry into state i is infinite. Explain why this doesnot mean that an infinite number of transitions can occur in a finite time.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.