Consider an inventory system which the sequence of events during each period is as follows.
(i) We observe the inventory level (call it i) at the beginning of the period.
(ii) If i <= 1, 4 – i units are ordered. If i >= 2, 0 units are ordered. Delivery of all ordered units is immediate.
(iii) With probability 1/3, 0 units are demanded during the period; with probability of 1/3, 1 unit is demanded during the period; and with probability 1/3, 2 units are demanded during the period.
(iv) We observe the inventory level at the beginning of the next period.
a) Defines a period state to be period’s beginning inventory level. Determine the transition matrix and diagram that could be used to model this inventory system as a Markov chain
b) Obtain the probability of all states 3 periods from now.