Let x be an ensemble with px 01 02 03 04 the ensemble is

Question

Let X be an ensemble with PX = {0.1, 0.2, 0.3, 0.4}. The ensemble is encoded using the symbol code C = {0001, 001, 01, 1}. Consider the code word corresponding to x ∈ X^N, where N is large.

(a) Compute the entropy of the fourth bit of transmission.

(b) Compute the conditional entropy of the fourth bit given the third bit.

(c) Estimate the entropy of the hundredth bit.

(d) Estimate the conditional entropy of the hundredth bit given the ninety-ninth bit.