Determine joint-conditional entropies for pair of integer

Question

Assignment:

1. Find the joint and conditional entropies for the pair of integer-valued random variables x, y with

A[x,y] = {[m,n] ε Z² : 1 ≤ n ≤ m ≤ 4}, P_x,y(m,n) = 1/4m

What is the efficiency of the vector x, y ?

2. Consider an infinite message {xn} with iid symbols and equally likely values in {0, 1}, and the message {y_n} defined as y_n = x_nx_n-1.

3. Consider an unlimited binary message {xn} with iid symbols taking values in {0, 1} with equal probability, and another message {yn} obtained from it as y_n = x_n - x_n-1.

a) Find the entropy of the rv y0.

b) Find the entropy of the rve [y-1, y₀, y₁].

c) Find the entropy per symbol and the efficiency of the message {y_n}.