Determinining entropies and shanon fano code for messages

Question

Question 1)a) What do you mean by information? What are its units? How does it relate to the entropy?

b) Suppose we have ten messages of probabilities: P(m₁) = 0.49, P(m₂) = 0.14,P(m₃) = 0.14, P(m₄) = 0.07, P(m₅) = 0.07, P(m₆)=.04, P(m7) = 0.02, P(m₈) = 0.02, P(m₉) = 0.005, P(m_l0) = 0.005. Determine the Shanon Fano code for the set of messages. Determine coding efficiency and redundancy.

Question 2) A transmitter has the alphabet consisting of five letters (x1 x2, x3, x4, x5) and receiver has the alphabet of four letters (y1, y2, y3, y4). The probabilities are P (x₁) = 0.2 5, P(x₂) = 0. 4, P(x₃) = 0.1 5, P(x₄) = 0.15, P(x₅) = 0.05 and

P(Y/X) =

         Y₁         Y₂        y₃       y₄
X₁      1          0          0          0
X₂    0.25     0.75       0          0
X₃      0        1/3       2/3        0
X₄      0          0         1/3      2/3
X₅      0          0           1          0

Compute entropies {H(Y), H(X, Y), H(X), H(X/Y), H(Y/X)} and I(X, Y).

Question 3)a) State and describe Sampling theorem in detail. Explain how is it useful in communication systems?

b) Explain Shanon’s Hartley theorem based on channel capacity. How does channel capacity modify if bandwidth is increased to infinity? Explain the orthogonal signalling performance on the basis of theorems.

Question 4) Describe the significance of the following:

a) Companding.

b) Prediction filters.

c) Adaptive filters.

d) Equalization