Assignment:
1. Prove the following statements
a) If x, y, z are discrete rvs
H(x,y|z) ≥ H(x|y,z)
b) If x is a discrete rv and w = g(x) with g an arbitrary real function of a real variable,
H(w)≤ H(x)
c) If x, y are discrete rvs and u = x + y
H(u|x) = H(y|x)
2. The Morse code used in telegraphy has the following coding map:
a) While in the written version words are separated by a blank space, in Morse code they are separated by '/'. Thus, the phrase "Morse code" would be written as
We can view the written Morse code as a variable length code with Ay = {·, -, /}. Is it a prefix code? How is it decodable? Find its average length and efficiency assuming that letters are statistically independent with the following distribution (empirically derived from their relative frequency in the text of this chapter).
b) For transmission, the symbols "·" and "-" are represented by "signal on" intervals with duration T and 3T , respectively. Consecutive symbols in the representation of a letter are separated by a "signal-off" interval with duration T. Consecutive letters in a same word and consecutive words are separated by "signal off" intervals with duration 3T and 7T, respectively. For example, the phrase "Morse code" would be transmitted as
Find the average duration of the representation for a letter in the case T = 0.12 s (corresponding to 10 words per minute), the efficiency and the information rate of the message s.