信息论-习题答案

1．设信源4.06.0)(21xxXPX通过一干扰信道，接收符号为Y={y1,y2}，信道转移矩阵为43416165，求：(1)信源X中事件x1和事件x2分别包含的自信息量；(2)收到消息yj(j=1,2)后，获得的关于xi(i=1,2)的信息量；(3)信源X和信宿Y的信息熵；(4)信道疑义度H(X/Y)和噪声熵H(Y/X)；(5)接收到信息Y后获得的平均互信息量。解：1)bitxpxIbitxpxI322.14.0log)(log)(737.06.0log)(log)(222221212)bitypxypyxIbitypxypyxIbitypxypyxIbitypxypyxIxypxpxypxpypxypxpxypxpyp907.04.04/3log)()/(log);(263.16.04/1log)()/(log);(263.14.06/1log)()/(log);(474.06.06/5log)()/(log);(4.0434.0616.0)/()()/()()(6.0414.0656.0)/()()/()()(2222222212121222122212111211222121221211113)symbolbitypypYHsymbolbitxpxpXHjjjiii/971.010log)4.0log4.06.0log6.0()(log)()(/971.010log)4.0log4.06.0log6.0()(log)()(224)symbolbitYHXYHXHYXHYXHYHXYHXHsymbolbitxypxypxpXYHijijiji/715.0971.0715.0971.0)()/()()/()/()()/()(/715.010log)43log434.041log414.061log616.065log656.0()/(log)/()()/(25)symbolbitYXHXHYXI/256.0715.0971.0)/()();(2．设二元对称信道的传递矩阵为32313132(1)若P(0)=3/4,P(1)=1/4，求H(X),H(X/Y),H(Y/X)和I(X;Y)；(2)求该信道的信道容量及其达到信道容量时的输入概率分布；解：1)symbolbitYXHXHYXIsymbolbitXYHYHXHYXHXYHYHYXHXHYXIsymbolbitypYHxypxpxypxpyxpyxpypxypxpxypxpyxpyxpypsymbolbitxypxypxpXYHsymbolbitxpXHjjijijijiii/062.0749.0811.0)/()();(/749.0918.0980.0811.0)/()()()/()/()()/()();(/980.0)4167.0log4167.05833.0log5833.0()()(4167.032413143)/()()/()()()()(5833.031413243)/()()/()()()()(/918.010log)32lg324131lg314131lg314332lg3243()/(log)/()()/(/811.0)41log4143log43()()(2222212122212212111121112222)21)(/082.010log)32lg3231lg31(2loglog);(max222imixpsymbolbitHmYXIC3．黑白气象传真图的消息只有黑色和白色两种，即信源X={黑，白}。设黑色出现的概率为P(黑)=0.3，白色出现的概率为P(白)=0.7。(1)假设图上黑白消息出现前后没有关联，求熵H(X)；(2)假设消息前后有关联，其依赖关系为P(白/白)=0.9，P(黑/白)=0.1，P(白/黑)=0.2，P(黑/黑)=0.8，求此一阶马尔可夫信源的熵H2(X);解：(1)symbolbitxpxpXHiii/881.0)7.0log7.03.0log3.0()(log)()((2)symbolbiteepeepepHepepepepepepepepepepepepeepepeepepepeepepeepepepijijiji/553.09.0log9.0321.0log1.0322.0log2.0318.0log8.031)/(log)/()(3/2)(3/1)(1)()()(2)()(2.0)(9.0)()(1.0)(8.0)()/()()/()()()/()()/()()(212112122211121222221211114．某一无记忆信源的符号集为{0,1}，已知P(0)=1/4，P(1)=3/4。(1)求符号的平均熵；(2)有100个符号构成的序列，求某一特定序列（例如有m个“0”和（100-m）个“1”）的自信息量的表达式；(3)计算(2)中序列的熵。解：(1)symbolbitxpxpXHiii/811.043log4341log41)(log)()((2)bitmxpxIxpmiimmmi585.15.4143log)(log)(434341)(100100100100100(3)symbolbitXHXH/1.81811.0100)(100)(1005．已知一个高斯信道，输入信噪比（比率）为3。频带为3kHz，求最大可能传输的消息率。若信噪比提高到15，理论上传送同样的信息率所需的频带为多少？解：HzPPCWsbitPPWCNXtNXt1500151log60001log/600031log30001log226．试求以下各信道矩阵代表的信道的容量：黑白p(黑/黑)=0.8e1e2p(白/白)=0.9p(白/白)=0.1p(白/黑)=0.2(1)[P]=0010100000010100(2)[P]=100100010010001001(3)[P]=3.0001.0002.0004.00007.0003.00004.0003.0002.0001.0解：1)这个信道是一一对应的无干扰信道ymbolbitnCs/24loglog222)这个信道是归并的无干扰信道ymbolbitmCs/585.13loglog223)这个信道是扩展的无干扰信道ymbolbitnCs/585.13loglog227．设信源01.01.015.017.018.019.02.0)(7654321xxxxxxxXPX(1)求信源熵H(X)；(2)编二进制香农码；(3)计算平均码长和编码效率。解：(1)symbolbitxpxpXHiii/609.2)01.0log01.01.0log1.015.0log15.017.0log17.018.0log18.019.0log19.02.0log2.0()(log)()(2222222712(2)xip(xi)pa(xi)ki码字x10.203000x20.190.23001x30.180.393011x40.170.573100x50.150.743101x60.10.8941110x70.010.9971111110(3)%1.8314.3609.2)()(14.301.071.0415.0317.0318.0319.032.03)(KXHRXHxpkKiii8．对信源01.01.015.017.018.019.02.0)(7654321xxxxxxxXPX编二进制费诺码，计算编码效率。解：xip(xi)编码码字kix10.200002x20.19100103x30.1810113x40.1710102x50.15101103x60.11011104x70.01111114%2.9574.2609.2)()(74.201.041.0415.0317.0218.0319.032.02)(KXHRXHxpkKiii9．对信源01.01.015.017.018.019.02.0)(7654321xxxxxxxXPX编二进制和三进制哈夫曼码，计算各自的平均码长和编码效率。解：二进制哈夫曼码：xip(xi)编码码字kis61s50.610s40.391s30.350s20.261x10.20102x20.191112x30.1800003x40.1710013x50.1500103s10.111x60.1001104x70.01101114%9.9572.2609.2)()(72.201.041.0415.0317.0318.0319.022.02)(KXHRXHxpkKiii10．设线性分组码的生成矩阵为110100011010101001G，求：（1）此（n，k）码的n=？k=？，写出此（n，k）码的所有码字。（2）求其对应的一致校验矩阵H。（3）确定最小码距，问此码能纠几位错？列出其能纠错的所有错误图样和对应的伴随式。（4）若接收码字为000110，用伴随式法求译码结果。解：1）n=6，k=3，由C=mG可得所有码字为：000000，001011，010110，011101，100101，101110，110011，1110002）此码是系统码，由G知，110011101P，则100101010110001011][IPHT3）由H可知，其任意2列线性无关，而有3列线性相关，故有3mind，能纠一位错。错误图样E伴随式TEHS1000001010100001100010000110001001000000100100000010014）由110TrHS知E＝010000，则010110ErR11．已知（7，4）循环码的生成多项式1)(3xxxg，求：（1）求该码的编码效率？（2）求其对应的一致校验多项式)(xh（3）写出该码的生成矩阵，校验矩阵。（4）若消息码式为21)(xxxm，求其码字。解：1）%14.57,4,7nkkn2）1)(1)(247xxxxgxxh3）1)(3124235346xxxxxxxxxxxxG，1101000011010000110100001101G而101110001011100010111H4）0110001,1)()()(45CxxxgxmxC12．在由1)(234xxxxg生成的(7,3)循环码中，求信息101的系统码字，并化画出编码电路图。解：因为462437)1()(xxxxxmx又因为1)()()(37xxrxgxmx所以1)(46xxxxc即该信息在所给循环中应编为（1010011）