7 - Linear codes

Summary

Introduction: The generator and parity-check matrices

We have already noted that the channel coding theorem (Theorem 2.4) is unsatisfactory from a practical standpoint. This is because the codes whose existence is proved there suffer from at least three distinct defects:

1. (a) They are hard to find (although the proof of Theorem 2.4 suggests that a code chosen “at random” is likely to be pretty good, provided its length is large enough).

2. (b) They are hard to analyze. (Given a code, how are we to know how good it is? The impossibility of computing the error probability for a fixed code is what led us to the random coding artifice in the first place!)

3. (c) They are hard to implement. (In particular, they are hard to decode: the decoding algorithm suggested in the proof of Theorem 2.4–search the region S(y) for codewords, and so on–is hopelessly complex unless the code is trivially small.)

In fact, virtually the only coding scheme we have encountered so far which suffers from none of these defects is the (7, 4) Hamming code of the Introduction. In this chapter we show that the Hamming code is a member of a very large class of codes, the linear codes, and in Chapters 7–9 we show that there are some very good linear codes which are free from the three defects cited above.