Flash and JavaScript are required for this feature.
Download the video from iTunes U or the Internet Archive.
Description: This lecture starts with historical applications of error control and convolutional codes in space programs. Convolutional codes are introduced along with examples and transformations in shift-register, state-machine, and trellis view.
Instructor: George Verghese

Lecture 6: Convolutional Codes
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR: I wanted to tell you a little bit about the use of digital communication schemes in the space program. And part of that is it wasn't just the use. A lot of coding theory was developed for use in this program. So in the early days, there was no error control coding. So they had very slow transmission rates and tried to compensate for not having error control coding by taking a long time to send a bit over. But in later years, with the Mariner and Viking probes, they started to use error control codes.
And linear block codes are what we're talking about. This would be the typical parameters for such a code. So we know how to read this. This is 32 bits per block. Six data bits. And minimum timing distance of 16. A particular kind of code called a bar orthogonal code or Hadamard code, which had specific characteristics and specific symmetries that actually helped with the decoding.
So for instance, on Mariner Nine, it's 1971. This went into Mars' orbit. And the code was used to encode the picture transmissions. So each data word was six bits to encode 64 gray levels and a picture. It turned out that, because of transmission issues, the safe number of bits for a block was 30 bits. And after that, you had to do a little bit of realigning or tweaking. So you could send 30 bits at a time safely. And so, that was a choice of n in that vicinity was a natural choice.
One thing you could have thought to do would be take the six bits and repeat them five times in the 30 bit window. And that would be a repetition code. It turned out that, with this particular Hadamard code, you could actually get the same data rates or comparable data rates. Let's see. What would the data rate be? It would be k over n, right? 6 over 32. But with much better error correction properties.
So let's see. How many errors could you correct in this code per block? Somebody? Seven. Yeah. Because you've got a minimum Hamming distance of 16. So you want D minus 1 over 2, the floor of that. So you could correct up to seven errors per block. And this code was actually used on space probes right into the 80s. And as I mentioned, this particular code has various symmetries that allow actually something called the Fast Fourier Transform to be used in the decoding. And so, that's really what drove this.
As you read about these probes, it's actually staggering how much they did with so little. Let's see. This thing went half a billion miles almost. It had an onboard computer with a memory of 512 words. So you can imagine the kind of engineering that went into organizing all of this. The transmitters-- and this is typical of these space probes-- you don't have a lot of energy generated from your solar panels necessarily. So 20 watt transmitters. So these have to transmit over this kind of distance the data that you want to send in the presence of noise and various other errors. OK?
So quite an engineering feat. Now the kinds of pictures that you would get? Well, these are pretty amazing actually, considering what the probes had to do. So over the lifetime, it sent over 7,000 images. Mariner 9 is still orbiting Mars, from what I understand. It's not sending back. It stopped sending back transmissions one or two years after this. But it's still in orbit until it slows down enough to crash in.
OK. So as I said, you're typically talking about low power. 20 watts. WNBR, what's a typical radio station power on a college campus? They advertise something on the order of 700 watts for their transmitter. So we're talking about doing a lot with a little here. A lot of the art is in the antenna. So you have an antenna that directs this power very sharply towards the intended receiver. But the more sharply you try to direct that, the bigger of a control problem you have. Because you've got to point that antenna all that more carefully. So all of these are coupled issues.
And then at the receiver end, you've got very high quality amplifiers and signal processing. But the data coding and error correction schemes are a key part of that. And it turns out that, as you got more ambitious with these transmissions, you had to go to more complicated codes. And these are the codes we're going to talk about today, what are called convolutional codes. We'll talk about the coding today. And then we'll talk about the decoding with what's called the Viterbi algorithm next lecture.
So this has been used extensively from late 1970s onwards. More recently, you have codes that are actually combinations of convolutional codes, what are called Turbo codes. And another family of codes, low density parity chat codes, which were developed in Bob Gallagher's PhD thesis here. Bob Gallagher's on our faculty. But convolutional codes were really a workhorse of the whole system.
OK. So an example is now Cassini, which is in orbit around Saturn. It's actively sending pictures. This, if I read the website correctly, is a picture from August 29. And I saw other pictures posted from June and July. So this is a picture of one of Saturn's moons. And you can see the rings and the shadows of the rings and so on. This is actually recreated in natural color from multiple images. This, I guess, is part picture and part artist's rendition. But that shows you what Cassini looks like. There's only one of them out there. I don't think there's something else to photograph Cassini.
So the kind of code that's used is a convolutional code. We'll learn what these parameters mean, how they enter into the definition of the code. And here is a typical code rate. You're talking about something on the order of 83,000 bits per second as the code rate here. Sorry, not the code rate. This is the data rate. OK? So the messages are coming. Let's see. You're sending six times this amount per second. But this is the rate at which the data is coming in. OK?
So convolutional codes. And again, I keep coming back to MIT names. Peter Elias was on our faculty here. He was a department head for a while. And in a short paper in 1955, he invented the idea of convolutional codes. So the idea here is not to divide up your data into blocks but to actually work on the streaming data. And as the data goes past, you generate parity bets at a regular rate. And what you transmit in most typical schemes are just the parity bits. You don't send the message bits. So this would be a non-systematic code, if you like. So there's no part of that message that's directly observing the message bits.
Now, you will actually generate and send multiple parity bids. So you'll have a message sequence, x zero xn. Sorry, x zero, x one. And from this, you derive parity bits. And you do that using the standard sorts of equations we've seen with block codes. Each parity bid here, for instance, parity bit zero at a time n will be some linear combination of message bits. But it's the message bits as they're streaming by. So you might have, for instance, this as your choice for parity but number zero.
OK. And then parity bit number one could be some other combination here. So for instance, xn plus x n minus 2, for instance. OK. So it's a linear combination of some set of messages, just the way we've been generating parity bits all along. The plus here, of course, is we're talking about binary messages. So this is addition in GF2. So it's exclusive or Modulo 2 addition. And you can imagine a whole bunch of such parity bits.
So in general, you would have R such parity bits computed off some set of message bits and transmitted instead of the message bits. So you might have, for each message been coming in, you might actually be sending out R parity bits. So what you do is just send these out in sequence. You'd send out the P zero value, the P1 value, time n. Then recompute a time n plus 1 and keep going. All right? Well, actually I have them here. I didn't see that.
So all this happens on a sliding window. This happens for a particular choice of n. And then it happens for the next choice of n and the next choice of n. So you're doing this on the fly with a streaming sequence. So let me just put up an equation that explains why this is called a convolutional code. It turns out the expressions of this type where you take a data stream coming in and generate new data streams of this form, it turns out that the operation that's being carried out here is something referred to as convolution.
So in general, what has P zero then? It's some weighted combination of x at the current time, x1 times step back, x2 time steps back. In general, k different values involved. So what I have is a P zero n being a summation from, let's say, J equals zero. So G zero J x of n minus J. All right? So this is just some set of numbers. Zero, one. Just as these bits are, these are zero, one. But this is the general form.
This particular kind of combination is referred to as a convolution operation on the input stream. And we'll see much more of this when we come later to modeling channels, the physical channels. We'll talk about convolution type models. So here, it's not so important that you must have this expression. We'll have plenty of opportunity to work with expressions like this. This is just for you to know that an expression of this type, wherever you see a summation with indices that are in this form, this is referred to as a convolution. OK?
So it's convolution of the message stream with some set of weights.
AUDIENCE: Professor?
PROFESSOR: Yeah?
AUDIENCE: What does G stand for?
PROFESSOR: The G is just a set of weights here. So in this particular case, for parity expression zero, G zero of zero would be 1. G zero of 1 would be 1. G zero of 2 would be 1.
AUDIENCE: OK.
PROFESSOR: It's just a set of weights. So yeah. This expression is a bit of overkill for the kind of use we're making of it. But it's just to explain the origin of the name. It turns out later, when we use it for channel modeling, the x's will not just be zeros or ones. They could take arbitrary real values. And the Gs could take arbitrary real values. So we'll be working with much more elaborate versions of this. OK?
The number k is referred to as the constraint length. And it's the maximum number of message bits involved when you look over all your parity expressions. So in this particular instance, k would be equal to 3. Right? It's the maximum window of data that you're using in a non-trivial way to generate the bits. So here, you are using up to 3 to generate this. Well, in this case also, you're using a window of three message bits. It happens that you're ignoring the one in the center. But the constraint length is the length of message that you're actually looking at.
OK. So in some sense, if you want to think of it this way, the number of parity expressions that you use? Well, that's straightforward. That's just telling you how much redundancy you're willing to put in. Whereas the constraint length is telling you how deeply you're folding that redundancy into the message. So the bigger the constraint length, the more message bits are involved in generating a parity bit. And so, the more you're scrambling up the message and spreading it over a large section of what's transmitted. And so, you might expect that you get a better error correction properties with larger constraint links. OK?
OK. This is not saying anything new. So how do we come to actually transmitting? Well, we generate the parity bits. And then, as I said, you send all the parity bits associated with your computation at time zero. Then all the parity bits associated with the computation at time 1, time 2, and so on. So in the case of the code used on the Cassini probe, that's 1 over 6 rate code. It's actually computing six parity expressions. So it's transmitting six parity bits for each message bit that comes in. What happens then at the next time instant is that you shift everything up by one. And we do the whole thing. OK?
Now, you can actually-- and I'll have this up on the slides-- you can actually crank through the equations. But it's not the most illuminating way to think of things. It's much easier to think of it visually through a block diagram of this type and using the idea of what's called a shift register. So what is a shift register? You may have encountered it in other places.
So we think of a shift register as, it's basically a box that can remember something. OK? That's the register part of it. A register is something that remembers a number. You've got some input stream that comes in and some output stream emerging. At any time, this stores a particular number which is available to the output. So whatever is stored in the register is available to the output. The shift part of this description is that whatever is of the input will get shifted into at the next clock cycle or the next time instance. OK? So the input gets shifted in at the next clock cycle. Whatever is in here is remembered for that one clock cycle and is available at the output. Right?
So if I have a sequence x n being fed in four n zero, one, two, three, and so on, if I'm seeing x n here, what must have gone into the previous time? If I see x n here at time n, if I'm seeing a particular input at time n, what must have gone in the previous time as x n minus 1? So what's sitting here is x minus 1. Right? And x n minus 1 is available to me at the output. The next clock cycle, the next input comes along. The xn goes in here. And the whole thing shifts. All right?
Now what you have up there is a cascade of shift registers. You've got some shift registers. So keep in mind, the operation that I described-- if this is xn, if I'm looking at this time at time n, xn sitting here-- what must be in this shift register is the input of the previous time. So that's x n minus 1. These are shown adjacent. What we really mean is that one shift register is feeding into the next one. They're just shown as adjacent. But what must be sitting here then is x n minus 2. And if I read off something from here, what I'm looking at is x n minus 1. Namely, what's sitting in the register. What I'm looking at here is x n minus 2.
All right. So do you see how this is working now? This is actually the same example that I had written up earlier, I guess, for the computational parity bit. So here's-- except it's-- yeah. It's the same one. P zero n. Maybe I have the equations. Let's see if I can display them for you. No, I can't. OK. So what's P zero n? P zero n is xn, that's connecting from here, plus x n minus 1 plus x n minus 2.
Again, by the way, in this diagram, what I showed as an arrow coming from the output of the shift register is just a shorthand here that shows the arrow coming out from the body of the register. It's the same thing we're talking about. OK? So P zero of n is the sum of these three message bits. So we're talking one constraint length three here. And what about P1 n? It's xn plus x n minus 2 with nothing of xn minus 1. All right? So imagine this being the picture for every n. So you start off at time zero and keep going. Right?
We refer to the state of the shift registers as the pair of numbers that we find in here. So if we're talking about x's that can be zeros or ones, the shift register combination here can be in one of four states. Right? Zero, zero. Zero, 1. 1, zero. Or 1, 1. So four states. So here's a four state shift register into which we're feeding in the stream. And what gets put out on the channel are these parity bits interleaved. That clear enough? OK.
Nothing I haven't said here. Right? So let's actually work through an example step by step. This is clear enough. But let's just see it concretely. Let's assume that I'm starting out with the shift registers in the zero state. And now, I've got this message sequence coming in that I want to send out. OK? So the sequence is 1, zero, 1, 1. So the first bit that appears here is the 1. And I've got to generate P zero and P1. Well, P zero is the exclusive OR of these three things. So it's 1. P1 is the exclusive OR of the first and the last. So it's again 1. So that defines P zero and P1 at time n.
The same way at the next time instant, the next message input bit comes in. So we had 1, zero, 1, 1. We took care of the 1 here. Now comes a zero. We do the same thing. So the exclusive OR of all three of them appears here. That's the 1. The exclusive OR of the first and the last appears there. And that's the zero. So you can see how things are getting folded together because the input that was here before is now sitting in here and plays a role in generation of the parity bit for the next step.
In fact, the word convolve means to fold together. And this is what it's actually trying to capture. You're folding together these two sets of weights, the weights on the top tier and the input sequence weight. And then the next two cases, similarly. OK. And that's what gets sent out at the bottom. So this is the transmitted sequence. So it's the 11100001. Right? That's all there is to it. The implementation of the shift register is very easy. And so, this is actually a very straightforward thing to implement.
Now there's another viewpoint that's also very useful here. Another way to look at what's going on, which is thinking in terms of the state of the register and how you move between the states. I guess, how many here are 004? Are those are the ones with smiles on their faces? OK. You see a lot of this there, I imagine. OK. So how do I read a diagram like this?
I've got a circle for each state that the shift register can be in. So the shift register can be in zero, zero. Zero, 1. 1, zero. 1, 1. Right? Each of these arcs represents a transition from one state to another. So let me ask you this. What does it take-- if I'm in the zero, zero state with my shift register-- so what you've got in the picture is your shift register sitting there with zero, zero. What does it take for me to get to the 1, zero state? What must my input have been to get to the 1, zero state?
Imagine how these shift registers operate. Right? If I'm going to get from zero, zero to 1, zero, I must have fed in a 1 at the previous time instance. So it takes an input of 1 to go from zero, zero to the 1, zero. So to go from zero, zero to 1, zero, use an input of 1. That's the number that we write before the slash. That's our labeling convention for the arcs. We put the input that it takes to make that transition. And then after the slash, we put the parity bits that are omitted. So what we've got for the 1, 1 is the parity bits that are omitted when you've got input 1 sitting here, zero, zero here, and you're using the parity computation that I had before.
Let's see here. So P zero is going to be x n plus x n minus 1 plus x n minus 2. So that gives you 1. And what about P1? P1 is x n plus xn minus 2. So that gives you another one. OK? So if you're in state zero-- the zero, zero state-- and you get an input of 1, you're going to transition to 1, zero. And you're going to omit 1, 1. OK. So the state diagram captures all that.
And similarly, all the way around. So I haven't checked each of these. But I hope there are no mistakes in it. But if you're in 1, zero? Oh well. By the way, if you're in zero, zero, there's no way to get to zero, 1. Right? So you don't see any arc from zero, zero to zero, 1. If you're in 1, zero, you can get to 1, 1. Or you can get to zero, 1 depending on what you feed in. OK? So it's very straightforward, then, to actually build out this diagram. Why don't we do a little bit more on here?
OK. So if I'm actually abstracting from the shift register picture to something that's more like the state picture, I'm going to say, here are my four states. I've just drawn it a little differently than I have in the upper picture. Instead of circles with these states in them, I prefer to think of them this way. So what we said is, if you get an input of 1, you limit 1, 1. And you'll get to that state. What does it take to get to the state? Somebody? Can I have a hand and a loud voice? Yeah?
AUDIENCE: Input zero.
PROFESSOR: OK. And then I guess you've got to go back to this to think about what's happening. So I'll allow you to think of a zero sitting at the input here. So what would the parity bits be? So the first parity bit will be the exclusive OR of the zero, 1, and zero. So it's going to give you a 1. Right? And then the next parity bit is going to be exclusive OR of what's here and there. So that's going to be a zero. I hope that matches with what I have upstairs. We're talking about going from 1, zero to zero, 1. It takes a zero input to do that. And what you omit is 1, zero. Right? So you can fill in all of these. This is the state transition diagram.
OK. Let's see. We say that, if you've got a constraint length of three then, of k equals three, for instance, or let's say if you've got a constraint length of k, you've got 2 to the k minus 1 states. well, that's because, in that constraint length, one of the bits involved is the input bit. That's not sitting in the shift registers. So you've got k minus 1 bits left over. So your shift register is k minus 1 stages long. And so, you've got 2 to the k minus 1 states. All right. So you could imagine generalizing this to more complicated sorts of situations.
Let's see. Just going back to the Cassini example, if you let me jump back a bit, there was a k there. What was it? k of 15. OK. So for Cassini, you're using one input bit and 14 more bits in your register. OK? So you've got 2 to the 14 possible states there. So in these codes, you're actually using very large constraint lengths.
OK. All right. I want to go from the state machine view to another view now, which is what's called-- so this is the state machine view-- to something called the trellis view. This is something-- by the way, there was a way of looking at things that was developed by someone else who was on our faculty, David Forni. In fact, if you visit his home, you'll see his garden. There's a nice trellis around it. And you'll see why when we draw this.
OK. So what's the trellis view? The trellis view says take the state machine but unfold it in time so that all your transitions over time are not happening here. At every time step, you draw the picture again and look to see where you get to. So let's do this. This is the one I want to be most careful with and where I'll introduce a few notational conventions so that our later life is simplified.
OK. So we've got state zero, zero. State zero, 1. State 1, zero. And state 1, 1. OK. Except, though, this is going to be the picture that I have at time-- let's say-- at time n equals zero. At time n equals 1, well, I've got the same shift registers. I'm going to draw this picture again. The easiest way to learn this is to just follow through one example. So please keep your attention here. And you'll have it sorted out. And then you won't have to worry about it again. It's the same thing as with LZW.
All right. So it looks kind of detailed. Maybe tedious. But it's actually very simple. Just hang in there and follow through one example. OK. So what does this say? At time n equals zero, I'm in zero, zero. Suppose I get the input zero. Suppose the input is zero. What state do I transition to? Here. Right? So if I have an input zero, I'm going to transition here. So this is with an input of zero. And what are my parity bits going to be? Both zeros. Right?
What about if I get an input of one? Where do I transition to? Well, we've already seen that here. If I get an input of 1, I'm going to transition to here. And what am I going to omit? Well, we've already calculated that. We're going to omit a 1, 1. Right? Let's do it for one more case. We're in zero, 1. What states can I transition to? I could go to zero, zero. And I would do that if my input was zero. Right? And what would my parity bits be? Well, that's another case for us to look at. If our input is zero and we're in state zero, 1, what would the parity bits be?
For this choice of parity bits, depends on what specific choice you made, of course. 1, 1? Do you agree? And if I get an input of 1 instead, where do I go? If I get an input of 1, I'm going to go to 1, zero, which is here. OK? So if I had an input of 1, I would go to 1, zero. My parity bits would be-- what would they be? Can I have a hand and a voice? Yeah. Zero, zero. Right.
OK. So it's that simple. That's all you have to do. Fill out this picture and you're seeing what this picture translates to at the next time instant. We're not using anything more than is in the state transition-- sorry-- in the state machine diagram. But we're unfolding things in time, which is actually very helpful. Now, this is a simplification we'll make in drawing this. Because I've arranged the states in natural binary counting order-- zero zero, zero 1, 1 zero, and 1, 1-- it's always the case that the upper arrow that emanates from a state corresponds to the input of zero. And the lower corresponds to an input of one. OK?
So I don't really need that first thing before the dash. I'm just going to dispense with it. So if you're going up, of the two choices that you have when you come out of a box, if you're going up, it's zero. The input is zero. And if you're going down, the input is 1. So I'm just going to label that as zero, zero. OK? I'm going to label this as 1, 1. And I guess I've forgotten already what some of these are. But you can see what the whole picture starts to look like. OK?
So let me actually, I'm not going to do these in detail. But let's just see how the next stage would differ, if at all. When I come to n equals 2, well, it's the same story all over again. So whatever pattern of arrows I had coming out of here, I have the same pattern at the next stage with the same labels. Right? Because there's nothing different.
So if you'll allow me, let me actually fill out a few of these. And you'll get practice drawing one of these when you do recitation maybe for another example. So I can keep going with these. Let's see here. This is going to be 1, 1. If I haven't found two arrows coming out of each box, then I'm not done. Oh, this is wrong. Right? Thank you. From zero, 1, I can go to 1, zero.
OK. So the two arrows coming out of each one, the upper arrow corresponds to the input having been zero. The lower arrow corresponds to an input having been 1. And there are two arrows going into each box, as well, corresponding to whether the bit that's going to get dropped off is a zero or a 1. All right? So there's a real symmetry to this. I'll draw one more stage just a little bit to make a point here.
OK. So you can keep going. So how do you generate a code word from a trellis diagram? You're starting in some state. Typically, it's the all zero state. In fact, what you'll usually do is have a header for your message stream which is all zero. So you force the shift register to be in the zero state once the real message bits come in. And then you move from here. So you're typically starting here. And then you navigate, depending on whether you've got a zero or a 1.
So if the first message bit is a 1, you're going to go down here. If the next message bit is a zero, you're going to go up here. If the next message bit is a zero, you're going to go up there. And the code word that you omit is going to be, in that case, 1, 1, 1, zero, 1, 1. And then all zeros. Right? Assuming you're staying at zero from then on. So depending on what the message sequence is, you can actually go through the trellis. It's infinitely long, or as long as your message sequence is. And figure out what the code word is that's omitted. So this is actually just a graphical way of displaying code words.
So the set of code words that I get, does that correspond to a linear code? Let's assume that, somewhere downstream, all these things come back down to zero, zero. OK? So I'm only considering a finite window of things. It's not going to go on forever. So suppose I'm going to end my input messages with zero, zero at the end and come back down to that state. So my messages will always start with zero, zero to force the register to the zero, zero state. And they'll end with zero, zero. OK?
The set of possible code words is a set of parity bits I omit along the way as I navigate through the trellis. Is the set of code words, does the set of code words constitute a linear code? That's the question. Maybe not obvious. Right? The way you answer that is actually thinking back to this setting. So one particular code word would correspond to a particular input sequence that generated it. A particular data message sequence that generated it. Another code would correspond to another message sequence. And the question is, is there a message sequence that would generate the sum of these two code words that you have?
And actually, it turns out that the answer is yes because these parity relationships are based on a nice linear operation. OK? So it turns out that the set of code words that you generate constitutes a linear code. So if you were going to think of a minimum Hamming distance for this code, what would you want to be thinking of? I don't know if I've actually drawn this correctly right now. Has anyone spotted any errors along the way? Or do I have it right? Seems to be OK.
What would, how would you look for a minimum Hamming distance in the set of code words generated over this window?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Sorry? I didn't hear where that came from. Yeah. Can you speak up? The minimum number of ones in a non-zero code word. Right? So it would be the weight, the minimum weight code word you'd find among all the number of code words. So you would have to find a path starting here. All my code words are going to start here and there. You have to find a path through this that picks up the minimum number of ones. A path that's different from the all zeros path.
OK? Find a path through there which has the minimum number of ones in it and the code word. So what would that be? In this particular case, maybe this path we highlighted in another color. I don't have-- this is not a proof. This is just a suggestion that this might be it. And you would have to draw in all the paths, explore all the other paths. But what would be the minimum weight along this one? Let's see. I've got 1, 1 there. 1, zero here. 1, 1 there. So I would get a weight of 5 on this path.
Now the question is whether you could find another path with a smaller number of ones attached to the code word. And I think if you work this out in detail on [INAUDIBLE], that you'll find that you're actually stuck with 5. OK? Now it turns out that the interpretation of this number is not quite as straightforward as the interpretation of the minimum Hamming distance in block codes. And the reason is that actually this is a more complicated kind of picture because it continues on with the structure. So we don't actually call it the minimum Hamming distance. We call it the free distance. So I'm just trying to evoke this.
So the minimum weight code word you find among the non-zero code words will indeed be a code word of eight 5. But the interpretation of that number may not be directly as simple as in the case of the Hamming distance. But it's close. OK? So what it really tells you is that, over a data length that is maybe not much longer than this, the code words that you have are separated by this distance minimum. So you might expect that you could correct two bit errors over data lengths that correspond to code words that are somewhat longer than this perhaps. OK?
So that's all very hand-wavy. But that's all we're going to do with the notion of free distance. So this is more complicated to deal with than a block code. But the free distance actually has that kind of intuition. It has the intuition of minimum Hamming distance locally over this window of data. Even if this went on for thousands of bits, if you got a burst of errors in this stretch that had up to two errors, you could correct them. Now we haven't talked about decoding. We're going to talk about that the next time.
OK. So that answers this piece. Now let me say one thing about decoding, just to set us up for next time. If I didn't have any noise in my channel, it actually turns out that decoding is pretty trivial. How is that? If I gave you the sequence of parity bits, can you think of a way that you could recover the input sequence? Yeah?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Good. Yeah. You see, if I add these two, I get x n minus 1 equals P zero n plus P1 n. So if you give me the parity bit stream, I can reconstruct for you exactly the input with a one step delay. That's pretty good. I'm happy with that. If it's taken me minutes for the signal to reach me from Saturn, I'm happy with a one step delay here in decoding. All right?
So in the absence of noise, the inversion is simple. The inversion meaning deducing the input message bits from the output, the parity bits. And this is a theme you'll see in many other settings. If there's no noise, inversion is easy. You can look at the output of a system and figure out what the input was. If you know exactly how the system was creating the output from the input. But in the presence of noise, you've got a problem. Because you see, if you have these parity bits corrupted at some rate-- every few bits, you've got errors-- well, you're interpreted message is going to have that same kind of error rate.
OK? So it's really, in the presence of noise, it's an unsatisfactory way to do it. So this doesn't work. We'll be looking at something more careful next time. OK? So well actually, since I have you here, let me put up the spot quiz. We haven't quite hit the mark. So can you answer these for me? What's the constraint length of this code? Anyone? Who hasn't answered? Yeah? 4. Right? Because you've got xn and xn minus 1, xn minus 2, xn minus 3. That's the largest window over which you're picking things.
What about the code rate? 1 over 3? Right? Because for every message bit, you're generating three parity bits. You're going to shift out those three parity bits before you do the shifting on the shift register. So the code rate is 1 over 3. The coefficients of the generators, of course, you can read up there. What about the number of states in the state machine here?
What? 8. Right? Because constraint length is 4. But one of those is the input. So you've got three bits that you're storing in memory. 2 to the 3 is 8. So the number of states in the machine is 8. OK? So more complicated picture than this. But the same principle. All right. We'll continue next time.