The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
CHARLES LEISERSON: Hey, everybody.
Let's get started here.
So last time we had the skull and crossbones,
this time we're going to have double skull and crossbones.
This stuff is really hard and really fun.
And we're going to talk about synchronization without locks.
And to start out, I want to talk about memory models.
And in particular, the most important memory model
from a theoretical point of view, which is sequential
consistency.
And to introduce it, I want to use an example
to introduce the notion of a memory model.
So suppose you have two variables, a
and b, which are initially 0, and those variables
are stored in memory.
And processor 0 moves a 1 into a, then
it moves the contents of ebx into b.
And meanwhile processor 1 moves a 1 into b,
and moves the contents of a into eax.
I just chose different registers just so we
can distinguish the two things.
Now let's think about this code.
We have these two things going on.
Is it possible that processor 0's ebx and processor
1's eax both contain the value 0 after the processors have both
executed their code?
They're executing in parallel.
So think about a little bit.
This is a good lecture to think about because, well, you'll
see in a minute.
So can they both have the value of 0?
So you're shaking your head.
Explain why?
STUDENT: So if ebx is greater than [INAUDIBLE]
then it's [INAUDIBLE].
CHARLES LEISERSON: OK, good.
And that's a correct argument, but you're
making a huge assumption.
Yeah, so the idea is that, well, if you're moving a 1 into it,
you're not looking at it.
It may be that one of them gets 0, and the other gets 1,
but it actually turns out to depend on what's
called the memory model.
And it took a long time before people realized that there
was actually an issue here.
So this depends upon the memory model.
And what you were reasoning about
was what's called sequential consistency.
You were doing happens before types of relationships
and saying, if this happened before that, then.
And so you had some global notion of time
that you were using to say what order these things happened in.
So let's take a look at the model that you were assuming.
It's interesting, because whenever I do this,
somebody always has the right answer,
and they always assume that it's sequentially consistent.
It's the most standard one.
So sequential consistency was defined
by Leslie Lamport who won the Turing Award, a few years ago.
And this is part of the reason he won it.
So what he said is, the result of any execution
is the same as if the operations of all the processors
were executed in some sequential order.
And the operations of each individual processor
appear in this sequence in the order specified by the program.
So let's just break that apart, because it's
a mouthful to understand.
So the sequence of instructions as defined by a processor's
program are interleaved with the corresponding sequences defined
by the other processors' programs
to produce a global linear order of all instructions.
So you take this processor, this processor,
and there's some way of interleaving them for us
to understand what happened.
That's the first part of what he's saying.
Then after you've done this interleaving,
a load instruction is going to get the value stored
to the address of the load.
That is, the value of the most recent
stored to that same location in that linear order.
So by most recent, I mean most recent in that linear order.
I'm going to give an example in just a second.
So it doesn't fetch one from way back,
it fetches the most recent one, the last write
that occurred to that location in that interleaved order
that you have picked.
Now there could be many different interleaved orders,
you can get many different behaviors.
After all, here we're talking about programs with races,
right?
We're reading stuff that other things are writing.
And so basically, the hardware can do whatever it wants.
But for the execution to be sequentially as consistent,
it must appear as if the loads and stores obeyed
some global linear order.
So there could be many different possible execution paths,
depending upon how things get interleaved.
But if you say, here's the result of the computation,
it better be that there exists one
of those in which every read occurred
to the most recent write according to some linear order.
Does that makes sense?
So let's do it for this example.
So here we have our setup again.
How many interleavings of four things are there?
Turns out there's six interleavings.
So those who've taken 6.042 will know that, right?
4 choose 2.
So the interleavings, you can do them
in the order 1, 2, 3, 4, 1, 3, 2, 4, 1, 3, 4, 2,
et cetera, et cetera.
But notice that in every one of these orders,
1 always comes before 2, and 3 always comes before 4.
So you have to respect the processor order.
The processor order, you have to respect it.
So if I execute in the first column, if that's the order,
what's the value that I end up with for eax and ebx?
STUDENT: 1 and 0.
CHARLES LEISERSON: 1 and 0.
Yep.
So it basically moves a 1 into a, then it moves b into ebx.
b is currently 0, so it's got a 0 in ebx.
Then processor 1 moves 1 into b.
And then it moves a into eax.
And a at that point has the value 1.
What about the second one?
STUDENT: 1, 1.
CHARLES LEISERSON: 1, 1.
Good.
Because they basically are both moving 1 into their registers,
then they're both storing.
What about the third one?
Yeah?
STUDENT: Same.
CHARLES LEISERSON: Same.
OK, fourth one?
We'll try to get everybody [INAUDIBLE]..
STUDENT: Same.
CHARLES LEISERSON: Same?
Yep.
Fifth one?
Same.
Last one?
STUDENT: 0, 1.
CHARLES LEISERSON: Yeah, 0, 1.
Good.
So this is the total number of ways
we could interleave things.
We don't know which one of these might
occur because, after all, the output is
going to be non-deterministic upon, which it is.
But one thing that we can say for certain
is that if you have sequential consistency,
there's no execution that ends with them both
being 0, which is exactly your intuition and correct
rationalization.
But it turns out interestingly that of modern computers, none
implement sequential consistency.
Why?
Because life would be too easy then.
None of them do that.
So we'll get there, we'll talk about what modern machines do.
So let's reason about sequential consistency.
So the way that you can formally reason about this,
to make an argument as you might have for example
on a quiz, if we had a quiz coming up,
would be to understand that an execution induces
a happens before a relationship that we
will denote as a right arrow.
And the right arrow relation is linear,
meaning that for any two instructions
either one happens before the other or the other
happens before the one for any two different instructions.
This is the notion of a linear order.
The arrow relation has to respect.
The happens before relation has to respect processor order.
In other words, that within the instructions executed
by a processor the global order has
to have those same sequence of instructions
of whatever that processor thought that it was doing.
And then a load from a location in memory
reads the value written by the most
recent store to that location according to happens before.
And for the memory resulting from an execution
to be sequentially consistent, there
must be a linear order that yields that memory state.
If you're going to write code without locks,
it's really important to be able to reason about what
happened before what.
And with sequential consistency, you just
have to understand what are all the possible interleavings.
So if you have n instructions here and m instructions there,
you only have to worry about n times m possible interleavings.
Actually, is it n times m?
No, you've got more than that.
Sorry.
I used to have good math.
So one of the celebrated results early in concurrency theory
was that fact that you could do mutual exclusion without locks,
or test and set, or compare and swap,
or any of these special instructions.
Really remarkable result. And so I'd
like to show you that because it involves thinking
about sequential consistency.
So let's recall, we talked about mutual exclusion last time
and how locks could solve that problem.
But of course locks introduced a lot
of other things like deadlock, convoying,
and a variety of things, some of which
I didn't even get a chance to talk about,
but they're in the lecture notes.
So let's recall that a critical section is a piece of code that
accesses a shared data structure that you
don't want two separate threads to be executing
at the same time.
You want it to be mutually exclusive.
Most implementations use one of these special instructions,
such as the xchg, the exchange instructions we talked about
to implement locks last time.
Or they may use test and set, compare and swap,
load linked store conditional.
Are any of these familiar to people?
Or is this new stuff?
Who's this new for?
Just want to make sure.
OK, great.
So there are these special instructions
in the machine that do things like an atomic exchange,
or a test and set.
I can set a bit and test what the prior value was
of that bit as an atomic operation.
It's not two sections where I set it,
and then the value changed in between.
Or compare and swap, we'll talk more about compare and swap.
And load linked store conditional, which is even
a more sophisticated one.
So in the early days of computing back in the 1960s,
this problem of mutual exclusion came up.
And the question was, can mutual exclusion
be implemented with only the loads and stores
as the only memory operations.
Or do you need one of these heavy duty
instructions that does two things and calls it atomic?
Oops, yep, so I forgot to animate
the appearance of Edsgar.
So two fellows, Dekker and Dijkstra, showed that it can,
as long as the computer system is sequentially consistent.
And so I'm not going to give their algorithm, which
is a little bit complicated.
I'm going to give I what I think is boiled down
to the most simple and elegant version of that
uses their idea, and it's due to Peterson.
And for the life of me, I have not
been able to find a picture of Peterson.
Otherwise, I'd show you what Peterson looks like.
So here's Peterson's algorithm.
And I'm going to model it with Alice and Bob.
They have a shared widget.
And what Alice wants to do to the widget is to frob it.
And Bob wants to borf it.
So they're going to frob and borf it.
But we don't want them to be frobbing and borfing
at the same time, naturally.
You don't frob and borf widgets at the same time.
So they're mutually exclusive.
So here's Peterson's algorithm.
So we have widget x.
So I'm just going to read through the code here.
And I have a Boolean variable called wants.
I have an A_wants and a B_wants.
A means Alice wants to frob the widget.
B_wants means that Bob wants to borf the widget.
And we're also going to have a variable that has two values, A
or B, for whose turn it is.
And so we start out with that code,
and then we fork the two Alice and Bob branches
of our program to execute concurrently.
And what Alice does is she says, I want it.
She sets A_wants to true.
And I set the turn to be Bob's turn.
And then the next loop has an empty body, notice.
It's just a while with a semicolon.
That's an empty body.
It's just going to sit there spinning.
It's going to say, while B wants it, Bob wants it,
and it's Bob's turn, I'm going to just wait.
And if it turns out that either Bob does not want it
or it's not Bob's turn, then that's
going to free Alice to go into the critical section and frob
x.
And then when she's done she says, I don't want it anymore.
And if you look at Bob's code, it's exactly the same thing.
And when we're done with this code,
we're going to then loop to do it again, because they just
want to keep frobbing and borfing
until their eyes turn blue or red, whatever color eyes they
have there.
Yeah, question?
I didn't explain why this works yet.
I'm going to explain why it works.
STUDENT: OK.
CHARLES LEISERSON: You're going to ask why it works?
STUDENT: I was going to ask why those aren't locks.
CHARLES LEISERSON: Why are they not locks?
STUDENT: [INAUDIBLE]
CHARLES LEISERSON: Well, a lock says
that if you can acquire it, then you stop the other person
from acquiring it.
There's no locking here, there's no waiting.
We're implementing a mutual exclusion region.
But a lock has a particular span--
it's got an acquire and a release.
So when you say A wants to be true,
I haven't acquired the lock at that point, have I?
Or if I set the turn to be the other character,
I haven't acquired a lock.
Indeed, I then do some testing and so forth and hopefully
end up with mutual exclusion, which is effectively
what locking does.
But this is a different way of getting you there.
It's only using loads and stores.
With a lock, there's an atomic--
I got the lock.
And if it wasn't available, I didn't get the lock.
Then I wait.
So let's discuss, let's figure out what's going on.
And I'm going to do it two ways.
First, I'm going to do the intuition,
and then I'm going to show you how you reason through it
with a happens before relation.
Question?
STUDENT: No.
CHARLES LEISERSON: No, OK.
Good.
Not good that there's no questions.
It's good if there are questions.
But good we'll move on.
So here's the idea.
Suppose Alice and Bob dropped both tried
to enter the critical section.
And we have sequential consistency.
So we can talk about who did things in what order.
So whoever is the last one to write to the variable turn,
that one's not going to enter.
And the other one will enter.
And then if Alice tries to enter the section,
then she progresses because at that point
she knows that B_wants is false.
And if only Bob tries to enter it,
then he's going to go because he's going
to see that A_wants is false.
Does that makes sense?
So only one of them is going to be in there at a time.
It's also the case that you want to verify that if you
want to enter, you can enter.
Because otherwise, a very simple protocol
would be not to bother looking at things but just take turns.
It's Alice's turn, it's Bob's turn, it's Alice's turn,
it's Bob turn.
And we don't want a solution like that
because if Bob doesn't want a turn, Alice can't go.
She can go once, and then she's stuck.
Whereas we want to be able to have somebody,
if they're the only one who wants
to go to execute the critical section,
Alice can frob, frob, frob, frob, frob.
Or Bob can borf, borf, borf, borf, borf.
We don't want to force them to go if they don't need to go.
So the intuition is that only one of them
is going to get in there because you
need the other one either to say you want to go in,
or else their value for wants is going to be 0.
And it's going to be false and you're
going to go through anyway.
But this is not a good argument, because this is handwaving.
We're at MIT, right, so we can do proofs.
And this proof isn't so hard.
But I want to show it to you because it
may be different from other proofs that you've seen.
So here's the theorem.
Peterson's algorithm achieves mutual exclusion
on the critical section.
The setup for the proof is, assume
for the purposes of contradiction
that both Alice and Bob find themselves
in the critical section together.
And now we're going to look at the series of instructions
that got us there, and then argue
there must be a contradiction.
That's the idea.
And so let's consider the most recent time that each of them
executed the code before entering the critical section.
So we're not interested in what happened long ago.
What's the very, very last pieces
of code as they entered the critical section?
And we'll derive a contradiction.
So here we go.
So without loss of generality, let's assume that Bob--
we have some linear order.
And to execute, noticed a B in the critical section, Alice
and Bob both had to set the variable turn.
So one of them had to do it first.
I'm going assume without loss of generality
that it was Bob because I can otherwise make exactly
the same argument for Alice.
So let's assume that Bob is the last one to write to turn.
So therefore, if Bob was the last one,
that means that Alice writing to turn,
so she got in there so she wrote to turn.
So her writing B to turn preceded Bob writing A to turn.
So we have that happens before relationship.
Everybody with me?
Do you understand the notation I'm using
and the happens before relationship?
Now Alice's program order says that true to A_wants
comes before her writing turn equals
B. That's just program order.
So we have that.
And similarly, we have Bob's program order.
And Bob's program order says, well, I wrote turn to A.
So Bob wrote, turn equals A. And then Bob,
in this case I'm going to do Bob read A_wants.
And then he reads turn.
So the second instruction here, up here,
so this is a conditional and.
So we basically are doing this.
And then if that's true, then we do this.
So this turn equals equals A. That's
reading turn and checking if it's A happens after A_wants.
So that's why I get these three things in order.
Does that makes sense?
Any question about that?
Is that good?
So I've established these two chains.
So I actually have three chains here
that I'm now going to combine.
Let's see.
So what's happening is let me look
to see what's the order of everything that happens.
So the earliest thing that happens
is that Alice wants to be true because--
where's that?
So Alice wants is true is, yes, is coming before.
That's the earliest thing that's happening here.
So that instruction is basically this--
Alice wants is true, it comes before the A turn equals
B. That comes before the A turn equals B.
So it comes before the write turn equals
A, write B turn equals A. And then B turn equals A.
So do you see the chain we've established?
You see the chain?
Yeah, yeah.
OK, good.
|
So it says A_wants is first.
A_wants equals true is first.
Then we have the turn equals B. That's all from the second line
here.
That's from this line here.
What's next?
Which instruction is next?
So turn equals A. That comes from the top line there.
What's next?
STUDENT: B [INAUDIBLE].
CHARLES LEISERSON: So I read B. Bob reads A_wants.
And then finally, Bob reads turn at A.
So this is all based on just the interleaving and the fact
that if you saw that we have the program order
and that Bob was the last to write.
That's all we're using.
And so why is that a contradiction?
Well, we know what the linear order is.
We know that when Bob read, what did Bob read?
What did Bob read when he read A_wants in step 4?
He read the last value in that chain,
the most recent value In that chain where it was stored to.
And what was stored there?
True.
Good.
And then Bob read turn.
And what was the most recent value
stored to turn in that chain?
STUDENT: [INAUDIBLE] A.
CHARLES LEISERSON: So then what?
STUDENT: Bob gets stuck.
CHARLES LEISERSON: Bob, if that were in fact
what he read in the while loop line, what
should be happening now?
He should be spinning there.
He shouldn't be in the loop.
Bob didn't obey.
His code did not obey the logic of the code.
Bob should be spinning.
That's the contradiction.
Because we said Bob was in the loop.
Does that makes sense?
Is that good?
So when you're confronted with synchronizing through memory,
as this is called, you really got
to write down the happens before things
in order to be careful about reviewing things.
I have seen in many, many cases engineers
think they got it right by an informal argument.
And in fact, for those people who have studied
model checking-- anybody have any interaction
with model checking?
What was the context?
STUDENT: 6.822.
CHARLES LEISERSON: Well, and were you
studying protocols and so forth?
STUDENT: Yeah.
CHARLES LEISERSON: So in 6.822, what class is that?
STUDENT: Formal programming.
CHARLES LEISERSON: Formal programing.
Good.
So for things like network protocols and security
protocols and for cache protocols in order
to implement things like MSI and MESI protocols and so forth,
these days they can't do it in their heads.
They have programs that look at all the possible ways
of executing what's called model checking.
And it's a great technology because it
helps you figure out where the bugs are and essentially reason
through this.
For simple things, you can reason it through.
For larger things, you use the same kind of
happens before analysis in those contexts in order
to try to prove that your program is correct,
that those protocols are correct.
So for example, in all the computers
you have in this room, every one of them,
there was a model checker checking
to make sure the cache analysis was done.
And many of the security protocols
that you're using as you access the web
have all been through model checking.
Good.
The other thing is it turns out that Peterson's algorithm
guarantees starvation freedom.
So while Bob wants to execute her critical session,
Bob cannot execute his critical section twice in a row,
and vise versa.
So it's got the property that one of the things that you
might worry about is Alice wants to go
and then Bob goes a gazillion times,
and Alice never gets to go.
Now that doesn't happen, as you can see,
from the code because every time you
go you set the turn to the other person.
So if they do want to go, they get to go through.
But proving that is a nice exercise.
And it will warm you up to this kind of analysis,
how you go about it.
Yeah?
STUDENT: Does it work with another [INAUDIBLE]??
CHARLES LEISERSON: This one does not.
And there has been wonderful studies
of what does it take to get n things to work together.
And this is one place where the locks have a big advantage
because you can use a single lock
to get the mutual exclusion among n things,
so constant storage.
Whereas if you just use atomic read and atomic write,
it turns out the storage grows.
And there's been wonderful studies.
Also, wonderful studies of these other operations,
like compare and swap and so forth.
And we'll do a little bit of that.
We'll do a little bit of that.
So often, in order to get performance,
you want to synchronize through memory.
Not often, but occasionally you want
to synchronize through memory to get performance.
But then you have to be able to reason about it.
And so the happens before sequential consistency,
great tools for doing it.
The only problem with sequential consistency is what?
Who is listening?
Yeah?
STUDENT: It's not real.
CHARLES LEISERSON: It's not real.
No, we have had machines historically
that implemented sequential consistency.
Today, no machines support sequential consistency,
at least that I'm aware of.
Instead they report what's called relaxed memory
consistency.
And let's take a look at what the motivation is
for why you would want to make it a nightmare for programmers
to synchronize through memory.
This has also led software people
to say, never synchronize through memory.
Why?
Because it is so hard to get it correct.
Because you don't even have sequential consistency
at your back.
So today, no modern day processor
implements sequential consistency.
They all implement some form of relaxed consistency.
And in this context, hardware actively reorders instructions,
and compilers may reorder instructions too.
And that leads you not to have the property
that the order of instructions that you specify in a processor
is the same as the order that they get executed in.
So you say do A and then B. The computer does B and then A.
So let's see instruction reordering.
So I have on the left the order that the programmer specified,
and the order on the right what the hardware did.
Or it may have been that the compiler reordered them.
Now if you look, why might the hardware or compiler
decide to reorder these instructions?
What's going on in these instructions?
You have to understand what these instructions are doing.
So in the first case, I'm doing a store and then a load.
And in the second case, I have reversed the order
to do the load first.
Now if you think about it, if you only
had one thing going on, what's the impact
here of this reordering?
Is there any reason the compiler or somebody
couldn't reorder these?
STUDENT: I think we reorder them is the reason that it
affects the pipeline.
If you have to store first, the write [INAUDIBLE]
you have to [INAUDIBLE].
CHARLES LEISERSON: Yeah, in what way
does it affect the pipeline?
STUDENT: That basically the load doesn't do anything
in the [INAUDIBLE],, whereas the store does.
CHARLES LEISERSON: I think you're on the right track.
There's a higher level reason why you might want
to put loads before stores.
Why might you want to put loads?
These are two instructions that normally if I only
had one thread, reordering them would be perfectly fine.
Well, it's not necessarily perfectly fine.
When might there be an issue?
It's almost perfectly fine.
STUDENT: [INAUDIBLE]
CHARLES LEISERSON: If A was equal to B.
But if A and B are different, than reordering
them is just fine.
If A and B are the same, if that's the same location,
uh-oh, I can't reorder them because one is using the other.
So why might the hardware prefer to put the load earlier?
Yeah?
STUDENT: There might be a later instruction which depends on B.
CHARLES LEISERSON: There might be a later instruction that
depends on B. And so why would it put the load earlier?
STUDENT: So by doing the load earlier,
the pipeline [INAUDIBLE] happens.
Earlier on, [INAUDIBLE].
CHARLES LEISERSON: Yeah, you're basically covering over latency
in a load.
When I do a load, I have to wait for the result
before I can use it.
When I do a store, I don't have to wait for the result
because it's not being used, I'm storing it.
And so therefore if I do loads earlier,
If I have other work to do such as doing the store,
then the instruction that needs the value of B
doesn't have to necessarily wait as long.
I've covered over some of the latency.
And so the hardware will execute faster.
So we've got higher performance by covering load latency.
Does that makes sense?
It's helpful to know what's going on in the hardware
here to reason about the software.
This is a really great example of that lesson
is what the compiler is doing there
that it chooses to reorder.
And frankly, in the era before 2004
when we were in the era of what's called
Dennard scaling, and we didn't worry.
All our computers just had one processor, it didn't matter.
Didn't have to worry about these issues.
These issues only come up for when you have
more than one thing going on.
Because if you're sharing these values, oops,
I changed the order.
So let's see, so when is it safe in this context
for the hardware compiler to perform
this particular reordering?
When can I do that?
So there's actually two answers here.
Or there's a combined answer.
So we've already talked about one of them.
Yeah?
STUDENT: When A is not B.
CHARLES LEISERSON: Yeah, when A is not B. If A and B are equal,
it's not safe to do.
And what's the second constraint where
it's safe to this reordering?
Yeah, go ahead.
STUDENT: If A equals B, but if you
have already one [INAUDIBLE].
CHARLES LEISERSON: Ooh, that's a nasty one.
Yeah, I guess that's true.
I guess that's true.
But more generally when is it safe?
That's a benign race in some sense, right?
Yeah?
Good.
Good, that's a good one.
What's the other case that this is safe to do?
Or what's the case where it's not safe?
Same question.
I just told you.
When might this be safe?
When is it safe to this reordering?
I can't do it if A is equal to B.
And also shouldn't do it when?
Yeah?
STUDENT: [INAUDIBLE] value of A.
CHARLES LEISERSON: Yeah, but.
Yeah?
STUDENT: [INAUDIBLE] if you have like
when a processor is operating.
CHARLES LEISERSON: Yeah, if there's no concurrency.
If there's no concurrency, it's fine.
The problem is when there's concurrency.
So let's take a look at how the hardware does reordering
so that we can understand what's going on.
Because in a modern processor, there's
concurrency all the time.
And yet the compiler still wants to be
able to cover overload latency, because usually it
doesn't matter.
So you can view hardware as follows.
So you have a processor on the left edge here,
and you have a network that connects it
to the memory system, a memory bus of some kind.
Now it turns out that the processor
can issue stores faster than the network can handle them.
So the processor can go store, store, store, store, store.
But getting things into the memory system,
that can take a while.
Memory system is big and it's slow.
But the hardware is usually not doing store on every cycle.
It's doing some other things, so there are bubbles
in that instruction stream.
And so what it does is it says, well, I'm
going to let you issue it because I
don't want to hold you up.
So rather than being held up, let's
just put them into a buffer.
And as long as there's room in the buffer,
I can issue them as fast as I need to.
And then the memory system can suck them out of the buffer
as it's going along.
And so in critical places where there's a bunch of stores,
it stores them in the buffer if the memory system can't handle.
On average, of course, it's going
to go at whatever the bottleneck is on the left or the right.
You can't go faster than whichever
is the bottleneck-- usually the memory system.
But we'd like to achieve that, and we
don't want to have to stall every time we try to do two
stores in a row, for example.
By putting a little bit of a buffer,
we can make it go faster.
Now if I try to do a load, that can stall the processor
until it's satisfied.
So whenever you do a load, if there's no more instructions
to execute it, if the next instruction to execute
requires the value that's being loaded,
the processor has to stall until it gets that value.
So they don't want the loads to go through the store buffer.
I mean, one solution would be just put everything
into the store buffer.
In some sense you'd be OK, but now I
haven't covered over my load latency.
So instead what they do is they do what's called a load bypass.
They go directly to the memory system for the load,
bypassing all the writes that you've done up to that point
and fetch it so that you get it to the memory system
and the load bypass takes priority over the store buffer.
But there's one problem with that hack, if you will.
What's the problem with that hack?
If I bypass the load, where could I run into trouble
in terms of correctness?
Yeah?
STUDENT: If one your stores is the thing
you're trying to load.
CHARLES LEISERSON: If one of your stores is the thing
you're trying to load.
Exactly.
And so what happens is, as the load bypass is going by,
it does an associative check in the hardware.
Is the value that I'm fetching one of the values that
is in the store buffer?
And if so, it responds out of the store buffer
directly rather than going into the memory system.
Makes sense?
So that's how the reordering happens within the machine.
But by this token, a load can bypass a store
to a different address.
So this is how the hardware ends up reordering it,
because the appearance is that the load occurred
before the store occurred if you are looking
at the memory from the point of view of the memory,
and in particular the point of view
of another processor that's accessing that memory.
So over here I said, store load.
Over here it looks like he did, load store.
And so that's why it doesn't satisfy sequential consistency.
Yeah, question?
STUDENT: So that store bumper would
be one for each processor?
CHARLES LEISERSON: Yeah, there's one for each processor.
It's the way it gets things into the memory, right?
So I'll tell you, computing would be so easy
if we didn't worry about performance.
Because if those guys didn't worry about performance,
they'd do the correct thing.
They'd just put them in in the right order.
It's because we care about performance that we make
our lives hard for ourselves.
And then we have these kludges to fix them up.
So that's what's going on in the hardware, that's
why things get reordered.
Makes sense?
But it's not as if all bets are off.
And in fact, x86 has a memory consistency model
they call total store order.
And here's the rules.
So it's a weaker model.
And some of it is kind of sequentially consistent type
of thing.
You're talking about what can be ordered.
So first of all, loads are never reordered with loads.
Let me see here.
Yeah, so you never reorder loads with loads.
That's not OK.
Always, you can count on loads being
seen by any external processor in the same order
that you issued the loads within a given processor.
So there is some rationale here.
Likewise, stores are not reordered with stores.
That never happens.
And then stores are not reordered with prior loads.
So you never move a store earlier past a load.
You wouldn't want to do that because generally it's
the other direction you're covering over latency.
But in fact, they guarantee it doesn't happen.
So you never move a store before a load.
It's always move a load before a store.
And then in general, a load may be reordered with a prior store
to a different location, but not with a prior load
to the same location.
So this is what were just talking about,
that A has to be not equal to B in order
for it to be reordered.
And at the point that you're executing this,
the hardware knows what the addresses
are that are being loaded and stored and can tell,
are they the same location or not.
And so it knows whether or not it's able to do that.
So the loads basically, you can move loads upwards.
But you don't reorder them.
And you only move it past a store
if it's a store to a different address.
And so here we have a bunch of things.
So this is basically weaker than sequential consistency.
There are a bunch of other things.
So for example, if I just go back here for a second.
The lock instructions respect a total order.
The stores respect a total order.
The lock instructions and memory ordering
preserves what they call transitive visibility.
In other words, causality, which is basically the happens--
says that the happens before a relation,
you can treat as if it's a linear order.
It's transitive as a binary relation.
So the main important ones are the ones at the beginning.
But it's helpful to know that locks are not
going to get reordered.
If you have a lock instruction, they're never
going to move it before things.
So here's the impact of reordering
on Peterson's algorithm.
Sorry, no, this is not Peterson's algorithm yet.
This impact of reordering on this
is that I may have written things in this order,
but in fact they execute in something like this order.
And therefore, the ordering, in this case,
2, 4, 1, 3 is going to produce the value 0,
0, which was exactly the value that you said
couldn't possibly appear.
Well, on these machines it can appear.
And also let me say, so instruction reordering violates
this sequential consistency.
And by the way, this can happen.
Not just in the hardware, this can happen in the compiler
as well.
The compiler can decide to reorder instructions.
It's like, oh my god, how can we be writing
correct code at all right.
But you've written some correct parallel code,
and you didn't have to worry about this.
So we'll talk about how we get there.
Yeah?
STUDENT: Is the hardware geared to even reorder [INAUDIBLE]??
Or [INAUDIBLE] it might happen?
CHARLES LEISERSON: It might happen.
No, there's no requirement that it move things earlier.
STUDENT: Why is it not always [INAUDIBLE]??
CHARLES LEISERSON: It may be that there
isn't enough register space.
Because as you move things earlier,
you're going to have to hold the values longer
before you're using them.
Yeah?
STUDENT: In the previous slide, [INAUDIBLE] load 3 [INAUDIBLE]
also.
CHARLES LEISERSON: That load 3 in the previous--
I'm sorry, I'm not following.
STUDENT: In the previous slide.
CHARLES LEISERSON: Oh, the previous slide, not this slide.
This one?
STUDENT: Yeah.
CHARLES LEISERSON: OK.
STUDENT: So [INAUDIBLE] load 3 [INAUDIBLE]..
CHARLES LEISERSON: Well, I had said
there's some things that I said we're no good, right?
So here it was, what did I do?
I moved the loads earlier in that example.
But there were some earlier ones.
Are you talking about even earlier than that?
STUDENT: Yeah, this one.
CHARLES LEISERSON: Oh, this one, OK.
STUDENT: So, load 3 can come before store [INAUDIBLE]..
CHARLES LEISERSON: So let's see.
So this is the original thing.
Store 3 is before store 4, and load 3 and load 4
are afterwards, right?
So the stores have to be in the same order
and the loads have to be in the same order.
But the loads can go before the stores
if they're to a different address.
So in this case, we moved load 3 up two,
and we moved load 4 up one.
We could have maybe move load 4 up before store 3,
but maybe they were to the same address.
STUDENT: OK, so load 3's store doesn't mean that they're
from the same address?
CHARLES LEISERSON: No, no, this is abstract.
You got it?
OK.
So this is why things can get reordering.
And in that case, we can end up with a reordering that gives us
something that we don't expect when we're
synchronizing through memory.
Never write non-deterministic code,
because you deal with this stuff--
unless you have to.
Unfortunately, sometimes, it's not fast enough otherwise.
Now let's go back and look at Peterson's algorithm
and what can go wrong with Peterson's algorithm.
So what reordering might happen here that would completely
screw up Peterson's algorithm?
A hint, we're looking for a load that
might happen before a store.
What load would be really bad to happen before a store?
Yeah?
STUDENT: If you load turn to [INAUDIBLE] before the store
turn [INAUDIBLE].
CHARLES LEISERSON: You load turn earlier.
Maybe.
Let me think, that's not the one I chose,
but maybe that could be right.
Well, you can't move it before the store to turn.
STUDENT: All right.
CHARLES LEISERSON: OK, yeah?
STUDENT: Maybe Alice loads B_wants to early?
CHARLES LEISERSON: Yeah, if Alice loads B_wants to early,
and if they both do, then they could be reordered
before the store of A_wants and B_wants,
because that's a load and B_wants-- well,
Alice isn't touching B_wants so why can't it just
move it earlier.
Those are not the same locations.
So suppose it reorders those, now what happens?
STUDENT: So [INAUDIBLE] B_wants [INAUDIBLE] too early?
CHARLES LEISERSON: Yeah, it would be false too early,
right?
STUDENT: And the same with A_wants.
CHARLES LEISERSON: And the same with A.
And now they discover they're in this critical section together.
And if there's one thing, we don't want Alice and Bob
in the same critical section.
Does that makes sense?
So you've got this problem.
There's reordering going on.
And, yikes, how could you possibly
write any parallel code and any concurrent code?
Well, they say, well, we'll put in a kludge.
They introduce some new instructions.
And this instruction is called a memory fence.
So don't get me wrong.
They need to do stuff like this.
There is an argument to say they should still build machines
with sequential consistency because it's
been done in the past.
It is hard work for the hardware designers to do that.
And so as long as the software people
say, well, we can handle weak consistency models,
[INAUDIBLE] says, OK, your problem.
So Mark Hill, who's a professor at University of Wisconsin,
has some wonderful essays saying why
he thinks that parallel machines should support
sequential consistency, and that the complaints of people not
having it supported, that those people they could support it
if they really wanted to.
And I tend to be persuaded by him.
He's a very good thinker, in my opinion.
But in any case, so what we have-- yeah, question?
STUDENT: How much of a difference
does it make to sacrifice?
CHARLES LEISERSON: So he talks about this
and what he thinks the differences is,
but it's apples and oranges.
Because sometimes part of it is what's
the price of having bugs in your code.
Because that's what happens is programmers
can't deal with this.
And so we end up with bugs in our code.
But they can reason about sequential consistency.
It's hard, but they can reason about it.
When you start having relaxed memory consistency, very
tricky.
So let's talk about what the solutions are.
And his argument is that the performance
doesn't have to be that bad.
There was a series of machines made
by a company called Silicon Graphics, which were all
sequentially consistent.
Parallel machines, all sequentially consistent.
And they were fine.
But they got killed in the market
because they couldn't implement processors as well as Intel
does.
And so they ended up getting killed in the market
and getting bought out, and so forth.
And now their people are all over,
and the people who were at Silicon Graphics, many of them
really understand parallel computing well,
the hardware aspects of it.
So a memory fence is a hardware action
that forces an ordering constraint
between the instructions before and after the fence.
So the idea is, you can put a memory fence in there and now
that memory fence can't be reordered
with things around it.
It maintains its relative ordering site to other things.
And that way you can prevent.
So one way you could make any code be sequentially consistent
is to put a memory fence between every instruction.
Not very practical, but there's a subset of those
that actually would matter.
So the idea is to put in just the run one.
You can issue them explicitly as an instruction.
In the x86, it's called the mfence instruction.
Or it can be performed implicitly,
so there are other things like locking, exchanging, and other
synchronizing instructions.
They implicitly have a memory fence.
Now the compiler that we're using
implements a memory fence via the function
atomic_thread_fence, which is defined in the C header file
stdatomic.h.
And you can take a look at the reference material
on that to understand a little bit more about that.
The typical cost on most machines
is comparable to that of an L2 cache access.
Now one of the things that is nice to see is happening
is they are bringing that down.
They're making that cheaper.
But it's interesting that Intel had one processor where
the memory fence was actually slower
than the lock instruction.
And you say, wait a minute, the lock instruction
has an implicit memory fence in it.
I mean, you've got a memory fence in the lock instruction.
How could the memory fence be slower?
So I don't know exactly how this happens, but here's my theory.
So you've got these engineering teams
that are designing the next processor.
And they of course want it to go fast.
So how do they know whether it's going to go fast?
They have a bunch of benchmark codes
and that they discover, well, now that we're
getting the age of parallelism, all these parallel codes,
they're using locking.
So they look and they say, OK, we're
going to put our best engineer on making locks go fast.
And then they see that, well, there's
some other codes that maybe go slow because they've
got fences.
But there aren't too many codes that just
need fences, explicit fences.
In fact, most of them use [INAUDIBLE]..
So they put their junior engineer on the fence code,
not recognizing that, hey, the left hand and the right hand
should know what each other is doing.
And so anyway, you get an anomaly
like that where it turned out that it was actually fastest--
we discovered as we're implementing the silk runtime--
to do a fence by just doing a lock on a location
that we didn't care about the lock.
We just did a lock instruction.
And that actually went faster than the fence instruction.
Weird.
But these systems are all built by humans.
So if we have this code and we want to restore consistency,
where might we put a memory fence?
Yeah?
STUDENT: After setting the turn?
CHARLES LEISERSON: After setting turn.
You mean like that?
STUDENT: Yeah.
CHARLES LEISERSON: Yeah.
OK, so that you can't end up loading it
before it's stored too.
And that kind of works, sort of.
You also have to make sure that the compiler doesn't screw you
over.
And the reason the compiler might
screw you over is that it looks at B_wants and turn B,
it says, oh, I'm in a loop here.
So let me load the value and keep using the value over.
And I don't see anybody using this value.
Right, so it loads the value.
And now it just keeps checking the value.
The value has changed on the outside,
but it's stored that in a register
so that that loop will go really fast.
And so it goes really fast, and you're spinning
and you're dead in the water.
So in addition to the memory fence,
you must declare variables as volatile
to prevent the compiler from optimizing them away.
When you declare something as volatile,
you say, even if you read it, if the compiler reads it.
When it reads it a second time, it's
still got to read it a second time from memory.
It cannot assume that the value is going to be stable.
You're saying it may change outside.
And then you also, it turns out, may need compiler fences
around frob and borf to prevent them reordering
some of frob and borf because that stuff can also sometimes
get moved outside the loop, the actual code in frob and borf,
because it wants to, it says, oh.
It doesn't realize always that there's no what's going on.
So the C11 language standard defines its own weak memory
model.
And you can declare things as atomic,
and there are a bunch of things there.
And here's a reference where you can
take a look at the atomic stuff that's
available if you want to do this dangerous programming.
In general for implementing general mutexes,
if you're going to use only load and store,
there's a very nice theorem by Burns and Lynch--
this is Nancy Lynch who's on the faculty here--
that says any n-thread deadlock-free mutual exclusion
algorithm using only load and store requires order n space--
the space is linear.
So this answers the question that I had answered orally
before.
And then it turns out that if you
want an n-thread deadlock-free mutual exclusion algorithm,
you actually have to use some kind of expensive operation,
such as a memory fence or an atomic compare-and-swap.
So in some sense, hardware designers
are justified when they implement special operations
to support animosity, as opposed to just doing
using these clever algorithms.
Those algorithms are really at some level
of theoretical interest.
So let's take a look at one of these special instructions.
And the one I picked is compare-and-swap
because it's the one that's probably most available.
There are others like test-and-set, and so forth.
And so when you do lock-free algorithms,
when you want to build algorithms that are lock free,
and we'll talk about why you might
want to do lock-free algorithms, there's loads and store,
and then there's this CAS instruction, Compare-and-Swap.
In stdatomic.h, it is called atomic_compare_exchange_strong.
And it can operate on various integer types.
It cannot compare and swap floating point numbers.
It can only compare and swap integers,
and sometimes that's a pain.
And so here's the definition of the CAS instruction.
Basically, what it does is it has an address.
And then it has two values, the old value and the new value.
And what it does is it checks to see,
is the value that is in that memory location
the same as the old value.
And if it is, it sets it to the new value and says,
I succeeded.
And otherwise, it says I failed.
So it swaps it if the value that I'm holding, the old value,
is the same as what's in there.
So I can read the value, if I want,
then do whatever I want to do.
And then before I update it, I can say,
update it only if the value hasn't changed.
And that's what the compare and swap does.
Does that makes sense?
And it does that all atomically,.
And there's an implicit fence in there
so things don't get reordered around it.
It's all done as one.
The hardware ensures that nothing can interfere
in the middle of this.
It's actually comparing the old value to what's in there,
and swapping in the new, all as one operation.
Or it says, nope, the value changed, therefore
it just returned false, and the value didn't get updated.
So it turns out that you can do an n-thread deadlock-free
mutual exclusion algorithm with compare-and-swap
using only constant space.
And here's the way you do it.
And this is basically just the space for the new text itself.
So you take a look at the lock instruction, and what
you do is you spin, which is to say you block, until you
finally get the value true.
So you're trying to swap in true.
So true says that somebody holds the lock.
I say the old value was false.
If it's true, then the swap doesn't succeed
and you just keep spinning.
And then otherwise, you swap in the value
and now you're ready to go.
And to unlock it, you just have to set it to false.
Question?
STUDENT: Why does it de-reference
the pointer in the lock?
CHARLES LEISERSON: Why does it de-reference the pointer?
Because you're saying, what memory location
are you pointing to.
You're interested in comparing with the value
in that location.
So it is a memory operation.
So I'm naming the memory location.
I'm saying, if the value is false, swap in the value
true and return true.
And if it's true, then don't do anything and tell me
that you didn't succeed, in which case in this loop
it'll just keep trying again and again and again.
It's a spinning lock.
Question?
STUDENT: [INAUDIBLE] when you saying
that you're [INAUDIBLE] the value at that address
before passing it into CAS.
Yeah, there shouldn't be a pointer de-reference after
[INAUDIBLE].
CHARLES LEISERSON: Oh, you're right.
A bug.
Gotcha, yep, gotcha, I'll fix it.
So let's take a look at a way that you might want to use CAS.
So here's a summing problem.
So suppose I want to compute on some variable of type x.
And I've got an array that's--
what is that-- that's a million elements long.
And what I'm going to do is basically
run through my array in parallel and accumulate things
into the result.
And so this is actually incorrect code.
Why is this incorrect code?
Yeah?
STUDENT: Extra like a floating point taken [INAUDIBLE]
and so forth?
CHARLES LEISERSON: Maybe, let's assume we have fast math.
Yeah?
STUDENT: You have multiple transfer and updated results
at the same time?
CHARLES LEISERSON: Which means what?
STUDENT: Which means you have a race.
CHARLES LEISERSON: You have a race.
Everybody is trying to update result.
You've got a gazillion strands in parallel all trying
to pound on updating result.
So one way you could solve this is with mutual exclusion.
So I introduce a mutex L. And I lock before I update
the result, and then I unlock.
Why did I put the computation on my array of i?
Why did I put that outside the lock?
STUDENT: It's [INAUDIBLE] function is very expensive.
That way, you're only locking the--
CHARLES LEISERSON: Yeah, whenever you lock,
you want to lock for the minimum time possible.
Because otherwise you're locking everybody else
out from doing anything.
So that was a smart thing in that particular code.
So that's the typical locking solution.
But look at what might happen.
What if the operating system decides
to swap out a loop iteration just after it acquires mutext?
As you go down, it says lock.
You get the lock, and now the operating says, oops,
your time quantum is up.
Somebody else comes in and starts to compute.
What's going to happen now?
What's the problem that you might observe?
Yeah?
STUDENT: [INAUDIBLE] if they're [INAUDIBLE] computation
[INAUDIBLE] have to [INAUDIBLE].
CHARLES LEISERSON: Yeah, everybody's
going to basically just sit there waiting
to acquire the lock because the strand that has the lock
is not making progress, because it's sitting on the side.
It's been these scheduled.
That's bad, generally.
You'd like to think that everybody who's running
could continue to run.
Yeah?
STUDENT: Well, I guess under what circumstances
might be useful for a processor to have
this running on multi-threads instead of multiple processors
simultaneously?
CHARLES LEISERSON: No, so this the multiple threads
are running on multiple processors, right?
STUDENT: What do you mean by the time quantum?
CHARLES LEISERSON: So one of these guys
says, so I'm running a thread, and that thread's time quantum
expires.
STUDENT: Oh, that processor's multiple threads.
CHARLES LEISERSON: Right.
STUDENT: OK.
CHARLES LEISERSON: So I've got a whole bunch of processors
with a thread on each, let's say.
And I've got a bunch of threads.
The operating system has several threads
that are standing by waiting for their turn.
And one of them grabs the lock and then the scheduler
comes in and says, oops, I'm going to take you off,
put somebody else in.
But meanwhile, everybody else is there trying to make progress.
And this guy is holding the key to going forward.
You thought you were only grabbing the lock
for a short period of time.
But instead, the operating system
came in and made you take a long time.
So this is the kind of system issue
that you get into when you start using things like locks.
So all the other loop iterations have to wait.
So it doesn't matter if--
yeah, question?
STUDENT: How does the [INAUDIBLE] reducer
have [INAUDIBLE]?
CHARLES LEISERSON: So that's one solution to this, yep.
STUDENT: How does it do it?
CHARLES LEISERSON: How does it do it?
We have the paper online.
I had the things for explaining how reducers work.
And there's too much stuff.
I always have way more stuff to talk about than I ever
get a chance to talk about.
So that was one where I said, OK, yeah.
STUDENT: OK.
CHARLES LEISERSON: OK.
So all we want to do is atomically
execute a load of x followed by a store of x.
So instead of doing it with locks,
I can use CAS to do the same thing,
and I'll get much better properties.
So here's the CAS solution.
So what I do is I also compute a temp,
and then I have these variables old and new.
I store the old result. And then I add the temporary result
that I've computed to the old to get the new value.
And if it turns out that the old value is exactly the same as it
used to be, then I can swap in the new value,
which includes that increment.
And if not, then I go back and I do it again.
I once again load, add, and try to swap in again.
And so now what happens if the operating system swaps out
a loop iteration?
Yeah?
STUDENT: It's OK because whenever this is put back on,
then you know it'll be different.
CHARLES LEISERSON: It'll be new values, it'll ignore it,
and all the other guys can just keep going.
So that's one of the great advantages
of lock-free algorithms.
And I have in here several other lock-free algorithms.
The thing you should pay attention in here
is to what's called the ABA problem, which
is an anomaly with compare-and-swap
that you can get into.
This is a situation where you think
you're using compare-and-swap, you say is it the old value.
It turns out that the value is the same,
but other people have come in and done stuff but happened
to restore the same value.
But you assume it's the same situation,
even though the situation has changed
but the value is the same.
That's called the ABA problem.
So you can take a look at it in here.
So the main thing for all this stuff
is, this is really interesting stuff.
Professor Nir Shavit teaches a class where
this is the content of the class for the semester
is all these really dangerous algorithms.
And so I encourage you, if you're interested in that.
The world needs more people who understand
these kinds of algorithms.
And it needs to find ways to help people program fast
where people don't have to know this stuff,
because this is really tricky stuff.
So we need both--
both to make it so that we have people
who are talented in this way, and also
that we don't need their talents.
OK, thanks, everybody.