Conditional probabilities are probabilities associated with
a revised model that takes into account some additional
information about the outcome of a probabilistic experiment.
The question is how to carry out this
revision of our model.
We will give a mathematical definition of conditional
probabilities, but first let us motivate this definition by
examining a simple concrete example.
Consider a probability model with 12 equally likely
possible outcomes, and so each one of them has probability
equal to 1/12.
We will focus on two particular events, event A and
B, two subsets of the sample space.
Event A has five elements, so its probability is 5/12, and
event B has six elements, so it has probability 6/12.
Suppose now that someone tells you that event B has occurred,
but tells you nothing more about the outcome.
How should the model change?
First, those outcomes that are outside event B
are no longer possible.
So we can either eliminate them, as was done in this
picture, or we might keep them in the picture but assign them
0 probability, so that they cannot occur.
How about the outcomes inside the event B?
So we're told that one of these has occurred.
Now these 6 outcomes inside the event B were equally
likely in the original model, and there is no reason to
change their relative probabilities.
So they should remain equally likely in revised model as
well, so each one of them should have now probability
1/6 since there's 6 of them.
And this is our revised model, the
conditional probability law.
0 probability to outcomes outside B, and probability 1/6
to each one of the outcomes that is inside the event B.
Let us write now this down mathematically.
We will use this notation to describe the conditional
probability of an event A given that some other event B
is known to have occurred.
We read this expression as probability of A given B. So
what are these conditional probabilities in our example?
So in the new model, where these outcomes are equally
likely, we know that event A can occur in
two different ways.
Each one of them has probability 1/6.
So the probability of event A is 2/6 which
is the same as 1/3.
How about event B. Well, B consists of 6 possible
outcomes each with probability 1/6.
So event B in this revised model should have probability
equal to 1.
Of course, this is just saying the obvious.
Given that we already know that B has occurred, the
probability that B occurs in this new model
should be equal to 1.
How about now, if the sample space does not consist of
equally likely outcomes, but instead we're given the
probabilities of different pieces of the sample space, as
in this example.
Notice here that the probabilities are consistent
with what was used in the original example.
So this part of A that lies outside B has probability
3/12, but in this case I'm not telling you how that
probability is made up.
I'm not telling you that it consists of 3
equally likely outcomes.
So all I'm telling you is that the collective probability in
this region is 3/12.
The total probability of A is, again, 5/12 as before.
The total probability of B is 2 plus 4 equals
6/12, exactly as before.
So it's a sort of similar situation as before.
How should we revise our probabilities and create--
construct--
conditional probabilities once we are told
that event B has occurred?
First, this relation should remain true.
Once we are told that B has occurred, then B is certain to
occur, so it should have conditional
probability equal to 1.
How about the conditional probability of A given that B
has occurred?
Well, we can reason as follows.
In the original model, and if we just look inside event B,
those outcomes that make event A happen had a collective
probability which was 1/3 of the total probability assigned
to B. So out of the overall probability assigned to B, 1/3
of that probability corresponds to outcomes in
which event A is happening.
So therefore, if I tell you that B has occurred, I should
assign probability equal to 1/3 that event A is
also going to happen.
So that, given that B happened, the conditional
probability of A given B should be equal to 1/3.
By now, we should be satisfied that this approach is a
reasonable way of constructing conditional probabilities.
But now let us translate our reasoning into a formula.
So we wish to come up with a formula that gives us the
conditional probability of an event given another event.
The particular formula that captures our way of thinking,
as motivated before, is the following.
Out of the total probability assigned to B--
which is this--
we ask the question, which fraction of that probability
is assigned to outcomes under which event A also happens?
So we are living inside event B, but within that event, we
look at those outcomes for which event A also happens.
So this is the intersection of A and B. And we ask, out of
the total probability of B, what fraction of that
probability is allocated to that intersection of A with B?
So this formula, this definition, captures our
intuition of what we did before to construct
conditional probabilities in our particular example.
Let us check that the definition indeed does what
it's supposed to do.
In this example, the probability of the
intersection was 2/12 and the total probability of B was
6/12, which gives us 1/3, which is the answer that we
had gotten intuitively a little earlier.
At this point, let me also make a comment that this
definition of conditional probabilities makes sense only
if we do not attempt to divide by zero.
That this, only if the event B on which we're conditioning,
has positive probability.
If B, if an event B has 0 probability, then conditional
probabilities given B will be left undefined.
And one final comment.
This is a definition.
It's not a theorem.
What does that mean?
It means that there is no question whether this equality
is correct or not.
It's just a definition.
There's no issue of correctness.
The earlier argument that we gave was just a motivation of
the definition.
We tried to figure out what the definition should be if we
want to have a certain intuitive and meaningful
interpretation of the conditional probabilities.
Let us now continue with a simple example.