Concepts: Conditional probability and the multiplication theorem

2.1 Information changes the sample space

A random experiment gives a sample space and a probability to each outcome. Once we learn that an event F has happened, every outcome outside F is ruled out and the new sample space is the set F. Probabilities of other events must then be recalculated inside this smaller world.

Take three fair coins tossed together, so there are eight equally likely outcomes. Let E be the event that at least two heads appear, and let F be the event that the first coin shows a tail. Without information, E has probability one half. If we are told that F occurred, only the four outcomes with a first tail remain possible, and among them just one belongs to E, so the new probability is one quarter. This reduced-sample-space probability is the conditional probability of E given F.

The definition comes from the same counting. The outcomes favourable to E inside F are the elements of the overlap, so the conditional probability is the number in the overlap divided by the number in F. Dividing the top and bottom of that fraction by the size of the whole sample space gives the working definition shown in the next block. The definition requires that F has nonzero probability, since otherwise the condition is impossible.

2.2 What a tree diagram shows

A tree diagram draws the experiment in stages. Each branch is labelled with the probability of taking that branch from the point where it starts, and the probability of a path is the product of the branch probabilities along it. The tree makes the conditional structure visible: the second-stage probabilities are conditional probabilities given the first-stage outcome, and the product along a path is the multiplication theorem in action. Figures 13.1 and 13.2 in the slideshow on the Bayes page are exactly such trees, first with outcomes only and then with probabilities attached.

2.3 The three properties

Conditioning on a fixed event behaves like a probability on the new sample space, so the familiar rules survive. The certain event has conditional probability one, and so does the condition itself. The addition rule holds for any two events after conditioning on the same event, and the last term disappears when the two events are disjoint. The complement rule also survives: the probability of the complement is one minus the conditional probability. All three are proved directly from the definition, and they let conditional probabilities be handled like ordinary probabilities as long as the condition stays fixed throughout.

2.4 The multiplication theorem

Rearranging the definition gives a rule for the probability of the overlap, which is the multiplication theorem. The symmetric version follows by exchanging the roles of the two events, and the useful form is the one whose conditional probability the data supplies. The theorem extends to a chain: for three events the probability of all three happening is the probability of the first, times the conditional probability of the second given the first, times the conditional probability of the third given the first two. That chain rule is why tree diagrams work, because each stage multiplies the previous product by the next conditional probability.

2.5 Reading a question correctly

The main danger is the direction of the condition. The two conditional probabilities of the same pair of events are different numbers, and swapping them is the most common error. In a two-way table or a tree, identify the conditioned event first: it is the one whose outcomes form the new sample space, and it sits after the bar. Then read the overlap from the intersection. Questions often supply counts of a population and ask for a conditional probability; in that case the denominator is the size of the given group and the numerator is the part of that group satisfying the other event.

2.6 Worked example

Two cards are drawn one after the other from a well-shuffled pack of 52, without replacement. What is the probability that the second card is a king, given that the first is a king?

Let A be the event that the first card is a king and B the event that the second card is a king. The condition is A, so the new sample space has 51 cards left, and three kings remain. The conditional probability is therefore 3/51 = 1/17. The same answer comes from the definition: the overlap probability is (4/52)(3/51) = 1/221, the probability of A is 4/52 = 1/13, and dividing gives 1/17. If the cards were drawn with replacement, the pack would be restored and the conditional probability would be 4/52 again; this contrast between the two cases is exactly what the multiplication theorem formalises.