STAT2115›Conditional Probability and Bayes' Theorem

Introduction to StatisticsTopic 15 of 24

Conditional Probability and Bayes' Theorem

6 minread

1,092words

Intermediatelevel

1. Conditional Probability

Conditional probability measures the probability of an event A occurring given that another event B has already occurred.

P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad \text{if } P(B) > 0

$P(A|B)$ = probability of A given B
$P(A \cap B)$ = probability that both A and B occur
$P(B)$ = probability of B

Key Points

Conditional probability changes the sample space to B.
If A and B are independent, then $P(A|B) = P(A)$

Example:

A card is drawn from a deck of 52 cards.
Let $A$ = "card is a King" ( $4/52$ )
Let $B$ = "card is a face card" ( $12/52$ )

P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{4/52}{12/52} = \frac{1}{3}

2. Multiplication Rule

Conditional probability leads to the general multiplication rule:

P(A \cap B) = P(B) \cdot P(A|B) = P(A) \cdot P(B|A)

Useful for dependent events.
If A and B are independent, $P(A \cap B) = P(A) \cdot P(B)$ .

3. Bayes’ Theorem

Bayes’ Theorem allows us to update the probability of an event based on new information.

Suppose $E_1, E_2, ..., E_n$ are mutually exclusive and exhaustive events (partition of sample space) and A is an event. Then:

P(E_i | A) = \frac{P(A|E_i) \cdot P(E_i)}{\sum_{j=1}^{n} P(A|E_j) \cdot P(E_j)}

$P(E_i)$ = prior probability of $E_i$
$P(A|E_i)$ = probability of observing A given $E_i$
$P(E_i|A)$ = posterior probability of $E_i$ after observing A

Steps to Use Bayes’ Theorem

Identify the partition events $E_1, E_2, ..., E_n$ .
Determine the prior probabilities $P(E_i)$ .
Determine likelihoods $P(A|E_i)$ .
Apply formula to find posterior probability $P(E_i|A)$ .

Example: Medical Test

Disease prevalence: $P(D) = 0.01$
Test positive if diseased: $P(T+|D) = 0.99$
Test positive if healthy: $P(T+|D') = 0.05$

Probability that a person has the disease given a positive test:

P(D|T+) = \frac{P(T+|D)P(D)}{P(T+|D)P(D) + P(T+|D')P(D')}

P(D|T+) = \frac{0.99 \cdot 0.01}{0.99 \cdot 0.01 + 0.05 \cdot 0.99} \approx 0.166

Even with a positive test, probability of disease ≈ 16.6%
Shows importance of prior probability (prevalence).

4. Summary Table

Concept	Formula	Notes
Conditional Probability	$$ P(A	B) = \frac{P(A \cap B)}{P(B)} $$	Changes sample space to B
Multiplication Rule	$P(A \cap B) = P(A) P(B	A) = P(B) P(A	B)$	For dependent events
Bayes' Theorem	$P(E_i	A) = \frac{P(A	E_i)P(E_i)}{\sum P(A	E_j)P(E_j)}$	Updates probability based on new info

Key Points to Remember

Conditional probability adjusts probabilities based on known events.
Multiplication rule links joint probability to conditional probability.
Bayes’ theorem is widely used in diagnostics, risk analysis, and machine learning.

Previous topic 14

Probability Spaces and Laws of Probability

Next topic 16

Discrete and Continuous Random Variables

Past Papers

Open this section to load past papers

Click on Show Past Papers to see past papers.

STAT2115›Conditional Probability and Bayes' Theorem

Introduction to StatisticsTopic 15 of 24

Conditional Probability and Bayes' Theorem

6 minread

1,092words

Intermediatelevel

1. Conditional Probability

Conditional probability measures the probability of an event A occurring given that another event B has already occurred.

P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad \text{if } P(B) > 0

$P(A|B)$ = probability of A given B
$P(A \cap B)$ = probability that both A and B occur
$P(B)$ = probability of B

Key Points

Conditional probability changes the sample space to B.
If A and B are independent, then $P(A|B) = P(A)$

Example:

A card is drawn from a deck of 52 cards.
Let $A$ = "card is a King" ( $4/52$ )
Let $B$ = "card is a face card" ( $12/52$ )

P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{4/52}{12/52} = \frac{1}{3}

2. Multiplication Rule

Conditional probability leads to the general multiplication rule:

P(A \cap B) = P(B) \cdot P(A|B) = P(A) \cdot P(B|A)

Useful for dependent events.
If A and B are independent, $P(A \cap B) = P(A) \cdot P(B)$ .

3. Bayes’ Theorem

Bayes’ Theorem allows us to update the probability of an event based on new information.

Suppose $E_1, E_2, ..., E_n$ are mutually exclusive and exhaustive events (partition of sample space) and A is an event. Then:

P(E_i | A) = \frac{P(A|E_i) \cdot P(E_i)}{\sum_{j=1}^{n} P(A|E_j) \cdot P(E_j)}

$P(E_i)$ = prior probability of $E_i$
$P(A|E_i)$ = probability of observing A given $E_i$
$P(E_i|A)$ = posterior probability of $E_i$ after observing A

Steps to Use Bayes’ Theorem

Identify the partition events $E_1, E_2, ..., E_n$ .
Determine the prior probabilities $P(E_i)$ .
Determine likelihoods $P(A|E_i)$ .
Apply formula to find posterior probability $P(E_i|A)$ .

Example: Medical Test

Disease prevalence: $P(D) = 0.01$
Test positive if diseased: $P(T+|D) = 0.99$
Test positive if healthy: $P(T+|D') = 0.05$

Probability that a person has the disease given a positive test:

P(D|T+) = \frac{P(T+|D)P(D)}{P(T+|D)P(D) + P(T+|D')P(D')}

P(D|T+) = \frac{0.99 \cdot 0.01}{0.99 \cdot 0.01 + 0.05 \cdot 0.99} \approx 0.166

Even with a positive test, probability of disease ≈ 16.6%
Shows importance of prior probability (prevalence).

4. Summary Table

Concept	Formula	Notes
Conditional Probability	$$ P(A	B) = \frac{P(A \cap B)}{P(B)} $$	Changes sample space to B
Multiplication Rule	$P(A \cap B) = P(A) P(B	A) = P(B) P(A	B)$	For dependent events
Bayes' Theorem	$P(E_i	A) = \frac{P(A	E_i)P(E_i)}{\sum P(A	E_j)P(E_j)}$	Updates probability based on new info

Key Points to Remember

Conditional probability adjusts probabilities based on known events.
Multiplication rule links joint probability to conditional probability.
Bayes’ theorem is widely used in diagnostics, risk analysis, and machine learning.

Previous topic 14

Probability Spaces and Laws of Probability

Next topic 16

Discrete and Continuous Random Variables

Past Papers

Open this section to load past papers

Click on Show Past Papers to see past papers.