Conditional Probability

Introduction

Conditional Probability

Suppose $A,B$ are subsets of some sample space $S$, written mathematically as $A,B \subseteq S$. The probability of $A$ given $B$ is defined as

$$\mathbb{P}[A | B] = \frac{\mathbb{P}[A \cap B]}{\mathbb{P}[B]}$$

so long as $\mathbb{P}[B] > 0$.

We read the conditional probability statement $\mathbb{P}[A | B]$ as the probability of $A$ given, or conditioned on, $B$. We think about this calculation as updating the probability of the probability of $A$ using the information contained in $B$. If the set $B$ contains any information about $A$, then $\mathbb{P}[A | B] \ne \mathbb{P}[A]$.

Notice that conditional probability connects back to our definition of independent events; for $A,B \subset S$, $A$ and $B$ are said to be independent if $$\mathbb{P}[A \cap B] = \mathbb{P}[A] \mathbb{P}[B]$$ So if sets $A,B$ are independent, then the conditional probability of $A$ given $B$ is equal to the probability of $A$ $$\mathbb{P}[A | B] = \frac{\mathbb{P}[A \cap B]}{\mathbb{P}[B]} = \frac{\mathbb{P}[A]\mathbb{P}[B]}{\mathbb{P}[B]} = \mathbb{P}[A]$$ Notice then that if $A,B$ are independent, then $B$ contains no information about $A$ and so $\mathbb{P}[A | B] = \mathbb{P}[A]$.

Conditional Probability Examples

1. In a school, there are 200 students. Among them, 120 are studying Mathematics, and 80 are studying Science. It is known that 60 students are studying both Mathematics and Science. You randomly pick a student who is studying Mathematics. What is the probability that the selected student is also studying Science?
2. Consider the following table giving counts for the types of each coffee purchased at my favorite coffee shop within a weekend.

   	Small	Medium	Large
   regular	19	23	22
   decaf	20	15	17

   1. What is the probability that someone orders a decaf cup of coffee?
   2. Given that someone ordered a medium, what is the probability that their order is for a decaf cup of coffee?
   3. What is the probability that someone orders a small cup of coffee?
   4. Given that some ordered a regular cup of coffee, what is the probability that their order is for a small cup of coffee?

Bayes' Theorem

Baye's theorem is a combination of conditional probability and the Law of Total Probability.

Law of Total Probability

For an arbitrary sample space $S$ and an arbitrary subset $B \subseteq S$, let $\{A_n\}$ be a partition of $S$. Then $$\mathbb{P}[B] = \sum_n \mathbb{P}[B \cap A_n]$$

The statement of the Law of Total Probability can be summarized by the following picture. In picture, it's a little easier to see that the area of $B$, otherwise known as the probability of $B$, $\mathbb{P}[B]$, is composed of four subsets of $B$. The four subsets are $B \cap A_1$, $B \cap A_2$, $B \cap A_3$, and $B \cap A_4$.

Because the sets $B \cap A_n$ for $n \in \{1,2,3,4\}$ are mutually exclusive, we can use the third condition of distributions to find the probability of $B$ as the sum of the probability of these sets.

Theoretically, the collection of sets $\{A_n\}$ can be a partition of any super set of $B$, $C \supseteq B$. Most often though, the Law of Total Probability is framed as in the picture above, relative to the sample space $S$.

Bayes' Theorem

Suppose $A,B \subseteq S$ and that $\mathbb{P}[B] > 0$. Bayes' Theorem states that $$\mathbb{P}[A | B] = \frac{\mathbb{P}[B|A]\mathbb{P}[A]}{\mathbb{P}[B]}$$

If necessary, the Law of Total Probability can be used to calculate the denominator. Bayes' Theorem is the formula that allows you to compute the probability of $A$ given $B$ using the probability $B$ given $A$; effecitively, the set being conditioned on is swapped for the set on which probability is being calculated, $A \leftrightarrow B$.

Bayes' Theorem Examples

1. A factory produces three types of widgets: Type A, Type B, and Type C. The production ratios are such that 50% of the widgets produced are Type A, 30% are Type B, and 20% are Type C. Each widget is then independently tested, and it is found that 5% of Type A widgets, 3% of Type B widgets, and 10% of Type C widgets fail the test. A widget is randomly selected from the production line and is found to have failed the test. What is the probability that this widget is a Type C widget?
2. In a certain population, 1% of people have a specific genetic disorder. There is a diagnostic test for this disorder, which has a 95% sensitivity (meaning that it correctly identifies 95% of those who have the disorder) and a 98% specificity (meaning that it correctly identifies 98% of those who do not have the disorder). A medical student picks a random individual from this population and administers the test, which comes back positive. Calculate the probability that this individual actually has the genetic disorder, given that their test result is positive.
3. A civil engineering company is working on a large infrastructure project. Historical data shows that there's a 70% chance of encountering significant geological challenges when building in this region. If such challenges are encountered, the probability of project delays is 80%. However, if no geological challenges are encountered, the probability of delays due to other factors is only 10%. Given that a project is delayed, what is the probability that it was due to geological challenges?

References

Conditional probability. Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. Accessed 2024-04-11.

Law of total probability. Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. Accessed 2024-04-11.

Partition of a set. Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. Accessed 2024-04-11.