Conditional probability: Refresher

https://arbital.com/p/conditional_probability_refresher

by Nate Soares Jul 6 2016 updated Oct 10 2016

Is P(yellow | banana) the probability that a banana is yellow, or the probability that a yellow thing is a banana?


[summary: $~$\mathbb P(\text{left} \mid \text{right})$~$ is the probability that the thing on the left is true assuming the thing on the right is true, and it's defined as $~$\frac{\mathbb P(\text{left} \land \text{right})}{\mathbb P(\text{right})}.$~$

Thus, $~$\mathbb P(yellow \mid banana)$~$ is the probability that a banana is yellow ("the probability of yellowness given banana"), while $~$\mathbb P(banana \mid yellow)$~$ is the probability that a yellow object is a banana ("the probability of banana, given yellowness").]

$~$\mathbb P(\text{left} \mid \text{right})$~$ is the probability that the thing on the left is true assuming the thing on the right is true, and it's defined as $~$\frac{\mathbb P(\text{left} \land \text{right})}{\mathbb P(\text{right})}.$~$

Thus, $~$\mathbb P(yellow \mid banana)$~$ is the probability that a banana is yellow ("the probability of yellowness given banana"), while $~$\mathbb P(banana \mid yellow)$~$ is the probability that a yellow object is a banana ("the probability of banana, given yellowness").%%note: In general, $~$\mathbb P(v)$~$ is an abbreviation of $~$\mathbb P(V = v)$~$ for some variable $~$V$~$, which is assumed to be known from the context. For example, $~$\mathbb P(yellow)$~$ might stand for $~$\mathbb P({ColorOfNextObjectInBag}=yellow)$~$ where $~$ColorOfNextObjectInBag$~$ is a [random_variable variable] in our [-probability_distribution] $~$\mathbb P,$~$ and $~$yellow$~$ is one of the [random_variable_value values] that that variable can take on.%%

$~$\mathbb P(x \land y)$~$ is used to denote the probability of both $~$x$~$ and $~$y$~$ being true simultaneously (according to some [-probability_distribution] $~$\mathbb P$~$). $~$\mathbb P(x\mid y)$~$, pronounced "the conditional probability of x, given y", is defined to be the quantity

$$~$\frac{\mathbb P(x \wedge y)}{\mathbb P(y)}.$~$$

For example, in the Diseasitis problem, $~$\mathbb P({sick}\mid {positive})$~$ is the probability that a patient is sick given a positive test result, and it's calculated by taking the 18% patients who are sick and have positive test results, and dividing by all 42% of the patients who got positive test results. That is, $~$\mathbb P({sick}\mid {positive})$~$ $~$=$~$ $~$\frac{\mathbb P({sick} \land {positive})}{\mathbb P({positive})}.$~$

Using a [bayes_frequency_diagrams frequency diagram], we can visualize $~$\mathbb P(sick \mid positive)$~$ as the probability of drawing a $~$sick$~$ result from a bag of only those people in the population who got a $~$positive$~$ result.

diseasitis frequency

bag of 18 and 24 patients

The "given" operator in $~$\mathbb P(x\mid y)$~$ tells us to assume that $~$y$~$ is true, to restrict our attention to only possible cases where $~$y$~$ is true, and then ask about the probability of $~$x$~$ within those cases.

Note that $~$\mathbb P(positive \mid sick)$~$ is not the same as $~$\mathbb P(sick \mid positive).$~$ To find the probability that a patient has a positive result given that they're sick, we can visualize taking the 20 sick patients and putting them in a group, and then asking the probability that a randomly selected one will have a positive result, which will be $~$\frac{18}{20} = 0.9$~$ — so $~$\mathbb P(positive \mid sick) = 90\%,$~$ while $~$\mathbb P(sick \mid positive) \approx 43\%.$~$ Mixing up which one is which is an unfortunate source of of many practical errors when you're trying to do these calculations using only the formal notation, at least until you get used to it. Just remember that $~$\mathbb P(\text{left} \mid \text{right})$~$ is the probability of the thing on the left given that the thing on the right is true.