Bayes' rule: Proportional form

https://arbital.com/p/bayes_rule_proportional

by Eliezer Yudkowsky Feb 13 2016 updated Oct 11 2016

The fastest way to say something both convincing and true about belief-updating.


[summary: Suppose that Professor Plum and Miss Scarlet are two suspects in a murder, and that we start out thinking that Professor Plum is twice as likely to have committed the murder as Miss Scarlet. We then discover that the victim was poisoned. We think that Professor Plum is around one-fourth as likely to use poison as Miss Scarlet. Then after observing the victim was poisoned, we should think Plum is around half as likely to have committed the murder as Scarlet: $~$2 \times \dfrac{1}{4} = \dfrac{1}{2}.$~$ This reasoning is valid by Bayes' rule.]

If $~$H_i$~$ and $~$H_j$~$ are hypotheses and $~$e$~$ is a piece of evidence, Bayes' rule states:

$$~$\dfrac{\mathbb P(H_i)}{\mathbb P(H_j)} \times \dfrac{\mathbb P(e\mid H_i)}{\mathbb P(e\mid H_j)} = \dfrac{\mathbb P(H_i\mid e)}{\mathbb P(H_j\mid e)}$~$$

%%if-after(Frequency diagrams: A first look at Bayes): In the Diseasitis problem, we use this form of Bayes' rule to justify calculating the posterior odds of sickness via the calculation $~$(1 : 4) \times (3 : 1) = (3 : 4).$~$ %%

%%!if-after(Frequency diagrams: A first look at Bayes): In the Diseasitis problem, 20% of the patients in a screening population have Diseasitis, 90% of sick patients will turn a chemical strip black, and 30% of healthy patients will turn a chemical strip black. We can use the form of Bayes' rule above to justify solving this problem via the calculation $~$(1 : 4) \times (3 : 1) = (3 : 4).$~$ %%

If instead of treating the ratios as odds, we actually calculate out the numbers for each term of the equation, we instead get the calculation $~$\frac{1}{4} \times \frac{3}{1} = \frac{3}{4},$~$ or $~$0.25 \times 3 = 0.75.$~$

If we try to directly interpret this, it says: "If a patient starts out 0.25 times as likely to be sick as healthy, and we see a test result that is 3 times as likely to occur if the patient is sick as if the patient is healthy, we conclude the patient is 0.75 times as likely to be sick as healthy."

This is valid reasoning, and we call it the proportional form of Bayes' rule. To get the probability back out, we reason that if there's 0.75 sick patients to every 1 healthy patient in a bag, the bag comprises 0.75/(0.75 + 1) = 3/7 = 43% sick patients.

Spotlight visualization

One way of looking at this result is that, since odds ratios are equivalent under multiplication by a positive constant, we can fix the right side of the odds ratio as equaling 1 and ask about what's on the left side. This is what we do when seeing the calculation as $~$(0.25 : 1) \cdot (3 : 1) = (0.75 : 1),$~$ the form suggested by the theorem proved above.

We could visualize Bayes' rule as a pair of spotlights with different starting intensities, that go through lenses that amplify or reduce each incoming unit of light by a fixed multiplier. In the Diseasitis case, if we fix the right-side blue beam as having a starting intensity of 1 and a multiplying lens of 1, and we fix the left-side beam of having a starting intensity of 0.25 and a multiplying lens of 3x, then the result gives us a visualization of the calculation prescribed by Bayes' rule:

bayes lights

Note the similarity to a waterfall diagram. The main thing the spotlight visualization adds is that we can imagine varying the absolute intensities of the lights and lenses, while preserving their relative intensities, in such a way as to make the right-side beams and lenses equal 1.

[fixme: draw the pre-proportional, odds form of the spotlight visualization.]

%todo: add example problem in proportional/spotlight form%

Usefulness in informal argument

The proportional form of Bayes' rule is perhaps the fastest way of describing Bayesian reasoning that sounds like it ought to be true. If you were having a fictional character suddenly give a Bayesian argument in the middle of a story being read by many people who'd never heard of Bayes' rule, you might have them say:

"Suppose the Dark Mark is certain to continue while the Dark Lord's sentience lives on, but a priori we'd only have guessed a twenty percent chance of the Dark Mark continuing to exist after the Dark Lord dies. Then the observation, "The Dark Mark has not faded" is five times as likely to occur in worlds where the Dark Lord is alive as in worlds where the Dark Lord is dead. Is that really commensurate with the prior improbability of immortality? Let's say the prior odds were a hundred-to-one against the Dark Lord surviving. If a hypothesis is a hundred times as likely to be false versus true, and then you see evidence five times more likely if the hypothesis is true versus false, you should update to believing the hypothesis is twenty times as likely to be false as true."

Similarly, if you were a doctor trying to explain the meaning of a positive test result to a patient, you might say: "If we haven't seen any test results, patients like you are a thousand times as likely to be healthy as sick. This test is only a hundred times as likely to be positive for sick as for healthy patients. So now we think you're ten times as likely to be healthy as sick, which is still a pretty good chance!"

Visual diagrams and special notation for odds and relative likelihoods might make Bayes' rule more intuitive, but the proportional form is probably the most valid-sounding thing that is quantitatively correct that you can say in three sentences.

[fixme: write a from-scratch Standalone Intro of the proportional form of Bayes' rule in particular, using the Diseasitis example and going from frequency diagram to waterfall to spotlight, with no proofs, just to justify the proportional form. add to Main a statement that if you can phrase things in proportional form, there exists a Standalone Intro that justifies it quickly.]


Comments

Eric Rogstad

We could visualize Bayes' rule as a pair of spotlights with different starting intensities, that go through lenses that amplify or reduce each incoming unit of light by a fixed multiplier\. In the Diseasitis case, if we fix the right\-side blue beam as having a starting intensity of 1 and a multiplying lens of 1, the result visualizes the calculation 0\.25 \* 3 = 0\.75:

Is this sentence supposed to have a part saying, "and if we set the left-side red beam… starting intensity of .25 and a multiplying lens of 3…" ?

Eric Rogstad

Arguably, the proportional form of Bayes's Rule is the fastest way of describing a Bayesian argument, without using any special notation like odds ratios, which at least sounds like it maybe ought to be true\. If you were having a fictional character suddenly give a Bayesian argument in the middle of a story being read by many people who'd never heard of Bayes, you might have them say:

Confused about the role this clause is playing. What does "which" refer to --

the proportional form of Bayes's Rule?

the whole rest of the sentence?