[summary: A form of Bayes' rule that uses relative odds.

Suppose we're trying to solve a mysterious murder, and we start out thinking the odds of Professor Plum vs. Miss Scarlet committing the murder are 1 : 2, that is, Scarlet is twice as likely as Plum to have committed the murder. We then observe that the victim was bludgeoned with a lead pipe. If we think that Plum, *if* he commits a murder, is around 60% likely to use a lead pipe, and that Scarlet, *if* she commits a murder, would be around 6% likely to us a lead pipe, this implies relative likelihoods of 10 : 1 for Plum vs. Scarlet using the pipe.

The posterior odds for Plum vs. Scarlet, after observing the victim to have been murdered by a pipe, are $~$(1 : 2) \times (10 : 1) = (10 : 2) = (5 : 1)$~$. We now think Plum is around five times as likely as Scarlet to have committed the murder.]

One of the more convenient forms of Bayes' rule uses relative odds. Bayes' rule says that, when you observe a piece of evidence $~$e,$~$ your posterior odds $~$\mathbb O(\boldsymbol H \mid e)$~$ for your hypothesis [-vector] $~$\boldsymbol H$~$ given $~$e$~$ is just your prior odds $~$\mathbb O(\boldsymbol H)$~$ on $~$\boldsymbol H$~$ times the Likelihood function $~$\mathcal L_e(\boldsymbol H).$~$

For example, suppose we're trying to solve a mysterious murder, and we start out thinking the odds of Professor Plum vs. Miss Scarlet committing the murder are 1 : 2, that is, Scarlet is twice as likely as Plum to have committed the murder a priori. We then observe that the victim was bludgeoned with a lead pipe. If we think that Plum, *if* he commits a murder, is around 60% likely to use a lead pipe, and that Scarlet, *if* she commits a murder, would be around 6% likely to us a lead pipe, this implies relative likelihoods of 10 : 1 for Plum vs. Scarlet using the pipe. The posterior odds for Plum vs. Scarlet, after observing the victim to have been murdered by a pipe, are $~$(1 : 2) \times (10 : 1) = (10 : 2) = (5 : 1)$~$. We now think Plum is around five times as likely as Scarlet to have committed the murder.

# Odds functions

Let $~$\boldsymbol H$~$ denote a [-vector] of hypotheses. An odds function $~$\mathbb O$~$ is a function that maps $~$\boldsymbol H$~$ to a set of Odds. For example, if $~$\boldsymbol H = (H_1, H_2, H_3),$~$ then $~$\mathbb O(\boldsymbol H)$~$ might be $~$(6 : 2 : 1),$~$ which says that $~$H_1$~$ is 3x as likely as $~$H_2$~$ and 6x as likely as $~$H_3.$~$ An odds function captures our *relative* probabilities between the hypotheses in $~$\boldsymbol H;$~$ for example, (6 : 2 : 1) odds are the same as (18 : 6 : 3) odds. We don't need to know the absolute probabilities of the $~$H_i$~$ in order to know the relative odds. All we require is that the relative odds are proportional to the absolute probabilities:
$$~$\mathbb O(\boldsymbol H) \propto \mathbb P(\boldsymbol H).$~$$

In the example with the death of Mr. Boddy, suppose $~$H_1$~$ denotes the proposition "Reverend Green murdered Mr. Boddy", $~$H_2$~$ denotes "Mrs. White did it", and $~$H_3$~$ denotes "Colonel Mustard did it". Let $~$\boldsymbol H$~$ be the vector $~$(H_1, H_2, H_3).$~$ If these propositions respectively have prior probabilities of 80%, 8%, and 4% (the remaining 8% being reserved for other hypotheses), then $~$\mathbb O(\boldsymbol H) = (80 : 8 : 4) = (20 : 2 : 1)$~$ represents our *relative* credences about the murder suspects — that Reverend Green is 10 times as likely to be the murderer as Miss White, who is twice as likely to be the murderer as Colonel Mustard.

# Likelihood functions

Suppose we discover that the victim was murdered by wrench. Suppose we think that Reverend Green, Mrs. White, and Colonel Mustard, *if* they murdered someone, would respectively be 60%, 90%, and 30% likely to use a wrench. Letting $~$e_w$~$ denote the observation "The victim was murdered by wrench," we would have $~$\mathbb P(e_w\mid \boldsymbol H) = (0.6, 0.9, 0.3).$~$ This gives us a Likelihood function defined as $~$\mathcal L_{e_w}(\boldsymbol H) = P(e_w \mid \boldsymbol H).$~$

# Bayes' rule, odds form

Let $~$\mathbb O(\boldsymbol H\mid e)$~$ denote the posterior odds of the hypotheses $~$\boldsymbol H$~$ after observing evidence $~$e.$~$ Bayes' rule then states:

$$~$\mathbb O(\boldsymbol H) \times \mathcal L_{e}(\boldsymbol H) = \mathbb O(\boldsymbol H\mid e)$~$$

This says that we can multiply the relative prior credence $~$\mathbb O(\boldsymbol H)$~$ by the likelihood $~$\mathcal L_{e}(\boldsymbol H)$~$ to arrive at the relative posterior credence $~$\mathbb O(\boldsymbol H\mid e).$~$ Because odds are invariant under multiplication by a positive constant, it wouldn't make any difference if the *likelihood* function was scaled up or down by a constant, because that would only have the effect of multiplying the final odds by a constant, which does not affect them. Thus, only the relative likelihoods are necessary to perform the calculation; the absolute likelihoods are unnecessary. Therefore, when performing the calculation, we can simplify $~$\mathcal L_e(\boldsymbol H) = (0.6, 0.9, 0.3)$~$ to the relative likelihoods $~$(2 : 3 : 1).$~$

In our example, this makes the calculation quite easy. The prior odds for Green vs White vs Mustard were $~$(20 : 2 : 1).$~$ The relative likelihoods were $~$(0.6 : 0.9 : 0.3)$~$ = $~$(2 : 3 : 1).$~$ Thus, the relative posterior odds after observing $~$e_w$~$ = Mr. Boddy was killed by wrench are $~$(20 : 2 : 1) \times (2 : 3 : 1) = (40 : 6 : 1).$~$ Given the evidence, Reverend Green is 40 times as likely as Colonel Mustard to be the killer, and 20/3 times as likely as Mrs. White.

Bayes' rule states that this *relative* proportioning of odds among these three suspects will be correct, regardless of how our remaining 8% probability mass is assigned to all other suspects and possibilities, or indeed, how much probability mass we assigned to other suspects to begin with. For a proof, see Proof of Bayes' rule.

# Visualization

Frequency diagrams, waterfall diagrams, and spotlight diagrams may be helpful for explaining or visualizing the odds form of Bayes' rule.

## Comments

Emile Kroeger

This page asks me if I learnt the concept of "Odds ratio" - but nowhere in the page does it actually explicitly

talkabout odds ratios, only about odds.