Inductive prior

by Eliezer Yudkowsky Feb 17 2016 updated Mar 3 2016

Some states of pre-observation belief can learn quickly; others never learn anything. An "inductive prior" is of the former type.

An "inductive prior" is a state of belief, before seeing any evidence, which is conducive to learning when the evidence finally appears. A classic example would be observing a coin come up heads or tails many times. If the coin is biased to come up heads 1/4 of the time, the inductive prior from Laplace's Rule of Succession will start predicting future flips to come up tails with 3/4 probability. The [ maximum entropy prior] for the coin, which says that every coinflip has a 50% chance of coming up heads and that all sequences of heads and tails are equally probable, will never start to predict that the next flip will be heads, even after observing the coin come up heads thirty times in a row.

The prior in Solomonoff induction is another example of an inductive prior - far more powerful, far more complicated, and entirely unimplementable on physically possible hardware.