Sample spaces are too large

https://arbital.com/p/sample_spaces_are_too_large

by Tsvi BT May 25 2016 updated May 25 2016

Sample spaces are often large, so it is hard to do probabilistic computations using a raw distribution over the sample space.


[summary: Sample spaces enumerate everything that could possibly happen, so that we can reason consistently under uncertainty. But this enumeration usually enumerates too many things, making it difficult to do computations using the raw sample space.]

When we are reasoning about an uncertain world using probability theory, we have a sample space of possible ways the world could be, and we assign probabilities to outcomes in the sample space with a probability distribution. Unfortunately, often the sample space (which is supposed to contain every possible way the world might turn out) is very large and possibly infinite.

For example, if we are thinking about a box with some particles in it, our sample space is the enormous set of all possible arrangements of the particles. To reason usefully about the box, we'd have to do computations using a gigantic table specifying a separate number for every single distinct arrangement of particles. Two arrangements that are pretty much the same, as far as we care, take up just as much of our computational resources as two importantly different arrangements. So, even though we could in principle reason consistently under uncertainty just using a probability distribution over a sample space, we cannot do so easily, because we are trying to keep track of too much.

[todo: examples]