Neutral genie metaphor

https://arbital.com/p/neural_genie_metaphor

by Alexei Andreev Dec 16 2015 updated Dec 28 2015


Definition. A neutral-genie metaphor is an attempt to illustrate a possible formal problem via an informal analogies involving neutral genies who obey English definitions of terms inside their [ goal systems], and otherwise tend to behave as bounded [ expected utility optimizers] unless otherwise specified. The genie is not assumed to have any goals other than those described in its stated goal system; it neither loves you nor hates you apart from that. In particular, a neutral genie does not care one way or another 'what you really meant' or 'what you had in mind' (unless we have otherwise constructed a [ Do What I Mean goal system] for it, which will not be part of most metaphors).