# Random utility function

https://arbital.com/p/random_utility_function

by Eliezer Yudkowsky Jan 11 2017 updated Feb 8 2017

A 'random' utility function is one chosen at random according to some simple probability measure (e.g. weight by Kolmorogov complexity) on a logical space of formal utility functions.

[summary: A 'random utility function' is a utility function that's been drawn according to some simple probability measure over a logical space of formal, compact specifications for utility functions. For example, we might say that a random utility function is a utility function specified by a random program drawn from the algorithmic complexity prior on programs.]

A 'random utility function' is a utility function selected according to some simple probability measure over a logical space of formal, compact specifications of utility functions.

For example: suppose utility functions are specified by computer programs (e.g. a program that maps an output description to a rational number). We then draw a random computer program from the standard universal prior on computer programs: $2^{-\operatorname K(U)}$ where $\operatorname K(U)$ is the algorithmic complexity (Kolmogorov complexity) of the utility-specifying program $U.$

This obvious measure could be amended further to e.g. take into account non-halting programs; to not put almost all of the probability mass on extremely simple programs; to put a satisficing criterion on whether it's computationally tractable and physically possible to optimize for $U$ (as assumed in the Orthogonality Thesis); etcetera.

Complexity of value is the thesis that the attainable optimum of a random utility function has near-null goodness with very high probability. That is: the attainable optimum configurations of matter for a random utility function are, with very high probability, the moral equivalent of paperclips. This in turn implies that a superintelligence with a random utility function is with very high probability the moral equivalent of a paperclip maximizer.

A 'random utility function' is not:

• A utility function randomly selected from whatever distribution of utility functions may actually exist among agents within the generalized universe. That is, a random utility function is not the utility function of a random actually-existing agent.
• A utility function with maxentropy content. That is, a random utility function is not one that independently assigns a uniform random value between 0 and 1 to every distinguishable outcome. (This utility function would not be tractable to optimize for--we couldn't optimize it ourselves even if somebody paid us--so it's not covered by e.g. the Orthogonality Thesis.)