"Eliezer goes back and forth between "sapient" a..."

https://arbital.com/p/918

by Phil Goetz Mar 13 2018 updated Mar 13 2018


Eliezer goes back and forth between "sapient" and "sentient", which are not synonyms. Neither is obviously a justification for claiming moral status as an agent.

It is important either to state clearly what one presumes gives an agent moral status (and hence what constitutes mindcrime), or to change each occurence of "sapient", "sentient", or "personhood" to all use the same word. I recommend stating the general case using personhood(X), a function to be supplied by the user and not defined here. Addressing the problem depends critically on what that function is--but the statement of the general case shouldn't be bound up with the choice of personhood predicate.

Choosing either "sapient" or "sentient" is problematic: "sentient" because it includes at least all mammals, and "sapient" because it really just means "intelligent", and the AI is going to be equally intelligent (defined as problem-solving or optimizing ability) whether it simulates humans or not. If intelligence grants moral standing (as it seems to here), and mindcrime means trapping an agent with moral standing in the AI's world, then the construction of any AI is inherently mindcrime.