Abortable plans


by Eliezer Yudkowsky Mar 22 2016

Plans that can be undone, or switched to having low further impact. If the AI builds abortable nanomachines, they'll have a quiet self-destruct option that includes any replicated nanomachines.

"Abortable" plans are those which can readily be switched to having very low net impact on the world. Suppose an AI is told to paint a car pink. The AI starts to do so by constructing replicating nanomachines that will paint the car pink. If the AI has successfully been given a shutdown utility function and shutdown button, we can press the button to have the AI switch off or suspend itself to disk and take no further action, but this might not affect the nanomachines already made. An AI with an abort button will have constructed the nanomachines such that at any time the AI can be given the "abort" instruction, which will with a minimum of further action on the AI's part cause all the nanomachines (including any replicated ones) to quietly self-destruct. That is, the AI has already planned such that the partial execution of the original plan, plus the activation midway of the abort subplan, will together have minimum impact on the world.