The plan experiment

by Alexei Andreev Jan 18 2017 updated Jan 31 2017

Root page for describing the reason and the process for planning how to approach and navigate through AGI development.

"The plan" is an experiment in collaborative discussion. The central topic is helping humans navigate development of AGI. Its development is a hard technical problem, and making sure that the way it's developed is beneficial to all humans makes it even harder. In the meantime, there is a perceived aura of pressure. This pressure comes from multiple sources:

This experiment exists to address all of these issues by recognizing their manifestations in each individual. To fully address these issues, the plan has to be public.

What's in the plan?

National response framework%%note: The National Response Framework describes not only how the Federal government organizes itself to respond to natural disasters, terrorist attacks, and other catastrophic events but also the importance of the whole community in assisting with response efforts. The intended audience for this document is individuals, families, communities, the private and nonprofit sectors, faith-based organizations, and local, state, tribal, territorial, insular area, and Federal governments.%% provides a detailed document describing how various organizations and federal governments can respond to an event of emergency. I imagine the final output of the plan to be something along these lines, although it'll be framed in a more proactive way.%%note: It's possible NIMS is closer to what I'm looking for. It's "a systematic, proactive approach to guide departments and agencies at all levels of government, nongovernmental organizations, and the private sector to work together seamlessly and manage incidents involving all threats and hazards—regardless of cause, size, location, or complexity—in order to reduce loss of life, property and harm to the environment."%%

Specifically, the plan will have these components:



  1. Go through existing literature and extract the relevant parts. Use those to write the first version of the plan.
  1. I will show the current version of the plan to one person and get their feedback.
  1. I will add as much of their feedback to the plan as possible, keeping their identity anonymous if requested, though I have a strong preference for people's opinions to be public.
  2. Iterate on this process and the plan.

Non-interference stance

It's hard to understand a person's point of view when your own gets in the way. This is why I'm going to follow these guidelines when having one-on-ones with people:

  1. My goal is not to make anyone do anything they don't want to do.
  2. My goal is not to get the person to adopt the plan.
  3. My goal is to explain various parts of the plan to a person, then understand and integrate their feedback into the plan. (Steelmanning their argument in the process.)
  4. My goal is for the person to be happy with the way their feedback was integrated into the plan.

In fact, if this approach works well, I'd be happy to take this to its logical conclusion of creating a new class of x-risks participants who are forbidden from interfering in AI development landscape.

The plan keeper

A lot of the plan will come from one-on-one discussions. If all the conversations are stored in one person's mind, they can have future conversations much better. However, if the past conversations were done by two people, than each person can only bring half of all the relevant information. So, naively, this task is much better done by one person. However, that's probably unrealistic. In either case, keeping the group small seems best.

Measure of success

Initial measure of success is: how many people's opinions are integrated into the plan and how happy those people are with their views being represented.

Final measure of success is: are people referring to and following the plan.

(It's possible we should only use the initial measure.)