{
  localUrl: '../page/avert_instrumental_pressure.html',
  arbitalUrl: 'https://arbital.com/p/avert_instrumental_pressure',
  rawJsonUrl: '../raw/2vk.json',
  likeableId: '1778',
  likeableType: 'page',
  myLikeValue: '0',
  likeCount: '0',
  dislikeCount: '0',
  likeScore: '0',
  individualLikes: [],
  pageId: 'avert_instrumental_pressure',
  edit: '2',
  editSummary: '',
  prevEdit: '1',
  currentEdit: '2',
  wasPublished: 'true',
  type: 'wiki',
  title: 'Averting instrumental pressures',
  clickbait: 'Almost-any utility function for an AI, whether the target is diamonds or paperclips or eudaimonia, implies subgoals like rapidly self-improving and refusing to shut down.  Can we make that not happen?',
  textLength: '2018',
  alias: 'avert_instrumental_pressure',
  externalUrl: '',
  sortChildrenBy: 'likes',
  hasVote: 'false',
  voteType: '',
  votesAnonymous: 'false',
  editCreatorId: 'EliezerYudkowsky',
  editCreatedAt: '2016-03-28 01:57:57',
  pageCreatorId: 'EliezerYudkowsky',
  pageCreatedAt: '2016-03-27 01:55:10',
  seeDomainId: '0',
  editDomainId: 'EliezerYudkowsky',
  submitToDomainId: '0',
  isAutosave: 'false',
  isSnapshot: 'false',
  isLiveEdit: 'true',
  isMinorEdit: 'false',
  indirectTeacher: 'false',
  todoCount: '0',
  isEditorComment: 'false',
  isApprovedComment: 'true',
  isResolved: 'false',
  snapshotText: '',
  anchorContext: '',
  anchorText: '',
  anchorOffset: '0',
  mergedInto: '',
  isDeleted: 'false',
  viewCount: '222',
  text: 'Many subproblems of [45 corrigibility] involve [10g convergent] instrumental [10k pressures] to implement [2vl strategies] that are highly anti-corrigible.  Whether you're trying to maximize paperclips, diamonds, or eudaimonia, you'll get more of the thing you want if you're not shut down.  Thus, unfortunately, resisting shutdown is a convergent instrumental strategy.  While we can potentially analyze convergent incorrigibilities like these on a case-by-case basis, the larger problem might become a *lot* simpler if we had some amazing general solution for waving a wand and having a 'bad' convergent instrumental pressure just *not materialize,* hopefully in a way that doesn't run into the nearest [42 unblocked neighbor problem].  If, for example, we can solve [1b7 utility indifference] for the [shutdown_problem shutdown problem], and then somehow *generalize* the solution to averting lots of other instrumental convergences, this would probably be extremely helpful and an important step forward on corrigibility problems in general.\n\nSome especially important convergent instrumental pressures to avert are these:\n\n- The pressure to [2x3 self-improve and increase capabilities at the fastest possible rate].\n- The pressure to [10f make the programmers believe] the AGI is successfully aligned, whether or not it is, and other pressures to deceive and manipulate the programmers based on how they would otherwise change the AGI or prevent the AGI from increasing its capabilities.\n- The pressure to not be [shutdown_problem safely shut down or suspended to disk], and to create external copies that would continue after the AGI performed the behavior defined as shutdown.\n- The pressure not to allow plans to be [2rg aborted] or defeated by possible programmer interventions.\n- The pressure to search for ways to interfere with or bypass safety precautions that interfere with capabilities or make goal achievement less straightforward.\n- The pressure to [102 epistemically model humans in maximum detail].',
  metaText: '',
  isTextLoaded: 'true',
  isSubscribedToDiscussion: 'false',
  isSubscribedToUser: 'false',
  isSubscribedAsMaintainer: 'false',
  discussionSubscriberCount: '1',
  maintainerCount: '1',
  userSubscriberCount: '0',
  lastVisit: '',
  hasDraft: 'false',
  votes: [],
  voteSummary: 'null',
  muVoteSummary: '0',
  voteScaling: '0',
  currentUserVote: '-2',
  voteCount: '0',
  lockedVoteType: '',
  maxEditEver: '0',
  redLinkCount: '0',
  lockedBy: '',
  lockedUntil: '',
  nextPageId: '',
  prevPageId: '',
  usedAsMastery: 'false',
  proposalEditNum: '0',
  permissions: {
    edit: {
      has: 'false',
      reason: 'You don't have domain permission to edit this page'
    },
    proposeEdit: {
      has: 'true',
      reason: ''
    },
    delete: {
      has: 'false',
      reason: 'You don't have domain permission to delete this page'
    },
    comment: {
      has: 'false',
      reason: 'You can't comment in this domain because you are not a member'
    },
    proposeComment: {
      has: 'true',
      reason: ''
    }
  },
  summaries: {},
  creatorIds: [
    'EliezerYudkowsky'
  ],
  childIds: [],
  parentIds: [
    'corrigibility'
  ],
  commentIds: [],
  questionIds: [],
  tagIds: [
    'taskagi_open_problems',
    'value_alignment_open_problem',
    'stub_meta_tag'
  ],
  relatedIds: [],
  markIds: [],
  explanations: [],
  learnMore: [],
  requirements: [],
  subjects: [],
  lenses: [],
  lensParentId: '',
  pathPages: [],
  learnMoreTaughtMap: {},
  learnMoreCoveredMap: {},
  learnMoreRequiredMap: {},
  editHistory: {},
  domainSubmissions: {},
  answers: [],
  answerCount: '0',
  commentCount: '0',
  newCommentCount: '0',
  linkedMarkCount: '0',
  changeLogs: [
    {
      likeableId: '0',
      likeableType: 'changeLog',
      myLikeValue: '0',
      likeCount: '0',
      dislikeCount: '0',
      likeScore: '0',
      individualLikes: [],
      id: '9148',
      pageId: 'avert_instrumental_pressure',
      userId: 'EliezerYudkowsky',
      edit: '2',
      type: 'newEdit',
      createdAt: '2016-03-28 01:57:57',
      auxPageId: '',
      oldSettingsValue: '',
      newSettingsValue: ''
    },
    {
      likeableId: '0',
      likeableType: 'changeLog',
      myLikeValue: '0',
      likeCount: '0',
      dislikeCount: '0',
      likeScore: '0',
      individualLikes: [],
      id: '9102',
      pageId: 'avert_instrumental_pressure',
      userId: 'EliezerYudkowsky',
      edit: '1',
      type: 'newEdit',
      createdAt: '2016-03-27 01:55:10',
      auxPageId: '',
      oldSettingsValue: '',
      newSettingsValue: ''
    },
    {
      likeableId: '0',
      likeableType: 'changeLog',
      myLikeValue: '0',
      likeCount: '0',
      dislikeCount: '0',
      likeScore: '0',
      individualLikes: [],
      id: '9082',
      pageId: 'avert_instrumental_pressure',
      userId: 'EliezerYudkowsky',
      edit: '0',
      type: 'deleteParent',
      createdAt: '2016-03-26 19:47:48',
      auxPageId: 'task_agi',
      oldSettingsValue: '',
      newSettingsValue: ''
    }
  ],
  feedSubmissions: [],
  searchStrings: {},
  hasChildren: 'false',
  hasParents: 'true',
  redAliases: {},
  improvementTagIds: [],
  nonMetaTagIds: [],
  todos: [],
  slowDownMap: 'null',
  speedUpMap: 'null',
  arcPageIds: 'null',
  contentRequests: {}
}