This page is an experiment. It may violate some conventions. I'm sorry.
(Our dithering between the authorial "I" and the authorial "we" is meaningful. Do not dismiss it or become inured to it.)
Self-Help for Animate Matter
If you are reading this page and it has the potential to influence your actions, then you are animate matter that has not reduced itself to an optimization process. This is good, maybe.
We can make the very weak assumption that you are human, or human-like enough (tulpa, Homo sapiens with unusual brain organization/damage/thal-ishness, etc.), but why should this matter? How do I know I'm not giving you the rope you'll need to hang us all?
A thread running through this entire exploration is the question of whether rationality (or the collection of rationality-adjacent concepts) is safe and ought to be propagated. So if you are reading this, we have evidence that you should be reading this. We just haven't exposed you to it, yet.
This is an idiosyncratic organization of rationality materials. Rationality is here understood as optimization: systematically moving toward a goal. Finding a desired option in a large space of options. Optimization in this sense is strongly entangled with the notion of a selection effect. A major goal here is to elucidate that connection. Another goal is to make the principles of rationality seem as basic and necessary as possible. Following the idea of practical advice backed by deep theories, we wish to make rationality sound like a branch of physics.
The reason for this is to dissolve mistakes. Our business here is largely to reduce cognitive bias. It is one thing to helpfully point out a mistake. It is quite another to dissolve the mistake. If you see someone making a mistake but you don't know why they are making it, you should be unsettled. It is all too easy to approach a problem with a simplified view and shout "Bias!" when the decisions of others differ from what you compute to be optimal. One should ask: "is there a reason for their action which I've not yet determined?" Let others try to give helpful advice; our business here is the more careful course.
Therefore, before any cognitive bias is introduced, the deep theory should be in place to explain why it occurs. Following this, practical advice for removing the bias from oneself should be given. However, by this point the advice should seem almost obvious based on the deep theory. What remains is only experimental validation that the advice works.
Above all, write what is needed. We write the above as a way of defining the highest standard we can so far conceive for a text on rationality. We will not always live up to this standard. To give good advice is still better than not to give it. To explain why something is a mistake without being able to explain why the mistake is made still offers a path to improvement.
The following outline is a plan of pages, with descriptions of planned content in some cases. Some bullets may stand for messy sets of pages, IE, portions of the outline to be filled in. Feel free to edit without worrying about stepping on feet.
- Being Strategic: The Very Idea
- Noticing Preferences
- Discuss the basic idea of noticing what you want and trying to get it. Discuss the research on value affirmation.
- Trying Things
- Optimization requires feedback. Discuss trial-and-error optimization processes, avoiding failures of the optimization feedback loop, seeking improvements by optimizing the feedback loop, optimization processes as a special case of selection processes.
- Trial and Error
- Exploration vs Exploitation
- Information and Control
- stuff about evolution which I need to research more
- There is work connecting thermodynamics and evolution/ecology which seems like it would fit well here. The work starts with work by Schrodinger, titled What is Life. Among other things, the book introduced the idea that life persists by negentropy. More recent work by Jeremy England pursues this line of thinking in a mathematical model of evolution. It's possible that some or all of this is nonsense, but if it turns out to be good, it seems like an interesting way to ground things in general principles, and also an excuse to talk about some information theory concepts.
- Every Good Key Must Be a Model of the Lock It Opens
- Another possibly interesting mathematical framework, which says something about why control systems need to accurately model the world if they do anything very complex.
- Truth: The Very Idea
- Discussion of basic epistemic biases??
- Selection Effects
- Selection bias; Bayesian solution
- Cherry-picking evidence and other sins
- (a bunch of biases can be fit in here)
- Availability Heuristic
- Probability Neglect
- Base-Rate Neglect
- Normalcy Bias
- Frequency Illusion
- Mere Exposure Effect, Illusory Truth Effect
- Information Cascade, Availability Cascade
- Inspection Paradox
- (lots more specific biases can fit under this one too)
- Tails Come Apart
- Optimizer's Curse
- Simultaneous Underestimation and Overestimation
- Principle-Agent Problem
- Lost Purposes