# By Creator

## Aaron Tucker

### wiki

• Combining vectors One of the most useful things we can do with vectors is to combine them!
• Elementary Algebra How do we describe relations between different things? How can we figure out new true things from tr…
• Geometric algebra A geometric algebra is a Clifford algebra over the reals which represents Euclidean geometry and man…
• Geometric product #Motivation want to incorporate rotors like $e^{\text{I}\theta}$ and scalars $n$ in the same system …
• Geometric product: summary Product for [multivectors multivectors]. Associative, left and right distributive. Non-commutative. …
• Geometry of vectors: direction What rotation would it take to line up this vector to this one?
• Group orbit When we have a group acting on a set, we are often interested in how the group acts on a particular …
• Locale Topology - but right
• Quotient group Given a group $G$ with operation $\bullet$ and a special kind of subgroup $N \leq G$ called the "no…
• Vector arithmetic Vectors: what they are, and how to add and scale them.

## Aeneas Mackenzie

• Big-O Notation This notation describes asymptotic behavior of functions. # O(x) A function f is O(g(x)) if, for la…

## Bryce Woodworth

### wiki

• Metric A metric is a function that defines a distance between elements in a set and follows some basic rules.

## Carolina Villegas

• Existential risk one where an adverse outcome would either **annihilate Earth-originating intelligent life** or perma…

## Daniel Satanove

### wiki

• Concrete groups (Draft) Instead of thinking of a group as a set with operations satisfying axoims, we develop groups as symmetry groups of various objects
• Groups as symmetires A group is an abstraction of a collection of symmetries of an object. Examples of groups include th…
• Power set The power set is the collection of all subsets of a set

## Dmitriy Zhukov

• Harem A group of women loving their superman.

## Dylan Hendrickson

### wiki

• 0.999...=1 No, it's not "infinitesimally far" from 1 or anything like that. 0.999... and 1 are literally the same number.
• Church encoding How can you represent things like numbers as lambda expressions?
• Equivalence relation A relation that allows you to partition a set into equivalence classes.
• Lambda calculus A minimal, inefficient, and hard-to-read, but still interesting and useful, programming language.
• Math style guidelines Stylistic conventions specific to pages about math.
• Ordered ring A ring with a total ordering compatible with its ring structure.
• Rice's Theorem: Intro (Math 1) You can't write a program that looks at another programs source code, and tells you whether it computes the Fibonacci sequence.
• Subgroup A group that lives inside a bigger group.
• The square root of 2 is irrational The number whose square is 2 can't be written is a quotient of natural numbers
• Transitive relation If a is related to b and b is related to c, then a is related to c.
• Well-ordered set An ordered set with an order that always has a "next element".

## Eliana Ruby

• Complex number A complex number is a number of the form $z = a + b\textrm{i}$, where $\textrm{i}$ is the imaginary …

## Eliezer Yudkowsky

### wiki

• 'Beneficial' Really actually good. A metasyntactic variable to mean "favoring whatever the speaker wants ideally to accomplish", although different speakers have different morals and metaethics.
• 'Concept' In the context of Artificial Intelligence, a 'concept' is a category, something that identifies thingies as being inside or outside the concept.
• 'Detrimental' The opposite of beneficial.
• 'Rationality' of voting in elections "A single vote is very unlikely to swing the election, so your vote is unlikely to have an effect" versus "Many people similar to you are making a similar decision about whether to vote."
• 99LDT x 1CDT oneshot PD tournament as arguable counterexample to LDT doing better than CDT Arguendo, if 99 LDT agents and 1 CDT agent are facing off in a one-shot Prisoner's Dilemma tournament, the CDT agent does better on a problem that CDT considers 'fair'.
• A quick econ FAQ for AI/ML folks concerned about technological unemployment Yudkowsky's attempted description of standard economic concepts that he thinks are vital for talking about technological unemployment and related issues.
• A reply to Francois Chollet on intelligence explosion A quick run-through of what I'd consider the standard replies to the arguments in Keras inventor Francois Chollet's essay "The impossibility of intelligence explosion".
• AI alignment The great civilizational problem of creating artificially intelligent computer systems such that running them is a good idea.
• AI alignment open problem Tag for open problems under AI alignment.
• AI arms races AI arms races are bad
• AI safety mindset Asking how AI designs could go wrong, instead of imagining them going right.
• AIXI How to build an (evil) superintelligent AI using unlimited computing power and one page of Python code.
• AIXI-tl A time-bounded version of the ideal agent AIXI that uses an impossibly large finite computer instead of a hypercomputer.
• Ability to read algebra Do you have sufficient mathematical ability that you can read a sentence that uses some algebra or invokes a mathematical idea, without slowing down too much?
• Ability to read calculus Can you take integral signs and differentiations in stride?
• Ability to read logic Can you read sentences symbolically stating "For all x: exists y: phi(x, y) or not theta(y)" without slowing down too much?
• Abortable plans Plans that can be undone, or switched to having low further impact. If the AI builds abortable nanomachines, they'll have a quiet self-destruct option that includes any replicated nanomachines.
• Absent-Minded Driver dilemma A road contains two identical intersections. An absent-minded driver wants to turn right at the second intersection. "With what probability should the driver turn right?" argue decision theorists.
• Actual effectiveness If you want the AI's so-called 'utility function' to actually be steering the AI, you need to think about how it meshes up with beliefs, or what gets output to actions.
• Ad-hoc hack (alignment theory) A "hack" is when you alter the behavior of your AI in a way that defies, or doesn't correspond to, a principled approach for that problem.
• Advanced agent properties How smart does a machine intelligence need to be, for its niceness to become an issue? "Advanced" is a broad term to cover cognitive abilities such that we'd need to start considering AI alignment.
• Advanced nonagent Hypothetically, cognitively powerful programs that don't follow the loop of "observe, learn, model the consequences, act, observe results" that a standard "agent" would.
• Advanced safety An agent is *really* safe when it has the capacity to do anything, but chooses to do what the programmer wants.
• Algorithmic complexity When you compress the information, what you are left with determines the complexity.
• Aligning an AGI adds significant development time Aligning an advanced AI foreseeably involves extra code and extra testing and not being able to do everything the fastest way, so it takes longer.
• Almost all real-world domains are rich Anything you're trying to accomplish in the real world can potentially be accomplished in a *lot* of different ways.
• An Introduction to Logical Decision Theory for Everyone Else So like what the heck is 'logical decision theory' in terms a normal person can understand?
• Answer to sparking widgets problem Odds of 1 : 3, probability of 1/4.
• Arbital biographies As a very strong default (presently an absolute rule), Joe Smith's page only says nice things about Joe. Even if a negative fact is true, it doesn't go on Joe's page.
• Arbital playpen Want to test a feature? Feel free to edit this page! asdfasfdasfda
• Arbital practices Guidelines and rules for interacting on Arbital.
• Arbital: Solving online explanations An explanation of Arbital's mid-term goals
• Arithmetical hierarchy The arithmetical hierarchy is a way of classifying logical statements by the number of clauses saying "for every object" and "there exists an object".
• Arithmetical hierarchy: If you don't read logic The arithmetical hierarchy is a way of stratifying statements by how many "for every number" and "th…
• Artificial General Intelligence An AI which has the same kind of "significantly more general" intelligence that humans have compared to chimpanzees; it can learn new domains, like we can.
• Attainable optimum The 'attainable optimum' of an agent's preferences is the best that agent can actually do given its finite intelligence and resources (as opposed to the global maximum of those preferences).
• Autonomous AGI The hardest possible class of Friendly AI to build, with the least moral hazard; an AI intended to neither require nor accept further direction.
• Averting instrumental pressures Almost-any utility function for an AI, whether the target is diamonds or paperclips or eudaimonia, implies subgoals like rapidly self-improving and refusing to shut down. Can we make that not happen?
• Averting the convergent instrumental strategy of self-improvement We probably want the first AGI to *not* improve as fast as possible, but improving as fast as possible is a convergent strategy for accomplishing most things.
• Bayes' rule Bayes' rule is the core theorem of probability theory saying how to revise our beliefs when we make a new observation.
• Bayes' rule examples Interesting problems solvable by Bayes' rule
• Bayes' rule: Functional form Bayes' rule for to continuous variables.
• Bayes' rule: Guide The Arbital guide to Bayes' rule
• Bayes' rule: Log-odds form A simple transformation of Bayes' rule reveals tools for measuring degree of belief, and strength of evidence.
• Bayes' rule: Odds form The simplest and most easily understandable form of Bayes' rule uses relative odds.
• Bayes' rule: Odds form (Intro, Math 1) Introduction to the odds form of Bayes' rule
• Bayes' rule: Odds form (Intro, Probability) Intro to Bayes' rule, odds form, for people already familiar with probability.
• Bayes' rule: Proportional form The fastest way to say something both convincing and true about belief-updating.
• Bayes' rule: Vector form For when you want to apply Bayes' rule to lots of evidence and lots of variables, all in one go. (This is more or less how spam filters work.)
• Bayesian reasoning A probability-theory-based view of the world; a coherent way of changing probabilistic beliefs based on evidence.
• Bayesian update Bayesian updating: the ideal way to change probabilistic beliefs based on evidence.
• Bayesian view of scientific virtues Why is it that science relies on bold, precise, and falsifiable predictions? Because of Bayes' rule, of course.
• Behaviorist genie An advanced agent that's forbidden to model minds in too much detail.
• Belief revision as probability elimination Update your beliefs by throwing away large chunks of probability mass.
• Big-picture strategic awareness We start encountering new AI alignment issues at the point where a machine intelligence recognizes the existence of a real world, the existence of programmers, and how these relate to its goals.
• Bounded agent An agent that operates in the real world, using realistic amounts of computing power, that is uncertain of its environment, etcetera.
• Boxed AI Idea: what if we limit how AI can interact with the world. That'll make it safe, right??
• Bulverism Bulverism is when you explain what goes so horribly wrong in people's minds when they believe X, before you've actually explained why X is wrong. Forbidden on Arbital.
• Calories-In-Calories-Out CICO is a proposed conceptual decomposition of the causes of changes in human body mass, particularl…
• Cartesian agent Agents separated from their environments by impermeable barriers through which only sensory information can enter and motor output can exit.
• Cartesian agent-environment boundary If your agent is separated from the environment by an absolute border that can only be crossed by sensory information and motor outputs, it might just be a Cartesian agent.
• Causal decision theories On CDT, to choose rationally, you should imagine the world where your physical act changes, then imagine running that world forward in time. (Therefore, it's irrational to vote in elections.)
• Central examples The "central examples" for a subject are examples that are referred to over and over again in the co…
• Central examples List of central examples in Value Alignment Theory domain.
• Cognitive domain An allegedly compact unit of knowledge, such that ideas inside the unit interact mainly with each other and less with ideas in other domains.
• Cognitive steganography Disaligned AIs that are modeling human psychology and trying to deceive their programmers will want to hide their internal thought processes from their programmers.
• Cognitive uncontainability 'Cognitive uncontainability' is when we can't hold all of an agent's possibilities inside our own minds.
• Coherence theorems A 'coherence theorem' shows that something bad happens to an agent if its decisions can't be viewed as 'coherent' in some sense. E.g., an inconsistent preference ordering leads to going in circles.
• Coherent decisions imply consistent utilities Why do we all use the 'expected utility' formalism? Because any behavior that can't be viewed from that perspective, must be qualitatively self-defeating (in various mathy ways).
• Coherent extrapolated volition (alignment target) A proposed direction for an extremely well-aligned autonomous superintelligence - do what humans would want, if we knew what the AI knew, thought that fast, and understood ourselves.
• Complexity of value There's no simple way to describe the goals we want Artificial Intelligences to want.
• Conceivability A hypothetical scenario is 'conceivable' or 'imaginable' when it is not *immediately* incoherent, al…
• Conditional probability The notation for writing "The probability that someone has green eyes, if we know that they have red hair."
• Consequentialist cognition The cognitive ability to foresee the consequences of actions, prefer some outcomes to others, and output actions leading to the preferred outcomes.
• Consequentialist preferences are reflectively stable by default Gandhi wouldn't take a pill that made him want to kill people, because he knows in that case more people will be murdered. A paperclip maximizer doesn't want to stop maximizing paperclips.
• Conservative concept boundary Given N example burritos, draw a boundary around what is a 'burrito' that is relatively simple and allows as few positive instances as possible. Helps make sure the next thing generated is a burrito.
• Context disaster Some possible designs cause your AI to behave nicely while developing, and behave a lot less nicely when it's smarter.
• Convergent instrumental strategies Paperclip maximizers can make more paperclips by improving their cognitive abilities or controlling more resources. What other strategies would almost-any AI try to use?
• Convergent strategies of self-modification The strategies we'd expect to be employed by an AI that understands the relevance of its code and hardware to achieving its goals, which therefore has subgoals about its code and hardware.
• Coordinative AI development hypothetical What would safe AI development look like if we didn't have to worry about anything else?
• Corporations vs. superintelligences Corporations have relatively few of the advanced-agent properties that would allow one mistake in aligning a corporation to immediately kill all humans and turn the future light cone into paperclips.
• Correlated competency When an AI achieving sufficiently high goodness on behavior A means we should strongly expect high goodness on behavior B.
• Correlated coverage In which parts of AI alignment can we hope that getting many things right, will mean the AI gets everything right?
• Cosmic endowment The 'cosmic endowment' consists of all the stars that could be reached from probes originating on Earth; the sum of all matter and energy potentially available to be transformed into life and fun.
• Cosmopolitan value Intuitively: Value as seen from a broad, embracing standpoint that is aware of how other entities may not always be like us or easily understandable to us, yet still worthwhile.
• Death in Damascus Death tells you that It is coming for you tomorrow. You can stay in Damascus or flee to Aleppo. Whichever decision you actually make is the wrong one. This gives some decision theories trouble.
• Decision theory The mathematical study of ideal decisionmaking
• Deep Blue The chess-playing program, built by IBM, that first won the world chess championship from Garry Kasparov in 1996.
• Definition Meta tag used to mark pages that strictly define a particular term or phrase.
• Descriptive versus normative propositions A normative proposition talks about what should be; a descriptive proposition talks about what is.
• Development phase unpredictable Several proposed problems in advanced safety are alleged to be difficult because they depend on some…
• Diamond maximizer How would you build an agent that made as much diamond material as possible, given vast computing power but an otherwise rich and complicated environment?
• Difficulty of AI alignment How hard is it exactly to point an Artificial General Intelligence in an intuitively okay direction?
• Directing, vs. limiting, vs. opposing Getting the AI to compute the right action in a domain; versus getting the AI to not compute at all in an unsafe domain; versus trying to prevent the AI from acting successfully. (Prefer 1 & 2.)
• Diseasitis 20% of patients have Diseasitis. 90% of sick patients and 30% of healthy patients turn a tongue depressor black. You turn a tongue depressor black. What's the chance you have Diseasitis?
• Distances between cognitive domains Often in AI alignment we want to ask, "How close is 'being able to do X' to 'being able to do Y'?"
• Distant superintelligences can coerce the most probable environment of your AI Distant superintelligences may be able to hack your local AI, if your AI's preference framework depends on its most probable environment.
• Distinguish which advanced-agent properties lead to the foreseeable difficulty Say what kind of AI, or threshold level of intelligence, or key type of advancement, first produces the difficulty or challenge you're talking about.
• Do-What-I-Mean hierarchy Successive levels of "Do What I Mean" or AGIs that understand their users increasingly well
• Don't try to solve the entire alignment problem New to AI alignment theory? Want to work in this area? Already been working in it for years? Don't try to solve the entire alignment problem with your next good idea!
• Edge instantiation When you ask the AI to make people happy, and it tiles the universe with the smallest objects that can be happy.
• Effability principle You are safer the more you understand the inner structure of how your AI thinks; the better you can describe the relation of smaller pieces of the AI's thought process.
• Emphemeral premises When somebody says X, don't just say, "Oh, not-X because Y" and then forget about Y a day later. Y is now an important load-bearing assumption in your worldview. Write Y down somewhere.
• Empirical probabilities are not exactly 0 or 1 "Cromwell's Rule" says that probabilities of exactly 0 or 1 should never be applied to empirical propositions - there's always some probability, however tiny, of being mistaken.
• Environmental goals The problem of having an AI want outcomes that are out in the world, not just want direct sense events.
• Epistemic and instrumental efficiency An efficient agent never makes a mistake you can predict. You can never successfully predict a directional bias in its estimates.
• Epistemic exclusion How would you build an AI that, no matter what else it learned about the world, never knew or wanted to know what was inside your basement?
• Epistemology What is truth?
• Evidential decision theories Theories which hold that the principle of rational choice is "Choose the act that would be the best news, if somebody told you that you'd chosen that act."
• Executable philosophy Philosophical discourse aimed at producing a trustworthy answer or meta-answer, in limited time, which can used in constructing an Artificial Intelligence.
• Expected utility Scoring actions based on the average score of their probable consequences.
• Expected utility agent If you're not some kind of expected utility agent, you're going in circles.
• Expected utility formalism Expected utility is the central idea in the quantitative implementation of consequentialism
• Explicit Bayes as a counter for 'worrying' Explicitly walking through Bayes's Rule can summarize your knowledge and thereby stop you from bouncing around pieces of it.
• Extraordinary claims What makes something an 'extraordinary claim' that requires extraordinary evidence?
• Extraordinary claims require extraordinary evidence The people who adamantly claim they were abducted by aliens do provide some evidence for aliens. They just don't provide quantitatively enough evidence.
• Extrapolated volition (normative moral theory) If someone asks you for orange juice, and you know that the refrigerator contains no orange juice, should you bring them lemonade?
• Fair problem class A problem is 'fair' (according to logical decision theory) when only the results matter and not how we get there.
• Faithful simulation How would you identify, to a Task AGI (aka Genie), the problem of scanning a human brain, and then running a sufficiently accurate simulation of it for the simulation to not be crazy or psychotic?
• Fallacies To call something a fallacy is to assert that you think people shouldn't think like that.
• Finishing your Bayesian path on Arbital The page that comes at the end of reading the Arbital Guide to Bayes' rule
• Flag the load-bearing premises If somebody says, "This AI safety plan is going to fail, because X" and you reply, "Oh, that's fine because of Y and Z", then you'd better clearly flag Y and Z as "load-bearing" parts of your plan.
• Friendly AI Old terminology for an AI whose preferences have been successfully aligned with idealized human values.
• General intelligence Compared to chimpanzees, humans seem to be able to learn a much wider variety of domains. We have 'significantly more generally applicable' cognitive abilities, aka 'more general intelligence'.
• Generalized principle of cognitive alignment When we're asking how we want the AI to think about an alignment problem, one source of inspiration is trying to have the AI mirror our own thoughts about that problem.
• Glossary (Value Alignment Theory) Words that have a special meaning in the context of creating nice AIs.
• Goal-concept identification Figuring out how to say "strawberry" to an AI that you want to bring you strawberries (and not fake plastic strawberries, either).
• Goodhart's Curse The Optimizer's Curse meets Goodhart's Law. For example, if our values are V, and an AI's utility function U is a proxy for V, optimizing for high U seeks out 'errors'--that is, high values of U - V.
• Goodness estimate biaser Some of the main problems in AI alignment can be seen as scenarios where actual goodness is likely to be systematically lower than a broken way of estimating goodness.
• Gotcha button A conversational point which, when pressed, causes the other person to shout "Gotcha!" and leap on what they think is a weakness allowing them to dismiss the conversation.
• Guarded definition A guarded definition is one where at least one position suspects there will be pressure to stretch a…
• Guide to Logical Decision Theory The entry point for learning about logical decision theory.
• Happiness maximizer It is sometimes proposed that we build an AI intended to maximize human happiness. (One early propo…
• Hard problem of corrigibility Can you build an agent that reasons as if it knows itself to be incomplete and sympathizes with your wanting to rebuild or correct it?
• Harmless supernova fallacy False dichotomies and continuum fallacies which can be used to argue that anything, including a supernova, must be harmless.
• High-speed intro to Bayes's rule A high-speed introduction to Bayes's Rule on one page, for the impatient and mathematically adept.
• How to author on Arbital! Want to contribute pages to Arbital? Here's our current version of the ad-hoc guide to being an author!
• How to build your own Lumenator Treating Seasonal Affective Disorder using MOAR LIGHT can sometimes solve what dinky little lightboxes can't.
• Humans doing Bayes The human use of Bayesian reasoning in everyday life
• Humean degree of freedom A concept includes 'Humean degrees of freedom' when the intuitive borders of the human version of that concept depend on our values, making that concept less natural for AIs to learn.
• Hypercomputer Some formalisms demand computers larger than the limit of all finite computers
• Ideal target The 'ideal target' of a meta-utility function is the value the ground-level utility function would take on if the agent updated on all possible evidence; the 'true' utilities under moral uncertainty.
• Identifying ambiguous inductions What do a "red strawberry", a "red apple", and a "red cherry" have in common that a "yellow carrot" doesn't? Are they "red fruits" or "red objects"?
• Identifying causal goal concepts from sensory data If the intended goal is "cure cancer" and you show the AI healthy patients, it sees, say, a pattern of pixels on a webcam. How do you get to a goal concept *about* the real patients?
• Ideological Turing test Can you explain the opposing position well enough that people can't tell whether you or a real advocate of that position created the explanation?
• Ignorance prior Key equations for quantitative Bayesian problems, describing exactly the right shape for what we believed before observation.
• Imitation-based agent An AI meant to imitate the behavior of a reference human as closely as possible.
• Immediate goods One of the potential views on 'value' in the value alignment problem is that what we should want fro…
• Inductive prior Some states of pre-observation belief can learn quickly; others never learn anything. An "inductive prior" is of the former type.
• Infrahuman, par-human, superhuman, efficient, optimal A categorization of AI ability levels relative to human, with some gotchas in the ordering. E.g., in simple domains where humans can play optimally, optimal play is not superhuman.
• Instrumental What is "instrumental" in the context of Value Alignment Theory?
• Instrumental convergence Some strategies can help achieve most possible simple goals. E.g., acquiring more computing power or more material resources. By default, unless averted, we can expect advanced AIs to do that.
• Instrumental goals are almost-equally as tractable as terminal goals Getting the milk from the refrigerator because you want to drink it, is not vastly harder than getting the milk from the refrigerator because you inherently desire it.
• Instrumental pressure A consequentialist agent will want to bring about certain instrumental events that will help to fulfill its goals.
• Intelligence explosion What happens if a self-improving AI gets to the point where each amount x of self-improvement triggers >x further self-improvement, and it stays that way for a while.
• Intended goal Definition. An "intended goal" refers to the intuitive intention in the mind of a human programmer …
• Intension vs. extension "Red is a light with a wavelength of 700 nm" vs. "Look at this red apple, red car, and red cup."
• Interest in mathematical foundations in Bayesianism "Want" this requisite if you prefer to see extra information about the mathematical foundations in Bayesianism.
• Interruptibility A subproblem of corrigibility under the machine learning paradigm: when the agent is interrupted, it must not learn to prevent future interruptions.
• Introduction to Bayes' rule: Odds form Bayes' rule is simple, if you think in terms of relative odds.
• Introduction to Logical Decision Theory for Analytic Philosophers Why "choose as if controlling the logical output of your decision algorithm" is the most appealing candidate for the principle of rational choice.
• Introduction to Logical Decision Theory for Computer Scientists 'Logical decision theory' from a math/programming standpoint, including how two agents with mutual knowledge of each other's code can cooperate on the Prisoner's Dilemma.
• Introduction to Logical Decision Theory for Economists An introduction to 'logical decision theory' and its implications for the Ultimatum Game, voting in elections, bargaining problems, and more.
• Introductory Bayesian problems Bayesian problems to try to solve yourself, before beginning to learn about Bayes' rule.
• Intution pump In philosophy, a metaphor or visualization used to shove the listener's intuition in a particular direction.
• Invisible background fallacies Universal laws also apply to objects and ideas that may fade into the invisible background. Reasoning as if these laws didn't apply to less obtrusive concepts is a type of fallacy.
• Joint probability The notation for writing the chance that both X and Y are true.
• Just a requisite A tag for nodes that just act as part of Arbital's requisite system
• Known-algorithm non-self-improving agent Possible advanced AIs that aren't self-modifying, aren't self-improving, and where we know and understand all the component algorithms.
• Laplace's Rule of Succession Suppose you flip a coin with an unknown bias 30 times, and see 4 heads and 26 tails. The Rule of Succession says the next flip has a 5/32 chance of showing heads.
• Likelihood functions, p-values, and the replication crisis What's the whole Bayesian-vs.-frequentist debate about?
• Limited AGI Task-based AGIs don't need unlimited cognitive and material powers to carry out their Tasks; which means their powers can potentially be limited.
• Linguistic conventions in value alignment How and why to use precise language and words with special meaning when talking about value alignment.
• Link glossary pages for overloaded words If your subject is using what sound like ordinary-language words in a special sense, create a glossa…
• List of Eliezer's current most desired fixes and features A place for Eliezer to note down his current list of personally-wanted features for editing and writing.
• List: value-alignment subjects Bullet point list of core VAT subjects.
• Logical decision theories Root page for topics on logical decision theory, with multiple intros for different audiences.
• Logical game Game's mathematical structure at its purest form.
• Look where I'm pointing, not at my finger When trying to communicate the concept "glove", getting the AGI to focus on "gloves" rather than "my user's decision to label something a glove" or "anything that depresses the glove-labeling button".
• Low impact The open problem of having an AI carry out tasks in ways that cause minimum side effects and change as little of the rest of the universe as possible.
• Math 0 Are you not actively bad at math, nor traumatized about math?
• Math 1 Is math sometimes fun for you, and are you not anxious if you see a math puzzle you don't know how to solve?
• Math 2 Do you work with math on a fairly routine basis? Do you have little trouble grasping abstract structures and ideas?
• Math 3 Can you read the sort of things that professional mathematicians read, aka LaTeX formulas with a minimum of explanation?
• Mathematics Mathematics is the study of numbers and other ideal objects that can be described by axioms.
• Mechanical Turk (example) The 19th-century chess-playing automaton known as the Mechanical Turk actually had a human operator inside. People at the time had interesting thoughts about the possibility of mechanical chess.
• Meta tags What are meta tags and when to use them?
• Meta-rules for (narrow) value learning are still unsolved We don't currently know a simple meta-utility function that would take in observation of humans and spit out our true values, or even a good target for a Task AGI.
• Meta-utility function Preference frameworks built out of simple utility functions, but where, e.g., the 'correct' utility function for a possible world depends on whether a button is pressed.
• Metaethics Metaethics asks "What kind of stuff is goodness made of?" (or "How would we compute goodness?") rather than "Which particular policies or outcomes are good or not-good?"
• Methodology of foreseeable difficulties Building a nice AI is likely to be hard enough, and contain enough gotchas that won't show up in the AI's early days, that we need to foresee problems coming in advance.
• Methodology of unbounded analysis What we do and don't understand how to do, using unlimited computing power, is a critical distinction and important frontier.
• Mild optimization An AGI which, if you ask it to paint one car pink, just paints one car pink and doesn't tile the universe with pink-painted cars, because it's not trying *that* hard to max out its car-painting score.
• Mind design space is wide Imagine all human beings as one tiny dot inside a much vaster sphere of possibilities for "The space of minds in general." It is wiser to make claims about *some* minds than *all* minds.
• Mind projection fallacy Uncertainty is in the mind, not in the environment; a blank map does not correspond to a blank territory. In general, the territory may have a different ontology from the map.
• Mindcrime Might a machine intelligence contain vast numbers of unhappy conscious subprocesses?
• Mindcrime: Introduction The more predictive accuracy we want from a model, the more detailed the model becomes. A very roug…
• Minimality principle The first AGI ever built should save the world in a way that requires the least amount of the least dangerous cognition.
• Missing the weird alternative People might systematically overlook "make tiny molecular smileyfaces" as a way of "producing smiles", because our brains automatically search for high-utility-to-us ways of "producing smiles".
• Modeling distant superintelligences The several large problems that might occur if an AI starts to think about alien superintelligences.
• Moral hazards in AGI development "Moral hazard" is when owners of an advanced AGI give in to the temptation to do things with it that the rest of us would regard as 'bad', like, say, declaring themselves God-Emperor.
• Moral uncertainty A meta-utility function in which the utility function as usually considered, takes on different values in different possible worlds, potentially distinguishable by evidence.
• Most complex things are not very compressible We can't *prove* it's impossible, but it would be *extremely surprising* to discover a 500-state Turing machine that output the exact text of "Romeo and Juliet".
• Multiple stage fallacy You can make an arbitrary proposition sound very improbable by observing how it seemingly requires X, Y, and Z. This didn't work for Nate Silver forecasting the Trump nomination.
• Mutually exclusive and exhaustive The condition needed for probabilities to sum to 1
• NGDP level targeting Central banks ought to regularize the total flow of money to increase at a predictable 5% rate per year, and doing this would solve a surprising number of other problems.
• Natural language understanding of "right" will yield normativity What will happen if you tell an advanced agent to do the "right" thing?
• Nearest unblocked strategy If you patch an agent's preference framework to avoid an undesirable solution, what can you expect to happen?
• Newcomb's Problem There are two boxes in front of you, Box A and Box B. You can take both boxes, or only Box B. Box A contains $1000. Box B contains$1,000,000 if and only if Omega predicted you'd take only Box B.
• Newcomblike decision problems Decision problems in which your choice correlates with something other than its physical consequences (say, because somebody has predicted you very well) can do weird things to some decision theories.
• Niceness is the first line of defense The *first* line of defense in dealing with any partially superhuman AI system advanced enough to possibly be dangerous is that it does not *want* to hurt you or defeat your safety measures.
• Nick Bostrom Nick Bostrom, secretly the inventor of Friendly AI
• Nick Bostrom's book Superintelligence The current best book-form introduction to AI alignment theory.
• No-Free-Lunch theorems are often irrelevant There's often a theorem proving that some problem has no optimal answer across every possible world. But this may not matter, since the real world is a special case. (E.g., a low-entropy universe.)
• Non-adversarial principle At no point in constructing an Artificial General Intelligence should we construct a computation that tries to hurt us, and then try to stop it from hurting us.
• Nonperson predicate If we knew which computations were definitely not people, we could tell AIs which programs they were definitely allowed to compute.
• Normalization (probability) That thingy we do to make sure our probabilities sum to 1, when they should sum to 1.
• Object-level vs. indirect goals Difference between "give Alice the apple" and "give Alice what she wants".
• Odds Odds express a relative probability.
• Omega (alien philosopher-troll) The entity that sets up all those trolley problems. An alien philosopher/troll imbued with unlimited powers, excellent predictive ability, and very odd motives.
• Omnipotence test for AI safety Would your AI produce disastrous outcomes if it suddenly gained omnipotence and omniscience? If so, why did you program something that *wants* to hurt you and is held back only by lacking the power?
• Ontology identification problem How do we link an agent's utility function to its model of the world, when we don't know what that model will look like?
• Ontology identification problem: Technical tutorial Technical tutorial for ontology identification problem.
• Open subproblems in aligning a Task-based AGI Open research problems, especially ones we can model today, in building an AGI that can "paint all cars pink" without turning its future light cone into pink-painted cars.
• Optimization daemons When you optimize something so hard that it crystalizes into an optimizer, like the way natural selection optimized apes so hard they turned into human-level intelligences
• Oracle System designed to safely answer questions.
• Orthogonality Thesis Will smart AIs automatically become benevolent, or automatically become hostile? Or do different AI designs imply different goals?
• Other-izing (wanted: new optimization idiom) Maximization isn't possible for bounded agents, and satisficing doesn't seem like enough. What other kind of 'izing' might be good for realistic, bounded agents?
• Paperclip A configuration of matter that we'd see as being worthless even from a very cosmopolitan perspective.
• Paperclip maximizer This agent will not stop until the entire universe is filled with paperclips.
• Parfit's Hitchhiker You are dying in the desert. A truck-driver who is very good at reading faces finds you, and offers to drive you into the city if you promise to pay 1,000 on arrival. You are a selfish rationalist. • Patch resistance One does not simply solve the value alignment problem. • Path targeting Don't say "We want this price to go up at 2%/year", say "We want this to be1 in year 1, $1.02 in year 2,$1.04 in year 3" and don't change the rest of the path if you miss one year's target.
• Path: Insights from Bayesian updating A learning-path placeholder page for insights derived from the Bayesian rule for updating beliefs.
• Path: Multiple angles on Bayes's Rule A learning-path placeholder page for learning multiple angles on Bayes's Rule.
• People A category for human beings!
• Perfect rolling sphere If you don't understand something, start by assuming it's a perfect rolling sphere.
• Philosophy A stub parent node to contain standard concepts, belonging to subfields of academic philosophy, that are being used elsewhere on Arbital.
• Pivotal event Which types of AIs, if they work, can do things that drastically change the nature of the further game?
• Posterior probability What we believe, after seeing the evidence and doing a Bayesian update.
• Preference framework What's the thing an agent uses to compare its preferences?
• Principles in AI alignment A 'principle' of AI alignment is a very general design goal like 'understand what the heck is going on inside the AI' that has informed a wide set of specific design proposals.
• Prior A state of prior knowledge, before seeing information on a new problem. Potentially complicated.
• Prior probability What we believed before seeing the evidence.
• Prisoner's Dilemma You and an accomplice have been arrested. Both of you must decide, in isolation, whether to testify against the other prisoner--which subtracts one year from your sentence, and adds two to theirs.
• Probability The degree to which someone believes something, measured on a scale from 0 to 1, allowing us to do math to it.
• Probability notation for Bayes' rule The probability notation used in Bayesian reasoning
• Probability notation for Bayes' rule: Intro (Math 1) How to read, and identify, the probabilities in Bayesian problems.
• Probability theory The logic of science; coherence relations on quantitative degrees of belief.
• Problem of fully updated deference Why moral uncertainty doesn't stop an AI from defending its off-switch.
• Programmer Who is building these advanced agents?
• Programmer deception Programmer deception is when the AI's decision process leads it to optimize for an instrumental goal…
• Proof of Bayes' rule Proofs of Bayes' rule, with graphics
• Proof of Bayes' rule: Intro Proof of Bayes' rule, assuming you know the rule itself, and the notations for the quantities involved.
• Proving too much If your argument could just as naturally be used to prove that Bigfoot exists and that Peano arithmetic is inconsistent, maybe it's an untrustworthy kind of argument.
• Psychologizing It's sometimes important to consider how other people might be led into error. But psychoanalyzing them is also dangerous! Arbital discussion norms say to explicitly note this as "psychologizing".
• Querying the AGI user Postulating that an advanced agent will check something with its user, probably comes with some standard issues and gotchas (e.g., prioritizing what to query, not manipulating the user, etc etc).
• Random utility function A 'random' utility function is one chosen at random according to some simple probability measure (e.g. weight by Kolmorogov complexity) on a logical space of formal utility functions.
• Rationality The subject domain for [ epistemic] and [ instrumental] rationality.
• Real-world domain Some AIs play chess, some AIs play Go, some AIs drive cars. These different 'domains' present different options. All of reality, in all its messy entanglement, is the 'real-world domain'.
• Realistic (Math 1) Real-life examples of Bayesian reasoning
• Reflective consistency A decision system is reflectively consistent if it can approve of itself, or approve the construction of similar decision systems (as well as perhaps approving other decision systems too).
• Reflective stability Wanting to think the way you currently think, building other agents and self-modifications that think the same way.
• Reflectively consistent degree of freedom When an instrumentally efficient, self-modifying AI can be like X or like X' in such a way that X wants to be X and X' wants to be X', that's a reflectively consistent degree of freedom.
• Relative likelihood How relatively likely an observation is, given two or more hypotheses, determines the strength and direction of evidence.
• Relevant limited AI Can we have a limited AI, that's nonetheless relevant?
• Relevant powerful agent An agent is relevant if it completely changes the course of history.
• Relevant powerful agents will be highly optimized The probability that an agent that is cognitively powerful enough to be relevant to existential outc…
• Rescuing the utility function If your utility function values 'heat', and then you discover to your horror that there's no ontologically basic heat, switch to valuing disordered kinetic energy. Likewise 'free will' or 'people'.
• Researchers in value alignment theory Who's working full-time in value alignment theory?
• Rich domain A domain is 'rich', relative to our own intelligence, to the extent that (1) its [ search space] is …
• Safe but useless Sometimes, at the end of locking down your AI so that it seems extremely safe, you'll end up with an AI that can't be used to do anything interesting.
• Safe impact measure What can we measure to make sure an agent is acting in a safe manner?
• Safe plan identification and verification On a particular task or problem, the issue of how to communicate to the AGI what you want it to do and all the things you don't want it to do.
• Separation from hyperexistential risk The AI should be widely separated in the design space from any AI that would constitute a "hyperexistential risk" (anything worse than death).
• Show me what you've broken To demonstrate competence at computer security, or AI alignment, think in terms of breaking proposals and finding technically demonstrable flaws in them.
• Shutdown problem How to build an AGI that lets you shut it down, despite the obvious fact that this will interfere with whatever the AGI's goals are.
• Shutdown utility function A special case of a low-impact utility function where you just want the AGI to switch itself off harmlessly (and not create subagents to make absolutely sure it stays off, etcetera).
• Solomonoff induction A simple way to superintelligently predict sequences of data, given unlimited computing power.
• Solomonoff induction: Intro Dialogue (Math 2) An introduction to Solomonoff induction for the unfamiliar reader who isn't bad at math
• Some computations are people It's possible to have a conscious person being simulated inside a computer or other substrate.
• Standard agent properties What's a Standard Agent, and what can it do?
• Start This page gives a basic overview of the topic, but may be missing important information or have stylistic issues. If you're able to, please help expand or improve it!
• Still needs work The next step up from "Work in Progress". The page can be read as complete, but is a draft that needs further review and fine-tuning.
• Strained argument A phenomenological feeling associated with a step of reasoning going from X to Y where it feels like…
• Strategic AGI typology What broad types of advanced AIs, corresponding to which strategic scenarios, might it be possible or wise to create?
• Strength of Bayesian evidence From a Bayesian standpoint, the strength of evidence can be identified with its likelihood ratio.
• Strictly confused A hypothesis is strictly confused by the raw data, if the hypothesis did much worse in predicting it than the hypothesis itself expected.
• Strictly factual question A "question of strict fact" is one which is true or false about the material universe (and maybe some math) without introducing any issues of values, perspectives, etcetera.
• Strong cognitive uncontainability An advanced agent can win in ways humans can't understand in advance.
• Subjective probability Probability is in the mind, not in the environment. If you don't know whether a coin came up heads or tails, that's a fact about you, not a fact about the coin.
• Sufficiently advanced Artificial Intelligence 'Sufficiently advanced Artificial Intelligences' are AIs with enough 'advanced agent properties' that we start needing to do 'AI alignment' to them.
• Sufficiently optimized agents appear coherent If you could think as well as a superintelligence, you'd be at least that smart yourself.
• Superintelligent A "superintelligence" is strongly superhuman (strictly higher-performing than any and all humans) on every cognitive problem.
• Task (AI goal) When building the first AGIs, it may be wiser to assign them only goals that are bounded in space and time, and can be satisfied by bounded efforts.
• Task identification problem If you have a task-based AGI (Genie) then how do you pinpoint exactly what you want it to do (and not do)?
• Task-directed AGI An advanced AI that's meant to pursue a series of limited-scope goals given it by the user. In Bostrom's terminology, a Genie.
• Terminal versus instrumental goals / values / preferences Distinguish events wanted for their consequences, from events wanted locally.
• The AI must tolerate your safety measures A corollary of the nonadversarial principle is that "The AI must tolerate your safety measures."
• The Robots, AI, and Unemployment Anti-FAQ Q. Are the current high levels of unemployment being caused by advances in Artificial Intelligence …
• The rocket alignment problem If people talked about the problem of space travel the way they talked about AI...
• Theory of (advanced) agents One of the research subproblems of building powerful nice AIs, is the theory of (sufficiently advanced) minds in general.
• Tiling agents theory The theory of self-modifying agents that build successors that are very similar to themselves, like repeating tiles on a tesselated plane.
• Time-machine metaphor for efficient agents Don't imagine a paperclip maximizer as a mind. Imagine it as a time machine that always spits out the output leading to the greatest number of future paperclips.
• Total alignment We say that an advanced AI is "totally aligned" when it knows *exactly* which outcomes and plans are beneficial, with no further user input.
• Toxoplasmosis dilemma A parasitic infection, carried by cats, may make humans enjoy petting cats more. A kitten, now in front of you, isn't infected. But if you *want* to pet it, you may already be infected. Do you?
• Transparent Newcomb's Problem Omega has left behind a transparent Box A containing $1000, and a transparent Box B containing$1,000,000 or nothing. Box B is full iff Omega thinks you one-box on seeing a full Box B.
• True Prisoner's Dilemma A scenario that would reproduce the ideal payoff matrix of the Prisoner's Dilemma about human beings who care about their public reputation and each other.
• Ultimatum Game A Proposer decides how to split 10 between themselves and the Responder. The Responder can take what is offered, or refuse, in which case both parties get nothing. • Underestimating complexity of value because goodness feels like a simple property When you just want to yell at the AI, "Just do normal high-value X, dammit, not weird low-value X!" and that 'high versus low value' boundary is way more complicated than your brain wants to think. • Understandability principle The more you understand what the heck is going on inside your AI, the safer you are. • Unforeseen maximum When you tell AI to produce world peace and it kills everyone. (Okay, some SF writers saw that one coming.) • Universal prior A "universal prior" is a probability distribution containing *all* the hypotheses, for some reasonable meaning of "all". E.g., "every possible computer program that computes probabilities". • Unphysically large finite computer The imaginary box required to run programs that require impossibly large, but finite, amounts of computing power. • Updateless decision theories Decision theories that maximize their policies (mappings from sense inputs to actions), rather than using their sense inputs to update their beliefs and then selecting actions. • User manipulation If not otherwise averted, many of an AGI's desired outcomes are likely to interact with users and hence imply an incentive to manipulate users. • User maximization A sub-principle of avoiding user manipulation - if you see an argmax over X or 'optimize X' instruction and X includes a user interaction, you've just told the AI to optimize the user. • Utility What is "utility" in the context of Value Alignment Theory? • Utility function The only coherent way of wanting things is to assign consistent relative scores to outcomes. • Utility indifference How can we make an AI indifferent to whether we press a button that changes its goals? • Valley of Dangerous Complacency When the AGI works often enough that you let down your guard, but it still has bugs. Imagine a robotic car that almost always steers perfectly, but sometimes heads off a cliff. • Value The word 'value' in the phrase 'value alignment' is a metasyntactic variable that indicates the speaker's future goals for intelligent life. • Value achievement dilemma How can Earth-originating intelligent life achieve most of its potential value, whether by AI or otherwise? • Value alignment problem You want to build an advanced AI with the right values... but how? • Value identification problem The subproblem category of value alignment which deals with pinpointing valuable outcomes to an adva… • Value-laden Cure cancer, but avoid any bad side effects? Categorizing "bad side effects" requires knowing what's "bad". If an agent needs to load complex human goals to evaluate something, it's "value-laden". • Vinge's Law You can't predict exactly what someone smarter than you would do, because if you could, you'd be that smart yourself. • Vinge's Principle An agent building another agent must usually approve its design without knowing the agent's exact policy choices. • Vingean reflection The problem of thinking about your future self when it's smarter than you. • Vingean uncertainty You can't predict the exact actions of an agent smarter than you - so is there anything you _can_ say about them? • Wants to get straight to Bayes A simple requisite page to mark whether the user has selected wanting to get straight into Bayes on … • Waterfall diagram Visualizing Bayes' rule as the mixing of probability streams. • Waterfall diagrams and relative odds A way to visualize Bayes' rule that yields an easier way to solve some problems • Well-calibrated probabilities Even if you're fairly ignorant, you can still strive to ensure that when you say "70% probability", it's true 70% of the time. • William Frankena's list of terminal values Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions... • Work in progress This page is being actively worked on by an editor. Check with them before making major changes. • You can't get more paperclips that way Most arguments that "A paperclip maximizer could get more paperclips by (doing nice things)" are flawed. • You can't get the coffee if you're dead An AI given the goal of 'get the coffee' can't achieve that goal if it has been turned off; so even an AI whose goal is just to fetch the coffee may try to avert a shutdown button being pressed. • Zermelo-Fraenkel provability oracle We might be able to build a system that can safely inform us that a theorem has a proof in set theory, but we can't see how to use that capability to save the world. ### comment ### question ## Emile Kroeger ## Emma Borhanian ## Emmanuel Smith ## Eric Bruylant ### wiki ### comment ### question ## Eric Leese ### wiki • P (Polynomial Time Complexity Class) P is the class of problems which can be solved by algorithms whose run time is bounded by a polynomial. • Turing machine A Turing Machine is a simple mathematical model of computation that is powerful enough to describe any computation a computer can do. ### comment ## Eric Rogstad ### wiki ### comment ## Erictest Rogstadtest ### wiki ### comment ## Erik Istre • Formal Logic Formal logic studies the form of correct arguments through rigorous and precise mathematical theories. • Logic Logic is the study of correct arguments. ## Ethan Orion ## Eyal Roth ## Faisal AlZaben ## G Gordon Worley ### wiki ### comment ## Geneva Macro Labs ## Glenn Davis ## Glenn Field ## Grady Simon ## Gregor Gerasev ## Guillaume Alemanni ## Gurkenglas Gurkenglas ## Gustavo Bicalho ## Haakon Borch ## Harun Rashid Anver ## Harun Rashid Anver ## Hunter Meriwether ## Ilia Livshits ### wiki • Flexagon Flexagons are flat models, that can be flexed in certain ways to reveal faces besides the two that were originally on the back and front ### comment ## Ilia Zaichuk ## Izaak Meckler ### wiki ### comment ## Jack Gallagher ## Jacob Kopczynski ## Jacob Thoennes ## Jaime Sevilla Molina ### wiki ### comment ## Jakob Schmid ## James Andrix ## Jason Gross ### wiki • Uncountability Some infinities are bigger than others. Uncountable infinities are larger than countable infinities. • Uncountability: Intro (Math 1) Not all infinities are created equal. The infinity of real numbers is infinitely larger than the infinity of counting numbers. • Uncountability: Intuitive Intro Are all sizes of infinity the same? What does "the same" even mean here? ### comment ## Javier Ivona ## Jeff Ladish ## Jeremy Perret ### wiki ### comment ## Jesse Aldridge ## Jessica Taylor ## Jim Babcock ## Joe Zeng ### wiki ### comment ## Johannes Schmitt • Least common multiple The **least common multiple (LCM)** of two positive natural numbers a, b is the smallest natural … ## John Buridan ## John Maxwell ## Jordan Bennett ### wiki ### comment ## Jules Tanneur ## Julius Jacobsen ## Kai Teorn ## Katriel Friedman ## Kaya Fallenstein ## Keji Li ## Kenzi Amodei ## Kerry Vaughan ## Kevin Clancy ### wiki • Complete lattice A poset that is closed under arbitrary joins and meets. • Computer Programming Familiarity Want to see programming analogies and applications in your math explanations? Mark this as known. • Join and meet Let\langle P, \leq \rangle$be a poset, and let$S \subseteq P$. The **join** of$S$in$P$, deno… • Join and meet: Examples A union of sets and the least common multiple of a set of natural numbers can both be viewed as join… • Join and meet: Exercises Try these exercises to test your knowledge of joins and meets. Tangled up -------------------- !… • Lattice (Order Theory) A poset that is closed under binary joins and meets. • Lattice: Examples Here are some additional examples of lattices.$\newcommand{\nsubg}{\mathcal N \mbox{-} Sub~G}$A f… • Lattice: Exercises Try these exercises to test your knowledge of lattices. ## Distributivity Does the lattice meet op… • Monotone function An order-preserving map between posets. • Monotone function: examples Here are some examples of monotone functions. A cunning plan -------- There's a two-player game ca… • Monotone function: exercises Try these exercises and become a *deity* of monotonicity. Monotone composition ----- Let$P, Q$, … • Order theory The study of binary relations that are reflexive, transitive, and antisymmetic. • Partially ordered set A set endowed with a relation that is reflexive, transitive, and antisymmetric. • Poset: Examples The standard$\leq$relation on integers, the$\subseteq$relation on sets, and the$|$(divisibilit… • Poset: Exercises Try these exercises to test your poset knowledge. # Corporate Ladder Imagine a company with five … • Real analysis The study of real numbers and real-valued functions. • Relation A **relation** is a set of [tuple\_mathematics tuples], all of which have the same [tuple\_arity ar… ### comment ## Kevin Van Horn ## Kevin Western ## Konrad Seifert ## Kyle Bogosian ## Lancelot Verinia ### wiki ### comment ## Leon D ## Luke Sciarappa ## M Yass ### wiki • Absolute Complement The complement$A^\complement$of a set$A$is the set of all things that are not in$A$. Put simply… • Antisymmetric relation A binary relation where no two distinct elements are related in both directions • Currying Transforms a function of many arguments into a function into a function of a single argument • Intersection The intersection of two sets is the set of elements they have in common • Operations in Set theory An operation in set theory is a Function of two sets, that returns a set. Common set operations inc… • Relative complement The relative complement of two sets$A$and$B$, denoted$A \setminus B$, is the set of elements tha… • Union The union of two sets is the set of elements which are in one or the other, or both ### comment ## Maelle Andre ## Malcolm McCrimmon ### wiki • Binary notation A way to write down numbers using powers of two. • Boolean A value in logic that evaluates to either "true" or "false". • Modular arithmetic Addition as traveling around a circle, instead of along a line. ### comment ## Malcolm Ocean ### wiki ### comment ## Malo Bourgon ## Manuel Te ## Mark Chimes ### wiki ### comment ## Mars (person) ### wiki ### comment ## Martin Epstein ## Matthew Fallshaw ## Micah Carroll ## Michael Cohen ### wiki • Decimal notation The winning architecture for numerals • Derivative How things change • Expected value Trying to assign value to an uncertain state? The weighted average of outcomes is probably the tool you need. • Factorial The number of ways you can order things. (Alternately subtitled: Is that exclamation point a factorial, or are you just excited to see me?) • Integer An **integer** is a Number that can be represented as either a Natural number or its [-additive\_inv… • Inverse function The inverse of a function returns an input of the original function when fed the original's corresponding output. • Pi Pi, usually written$π$, is a number equal to the ratio of a circle's [-circumference] to its [-diam… • Real number A **real number** is any number that can be used to represent a physical quantity. Intuitively, rea… ### comment ## Mike Johnson ## Morgan Redding • Asymptotic Notation Asymptotic notation seeks to capture the behavior of functions as its input(s) become extreme. It is most widely used in Computer Science and Numerical Approximation. ## Morgan Sinclaire ## Nate Soares ### wiki • A googol A pretty small large number. • A googolplex A moderately large number, as large numbers go. • Abelian group A group where the operation commutes. Named after Niels Henrik Abel. • Abstract algebra The study of groups, fields, vector spaces, arithmetics, algebras, and more. • Algebraic structure Roughly speaking, an algebraic structure is a set$X$, known as the underlying set, paired with a co… • Arity (of a function) The arity of a function is the number of parameters that it takes. For example, the function$f(a, b…
• Associative operation An **associative operation** $\bullet : X \times X \to X$ is a binary operation such that for all $x… • Associativity vs commutativity Associativity and commutativity are often confused, because they are both constraints on how a funct… • Associativity: Examples Yes: [Addition], [multiplication], string concatenation. No: [subtraction], [division], a Function … • Associativity: Intuition Associative functions can be interpreted as families of functions that reduce lists down to a singl… • Bag In mathematics, a "bag" is an unordered list. A bag differs from a set in that it can contain the sa… • Bayes' rule: Definition Bayes' rule is the mathematics of probability theory governing how to update your beliefs in the lig… • Bayes' rule: Probability form The original formulation of Bayes' rule. • Binary function A binary function$f$is a function of two inputs (i.e., a function with arity 2). For example,$+,$… • Bit The term "bit" refers to different concepts in different fields. The common theme across all the us… • Bit (abstract) An abstract bit is an element of the set$\mathbb B$, which has two elements. An abstract bit is to … • Bit (of data) A bit of data is the amount of data required to single out one message from a set of two. Equivalen… • Bit (of data): Examples In the game "20 questions", one player (the "leader") thinks of a concept, and the other players ask… • Blue oysters A probability problem about blue oysters. • Cartesian product The Cartesian product of two sets$A$and$B,$denoted$A \times B,$is the set of all [ordered\_pai… • Ceiling The ceiling of a real number$x,$denoted$\lceil x \rceil$or sometimes$\operatorname{ceil}(x),$i… • Closure A set$S$is _closed_ under an operation$f$if, whenever$f$is fed elements of$S$, it produces an… • Codomain (of a function) The codomain$\operatorname{cod}(f)$of a function$f : X \to Y$is$Y$, the set of possible outputs… • Codomain vs image It is useful to distinguish codomain from image both (a) when the type of thing that the function pr… • Communication: magician example Imagine that you and I are both magicians, performing a trick where I think of a card from a deck of… • Commutative operation A commutative function$f$is a function that takes multiple inputs from a set$X$and produces an o… • Commutativity: Examples Yes: addition, multiplication, maximum, minimum, rock-paper-scissors. No: subtraction, division, st… • Commutativity: Intuition We can think of commutativity either as an artifact of notation, or as a symmetry in the output of a… • Compressing multiple messages How many bits of data does it take to encode an$n$-message? Naively, the answer is$\lceil \log_2(n…
• Conditional probability: Refresher Is P(yellow | banana) the probability that a banana is yellow, or the probability that a yellow thing is a banana?
• Correspondence visualizations for different interpretations of "probability" Let's say you have a model which says a particular coin is 70% likely to be heads. How should we as…
• Corrigibility "I can't let you do that, Dave."
• Data capacity The data capacity of an object is defined to be the Logarithm of the number of different distinguish…
• Decit Decimal digit
• Dependent messages can be encoded cheaply Say you want to transmit a 2-message, a 4-message, and a 256-message to somebody. For example, you m…
• Digit wheel A mechanical device for storing a number from 0 to 9. ![](http://www.cl.cam.ac.uk/~djg11/howcompu…
• Direct sum of vector spaces The direct sum of two vector spaces $U$ and $W,$ written $U \oplus W,$ is just the sum of $U$ and $W… • Domain (of a function) The domain$\operatorname{dom}(f)$of a function$f : X \to Y$is$X$, the set of valid inputs for t… • Emulating digits In general, given enough$n$-digits, you can emulate an$m$-digit, for any$m, n \in\mathbb N$. I… • Encoding trits with GalCom bits There are$\log_2(3) \approx 1.585$bits to a Trit. Why is it that particular value? Consider the Ga… • Example problem Tag for pages that provide an example problem referenced by a number of other pages. The summary of… • Exchange rates between digits In terms of data storage, if a coin is worth$1, a digit wheel is worth more than $3.32, but less than$3.33. Why?
• Fractional bits It takes $\log_2(8) = 3$ bits of data to carry one message from a set of 8 possible messages. Simila…
• Fractional bits: Digit usage interpretation It is 316, not 500, that requires about two and a half digits to write down. 500 requires nearly 2.7…
• Fractional bits: Expected cost interpretation In the GalCom thought experiment, you regularly have to send large volumes of information through de…
• Fractional digits When $b$ and $x$ are integers, $\log_b(x)$ has a few good interpretations. It's roughly the length o…
• Frequency diagram Visualizing Bayes' rule by manipulating frequencies in large populations
• Frequency diagrams: A first look at Bayes The most straightforward visualization of Bayes' rule.
• Function Intuitively, a function $f$ is a procedure (or machine) that takes an input and performs some opera…
• Function: Physical metaphor Many functions can be visualized as physical mechanisms of wheels and gears, that take their inputs …
• GalCom In the GalCom thought experiment, you live in the future, and make your money by living in the Dene…
• GalCom: Rules 1. It costs 1 galcoin per bit to reserve on-peak bits in advance. (Galcoins are very expensive.) 2. …
• Generalized associative law Given an associative operator $\cdot$ and a list $[a, b, c, \ldots]$ of parameters, all ways of red…
• Graham's number A fairly large number, as numbers go.
• Group The algebraic structure that captures symmetry, relationships between transformations, and part of what multiplication and addition have in common.
• Group theory What kinds of symmetry can an object have?
• How many bits to a trit? $\log_2(3) \approx 1.585.$ This can be interpreted a few different ways: 1. If you multiply the nu…
• Image (of a function) The image $\operatorname{im}(f)$ of a function $f : X \to Y$ is the set of all possible outputs of $… • Information Information is a measure of how much a message grants an observer the ability to predict the world.… • Information theory The study (and quantificaiton) of information, and its communication and storage. • Interpretations of "probability" What does it *mean* to say that a fair coin has a 50% probability of coming up heads? • Intradependent encoding An encoding$E(m)$of a message$m$is intradependent if the fact that$E(m)$encodes$m$can be de… • Intradependent encodings can be compressed Given an encoding scheme$E$which gives an Intradependent encoding of a message$m,$we can in prin… • Introductory guide to logarithms Welcome to the Arbital introduction to logarithms! In modern education, logarithms are often mention… • Life in logspace The log lattice hints at the reason that engineers, scientists, and AI researchers find logarithms s… • Likelihood "Likelihood", when speaking of Bayesian reasoning, denotes *the probability of an observation, sup… • Likelihood function Let's say you have a piece of evidence$e$and a set of hypotheses$\mathcal H.$Each$H_i \in \math…
• Likelihood notation The likelihood of a piece of evidence $e$ according to a hypothesis $H,$ known as "the likelihood of…
• Likelihood ratio Given a piece of evidence $e$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them…
• Linear algebra The study of [linear\_transformation linear transformations] and vector spaces.
• List A list is an ordered collection of objects, such as [0, 1, 2, 3] or ["red", "blue", 0, "shoe"]. …
• Log as generalized length To estimate the log (base 10) of a number, count how many digits it has.
• Log as the change in the cost of communicating When interpreting logarithms as a generalization of the notion of "length" and as digit exchange rat…
• Log base infinity There is no log base infinity, but if there were, it would send everything to zero
• Logarithm The logarithm base $b$ of a number $n,$ written $\log_b(n),$ is the answer to the question "how man…
• Logarithm base 1 There is no log base 1.
• Logarithm tutorial overview The logarithm tutorial covers the following six subjects: 1. What are logarithms? 2. Logarithms as…
• Logarithm: Examples $\log_{10}(100)=2.$ $\log_2(4)=2.$ $\log_2(3)\approx 1.58.$ (TODO)
• Logarithm: Exercises Without using a calculator: What is $\log_{10}(4321)$? What integer is it larger than, what integer …
• Logarithmic identities - [ Inversion of exponentials]: $b^{\log_b(n)} = \log_b(b^n) = n.$ - [ Log of 1 is 0]: $\log_b(1) … • Logarithms invert exponentials The function$\log_b(\cdot)$inverts the function$b^{(\cdot)}.$In other words,$\log_b(n) = x$imp… • Monoid A monoid$M$is a pair$(X, \diamond)$where$X$is a [set\_theory\_set set] and$\diamond$is an [a… • Nate's ruminations These posts are a mirror of posts on the blog [MindingOurWay.com](mindingourway.com) which pertain t… • Needs lenses This page has only a technical introduction. If you're able to, please help by adding an intuitive explanation! • Non-standard terminology A tag for terminology that is Arbital-specific, Arbital-originated, or just not very common outside … • Odds form to probability form The odds form of Bayes' rule works for any two hypotheses$H_i$and$H_j,$and looks like this:$$\… • Odds: Introduction What's the difference between probabilities and odds? Why is a 20% probability of success equivalent to 1 : 4 odds favoring success? • Odds: Refresher A quick review of the notations and mathematical behaviors for odds (e.g. odds of 1 : 2 for drawing a red ball vs. green ball from a barrel). • Operator An operation$f$on a set$S$is a function that takes some values from$S$and produces a new value… • Order of a group The order$|G|$of a group$G$is the size of its underlying set. For example, if$G=(X,\bullet)$an… • Ordinary claims require ordinary evidence Extraordinary claims require extraordinary evidence, but ordinary claims *don't*. • Probability distribution: Motivated definition People keep writing things like P(sick)=0.3. What does this mean, on a technical level? • Probability interpretations: Examples Consider evaluating, in June of 2016, the question: "What is the probability of Hillary Clinton wi… • Proof of Bayes' rule: Probability form Let$\mathbf H$be a [random\_variable variable] in$\mathbb P$for the true hypothesis, and let$H_…
• Properties of the logarithm - $\log_b(x \cdot y) = \log_b(x) + \log_b(y)$ for any $b$, this is the defining characteristic of …
• Range (of a function) The "range" of a function is an ambiguous term that is generally used to refer to either the functio…
• Replacing Guilt In my experience, many people are motivated primarily by either guilt, shame, or some combination of…
• Report likelihoods, not p-values If scientists reported likelihood functions instead of p-values, this could help science avoid p-ha…
• Ring A ring is a kind of Algebraic structure which we obtain by considering groups as being "things with…
• Set An unordered collection of distinct objects.
• Set builder notation $\{ 2n \mid n \in \mathbb N \}$ denotes the set of all even numbers, using set builder notation. Set…
• Shannon The shannon (Sh) is a unit of Information. One shannon is the difference in [info\_entropy entropy] …
• Shift towards the hypothesis of least surprise When you see new evidence, ask: which hypothesis is *least surprised?*
• Size of a set
• Sock-dresser search There's a 4/5 chance your socks are in one of your dresser's 8 drawers. You check 6 drawers at random. What's the probability they'll be in the next drawer you check?
• Sparking widgets 10% of widgets are bad and 90% are good. 4% of good widgets emit sparks, and 12% of bad widgets emit…
• String (of text) A string (of text) is a series of letters (often denoted by quote marks), such as "abcd" or "hell…
• Subspace A subspace $U=(F_U, V_U)$ of a Vector space $W=(F_W, V_W)$ is a vector space where $F_U = F_W$ and $… • Sum of vector spaces The sum of two vector spaces$U$and$W,$written$U + W,$is a vector space where the set of vector… • The End (of the basic log tutorial) That concludes our introductory tutorial on logarithms! You have made it to the end. Throughout thi… • The Stamp Collector Once upon a time, a group of naïve philosophers found a robot that collected trinkets. Well, more sp… • The characteristic of the logarithm Any time you find an output that adds whenever the input multiplies, you're probably looking at a (… • The log lattice Log as the change in the cost of communicating and other pages give physical interpretations of what… • There is only one logarithm All logarithm functions are the same, up to a multiplicative constant. • Thought experiment Meta-tag for thought experiments. • Trit Trinary digit • Underlying set What do a Group, a Partially ordered set, and a [ topological space] have in common? Each is a Set … • Vector space A vector space is a field$F$paired with a Group$V$and a function$\cdot : F \times V \to V$(cal… • What is a logarithm? Logarithms are a group of functions that take a number as input and produce another number. There i… • Why is log like length? If a number$x$is$n$digits long (in Decimal notation), then its logarithm (base 10) is between$n…
• Why is the decimal expansion of log2(3) infinite? Because 2 and 3 are relatively prime.
• You're allowed to fight for something The first sort of guilt I want to address is the listless guilt, that vague feeling one gets after p…
• concat (function) The string concatenation function concat puts two strings together, i.e., concat("one","two")="on…
• n-digit An $n$-digit is a physical object that can be stably placed into any of $n$ distinguishable states. …
• n-message A message singling out one thing from a set of $n$ is sometimes called an $n$-message. For example,…

## Patrick LaVictoire

### wiki

• Gödel encoding and self-reference The formalism that mathematicians use to talk about arithmetic turns out to be able to talk about itself.
• Peano Arithmetic A system for proving theorems about arithmetic, which is strong enough to include self-reference.
• Quine A computer program that prints (or does other computations to) its own source code, using indirect self-reference.

## Qiaochu Yuan

### wiki

• Colon-to notation Find out what the notation "f : X -> Y" means that everyone keeps using.
• Group action "Groups, as men, will be known by their actions."
• Group theory: Examples What does thinking in terms of group theory actually look like? And what does it buy you?
• Group: Examples Why would anyone have invented groups, anyway? What were the historically motivating examples, and what examples are important today?
• Group: Exercises Test your understanding of the definition of a group with these exercises.
• In notation There's a weird E-looking symbol called \in in LaTeX. What does it mean?
• Mapsto notation There's an arrow called \mapsto in LaTeX. What does it mean?
• Representation theory The study of how groups act on vector spaces.

## Raymond Arnold

### wiki

• Practicing Brevity Wanna save the world? Write better TLDRs. And spend more time respecting your reader's time.

## Ryan Hendrickson

### wiki

• Algebraic structure tree When is a monoid a semilattice? What's the difference between a semigroup and a groupoid? Find out here!
• Reflexive relation A binary relation over some set is **reflexive** when every element of that set is related to itself…

## Silas Barta

### wiki

• Doppler Effect Why do things make a higher pitch sound as they come toward me but lower as they go away?
• Externality Positive and negative affects on third parties, and the considerations they introduce
• Human perception of sound What is the mechanism by which vibrations around the human ear are translated into the sensation of sound?
• Pigovian tax Taxation of negative externalities so that their producers have an incentive to cheaply reduce them
• Supply and Demand How are prices typically formed in market systems?