Distances between cognitive domains


by Eliezer Yudkowsky Feb 18 2017 updated Feb 18 2017

Often in AI alignment we want to ask, "How close is 'being able to do X' to 'being able to do Y'?"

In the context of AI alignment, we may care a lot about the degree to which competence in two different cognitive domains is separable, or alternatively highly tangled, relative to the class of algorithms reasoning about them.

For example: If the domains X and Y are 'blue cars' and 'red cars', then it seems unlikely that X and Y would be well-separated domains because an agent that knows how to reason well about blue cars is almost surely extremely close to being an agent that can reason well about red cars, in the sense that:

In more complicated cases, which domains are truly close or far from each other, or can be compactly separated out, is a theory-laden assertion. Few people are likely to disagree that blue cars and red cars are very close domains (if they're not specifically trying to be disagreeable). Researchers are more likely to disagree in their predictions about:

Relation to 'general intelligence'

A key parameter in some such disagreements may be how much credit the speaker gives to the notion of General intelligence. Specifically, to what extent the natural or the most straightforward approach to get par-human or superhuman performance in critical domains, is to take relatively general learning algorithms and deploy them on learning the domain as a special case.

If you think that it would take a weird or twisted design to build a mind that was superhumanly good at designing cars including writing their software, without using general algorithms and methods that could with minor or little adaptation stare at mathematical proof problems and figure them out, then you think 'design cars' and 'prove theorems' and many other domains are in some sense naturally not all that separated. Which (arguendo) is why humans are so much better than chimpanzees at so many apparently different cognitive domains: the same competency, general intelligence, solves all of them.

If on the other hand you are more inspired by the way that superhuman chess AIs can't play Go and AlphaGo can't drive a car, you may think that humans using general intelligence on everything is just an instance of us having a single hammer and trying to treat everything as a nail; and predict that specialized mind designs that were superhuman engineers, but very far in mind design space from being a kind of mind that could prove Fermat's Last Theorem, would be a more natural or efficient way to create a superhuman engineer.

See the entry on General intelligence for further discussion.