14 • What is intelligence?
Looking at various ways of how we interact with our environment — with objects, solving problems, or communicate with other people — a fundamental pattern emerges.
We learned earlier in this series that we can’t rely purely on logical reasoning and only accept what is certain. We‘ve also seen that most challenges we face in our daily lives that require us to figure something out, are ill-defined, and when we manage to figure them out, we are often struck with insight.
Perhaps surprisingly, at this point, we haven’t yet touched on intelligence. You’d think intelligence plays some role in all of this…
What is it? What makes us intelligent?
Avoiding combinatorial explosion
Let's say you're playing a game of chess. On average, the number of operations you can perform on any turn […] is 30. Now don't say to me, “How many of those are stupid?” That's not the point. I'm trying to explain how you're intelligent. That's what I have to explain, it's not what I can assume.
So there's 30 legal moves and, on average, there's 60 turns — [30^60] is the number of pathways that are in the problem space. This is known as combinatorial explosion. […] This is a vast number. It's something like 4.29×10^88 in standard scientific notation. I want you to understand how big this is, how astronomically, incomprehensibly, large this is.
One thing you might say is, “I have many different neurons and they all work in parallel and they're all checking different alternative pathways, and that's how I do this — parallel distributed processing.” There's an important element in that, but that's not going to be enough, because you probably have around [10^10] many neurons. Now that’s a lot! But it's nowhere near [30^60] and you say, “But it's not the neurons, it's the number of connections.” And that's something like 5×10^15, and that's a big number. But you know what it's astronomically far away from? It's astronomically far away from [30^60].
Even if each synaptic connection is exploring a pathway, this is still overwhelming your brain. In fact, this is greater than the number of atomic particles that are estimated to exist in the universe. This is, in a literal sense, bigger than the universe.
It's big. What does that mean? That means you cannot do the following: You cannot search the whole space. For very, very many problems, the problems are combinatorially explosive and therefore you cannot search that space. You do not have the time, the machinery, the resources to search the whole space.
Realizing relevance
Now here is what is deeply, deeply interesting about you — this is sort of my professional obsession […] — this is what you do somehow: You can't search the whole space. I mean, you can't search, you can't look [at a part] and then reject. Because if you look at [one] part of the space and reject it, and then look at [another] part of it, then you end up searching the whole space. It's not a matter of checking and rejecting because that's searching the whole space.
What you do is somehow do this: you somehow zero in on only a very small subsection of that whole space and you search in there. And you often find a solution! You somehow zero in on the relevant information and you make that information effective in your cognition. You do what I call relevance realization: You realize what's relevant.
Now this fascinates me, and that fascination is due to the work of Newell and Simon. How do you do that? Computers are really fast. Even the fastest chess-playing computers don't check the whole space. They can’t! They're not powerful enough or fast enough. That's not how they play.
So this issue of avoiding combinatorial explosion is actually a central way of understanding your intelligence. And you probably hadn't thought of that before, that one of the things that makes you crucially intelligent is your ability to zero in on relevant information. And of course, you're experiencing that in two related, but different ways:
One way is — and the way this is happening so automatically and seamlessly for you — is the generation of obviousness, what's obvious. Well, obviously I should pick up my marker. Obviously I should go to the board. Obviousness is not a property of physics or chemistry or biology. Obviousness is not what explains your behavior. It explains your behavior in a “common sense” way, but obviousness is what I scientifically have to explain. How does your brain make things obvious to you?
And that's related to, but not identical to, this issue of how things are salient to you. How they stand out to you. How they grab your attention. And what we already know is that that process isn't static because sometimes how you zeroed in on things as relevant, what was obvious to you, what was salient to you, how you "join the nine dots" is obvious and salient to you, and yet you get it wrong. And part of your ability is to restructure what you find relevant and salient — you can dynamically self-organize what you find relevant and salient.
A key problem of artificial intelligence
Now Newell and Simon wrestled with this and there's a sense in which this is the key problem that the project of artificial general intelligence is trying to address right now. In fact, that's what I argued. I've argued in some work I did with Tim Lillicrap and Blake Richards, some work I've done with Leo Ferraro — there's related work by other people.
But Newell and Simon realised that in some way you have to deal with combinatorial explosion. To make a general problem solver, you have to give the machine, the system, the capacity to avoid combinatorial explosion.
We're going to see that this is probably the best way of trying to understand what intelligence is. People like Stanovich argue that what we're measuring when we're measuring your intelligence in psychometric tests is precisely your ability to deal with computational limitations to avoid combinatorial explosion. Christopher Cherniak argues something similar.
What makes you intelligent is your capability to avoid combinatorial explosion and realize what is relevant in a given situation. Your cognitive processing does this by:
Automatically and seamlessly generating obviousness
Dynamically self-organizing your salience landscape
It appears that these capabilities do not work algorithmically, but heuristically, and therefore this is one of the crucial challenges in creating artificial (general) intelligence (AGI).
Understanding better how we make things obvious and how we dynamically reorganize our salience landscape, will be key to making progress in AGI. Or you can turn it around: If we someday manage to create AGI, we will probably understand much better how our own cognitive system works — this seems to be a precondition to creating consciousness.
Problem (re)framing
Problem formulation helps you avoid combinatorial explosion and helps you deal with ill-definedness. And this process by which you move from a poor problem formulation to a good problem formulation is the process of insight. This is why we have seen throughout, insight is so crucial to you being a rational cognitive agent. And that means that, in addition to logic being essential for our rationality, those psycho-technologies that enhance our capacity for insight are also crucially important, indispensable.
We know that in insight relevance realization is recursively self-organizing, restructuring itself in a dynamic fashion. Insight, of course, it's not only important for changing ill-definedness — this is what problem formulation is doing, or as I'll later call it problem framing — it's also doing this: It’s helping me avoid combinatorial explosion. But it's also doing something else, something that we talked about and saw already before with the 9-dot problem: It's helping you to overcome the way in which your relevance realization machinery is making the wrong things salient and obvious for you. Insight is also the way in which this process is self-corrective, in which the problem formulation process is self-corrected, because you can mislead yourself, be misdirected.
Insight deals with converting ill-defined problems into well-defined problems by doing problem framing, problem formulation, or doing reframing when needed. It helps us avoid combinatorial explosion by doing problem formulation or reframing as with the person who shifted from a covering strategy to a parity strategy in the mutilated chess board. Or it also helps us correct how we limit, inappropriately, our attempts to solve a problem by what we consider salient or relevant. And it allows us to reformulate, reframe, and break out of the way we have boxed our cognition and our consciousness in. Insight is crucial in the cultivation of wisdom.
Insight is a feature of our cognition that emerges when it dynamically reconfigures our relevance realization machinery, to change the framing of a problem to make different things salient and obvious, so that we can find new solutions we couldn’t before. Again, relevance realization — making the right things stand out — is crucial for insight.
Categorization
So presumably one of the things that contributes, as I mentioned, to your capacity for being a general problem solver, is your capacity for categorizing things. […]
Your ability to categorize things massively increases your ability to deal with the world. If I couldn't categorize things, if I couldn't see these [two markers] both as markers, I would have to treat each one as a raw individual that I'm encountering for the first time. Kind of like we do when we meet people and we treat them with proper nouns. So this would be Tom, and then this would be Agnes, and meeting Tom doesn't tell me anything about what Agnes is going to be like. […]
But if I can categorize them together, I can make predictions about how any member of this category will behave. It massively speeds up my ability to make predictions, to abstract important and potentially relevant information. It allows me to communicate. I can communicate all of that categorical information with a common noun marker. Your ability to categorize is central to your ability to be intelligent.
So what is a category? A category isn't just any set of things. A category is a set of things that you sense belong together. Now we noted last time that your sense of things belonging together isn't necessarily because they share an essence. That's the common mistake.
How is it that we categorize things? How does this basic ability central to our intelligence operate? I'm not going to try and fully explain that, I don't know anyone that can do that right now. All I need to do is show you how this issue of relevance realization is at the center.
The standard explanation, the one that works from common sense, is the one you see in Sesame Street: You give the child [three markers and an eraser], “Here are three things. Three of these things, they're kind of the same. One of these things is not like the others. Three of these things are kind of the same.” And you have to pick out the one. And so these [markers] go together, this one [eraser] doesn’t. These [markers] are categorized as markers. That's the Sesame Street explanation.
What's the explanation? I noticed that these [three markers] are similar. I noticed that this one [eraser] is different. I mentally group together the things that are similar. I keep the things that are different mentally apart. And that's how I form categories. Isn't that obvious?
Explaining how it becomes obvious to you and how you make the correct properties salient is the crucial thing. Why? Well, this was a point made famous by the philosopher Nelson Goodman. […] Nelson pointed out that we're often — I'm going to use our language, I think this is fair — but we're often equivocating when we invoke "similarity" and how obvious it is, between a psychological sense and a logical sense. And in that sense we're deceiving ourselves that we're offering an explanation.
What do I mean? What does similarity mean in a logical sense? Remember the Sesame Street example: "kind of the same". Similarity is partial identity: kind of the same.
What does partial identity mean? You share properties, you share features. And the more features you share, the more identical you are, the more similar you are. There you go.
That’s pretty clear. Once you agree with that, Nelson Goodman is going to say, “Well, now you have a problem because any two objects are logically overwhelmingly similar”. Because pick any two objects, say a bison and a lawn-mower. I have to pick properties that they share in common:
They're both found in North America.
Neither one was found in North America 300 million years ago.
Both contain carbon.
Both can kill you if not properly treated.
Both have an odor.
Both weigh less than a ton.
Neither one makes a particularly good weapon.
In fact, the number of things that I can say truly that are shared by [a bison] and [a lawnmower] is indefinitely large. It's combinatorially explosive. It goes on and on and on and on.
And this is Goodman’s point: They share many, many indefinitely large number of properties. Now what I imagine you're saying is, "Yeah, that's all true." — I didn't say anything false! Notice how truth and relevance aren't the same thing. I didn't say anything false. But what you're saying to me is, “Yeah, but those aren't the important properties. You're picking trivial and…" Notice what you're doing. You're telling me that I haven't zeroed in on the relevant properties — the ones that are obvious to you, the ones that stand out to you as salient.
What you're now doing is you're moving from a logical to a psychological account of similarity. What matters for psychological similarity is not any true comparison, but finding the relevant comparisons. And the thing about that is that doesn't seem to be stable. Barsalou pointed this out.
I'm going to give you a set of things. Is it a category?
your wife/spouse
your children
pets
works of art
gasoline
explosive material
Is that a category? And you go, “No, they don't share enough in common.” Now here's what I say to you: “There's a fire!” Oh, right, all those things belong together now! Because I care about my wife, I care about my kids, because fire can kill them, and pets. And explosive stuff and flammable stuff is dangerous. Now it forms a category. In one context, not a category, and another context, a very tight and important category.
Now the logical sharing has not changed. What’s shared psychologically is what properties or features you consider relevant for making the comparison. Out of all of what is logically shared, you zero in on the relevant features for comparison.
You do the same thing when you're deciding that two things are different because any two objects, any objects you think are really the same also have an indefinitely large number of differences. And when you are holding things as different, because you have zeroed in… Here [with markers and eraser], shape and use are relevant differences.
At the core of your ability to form and use categories is your capacity, again, for zeroing in on relevant information.
Categorization is the “trick” we use to build hierarchies of abstraction: Abstract things become more and more the same, they fit in the same (more generic) category and the differences between them matter less. Going the other direction, we can make differences more relevant and distinguish similar things by looking at more and more details.
How we decide what properties matter for these comparisons is again a matter of relevance, and highly dependent on the context.
Communication
What about communication? Isn't that central to being a general problem solver? You bet! Especially if most of my intelligence is my ability to coordinate my behavior with myself and with others. Communication is vital to this. We see this even in creatures that don't have linguistic communication: Social communication makes many species behave in a more sophisticated fashion. There's a relationship between how intelligent an individual is and how social the species is. It's not an algorithm. There seem to be important exceptions like the octopus, but in general, communication is crucial to being an intelligent cognitive agent.
Let's try and use linguistic communication as our example, because that way we can also bring in the linguistics that's in cognitive science. The point is when you're using language to communicate, you're involved with a very particular problem. This was made really clear by the work of H. P. Grice. He pointed out that you always are conveying much beyond what you're saying. It's much more than what you're saying. It always has to be. And that communication depends on you being able to convey much more than you say.
Now, why is that? Because I have to depend on you to derive the implications — that's a logical thing — and then, what he also called implicature — which is not a directly logical thing — in order for me to convey above and beyond what I'm saying.
I drive up in my car. I put my window down. And I say, "Excuse me!" There's a person on the street. "I'm out of gas." The person comes over and says, "Oh, there's a gas station at the corner." And I go, "Thank you", and drive away.
Let’s go through this carefully: I rolled down the window and I just shout out, "Excuse me!" Now what would I actually need to be saying to capture everything that I'm conveying? I would have to say… I'm shouting this word "Excuse me" in the hope that anybody who hears it understands that the “me” refers to the speaker and that by saying “Excuse me” I'm actually requesting that you give me your attention, understanding, of course, that I'm not demanding that you'll give me your attention for an hour or three hours or 17 days, but for some minimal amount of time that's somehow relevant for a problem that I'm going to pose that's not too onerous.
"I" — and again, when I'm saying “I”, I mean, this person making the noises, who is actually the same as the one referred to by this other word, “me” — “I’m out of gas.” And, of course, I don't mean "me" or "I", the speaker, I mean the vehicle I'm actually in. I'm not asking you to make me more flatulent. I'm asking for you to help me find gasoline for my car. And I'm actually referring to gasoline by this short term "gas". And by saying this, I know you understand that my car isn't completely out of gasoline. There's enough in it that I can drive some relatively close distance to find a source of gasoline.
The other person: "Oh!” By uttering this otherwise meaningless term, I'm indicating that I accept the deal that we have here, that I'm going to give you a bit of my attention, and I understand that it's not going to be too long, too onerous. I can make a statement, seemingly out of the blue, that you will know how to connect to what you actually want, which is gasoline for your car, that's not completely out of gas. I will just say the statement, "There is a gasoline station at the corner."
You will figure out that that means that you can drive to it. I'm talking about a nearby corner. Somehow relevantly similar to the amount of gasoline, this isn't a corner halfway across the continent. There's a gasoline station. It will distribute gasoline for your car. It's not for giving helium to blimps. It's not a little model of a gas station. It's not a gas station that's closed and not has not been in business for ten years. It's a gasoline station that will accept Canadian currency or credit. It won't demand your firstborn or fruits from your field.
And you know how all of that's going on? Because if any of that is violated, you either find it funny or you get angry. If you say, “Excuse me, I'm out of gas” and the person comes up and blows some helium into your car, you don't go, "Oh, thank you! That's what I wanted! I wanted some gas. Helium. Yeah.” It's ridiculous! If you drive to the corner and there's a gas station that's been out of business for ten years, you go, “What? What's going on? What's wrong with that person?”
You're always conveying way more than you're saying. Now, notice something else: Notice, I gave you a whole bunch of sentences to try and explicate what I was conveying. But you know what? Each one of these sentences is also conveying more than it was saying. And if I was trying to unpack what it said, what it were conveying, and what they say, I would have to generate all of its sentences and so on and so on. And you see what this explodes into: You can't say everything you want to convey. You rely on people reading between the lines.
By the way, that is actually what this word [intelligence] means. At least one of the etymologies of “intelligence” is “inter-ledger”, which means "to read between the lines".
What did Grice say we do? Well, what we do is we follow a bunch of maxims. We assume that when we're trying to communicate, there's some basic level of cooperation — I don't mean social cooperation, just communicative cooperation — and we assume that people are following some maxims.
You're at a party, and you asked me, "How many kids do you have?" And I say, "I have one kid." And then later on I'm talking and you overhear me — somebody asked me the same — "How many kids do you have?" — "I have two sons." You come up to me and say, "What's wrong with you? Why did you lie?" — "What? I didn't lie. If I have two kids, I necessarily have one child. I didn't say anything false, saying I have one kid!" And you'll just say, "What an asshole!" Because I didn't provide you with the relevant amount of information. I didn't give you the information you needed in order to try and pick up on what I was conveying. You spoke the truth, the logical truth. Or I did in this example. But I didn't speak it in such a way — this is why you can't be perfectly logical — I didn't speak it in such a way that I aided you in determining what the relevant conveyance is.
Four maxims of conveyance
Grice said we follow four maxims:
We assume the person is trying to convey the truth. This is actually often called the maxim of quality, it has to do with truth though.
And then a maxim quantity — they're trying to give us the right amount of information.
And then there's a maxim of manner.
And then there's a maxim of relevance.
This is basically:
We assume that people are trying to tell the truth.
They're trying to give us the right amount of information.
They're trying to put it in the kind of format that's most helpful to us in getting what's conveyed beyond what's said.
And they're giving us relevant information. (There it is again, the word relevance.)
Then Sperber and Wilson come along writing a very important book, which I'll talk about later, and I have criticisms of, but the book is entitled "Relevance". And what's interesting is, they're proposing this not just as a linguistic phenomenon, but a more general cognitive phenomenon — they argue that all of these actually reduce to the one maximum: be relevant.
Ok, so manner: What is it to be helpful to somebody? Present the information in a way that's helpful. What you do is you try and make salient and obvious what is relevant. Ok, that was easy!
Quantity: Give the relevant amount of information. What about this? And you say, "John, I got ya! You can't reduce this one, because this is truth. And you have been hitting me over the head since the beginning of this series that truth and relevance are not the same thing." You're right.
What does Sperber and Wilson do about that? They do something really interesting: They say we don't actually demand that people speak the truth. Because if we did, we're screwed, because most of our beliefs are false. What are we actually asking people to do? We're asking people to be honest or sincere. That's not the same thing. You're allowed to say what you believe to be true, not what is true. "Ok, so what?" you say. That means the maxim is actually "be sincere". What does sincere mean? Convey what's in your mind. Everything that's in your mind? Everything that's going on?
When you asked me, "How many kids do you have?", I've got all this stuff going on in my mind. About this marriage is failing. What am I going to do to take care of these kids? I love this kid, but it's… Do I convey all of that? No! If have you been trapped with somebody that's getting drunk and they talk like that at a party, it’s horrible. You're trapped. If you say, "Do you have any kids?" And you're trapped for three hours. That's not what we mean.
We don't mean, “Tell me everything that's in your mind right now. Convey it all to me, John, give it to me all!” That's not what we mean. What do we mean?
We mean convey what is relevant to the conversation or context. Out of all of the possible implications and implicatures, zero in on those that you might think are relevant to me, our conversation and the context. So that also reduces to relevance. At the key of your ability to communicate is your ability to realize relevant information.
Notice what I'm doing here: I'm doing this huge convergence argument again, and again, and again. What's at the core of your intelligence? What's at the core of your intelligence, again, and again, and again, is your capacity for relevance realization.
Mirror of the Self is a weekly newsletter series trying to explain the connection between creators and their creations, and analyze the process of crafting beautiful objects, products, and art. Using recent works of cognitive scientist John Vervaeke and design theorist Christopher Alexander, we embark on a journey to find out what enables us to create meaningful things that inspire awe and wonder in the people that know, use, and love them.
If you are new to this series, start here: A secular definition of sacredness.
For an overview and synopsis of the first 13 articles, see: Previously… — A Recap.