This is the first in a series of posts about the relationship between language and the world, what machines can tell us about the structure of language, and the progress of science. While I’m further developing themes I touched on during my PhD, these posts are mostly thinking out loud — I’m not entirely sure where they’re going yet.

tl;dr for this post: We consider a thought experiment in which:

1. (Human-analogous) understanding is not necessary for linguistic competence
2. This is because knowledge of language can substitute for knowledge of the world
3. So a failure of linguistic competence is explainable without a lack of understanding
4. Understanding is required for competence in the world, but not every speaker is competent in every world

Bender & Koller (2020)¹ sets up the Octopus Test to demonstrate that the meaning of language cannot be learned from form alone. Let’s put aside the fact that Bender & Koller were writing in 2020 about language models much less capable than today’s, and that Bender has since clarified that even on their definition machine understanding is in fact possible: “when you ask a virtual assistant to set a timer, turn on a light, etc and it does, it has understood”². Instead, I’m going to develop their example further to distinguish ways that understanding of the world and knowledge of language can inform linguistic competence.

In the Octopus Test, two human interlocutors (A and B) are stranded on separate desert islands, and communicate via an underwater telegraph. An octopus (O), who is intelligent but knows nothing of the surface world, can tap into the telegraph line to hear their conversations, and ultimately can send its own signals pretending to be B.

Bender & Koller’s mashup between the Turing Test and a cryptography demonstration asks: under what circumstances will O be able to pretend to be B? Whatever they turn out to be, those are the circumstances under which meaning can be learned from the form of language alone. Conversely, if O cannot pretend to be B, then they have failed the test, which would demonstrate that knowledge of linguistic form is not sufficient to achieve linguistic competence.

It’s worth being specific about the two aspects of language use here:

• O is linguistically competent if O can pose as B without making A suspicious: their language is accepted by another speaker
• O produces meaning or has (human-analogous) understanding when their speech relates to communicative intents in the same way that B’s does. Here, communicative intents are some entity outside of language (e.g. in the external world or in the interlocutor’s heads). Without committing to the correctness or relevance of human-analogous understanding, I will follow Bender & Koller in just calling this understanding.

The authors claim that in fact, O “would fail a sufficiently sensitive test, because it lacks the ability to connect its utterances to the world”¹. In other words:

a. there is some circumstance under which O would not be linguistically competent b. this is because O does not have understanding.

When does an Octopus Test exist?

The authors don’t claim that O will always be unable to fool A. They grant that while O will be able to share social niceties about the day by imitating B’s messages, they will not be able to advise A on how to build a catapult or escape a bear. These latter require knowledge about catapults and bears which is only accessible from the world, not merely from language. It’s not that O can never fool A but, “O’s language use will eventually diverge from the language use of an agent who can ground their language.” But when exactly?

The Isles of Fruit

To illustrate where competence and understanding might diverge, let’s develop an even more impoverished environment for A and B. Rather than having a full-keyboard telegraph with which to communicate, they only have a Baby’s First Telegraph, in which messages consist of exactly two words: the name of a color and the name of a fruit.

red apple
green grape
yellow lemon

Fittingly, the islands A and B are stranded on are the twin Isles of Fruit. These two islands are identical in every way but the only vegetation (in fact the only objects at all) they contain are these:

red apple
green grape
yellow lemon

A and B are desperate for human connection and so despite the limitation of their devices, they want to understand what situation the other is in — or perhaps offer help. So they nevertheless send each other a series of messages:

A > B: red apple
A > B: green grape
B > A: red apple
B > A: green grape
A > B: yellow lemon
B > A: yellow lemon

A and B, as good language-users, have managed to send only messages which are both grammatically correct (naming exactly one color and exactly one fruit), and true of the Isles of Fruit world (only mentioning fruits in colors which exist on the Isles). Given the limitations of their world and their keyboards, they have in fact sent the only three true messages that are possible to send.

What happens to O in this situation? Clearly, by memorizing and repeating this small corpus, O is linguistically competent and is indistinguishable to A from B. In fact, O is perfectly competent: there is no true phrase they cannot say and no false phrase they will say.

Unlike in B&K’s social niceties example, the messages A and B send are about the world, and presumably they have understanding of which fruits they’re talking about. O does not have this understanding, so can achieve linguistic competence without understanding.

An Octopus Could Extrapolate

Of course, real languages are unlike this in many ways. Perhaps most saliently, linguistic corpora never contain all possible utterances in a language. More strongly, there are utterances in languages which are grammatically correct and true, but have never occurred in the world. A linguistically competent speaker must be able to say things that have never been said before.

Let’s see what happens when the corpus is smaller than the language. Say that Baby’s First Telegraph was in fact designed for very precocious children: instead of having only one key for each color, it has keys for any number of synonyms for the color terms. In addition, the Isles of Fruit in fact contain multiple fruits of the same color³:

red apple
red strawberry
green grape
green pear
yellow lemon

A and B can now have a much richer conversation:

A > B: green pear
B > A: green grape
...
A > B: red apple
B > A: vermillion apple
A > B: red strawberry

To be competent in this setting, O must be able to send a correct message they have not seen before such as vermillion strawberry. Could they learn to do this?

Very plausibly, yes: O could represent the corpus with a graph by connecting a color and a fruit whenever they have occurred together in a message:

In this diagram, the square nodes are color words which have appeared in the message corpus, the round nodes are fruit words. Solid lines denote pairs of words that have appeared in the corpus, and the dashed line is one that O could infer. O could impute the dashed line by learning the rule: whenever two colors have been used with the same fruit, they can be used with all the same fruits (exercising closure on each connected component of the graph). So because “red” has been used with both “strawberry” and “apple”, and “apple” has been used with “vermillion”, “vermillion” can also be used with “strawberry”, and O could say vermillion strawberry

So O does not need to just parrot existing messages, they can be competent in sending messages that have never been sent before, still without any direct access to the world.

What the Octopus Could Not Say

But there’s still a sense in which O is just repeating things they’ve seen before: they can never be the first to use a word from the telegraph keyboard, because such a word would not be connected to any other in their graph.

Of course, if B notices a new fruit on their Isle:

yellow papaya

B could be the first to send a message about it to A. O will never (confidently) do this, if O tried to use a word they haven’t seen there’s a good chance they would send a message that is not true about the world, like purple papaya. Either way, A can distinguish O from B.

Even with access only to this restricted world, as B&K put it, A and B can “constantly generate new communicative intents” but O cannot. Because the communicative intents precede the corpus, they will always contain more than the corpus and so the Octopus Test appears in messages about new things in the world.

Settling the score

But is this fair to O? In this example, the different participants have access to different resources: O is new to language, and has no access to the Isles, A and B are new to the Isles but have plenty of prior experience of both the world and of language. But this need not be the case: suppose A and B are not the first castaways O has observed. O could have a language corpus that includes messages from the prior castaways’ too.⁴ In this case, even before A and B have discovered what fruit is on their island, O could start to mention them. Similarly, O could make use of color words already in their graph. In this case, O becomes once more indistinguishable from B.

It might be argued that what makes O special is the fact that they only have this indirect access to the world via language — they rely on someone having been on the world of the Islands. But we can make the setting even more symmetrical. Maybe A’s Isle is full of fruit, but B’s is truly deserted. In that case, if B wants to send true messages, they can only repeat ones that A has sent before (or ones that can be inferred through color-synonymy from previous messages). In this case, not only does B only have indirect access but in terms of competence, O can use terms before B: and so A has more reason to suspect B than to suspect O.⁵

Where does this leave us? First, we’ve seen that in general O can be linguistically competent with no access to the world. And it seems that the larger O’s access to a linguistic corpus, the more likely they are to be competent: direct access to the world can be replaced with access to a corpus. And this corpus need not be complete, as long as the language has sufficient structure to permit some inferences.

Moreover, A’s seeming ability to judge O as linguistically incompetent is not because of O’s lack of understanding but because of A’s background knowledge about A and B’s situation (that A and B are on identical islands). Even in B&K’s original setting with a less limited telegraph, failing to be able to help with the bear is only a test on the assumption that B knows enough about bears (and what the word “bear” means).

On the Isles of Fruit, there is no Octopus Test. I’ll return in a later post to whether this is likely to be true of real human languages in the real world, but for now, let’s consider the second part of B&K’s claim: O’s failures to be linguistically competent are because they fail to understand.

Diagnosing failures

So far, we’ve seen that B could send messages that O cannot by drawing on the connection between words and the world. Let’s consider a specific example: Let’s say that A and B happen to have used non-overlapping sets of terms to describe two green fruits:

In this case, O has no way to know that they could say green pear, even though they have seen both words used, because there is no path through the network from “pear” to “green”.

But B is not similarly unable to send green pear. Why is this? It could be the case that there is also a green pear on B’s Isle:

Here, the added nodes show these aspects of the world that are available to B but not O: colors in cubes and fruits in cylinders. B can say green pear by drawing on the green pear in the world and their understanding of the relationship between this and the words “green” and “pear”.

There’s another way B could figure this out: even if they haven’t seen the pear (perhaps they’re even on the desert Isle). If A has described the pear as “jade”, B can use their prior knowledge of the terms’ synonymity to describe the pear as “green”:

The bold line shows a link that B has access to because of their knowledge of language outside of the messages.

If O fails to send green pear, it’s because their model is not like B’s. But we can either explain this failure as a lack of understanding or a lack of a big enough corpus (B’s prior knowledge of language operates just like O’s exposure to prior telegrams). So when O does fail, it cannot be exclusively attributed to their lack of understanding.

Competence in the World

Whatever their degree of linguistic competence, there is one capacity that O still unarguably lacks: the ability to point out named objects in the world:

“O has never observed these objects, and thus would not be able to pick out the referent of a word when presented with a set of (physical) alternatives.”¹

This situation is the most analogous to the “more constrained thought experiments” B&K later offer, like a model of language being exposed to photographs for the first time. Even if understanding is not required for linguistic competence, surely it’s required for competence in the world?

But consider what is being counted as a world “outside of language” here. O does in fact have non-linguistic correlates of the language: the signals in the electrical cable themselves. If B was brought under water, they would be equally incapable of knowing what pattern of electrical current was associated with “strawberry”. So again, B and O are in symmetrical positions, each incompetent in the others’ world.

Perhaps this is a limitation of the thought experiment. For B&K, an electrical signal is form rather than intent because it is an “observable realization of language” and so O is supposed to be understood as having truly no extralinguistic referent for the messages. This could be so, but there are still other worlds at play. Perhaps A is a super-taster and can distinguish by taste an Allstar strawberry from an Alice strawberry. If B cannot do this, they also cannot pick out the referent (in the world of taste) for the word. Does B then lack understanding?

The required distinction between form and intent is also tenuous. If O can hear A and B speaking by means of a talking tube, that must be form. But maybe O’s hearing is acute enough to echolocate which fruits A and B are looking at, in which case they have access to the world too. Maybe A and B communicate by writing on the skin of the fruits and floating them between their islands. On the other hand, what if (with a real telegraph), A mentions “the message I sent yesterday”. The referent of this phrase must either be the electrical signal in the telegraph or the linguistic message itself. In the first case, electrical signals are intent again. In the latter then, not only can communicative intent can also be something inside language but in O would have exactly the same access to the referent as B does because both observed messages.

In these circumstances, the communicative intents behind the words are at the very least multiple, and may only have partial overlap for A and B. While it’s true that O cannot pick out the referent of words in A and B’s world, claiming that this is a distinguishing failure on their part requires privileging the (visual) worlds of the Isles over other worlds that contain other referents. In general no speaker is expected to pick out the referent of a word in every world.

The Structure of the Language and the Structure of the World

Therefore, on the Isles of Fruit, form-only learning of language is possible. The only role of reference is to establish the correspondence between a language and some part of some world. Of course, this is a thought experiment with a toy environment and toy language, and we can’t directly claim these conclusions are true of real languages. The Isles do give us a restatement of the problem: if form-only learning of natural languages is not possible, what is the critical way in which they differ from the Isles of Fruit?

I’m not going to address this directly, because I think this thought experiment leads to some more interesting questions. When they send messages, A and B are talking about the world. And as we saw, For O to fool A while learning from a finite corpus, the language must have some extrapolatable structure. But in a natural language, what is this structure? And does its structure encode the structure of the world?