(And the untold story of Jane, The Human GPU¹)

Could a computer ever have a mind of its own? Philosophers, cognitive scientists, and computer scientists disagree about whether a machine could ever exhibit intentionality or have real mental states. Beliefs about artificial intelligence generally fall into two categories: weak AI and strong AI. Programs that exhibit weak AI can produce human-like answers for a single problem. In 2019, weak AI programs are everywhere in our day-to-day lives, processing speech into text, refining our search results, and powering the perception systems of self-driving cars. Strong AI programs are not just programs that behave like minds in specific situations; strong AI programs are minds. In 2019, strong AI does not exist. Many people think that it can never exist. A group of thinkers called Connectionists think that a class of programs called artificial neural networks could produce strong AI. I will argue in this paper that although an artificial neural network can exhibit extraordinary complexity, it never truly understands the symbols it processes, and therefore could never “think” like a mind.

One famous case against strong AI was proposed by philosopher John Searle in 1980. In his “Chinese Room” argument, Searle explained why such strong AI could never emerge from a computer program. The Chinese Room thought experiment goes like this: An English-speaking person, let’s call him John, is locked in an empty room with one large book. The room has two openings: one for receiving mail and one for delivering it. The book contains instructions in English on how to process each “input” character in Chinese and which “output” character to record. When a story written in Chinese is passed through the door, John looks up its characters in the book, follows the proper English instructions, and copies the corresponding Chinese characters onto a sheet of paper. He passes the paper through the other door. He can repeat this process for any Chinese input, of any length. He always produces output that is cogent, grammatically correct Chinese. Yet John doesn’t speak a word of Chinese.

The Chinese Room argument rests on the axiom that syntax can never produce semantics. Traditional computer programs manipulate only formal symbols, so they only know syntax. To be a mind, a program must make the jump from syntax to semantics: the program has to actually understand what the underlying symbols mean. Programs process symbols, but never know what they mean. Thus, a program could never contain a mind or be a mind.

Recent research has suggested that it may be possible for semantics to emerge from syntax. The rise of neural networks has produced results that increasingly suggest that a machine can, in fact, have a mind. This is the basis for Churchland’s “Connectionist Reply” to Searle’s Chinese Room. The Connectionist Reply agrees that a traditional program based on symbol manipulation cannot produce a mind, but claims that a new type of connectionist networks called an artificial neural network can.

Artificial neural networks follow a model inspired by the human brain. Each input passes through a system of layers of intertwined neurons. Artificial neural networks “learn” through a training process that iteratively nudges the parameters of each neuron until the system finds the best possible representation for some data. After training, the internal structure of the network contains information. It is this structure that allows artificial neural networks to generalize well, even to data that they have not seen before.

Connectionists argue that it’s these internal representations that contain meaning. A vast parallel network arranged and primed in the proper way can create semantics. According to a connectionist, classical programs can’t produce minds, but that is because they’re too simple. Connectionist programs, on the other hand, have the breadth, depth, and structural complexity required to build a mind. In fact, the brain is an example of a connectionist program. (It’s worth noting that the brain is the only connectionist program we know of that produces strong AI.)

Searle was not persuaded that connectionism can lead to strong AI. He countered with a new thought experiment: Expand the Chinese Room from one person to the size of mainland China. In it, place several billion English speakers, so each person can correspond one-to-one with a neuron from the human brain. Then teach each person to examine their inputs and pass along the proper outputs, just as a neuron would do in the brain. Properly organized and executed, this massive system can produce the same signals that propagate through the human brain (although at an appallingly low speed). So, although none of the people involved speak Chinese, this network can “read” and respond in perfect Chinese. But, as Searle points out, none of the humans involved understand Chinese. And it’s nonsensical to say that the entire mainland of China understands English. Thus, he says, connectionist networks cannot really understand English.

I don’t think that Searle’s amended thought experiment provides a strong basis for any believer in connectionism to change their mind. The meat of the connectionist theory is that although formal symbols alone are not enough to produce a mind, high-dimensional representations are. Someone who believes this will probably believe that this bizarre network Searle proposes is a mind; it requires far more people than exist on Earth, would be impossible to orchestrate, and would be unreasonably slow, yet it is a mind. After all, this system would contain all the same high-level meaningful representations as any connectionist network.

Although I think Searle’s response is flawed, I do not agree with the original Connectionist Reply. Connectionism implies that the only real barrier to writing a program that is a mind is complexity. But Searle’s Chinese Room does not stipulate the underlying program must be simple. In the Chinese Room experiment, Searle executes a “program” by performing a series of steps that he finds in a book. Connectionism implies that this program somehow could never be complex enough to constitute a mind, but a connectionist program could be.

Let’s discuss in brief what an artificial neural network actually does. The output of an artificial neural network is just the last thing of a long series of mathematical operations, adding and multiplying matrices full of numbers. The “neurons” in connectionism are just the weights of the matrices involved. This model– reducing the entire program to a series of matrix operations– allows connectionist programs (with enough matrices, or “layers” of neurons) to model data of arbitrary complexity. Connectionists think that this complexity is enough to match that of our own brain.

An artificial neural network takes in a list of Chinese characters, does a lot of math, and converts some output numbers back to Chinese characters. John’s job in the Chinese Room is to examine list of symbols, do some reading and some thinking, and write down another list of symbols. And the “program” that John executes is just some sequence of steps that begins with a sequence of Chinese characters and ends with a sequence of Chinese characters. It can be any program at all. The Chinese Room argument does not hinge on the stipulation that John execute a classical program that manipulates nothing but formal symbols.

Let’s consider a third thought experiment, also a modification of the original Chinese Room. Jane sits in an empty room with one “in” mailbox and one “out” mailbox. She sits in front of a book which contains three (quite long and boring) chapters. Jane’s book contains all the information necessary for her to do the work of an artificial neural network by hand. The book’s first chapter is a manual of operations: it details every step Jane must follow to transform input into output. The second chapter is a sort of table of contents: it contains every character in Chinese next to its unique ID number. The third chapter contains pages and pages of matrices. These matrices contain the parameters that define the network’s neurons, in the connectionist sense. If representations really can produce semantics, then it is in these pages that the semantics lie.

Jane’s job is a little more complicated than John’s. She doesn’t just flip through the pages of a book; she has to do quite a bit of math. Every time a new input comes along, Jane converts each Chinese character to the proper ID number, then performs a complicated series of multiplication and addition operations. This is grueling work, and she gets some serious practice in arithmetic in the meantime. But when she finishes, she ends up with one final list of numbers that she can easily convert back to Chinese (by consulting Chapter 2). For every input sequence, Jane can produce the exact same output as an artificial neural network.

After performing this process many times, Jane will probably memorize the mappings of Chinese characters to ID numbers. But she will certainly not speak any Chinese. Jane can perform all of the same functionality as a Chinese-speaking artificial neural network for as long as she wants, but she won’t learn a lick of Chinese. This shows that connectionist programs cannot “find a way” out of the Chinese Room.

There is nothing magical about an artificial neural network. It cannot manufacture semantics when it knows nothing but syntax. At the end of the (computationally intensive training process) day, all that happens inside an artificial network is a lot of addition and a lot of multiplication. An artificial neural network may be a very sophisticated program, but it still is a program. And programs– even exceedingly complex programs like connectionist ones– still produce syntax, from syntax. Therefore, the Connectionist Reply does not challenge axiom 3 of the Chinese Room. Even artificial neural networks cannot turn syntax into semantics.

This essay was written for “Minds, Machines, In Persons” class, taught by Prof. Zachary C. Irving, in Fall 2019.

Footnotes