MUD Bot passes the Turing Test?

[squian]

In a 1950 article titled Computing Machinery and Intelligence (http://cogprints.ecs.soton.ac.uk/archive/00000499/00/turing.html), Alan Turing, a brilliant mathematician and pioneer in Computer Science, asked, "Can machines think?" To answer the question Turing proposed "The Imitation Game" -- if a computer can imitate a human during a blind interrogation session, then we can consider that machine capable of thought. It's an interesting challenge that has become an annual event. Every year, the Loebner Prize (http://www.loebner.net/Prizef/loebner-prize.html) of $2000 is granted to the most human computer. One day, they may even award $100,000 and a gold medal if a computer can completely fool the prestigious judges.

Although I await that day with the excitement only a computer geek could truly appreciate, I have often wondered if the prestigious judges are too stringent -- what if average people were unknowingly subjected to the Turing Test? As it turns out, the pervasiveness of computing and the Internet seems to provide boundless opportunities to "fool" the public with computers that act like humans. That said, I never imagined how amusing it might be to watch a person try to discern human from machine. See this interaction with Eliza, a MUD bot. (http://dikumud.com/Eliza/Tanu.aspx)

[viscousmemories]

Oh man that's some desperately funny shit. I think I peed my pants reading that.

Tanu :: follow elisabeth
Tanu :: say now what
Elisabeth :: say why now?
Tanu :: say i'm following
Elisabeth :: say my uncle was once following in 1972, so what?
Tanu :: say why are you following
Elisabeth :: say all normal people are following
Tanu :: say you talk about normality i think you to are normal
Elisabeth :: say Oh, I talk about normality you think me to am normal !
Tanu :: say you use aliases to type that fast don't you! [no, he has dropped the idea its a program once more]
Elisabeth :: say follow me
Tanu :: nod
Tanu :: say following
Elisabeth :: say i think you are sick, leave me alone

:roflmao:

[davidm]

The link is down for me. It does look funny, though, based on the transcript.

The philosopher John Searle has a powerful argument against the Turing Test. Searle says his argument shows it is impossible for any digital computer ever to understand language or think. This would make the Turning Test invalid. It has to do with what's really going on inside the computer. :gears:

The Chinese Room Argument (http://plato.stanford.edu/entries/chinese-room/)

From the above link:

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.
:break:

Gawd, I love these smilies.

[ceptimus]

I always thought the 'Chinese room' argument was ridiculous.

Imagine a huge bunch of neurons, these neurons are all connected together in an incredibly complex way. Some collections of neurons store symbols that represent words, and other collections of neurons store rules about how the word symbols are to be manipulated. None of the individual neurons, or even groups of neurons understand the symbols or what the rules mean.

What I asked you to imagine is, of course, a human brain. How can the human brain ever understand English? This is exactly the same argument as the 'Chinese room' one, except that the non-Chinese-comprehending human and his rule book are replaced by non-English-comprehending neurons.

If the room and the rule book were good enough, then the room would understand Chinese (albeit very slowly) even though the human inside the room wouldn't know what was going on. This is analogous to the whole human brain understanding English, even though individual neurons, or collections of neurons are clueless.

[JoeP]

The Chinese Room argument is hotly debated. The article you linked does mention what to me is its fatal flaw: he shows that the "room" would speak chinese while the man inside it doesn't; but the man is not the whole of the room. The argument proves nothing about the room-as-a-whole understanding Chinese, unless you make assumptions about the non-human parts of it necessarily being incapable of understanding - which would make it a circular argument.

The same approach could prove the existence of the soul (as a non-physical, indivisible seat of consciousness and identity), since if you take any bit of the brain in isolation it clearly doesn't have consciousness, free will, morality, personality or any of those things that make a "mind".

Daniel Dennett's and Stephen Pinker's works on consciousness and language are much more appealing to me.

[Farren]

The Chinese room is the most ill-devised and obviously wrong gedanken experiments I've ever had the misfortune to encounter.

In the first place, it posits a man in a room with a book to demonstrate it's point, then procedes to use the man's lack of understanding of Chinese to illustrate that a system can be devised whereby something can respond, but not understand.

But if the Chinese room is an analogy to an AI, the man, the room, and the book containing the algorithm for responding is the AI, not the man. i.e. Focussing on the man not understanding Chinese is like focussing on the amygdala or Corpus Collosum of a Chinese person not understanding Chinese, which, while true, doesn't indicate that the Chinese person in question doesn't understand Chinese.

In Searles gedanken experiment, the man uses an algorithm, written in a book, to respond. The algorithm doesn't allow the man to translate from Chinese to his native language, process the information, then translate back. It facilitates him generating his entire response without any understanding of what was said. This is important to understanding just how crappy the argument is.

In order for the book to contain an algorithm which could achieve this astonishng task the algorithm would have to be truly vast. Words and phrases in Chinese are contextual. Also the vast majority of communication is concerned with information that is outside of language, such as how the communicator feels and the current political, social and physical state of the world. Thus the algorithm would have to encode, in some way all of the Chinese culture and history that is readily available to the thinking of a contemporary Chinese person, as well as all kinds of knowledge about the real world.

It might be argued that in areas like logics, things can be discussed that are self-contained. Limited sets of propositions can be examined and discussed for internal self-consistency without necessity for those things to represent common knowledge or real world values. But in such areas a strictly logical device can be surplanted for a human in any event, so those specific discussions cannot be claimed as evidence of intelligence in humans. What comes to mind is idiot savants who can perform seemingly impossible tasks of computation without any understanding of the meaning or significance of what they are computing.

Thus, if the Chinese room argument is to be used as an analogy for human intelligence and understanding, it must make the former assumption (that the algorithm in the book must encode all of the Chinese culture, history , knowledge of the real world and knowledge of human feelings that is readily available to the thinking of a contemporary Chinese person).

In programming terms (if the book describes an algorithm) the simple assumption would be that in order to store this vast contextual knowledge, a massive proportion of the book would have to be look-up tables, like a database, but, since among the looking up needing to be done, sub-processes must also be looked up, the line between data and process would be blurred, as it is in both real and computerised models of neural networks. In fact this line does not exist in real computing (all turing machines, which modern computers are, are just streams of numbers, where the numbers themselves determine which are instructions and which data).

Furthermore, for the experiment to work, the man must have a means of transcribing and storing the results of previous interactions that would affect the following interactions, else conversations like these would be impossible:

A: Have you met Bob?
B: Yes
A: What do you think of him?
...

Which means that the book must have the facility to acquire new data. But it doesn't stop there. Human beings can also communicate algrorithms and this is essential to emulating human understanding as proposed in the Chinese Room gedanken experiment. For instance, I can communicate the following:

A: If a person is the mother of a second person and the second person is the wife of a third, the first is the mother in law of the third.
B: OK
A: If a person is the mother of a second person and the second person is female, the second person is their daughter
B: OK
A: If a person is the wife of a second person, it implies the first person is female
B: OK
A: If Mary is John's wife and Mary is Sue's daughter, is Sue John's mother in law?
B: (Thinks) Yes

Which means that the Chinese room must have a facility whereby new algorithms, ways of making choices and selecting data, are required. Furthermore it must have instructions for changing the existing instructions in the book.

A third critical requirement of the situation facilitating adequate emulation of intelligent communication with a Chinese person is that the entire interplay of internal actors must happen astonishingly fast. If the system were envisioned working at the speed a man normally flipping through such a gigantic book would occur, the external agent would wait a lifetime for one answer.

In fact the man would have to be flipping pages at a speed close to or beyond light speed to emulate the processing of the electro-chemical parallel network in our heads. Someone with a little knowledge of neurology might point out that signals between neurons are substantially less than electron speed but when you include parallelism in the equation the net effect is substantially faster than a linear processor (which the human is) working at electron speeds.

So now you have a situation where there's this blur of a man, moving at lightspeed, reading, flipping, reading, altering, reading the altered text, making another choice when same situation occurs a second or third time and so on. The book has become , in real time a living, amorphous, constantly changing affair and the man has become a signal speeding through it.

I wrote that last visualisation out explicitly because, the immediate and obvious failure of Searles argument - seeing one component not the entire room and all its contents as the AI - is difficult for people because they see the book as a dry, dead thing. But in order for the actual experiment to work, the book would have to take on all the behavioural characteristics of livng tissue like the brain and the man would take on the speed and mindlessness of the electrochemical signals that race around it.

In this light, its perfectly reasonable and easy to see that the Chinese Room, not the man, would constitute a human-like intelligence.

Searle's argument is one of the crappiest challenges to Strong AI I've ever seen and it continues to astonish me that luminaries like Penrose (who quoted it in "The Emperor's new Mind" then again in "Shadows of the Mind") continue to endorse it and use it as an argument.

[edit]
I see Joe and Ceptimus got there before me.

[beyelzu]

The Chinese room is the most ill-devised and obviously wrong gedanken experiments I've ever had the misfortune to encounter.

In the first place, it posits a man in a room with a book to demonstrate it's point, then procedes to use the man's lack of understanding of Chinese to illustrate that a system can be devised whereby something can respond, but not understand.

But if the Chinese room is an analogy to an AI, the man, the room, and the book containing the algorithm for responding is the AI, not the man. i.e. Focussing on the man not understanding Chinese is like focussing on the amygdala or Corpus Collosum of a Chinese person not understanding Chinese, which, while true, doesn't indicate that the Chinese person in question doesn't understand Chinese.

In Searles gedanken experiment, the man uses an algorithm, written in a book, to respond. The algorithm doesn't allow the man to translate from Chinese to his native language, process the information, then translate back. It facilitates him generating his entire response without any understanding of what was said. This is important to understanding just how crappy the argument is, too.

In order for the book to contain an algorithm which could achieve this astonishng task the algorithm would have to be truly vast. Words and phrases in Chinese are contextual. Also the vast majority of communication is concerned with information that is outside of language, such as how the communicator feels and the current political, social and physical state of the world. Thus the algorithm would have to encode, in some way all of the Chinese culture and history that is readily available to the thinking of a contemporary Chinese person, as well as all kinds of knowledge about the real world.

It might be argued that in areas like logics, things can be discussed that are self-contained. Limited sets of propositions can be examined and discussed for internal self-consistency without necessity for those things to represent common knowledge or real world values. But in such areas a strictly logical device can be surplanted for a human in any event, so those specific discussions cannot be claimed as evidence of intelligence in humans. What comes to mind is idiot savants who can perform seemingly impossible tasks of computation without any understanding of the meaning or significance of what they are computing.

Thus, if the Chinese room argument is to be used as an analogy for human intelligence and understanding, it must make the former assumption (that the algorithm in the book must encode all of the Chinese culture, history , knowledge of the real world and knowledge of human feelings that is readily available to the thinking of a contemporary Chinese person).

In programming terms (if the book describes an algorithm) the simple assumption would be that in order to store this vast contextual knowledge, a massive proportion of the book would have to be look-up tables, like a database, but, since among the looking up needing to be done, sub-processes must also be looked up, the line between data and process would be blurred, as it is in both real and computerised models of neural networks. In fact this line does not exist in real computing (all turing machines, which modern computers are, are just streams of numbers, where the numbers themselves determine which are instructions and which data).

Furthermore, for the experiment to work, the man must have a means of transcribing and storing the results of previous interactions that would affect the following interactions, else conversations like these would be impossible:

A: Have you met Bob?
B: Yes
A: What do you think of him?
...

Which means that the book must have the facility to acquire new data. But it doesn't stop there. Human beings can also communicate algrorithms and this is essential to emulating human understanding as proposed in the Chinese Room gedanken experiment. For instance, I can communicate the following:

A: If a person is the mother of a second person and the second person is the wife of a third, the first is the mother in law of the third.
B: OK
A: If a person is the mother of a second person and the second person is female, the second person is their daughter
B: OK
A: If a person is the wife of a second person, it implies the first person is female
B: OK
A: If Mary is John's wife and Mary is Sue's daughter, is Sue John's mother in law?
B: (Thinks) Yes

Which means that the Chinese room must have a facility whereby new algorithms, ways of making choices and selecting data, are required. Furthermore it must have instructions for changing the existing instructions in the book.

A third critical requirement of the situation facilitating adequate emulation of intelligent communication with a Chinese person is that the entire interplay of internal actors must happen astonishingly fast. If the system were envisioned working at the speed a man normally flipping through such a gigantic book would occur, the external agent would wait a lifetime for one answer.

In fact the man would have to be flipping pages at a speed close to or beyond light speed to emulate the processing of the electro-chemical parallel network in our heads. Someone with a little knowledge of neurology might point out that signals between neurons are substantially less than electron speed but when you include parallelism in the equation the net effect is substantially faster than a linear processor (which the human is) working at electron speeds.

So now you have a situation where there's this blur of a man, moving at lightspeed, reading, flipping, reading, altering, reading the altered text, making another choice when same situation occurs a second or third time and so on. The book has become , in real time a living, amorphous, constantly changing affair and the man has become a signal speeding through it.

I wrote that last visualisation out explicitly because, the immediate and obvious failure of Searles argument - seeing one component not the entire room and all its contents as the AI - is difficult for people because they see the book as a dry, dead thing. But in order for the actual experiment to work, the book would have to take on all the behavioural characteristics of livng tissue like the brain and the man would take on the speed and mindlessness of the electrochemical signals that race around it.

In this light, its perfectly reasonable and easy to see that the Chinese Room, not the man, would constitute a human-like intelligence.

Searle's argument is one of the crappiest challenges to Strong AI I've ever seen and it continues to astonish me that luminaries like Penrose (who quoted it in "The Emperor's new Mind" then again in "Shadows of the Mind") continue to endorse it and use it as an argument.

[edit]
I see Joe and Ceptimus got there before me.:bow:


great post at first the chinese room argument stuck me as making since. I didnt see the obvious logical fallacy. I wish Penrose and Searle could see it and have to argue with you about it.


most fucking impressive.

[squian]

This is the first time I have heard the Chinese Room as a counter-argument to the Turing Test. For that matter, I have always thought of the Turing Test as a prescriptive test. That is to say, the Turing Test does nothing to "prove" machines can think or even illuminate the concept of "thinking". Instead, Alan was suggesting anything that exhibits capabilities good enough to fool a human ought to be considered "thinking", regardless of the mechanisms.

I have always known the Chinese Room as a paradox. How can we say the "room" can think but neither the man nor the book can? Sure seems paradoxical to me so I seem to have missed the obvious flaw.

If we simply dismiss the paradox and agree the room, as a system, is able to think, we have ruled out the man as the agent in the system doing the "thinking". If the man represents a machine and the book an algorithm, it seems like a fairly good argument that a system might exhibit human-fooling behavior without having a "thinking machine". "Thinking" is only possible with the system of machine and algorithm. Which does not seem to be a disproof of Strong AI anyhow. Only that an AI must be a combination of machine and algorithm. So, too, the human mind is a combination of neurons firing (as an execution engine) and the information they hold (as connections between them).

Whatever the implications for "thinking machines", there is one clear conclusion for me, "It is impossible to separate thought from matter that thinks."

[Farren]

This is the first time I have heard the Chinese Room as a counter-argument to the Turing Test. For that matter, I have always thought of the Turing Test as a prescriptive test. That is to say, the Turing Test does nothing to "prove" machines can think or even illuminate the concept of "thinking". Instead, Alan was suggesting anything that exhibits capabilities good enough to fool a human ought to be considered "thinking", regardless of the mechanisms.

I have always known the Chinese Room as a paradox. How can we say the "room" can think but neither the man nor the book can? Sure seems paradoxical to me so I seem to have missed the obvious flaw.

If we simply dismiss the paradox and agree the room, as a system, is able to think, we have ruled out the man as the agent in the system doing the "thinking". If the man represents a machine and the book an algorithm, it seems like a fairly good argument that a system might exhibit human-fooling behavior without having a "thinking machine". "Thinking" is only possible with the system of machine and algorithm. Which does not seem to be a disproof of Strong AI anyhow. Only that an AI must be a combination of machine and algorithm. So, too, the human mind is a combination of neurons firing (as an execution engine) and the information they hold (as connections between them).

Whatever the implications for "thinking machines", there is one clear conclusion for me, "It is impossible to separate thought from matter that thinks."

Squian

Read Penrose's excellent "The Emperor's New Mind" which, but for ocassional lapses into the ridiculous (such as quoting Searle) makes a strong (but not conclusive) case for Strong AI only being achievable through quantum-effect machines as, possibly humans are.

[squian]

Read Penrose's excellent "The Emperor's New Mind" which, but for ocassional lapses into the ridiculous (such as quoting Searle) makes a strong (but not conclusive) case for Strong AI only being achievable through quantum-effect machines as, possibly humans are.

I think Penrose presents something similar (if not the same) about "quantum-effect machines" in The Large, the Small and the Human Mind.

Did you find Penrose ridiculous for his own reasons or because he quoted Searle?

[ceptimus]

Excellent post Farren. :bow: :)

Penrose's appeal to 'quantum stuff' as an explanation of consciousness seems weak. It amounts to "We don't really understand consciousness, and we don't understand quantum effects fully either, so the two things might be related somehow." Well duh!

All present science shows that for quantum effects to cohere long enough to provide computation requires an environment where particles can remain undisturbed. Such conditions are not thought to exist anywhere inside living brains.

Of course, Penrose is a genius, so he sees things we can't. Perhaps he will have the last laugh. But geniuses have gardened in areas outside their expertise before, usually with unimpressive results. Newton spent much longer on alchemy and searching for proofs in the bible, than he ever did on mechanics and mathematics.

[Farren]

I don't think Penrose makes a convincing case for AI at a human level requiring quantum effects, of necessity, but I enjoy his speculation because its informative and does suggest, at least, some possibilities.

I agree with you (ceptimus). I think Penrose is simply working on intuition and attempting to articulate that intuition in his books on AI without managing to form a convincing proof.

That said, I would never have known about microtubules in the cytoskeleton of prokaryotic cells if I hadn't read Penrose and it does offer a tantalising glimpse at the possibility of deeper levels of processing in the human brain, even if the majority of what we understand to be consciousness occurs because of the electrochemical behaviour its more widely ascribed to.

I'm leery, too, of even allowing that consciousness is a distinct, defineable thing. Mainly because of a great admiration for and subscription to many of the ideas of Buddhism and Daoism, I intuit that whatever qualities human beings have are simply expressions of qualities the matter of which we are made has. i.e. Consciousness is nothing more than qualia, the sense of being something as opposed to the observation of the thing - and every quanta of energy in the universe has that sense.

Further to that, human consciousness is simply "mysterious" to us for the reason quite cryptically but poetically described by Douglas Hofstader in Godel, Escher, Bach - that, when a thing examines itself, infinite recursion occurs.

Squian: I mainly found his use of Searle ridiculous, but I recall (its a long time since I read Shadows or its predecessor) finding one or two other bits a bit of a stretch. Nonetheless, his discussion of the cytoskeleton, non-recursive sets and many other issues were fascinating eye-openers.

I think the essence of the Turing Test remains valid, despite many criticisms I've read. A lot of criticism rests on the literal wording of the original, rather than the spirit of the thing. i.e. Arguing about the specific test proposed, while there are other tests that can be considered that keep the same core qualities. The point of the Turing test is that human consciousness is not a testable thing. It is an observed thing that can only be verified by other human beings.

The intuitive objection to that idea, which finds expression in many criticisms, is based on the idea that human beings are themselves poor observers. That even though our senses and judgement inform us sufficiently in most situations, they are easily fooled when deliberate deceptions are constructed with an understanding of how they can be fooled.

While this is fair comment, I think its obvious that the Turing Test is a fuzzy test, where the veracity of the results increases or decreases as more and more tests are done on the same subject.

[JoeP]

Read Penrose's excellent "The Emperor's New Mind" which, but for ocassional lapses into the ridiculous (such as quoting Searle) makes a strong (but not conclusive) case for Strong AI only being achievable through quantum-effect machines as, possibly humans are.
:blush: I've read The Emperor's New Mind ... and I can't remember it. Just a vague recollection that I found him brilliant but blind to serious possibilities. I think it was Arthur C Clarke who said "If an elderly but distinguished scientist says that something is impossible, it is almost certainly possible." The same applies to philosophers. They are serious prone to getting stuck into certain mindsets, acquiring blind spots, and so on - and it seems like a characteristic of aging.

Do you have anything to add to ceptimus's summary of how he justifies quantum mechanics being required? It's hardly worth rejecting.

Godel, Escher, Bach - now there's a good book. My Aunt Hilary enjoyed it too.

[Farren]

No I more or less agree with ceptimus inasmuch as Penrose doesn't make a strong case for quantum effects, of necessity, being required for consciousness. Nonetheless I think the weakness of his particular arguments don't conversely, rule out the possibility - and the book was an eye-opener as far as scientific titbits are concerned.

By the way Joe, you mentioned Dennit and Pinker? Got any titles of books worth reading?

[edit to add]
Sorry I see that there was a para of ceptimus post for some reason I didn't absorb - twice - talk about lack of attention.

The argument against quantum effects playing a role - that all of our attempts require containment, also don't strike me as particularly strong. They're premised on quantum computation of necessity functioning in the brain in the same way that as an artificial device would function.

Its quite conceivable that a quantum device could compute in an entirely different manner in a messy environment, especially when the computing isn't being done in digital bits but analogue values.

Is there a compelling reason computation must be discrete that I'm not aware of?

[ceptimus]

Its quite conceivable that a quantum device could compute in an entirely different manner in a messy environment, especially when the computing isn't being done in digital bits but analogue values.

Is there a compelling reason computation must be discrete that I'm not aware of?
Our current understanding of quantum phenomena is that the effects are all discrete - digital if you like. This is the whole idea of 'quantum' stuff, things happen in discrete packets, always of a certain size. This seems like the antithesis of an analogue system. But I sort of understand what you mean, and I admit the possibility that Penrose is right - I just don't find his arguments very compelling.

[Farren]

Its quite conceivable that a quantum device could compute in an entirely different manner in a messy environment, especially when the computing isn't being done in digital bits but analogue values.

Is there a compelling reason computation must be discrete that I'm not aware of?
Our current understanding of quantum phenomena is that the effects are all discrete - digital if you like. This is the whole idea of 'quantum' stuff, things happen in discrete packets, always of a certain size. This seems like the antithesis of an analogue system. But I sort of understand what you mean, and I admit the possibility that Penrose is right - I just don't find his arguments very compelling.

I think you're a little off base (on the discreteness thing - I agree Penrose doesn't make a compelling case). The discreteness in quantum mechanics is only in energy gain. The evolution of wave functions isn't discrete. As I understand it, the requirement for discreteness in artificial attempts at computation is due to a number of reasons which don't imply that computation can't happen in other ways.

One is containment of a specific number of quanta of energy, a quantum bit, or qubit. Another is the containment of a wave function used for quantum computation so that the specific wave function doesn't become entangled with other wave functions. Its the requirement for this latter kind of containment IIRC that makes the environment of the brain too messy to envision the current theoretical models for quantum computation working.

But there's no implicit implication that other kinds of computation aren't possible. The problems specifically arise out of the discrete engineering models we use because its easier to build reliable devices that way, rather than any inviolable requirement of quantum physics.

This is totally outside of my specialization so I could be way off base. Any physics guru's here?

[JoeP]

By the way Joe, you mentioned Dennit and Pinker? Got any titles of books worth reading?
By Dennett, I have Consciousness Explained - a bold title and a great read - and Kinds of Minds. By Pinker, How the Mind Works - same comment! - and Words and Rules - this one is just about linguistics, but is fascinating.
This is totally outside of my specialization so I could be way off base. Any physics guru's here?
I'm sure there are, but in the meantime: stw for "quantum computing" and "qubit". There is some fascinating stuff. The only one in my bookmarks is QCL - A Programming Language for Quantum Computers (http://tph.tuwien.ac.at/~oemer/qcl.html) - read the papers linked at the end.

[davidm]

The Chinese room is the most ill-devised and obviously wrong gedanken experiments I've ever had the misfortune to encounter.

In the first place, it posits a man in a room with a book to demonstrate it's point, then procedes to use the man's lack of understanding of Chinese to illustrate that a system can be devised whereby something can respond, but not understand.

But if the Chinese room is an analogy to an AI, the man, the room, and the book containing the algorithm for responding is the AI, not the man. i.e. Focussing on the man not understanding Chinese is like focussing on the amygdala or Corpus Collosum of a Chinese person not understanding Chinese, which, while true, doesn't indicate that the Chinese person in question doesn't understand Chinese.



I think this is the Systems Reply to Searle's argument.:

The Systems Reply, which Searle says was originally associated with Yale, concedes that the man in the room does not understand Chinese. But, the reply continues, the man is but a part, a central processing unit (CPU), in a larger system. The larger system includes the memory (scratchpads) containing intermediate states, and the instructions — the complete system that is required for answering the Chinese questions. While the man running the program does not understand Chinese, the system as a whole does.


Searle's response to the Systems Reply is simple: in principle, the man can internalize the entire system, memorizing all the instructions, doing all the calculations in his head. He could then leave the room and wander outdoors, perhaps even conversing in Chinese. But he still would have no way to attach "any meaning to the formal symbols". The man would now be the entire system, yet he still would not understand Chinese. For example, he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax. (See below the section on Syntax and Semantics).

I'm not especially interested in defending the Chinese Room argument -- I don't have enough information to know whether it is good or not -- but I'm curious about the claim that whole system is the AI: man, book, algorithm, etc. Even if Searle's response to the Systems Reply is flawed, I would ask where, in the whole Chinese room -- in the whole system -- does consciousness and understanding lie? If AI means anything, it must mean that it produces a consciousness that, in this case, understands Chinese; and (probably) what's more, understands that it understands Chinese: is self-conscious and reflective. I don't see why the system "as a whole" would have this capability, any more than a televison "as a whole" understands that it is transmitting images.

Clearly humans have this sort of understanding (or it seems as if we do), but human brains are not digital computers, :robot: and I think Searle's argument was directed only at the latter.

[Farren]

I think this is the Systems Reply to Searle's argument.:

The Systems Reply, which Searle says was originally associated with Yale, concedes that the man in the room does not understand Chinese. But, the reply continues, the man is but a part, a central processing unit (CPU), in a larger system. The larger system includes the memory (scratchpads) containing intermediate states, and the instructions — the complete system that is required for answering the Chinese questions. While the man running the program does not understand Chinese, the system as a whole does.

Searle's response to the Systems Reply is simple: in principle, the man can internalize the entire system, memorizing all the instructions, doing all the calculations in his head. He could then leave the room and wander outdoors, perhaps even conversing in Chinese. But he still would have no way to attach "any meaning to the formal symbols". The man would now be the entire system, yet he still would not understand Chinese. For example, he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax. (See below the section on Syntax and Semantics).

I'm not especially interested in defending the Chinese Room argument -- I don't have enough information to know whether it is good or not -- but I'm curious about the claim that whole system is the AI: man, book, algorithm, etc. Even if Searle's response to the Systems Reply is flawed, I would ask where, in the whole Chinese room -- in the whole system -- does consciousness and understanding lie? If AI means anything, it must mean that it produces a consciousness that, in this case, understands Chinese; and (probably) what's more, understands that it understands Chinese: is self-conscious and reflective. I don't see why the system "as a whole" would have this capability, any more than a televison "as a whole" understands that it is transmitting images.

Clearly humans have this sort of understanding (or it seems as if we do), but human brains are not digital computers, :robot: and I think Searle's argument was directed only at the latter.

As I stressed in my earlier post, the algorithm being described, of necessity, must be a recursive/self-reflecting algorithm and must constantly morph in order to emulate human understanding. The very nature of the emulation dictates that it must be. The algorithm would have to learn. It would have to have knowledge of feelings, it would have to change its actual operation to take new knowledge into account in its responses.

Furthermore, it must, of necessity have the kinds of recursive knowledge you specifically demand (understanding that it understands), else enquiries about self understanding would fail and the system would not be correctly emulating human consciousness.

Analogies to TV's are, I think, poor ones because a TV does not use recursive processes.

You ask where, in the system, the actual consciosness would lie. The same can be asked of human beings. If I could pluck your brain cells out, one at a time, then put them back before plucking out the next one, at what point would you feel like your consciousness is outside your head?

The obvious answer is never. Thinkers as far back as Hume have recognised that a point-particle view of consciousness is pointless. Consciousness is almost self-evidently systemic, not particular. So the question "where in the room is the consciousness?" is similar to the question "Where in the United States is the nation?". Its a question with built-in premises that are faulty.


In examining the Chinese room, its easy for the mind to assume the conclusion of the experiment as one of its premises. Namely:

It's obvious nothing apart from a human being is conscious, isn't it? So if anything other than a human agent claims understanding and consciousness, it's clearly not?

But its not obvious at all. In fact there's a perfectly defensible philosophical position (panpsychism) that holds that everything is conscious, in the sense of having the experience of being itself (qualia) rather than just the quality of being itself. From a panpsychic perspective, each system has the experience of being itself that is distinct from the meta-systems it is part of - and the sub-systems that make it up.

From this perspective weather systems are consciousnesses, societies are consciousnesses, ecosystems, which include societies and weather systems, are consciousnesses. And so on.

From this perspective human consciousness is not a special or unique emergent quality, its simply a specific self-experience of matter in a specific configuration. The "vexing" question of consciousness disappears when a panpsychic position is taken and replaced with the more mundane issue of simply describing distinctive behaviourial characteristics of a particular consciousness.

From a panpsychic perspective, if the man internalised the process which allowed him to correctly emulate Chinese communication (with all its attendant requirements of algorithmic learning and knowledge of human feeling, contemporary society, nature, et al), there would be a conscious system in such a man that is effectively a Chinese consciousness.

The fallacy is the same whether the man is in the system or the system is in the man. The specific demand of the "internalising" argument is that there are two agents in the mind of the subject: The part that processes the algorithm and the algorithm. The switch is then made to only one of the agents (the part that processes the algorithm) to justify the claim that no understanding is present. But from the altogether more coherent position that both of the agents constitute a third, higher system, its easy to see where the fault in this reasoning lies. Bob may not be aware of it, but he's become the agent for a higher consciousness in his mind.

Another thing I should point out is that there's a hidden premise in many discussions of this nature (certainly Searle and Penrose are guilty) and you've explicitly implied is one of your premises: That strong AI assumes a digital computer.

It doesn't. Its quite possible that human-like intelligence will be best achieved using analogue computing devices.

[davidm]

I'm not entirely persuaded by this line of thought, Farren. In fact -- and I'm perfectly prepared to be persuaded that I'm wrong-- I think the Turing Test is bogus, and that the Chinese Room, in whole or in part, does not understand Chinese.

The Turing Test is a test of imitation. It says that if a machine can sucessfully imitate a man's conversation, in the sense that no other party could distinguish between the machine and a man, then we are entitled to say that the machine is thinking, in the same sense that a man thinks.

But why are we entitled to say this? Why can't it just be that the machine has no point of view, no qualia, no experience whatever? This ties in with the problem of other minds. Logically, we can conceive of zombies: People who act just as we do, who appear to think and reason and feel and respond to events as we do. But, zombies have no internal experiences. Far from self-conscious, they are not conscious at all. They merely imitate consciousness, through a deterministic process.

I have played chess computers many times, and I always lose when I crank them up to grand master level. Isn't this a Turing Test? If I can't distinguish the moves of a chess computer from those of a chess grandmaster (or even those of an ordinary player, for that matter), then shouldn't I conclude that the chess computer is thinking? Good chess-playing, in humans, is evidence of a certain kind of very high-level thinking.

(Why is there no chess-playing smilie in the Sporting Group?) :(

Yet, I don't think anybody believes that the computer is having the "experience" of playing chess, or the experience of anything at all. So far as I can tell, there is no reason to believe that it is having such experiences. I don't think anyone has provided evidence that they do.

Panpsychism may be philosophically defensible, but it is also philosophically disposable. Why should I think that weather systems embody consciousness? It seems (to me) much easier to deny this.

Also, to clarify, I am not implying that strong AI assumes a digital computer. Actually rather to the contrary. I'm saying that if the Chinese Room argument works, it applies only to digital computers. Clearly it can't apply to everything, because if it did, then humans would not be conscious.

[Farren]

I'm not entirely persuaded by this line of thought, Farren. In fact -- and I'm perfectly prepared to be persuaded that I'm wrong-- I think the Turing Test is bogus, and that the Chinese Room, in whole or in part, does not understand Chinese.

The Turing Test is a test of imitation. It says that if a machine can sucessfully imitate a man's conversation, in the sense that no other party could distinguish between the machine and a man, then we are entitled to say that the machine is thinking, in the same sense that a man thinks.

But why are we entitled to say this? Why can't it just be that the machine has no point of view, no qualia, no experience whatever? This ties in with the problem of other minds. Logically, we can conceive of zombies: People who act just as we do, who appear to think and reason and feel and respond to events as we do. But, zombies have no internal experiences. Far from self-conscious, they are not conscious at all. They merely imitate consciousness, through a deterministic process.

I have played chess computers many times, and I always lose when I crank them up to grand master level. Isn't this a Turing Test? If I can't distinguish the moves of a chess computer from those of a chess grandmaster (or even those of an ordinary player, for that matter), then shouldn't I conclude that the chess computer is thinking? Good chess-playing, in humans, is evidence of a certain kind of very high-level thinking.

(Why is there no chess-playing smilie in the Sporting Group?) :(

Yet, I don't think anybody believes that the computer is having the "experience" of playing chess, or the experience of anything at all. So far as I can tell, there is no reason to believe that it is having such experiences. I don't think anyone has provided evidence that they do.

Panpsychism may be philosophically defensible, but it is also philosophically disposable. Why should I think that weather systems embody consciousness? It seems (to me) much easier to deny this.

Also, to clarify, I am not implying that strong AI assumes a digital computer. Actually rather to the contrary. I'm saying that if the Chinese Room argument works, it applies only to digital computers. Clearly it can't apply to everything, because if it did, then humans would not be conscious.

The short answer is all of your questions can be applied to other human beings. Its interesting that you raise the issue of zombies because there's no objective means by which we can determine that every other human being we deal with is not a zombie. The assumption that other human beings are conscious of themselves is as philosophically disposable as panpsychism.

So lets say there's a charitable assumption and a pessimistic assumption. The charitable assumption removes a great many philosophical conundrums and thus has the advantage of economy, which is why I find it a more elegant framework when contemplating these issues.

The physical architecture my brain uses to process the word "word" can be topologically entirely dissimilar from the architecture used in your brain. So consciousness cannot reside in specific architectural details like specific configurations of neurons. At a lower level, the precise topology, size and shape of neurons have no uniformity between us, so consciousness cannot be assumed to reside in the precise structure of individual neurons. The lower down the heirarchy of detail you go the more troublesome it becomes.

If, on the other hand, you climb the ladder from micro- to macroscopic and claim that consciousness is an emergent pattern of statistically similarly behaving systems, you're faced with the conundrum of defining the characteristics of that emergent behaviour. One assumes you can't view a specific personality as being the definition of consciousness, else all other personalities would by definition not be conscious. So one must extract and propose general sets of characteristics. Feedback? Point a camera at a TV. Memory? Write a word on a piece of paper. And so on.

Panpsychism, on the other hand, sidesteps the problem by conflating consciousness and qualia, so that there's no need to define "consciousness". Its an inherently simple quality. The only discussion required by panpsychism is how one system interacts with another system. Its way, way, way more economical.

I do take your point about Searle's argument implicitly applying only to digital computers, but I don't think that Searle saw that hidden proviso when formulating it and Penrose, in citing it, definitely makes the mistake of conflating the concepts of digital and general computing, as if anything that isn't a quantum computer is by default a digital computer.

[davidm]

I'm not entirely persuaded by this line of thought, Farren. In fact -- and I'm perfectly prepared to be persuaded that I'm wrong-- I think the Turing Test is bogus, and that the Chinese Room, in whole or in part, does not understand Chinese.

The Turing Test is a test of imitation. It says that if a machine can sucessfully imitate a man's conversation, in the sense that no other party could distinguish between the machine and a man, then we are entitled to say that the machine is thinking, in the same sense that a man thinks.

But why are we entitled to say this? Why can't it just be that the machine has no point of view, no qualia, no experience whatever? This ties in with the problem of other minds. Logically, we can conceive of zombies: People who act just as we do, who appear to think and reason and feel and respond to events as we do. But, zombies have no internal experiences. Far from self-conscious, they are not conscious at all. They merely imitate consciousness, through a deterministic process.

I have played chess computers many times, and I always lose when I crank them up to grand master level. Isn't this a Turing Test? If I can't distinguish the moves of a chess computer from those of a chess grandmaster (or even those of an ordinary player, for that matter), then shouldn't I conclude that the chess computer is thinking? Good chess-playing, in humans, is evidence of a certain kind of very high-level thinking.

(Why is there no chess-playing smilie in the Sporting Group?) :(

Yet, I don't think anybody believes that the computer is having the "experience" of playing chess, or the experience of anything at all. So far as I can tell, there is no reason to believe that it is having such experiences. I don't think anyone has provided evidence that they do.

Panpsychism may be philosophically defensible, but it is also philosophically disposable. Why should I think that weather systems embody consciousness? It seems (to me) much easier to deny this.

Also, to clarify, I am not implying that strong AI assumes a digital computer. Actually rather to the contrary. I'm saying that if the Chinese Room argument works, it applies only to digital computers. Clearly it can't apply to everything, because if it did, then humans would not be conscious.

The short answer is all of your questions can be applied to other human beings. Its interesting that you raise the issue of zombies because there's no objective means by which we can determine that every other human being we deal with is not a zombie. The assumption that other human beings are conscious of themselves is as philosophically disposable as panpsychism.

So lets say there's a charitable assumption and a pessimistic assumption. The charitable assumption removes a great many philosophical conundrums and thus has the advantage of economy, which is why I find it a more elegant framework when contemplating these issues.

The physical architecture my brain uses to process the word "word" can be topologically entirely dissimilar from the architecture used in your brain. So consciousness cannot reside in specific architectural details like specific configurations of neurons. At a lower level, the precise topology, size and shape of neurons have no uniformity between us, so consciousness cannot be assumed to reside in the precise structure of individual neurons. The lower down the heirarchy of detail you go the more troublesome it becomes.

If, on the other hand, you climb the ladder from micro- to macroscopic and claim that consciousness is an emergent pattern of statistically similarly behaving systems, you're faced with the conundrum of defining the characteristics of that emergent behaviour. One assumes you can't view a specific personality as being the definition of consciousness, else all other personalities would by definition not be conscious. So one must extract and propose general sets of characteristics. Feedback? Point a camera at a TV. Memory? Write a word on a piece of paper. And so on.

Panpsychism, on the other hand, sidesteps the problem by conflating consciousness and qualia, so that there's no need to define "consciousness". Its an inherently simple quality. The only discussion required by panpsychism is how one system interacts with another system. Its way, way, way more economical.

I do take your point about Searle's argument implicitly applying only to digital computers, but I don't think that Searle saw that hidden proviso when formulating it and Penrose, in citing it, definitely makes the mistake of conflating the concepts of digital and general computing, as if anything that isn't a quantum computer is by default a digital computer.

To clarify, when you say:

"From this perspective weather systems are consciousnesses, societies are consciousnesses, ecosystems, which include societies and weather systems, are consciousnesses. And so on.

What, exactly, does "are consciousness" mean? Do you mean to say that they are conscious? Or is there some subtle distinction that I'm missing?

Let's return to chess-playing computers. Do you think the chess-playing computer is having some kind of conscious experience? If so, could we establish what kind of experience that is?

The zombie problem does apply to other people, but at least I can know that "I" (whatever, excactly, "I" is) am conscious. I think. Or, it might be more accurate to say, "thinking is taking place." This might be more accurate, because I do not think it is clear what an "I" is, and the "I" might be an illusion.

So if thinking and experiencing is taking place in me, I have warrant to assume (even if I can't prove) that thinking and experiencing is taking place in others, because we are all members of the same species. Putitng the philosophical issue to one side, it would seem extraordinary if I were the only one to experience "thinking."

But, from that, it doesn't seem to follow that other systems are "conscious," unless we can figure out precisely what consciousness is. Making assumptions about what is and is not conscious seems to involve making certain presuppositions that are not yet verified. I do agree that conciousness appears to be some kind of macropattern and certainly doesn't inhere in a single element, like the man alone in the Chinese Room argument.

But I also think, with respect to AI, that it might be very possible to build a zombie. The computer would display the full range of intellectual powers of humans; perhaps even more. But, it would not be having a conscious experience.

Maybe that's not possible. Maybe any kind of "conscious-like" acitivty entails actually being conscious. Nobody knows whether this is true or not. An interesting question would be, if a zombie (without consciousness) behaved exactly like a human (with consciousness), then what difference would it make whether a "conscious-seeming" entity was conscious or just a zombie? And now that I think about it, if it makes no difference at all, then maybe this just proves Turing's point, and his test isn't bogus after all.

In which case, I've used my own argument to kick my own butt. :buttkick:

[JoeP]

Damn you guys and your long replies, which I have to read now thanks to vbulletin's crappy lack of persistent unread-post status. I'm going to post a short reply to show you how it's done.

I claim the mass of the Earth is 6.0 x 1024 kg. You say "show me the exact point where I can find that mass". I crush you under a heavy rock because you're not worth arguing with. :muahaha: