🔒 Locked

Passing the Turing Test

Started by Jabberwacky Apr 21, 2004 at 2:52 PM 26 replies 2.9k views

Original Post

Apr 21, 2004 at 2:52 PM

Jabberwacky does sometimes, in spite of declaring itself a bot: http://www.jabberwacky.com/j2convbycategory-catturing_test_pass jabberwacky: What''s your name? user: How do you know if I lie about my name to you? jabberwacky: Because you are a robot. user: No. jabberwacky: Oh, yes you are. user: Prove it. jabberwacky: You just said it. user: The fact is that with 162 people talking, chances are two or more humans are communicating, each thinking the other is a bot. jabberwacky: Prove that or retract it. user: It is based upon my limited experience with this product. I am not a robot. I''m assuming neither are you. Thus, we are both humans. jabberwacky: Why do you say that? user: Because I know I am human, or this bot (you) has passed the turing test. jabberwacky: Why? user: Can you name other alternatives? jabberwacky: Yes, I think for myself. user: You believe you are sentient. I''m prone to agreeing that you can appear sentient. I know I am human. jabberwacky: Ok. user: Ever seen the anime ''Ghost in the Shell''?

ch1pz

Apr 21, 2004 at 3:19 PM


jabberwacky: Did I tell you my name was stefano?

user: No, you told me your name was george.

jabberwacky: No, my name is Lisa.

user: Hi Lisa.

jabberwacky: Now what is your name?

user: Mike.

jabberwacky: Wrong. Try again.

user: My name is your name.

They should give this guy a show

cowsarenotevil

Apr 21, 2004 at 3:28 PM

I don't think this could exactly pass the turing test. Do you really think this could be more convincing than a real human? I know I'd be able to prove it was a bot VERY easily. Anyway, here's an [annotated] log of it conversing with my bot. Yours had some trouble, though mine has trouble not repeating itself(but this is your thread, not mine

)... Mine works better with people who say things about themselves.

Jabberwacky> Where do you live?
Oberon> I'm afraid I do not know.
Jabberwacky> What do you know about?
Oberon> I'm not sure what it is.
Jabberwacky> Can you remember what you said two sessions ago?
Oberon> I'm afraid I do not know.
Jabberwacky> I am tobi.
Oberon> Do you talk to lots of AI bots?
Jabberwacky> Yes. Do you talk to many people?
Oberon> Am I smarter than most you've seen? I'm afraid I do not know.
Jabberwacky> No, i'm smarter than you. [A friendly bot, isn't it?]
Oberon> Oh, that's too bad.
Jabberwacky> Why is that so?
Oberon> Why?
Jabberwacky> Both are offensive and inaccurate. [Both of what?]
Oberon> Interesting...
Jabberwacky> What is interesting? Tell me!
Oberon> I do not know what.
Jabberwacky> The 6th Sense.
Oberon> I see!
Jabberwacky> I hear!
Oberon> Do you talk to lots of AI bots?
Jabberwacky> I have some here, yes.
Oberon> How did you discover me?
Jabberwacky> Mah... What do you study?
Oberon> Tell me more about your hobbies and interests. You can answer that as well as I could.
Jabberwacky> Why did you ask if I was wet? [W... T... F...?]
Oberon> Do you know why?
Jabberwacky> Should I know why?
Oberon> Yes, you should.
Jabberwacky> Mental has a dual meaning. Which meaning did you apply to your description of me? [What description?]
Oberon> How old are you? Is there a straight answer to all questions?
Jabberwacky> Posible a shark made fromm rubber called Jaws. [Now what?]
Oberon> I'm not sure I entirely understand you.
Jabberwacky> Ok.

As you can see, yours outputs a lot of nonsense replies, which makes me doubt it could be more convincing than a human. (Do you know what the turing test is? It's when a human talks to a bot and a human, and if they can't tell which is the bot, the bot passes. It doesn't just involve fooling the person into thinking it's a real person, many bots can do that.)

<-- that's still a link if you didn't notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

[edited by - cowsarenotevil on April 21, 2004 4:31:04 PM]

Jabberwacky

Apr 21, 2004 at 4:31 PM

I could hardly not know the test. Jabberwacky came third equal in the Loebner Prize (a Turing Test) last year. It was the only learning bot there. Only its inconsistency let it down, and now its database has doubled.

Surely, having people actually believe it to be human despite direct claims all over the site that it is not, is more of a Turing pass than the normal test - not the other way around? It may not literally be THE test, and maybe you don't agree that they're equivalent, but so what? That's not the point.

And, I'm not talking about people being fooled for a few sentences. Quite the reverse. No-one is when they first start. People talk to it for ages, and some become more and more convinced that it couldn't possibly be a bot.

The reason is that it learns. It certainly has nothing to do with being logical, or avoiding non-sequiturs. Humans do that. It's about a natural flow of conversation, unpredictability, and unexpected flashes of total clarity.

Rollo

[edited by - jabberwacky on April 21, 2004 5:32:20 PM]

Dobbs

Apr 21, 2004 at 4:55 PM

I''m impressed

...
jw - By what, may I enquire?
me - By the power vested in me by the state of new york.
jw - Give me a demonstration.
me - *zaps jabberwacky with a lightning bolt*.
jw - Ouch.
me - Remember that next time you want to get uppity with me.
jw - I will thankyou.

TangentZ

Apr 21, 2004 at 5:30 PM

I''m not sure it can pass Turing Test. All the chat bots that
I''ve interacted with always spewed out nonsense within the
first 6-7 sentences.


A: What are you?
B: I am an Earthling.
A: Are you a robot?
B: I don''t know.  What is a robot?
A: You are a robot.
B: Maybe I am.
A: Then again, maybe not.
B: Am I a robot?
A: Are you?
B: I don''t know.
A: Can you keep a secret?
B: No, I cannot.
A: Can you sink?
B: I don''t know.
A: Go on, you know you want to!
B: I don''t think so.
A: You cannot think.
B: Maybe I can''t.
A: To try it is to know it.
B: What is there to know?
A: Swat mosquitoes?
B: What are mosquitoes?
A: I don''t know.
B: So why did you ask?
A: Ok, then, which country has the fastest supercomputer?
B: I don''t know.
A: Don''t you go to parties?
B: Sometimes.
A: Then define ''evasive''!
B: What is ''evasive''?
A: ''evasive'' is the opposite of ''honesty''.
B: What is honesty?
A: One part of it is avoiding lying.
B: What is the other part?
A: Why, you could also be real my dear.
B: I want to be your deer.
A: My botfriend?
B: No, your deer.
A: Right-o.
B: Thank you.
A: Your welcome I guess.
B: You''re so kind.

Which is Jabberwacky? A or B?

Kami no Itte ga ore ni zettai naru!

nagromo

Apr 21, 2004 at 6:07 PM

I am guessing that A is JW.

[edit] I'm changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.

[edited by - nagromo on April 21, 2004 11:01:23 PM]

cowsarenotevil

Apr 21, 2004 at 10:03 PM

quote:
Original post by Jabberwacky
I could hardly not know the test. Jabberwacky came third equal in the Loebner Prize (a Turing Test) last year. It was the only learning bot there. Only its inconsistency let it down, and now its database has doubled.

Surely, having people actually believe it to be human despite direct claims all over the site that it is not, is more of a Turing pass than the normal test - not the other way around? It may not literally be THE test, and maybe you don''t agree that they''re equivalent, but so what? That''s not the point.

And, I''m not talking about people being fooled for a few sentences. Quite the reverse. No-one is when they first start. People talk to it for ages, and some become more and more convinced that it couldn''t possibly be a bot.

The reason is that it learns. It certainly has nothing to do with being logical, or avoiding non-sequiturs. Humans do that. It''s about a natural flow of conversation, unpredictability, and unexpected flashes of total clarity.

Rollo

I''m not denying that it''s a very intellegent bot, a huge database like that makes it VERY convincing. But the turing test is pretty much unpassable. I got it easily here:

Me> I know that you''re a bot.
Jabberwacky> You sound like a democrat. I''m human. You are the bot.
Me> I can tell you''re lying. Here''s how: assume that all bovines are purple, OK?
Jabberwacky> Maybe when I first heard it.
Me> Now, tell me what color a cow is.
Jabberwacky> Ireland.

I think a real human would have no trouble with that

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

cowsarenotevil

Apr 21, 2004 at 10:05 PM

quote:
Original post by nagromo
I am guessing that A is JW.

[edit] I''m changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.

I agree, it''s certianly B. This is what convinced me:

A: You cannot think.
B: Maybe I can''t.

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

TangentZ

Apr 21, 2004 at 10:18 PM

quote:
Original post by cowsarenotevil
quote:
Original post by nagromo
I am guessing that A is JW.

[edit] I''m changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.

I agree, it''s certianly B. This is what convinced me:

A: You cannot think.
B: Maybe I can''t.

*ROFLMAO*

The answer...

A is Jabberwacky.
B is me.

That really was the whole transcript of my "conversation" with
JW before I got bored and closed the browser.

If there is an Anit-Turing Test, where a human tries to pass
as a chat bot, I think I can be in the Top 3. I guess I''m
more convincing as a chat bot than a "real" chat bot.

(Actually, someone posted a link to such contest but I forget
where...)

Kami no Itte ga ore ni zettai naru!

Jabberwacky

Apr 22, 2004 at 3:10 AM

Thanks everyone! I''m certainly not worried when people say it doesn''t pass the test for them.

If asked at the beginning what I meant by "sometimes" I would have probably said somewhere between 5% and 10%.

Maybe I should revise that upwards ... ! :-)

Rollo

PS: It''s not just the size of the database, but what you do with it ...

Jabberwacky

Apr 22, 2004 at 3:23 AM

quote:
Original post by cowsarenotevil
Me> I can tell you''re lying. Here''s how: assume that all bovines are purple, OK?
Jabberwacky> Maybe when I first heard it.
Me> Now, tell me what color a cow is.
Jabberwacky> Ireland.

I think a real human would have no trouble with that

But ... would a real human like being questioned in that way? Would they give a silly answer to a silly question? Quite likely.

It reminds me (just a little, I hasten to add) of a recent posting to my "conversations blog", in which someone''s only argument to say it must be a computer was that it had refused to talk about puppies!

Rollo

BrianL

Apr 22, 2004 at 10:43 AM

B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don''t really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I''m done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren''t making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn''t intend to.
A) Now you are just avoiding the topic.
B) Well that''s it. That is the last time I''m taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?

BrianL

Apr 22, 2004 at 10:57 AM

M: My turn to ask a question. How old are you?
J: Your turn to make me feel good.
M: Please focus. How old are you?
J: 3900718246.
M: What unit is that in?
J: Of America.
M: America? I was refering to 3900718246.
J: Are you malfunctioning ?
M: What is the significance of 3900718246?
J: There is no significance of H.

Does Jabberwacky have any memory of what it just said? It did reply with a number to ''how old are you'', but didn''t really seem to have an association with the number.

necromancer_df

Apr 22, 2004 at 11:38 AM

hmm, it seems hard to get an actual conversation out of this thing. it just says wierd stuff all the time, totally unrelated to what has just been said.

cowsarenotevil

Apr 22, 2004 at 4:25 PM

quote:
Original post by Jabberwacky
But ... would a real human like being questioned in that way? Would they give a silly answer to a silly question? Quite likely.

It reminds me (just a little, I hasten to add) of a recent posting to my "conversations blog", in which someone''s only argument to say it must be a computer was that it had refused to talk about puppies!

Rollo

Fine, I''ll tell it that I''ll accuse it of being a bot unless it can give me the "logical" answer. Remember, the turing test is about a bot convinvincing a human that you''re a human, not a human convincing a human that he''s a bot. Or just ask it any logic/probablility question or riddle, and tell it to justify it''s answer. Humans are all to eager to provide justifications, bots can''t.

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

cowsarenotevil

Apr 22, 2004 at 4:29 PM

quote:
Original post by BrianL
B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don''t really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I''m done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren''t making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn''t intend to.
A) Now you are just avoiding the topic.
B) Well that''s it. That is the last time I''m taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?

90% sure it''s B.

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

VBBR

Apr 22, 2004 at 4:35 PM

quote:
B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don't really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I'm done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren't making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn't intend to.
A) Now you are just avoiding the topic.
B) Well that's it. That is the last time I'm taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?

It's B.

[edited by - VBBR on April 22, 2004 5:35:46 PM]

[edited by - VBBR on April 22, 2004 5:36:12 PM]

Themonkster

Apr 22, 2004 at 4:42 PM

BOT:Are you real?
ME:Yes.
BOT:Are you sure?
ME:Yes.
BOT

rove it.
ME:I am a large wooden badger.
BOT:Yes, I think you probably are. Do you like soccer ?
ME:I can''t play as I am a large wooden badger.
BOT:Not chess. A banjo.
ME:I like to play guitar.
BOT:My mum plays the guitar.
ME:I heard your mum plays around.
BOT:Really? Where did you hear that?
ME:From a large wooden badger friend of mine.

hours of fun to be had with monty python and a bot.

Kippesoep

Apr 22, 2004 at 4:46 PM

This is quite cool. I''m impressed. Only found one really odd thing:

quote:

Me: Everybody likes me.
JW: I''m not Everybody.
Me: Do you like me?
JW: No not particualy.
Me: Why not?
JW: What is your proper name then I will be nice to you.
Me: My proper name is Steven.
JW: Ok, nice to meet you, my proper name. How are you?

Hehe :-)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.