AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

What I’ve learned from the Loebner contest—and I haven’t even entered (yet)
 
 
  [ # 76 ]

yes smile

Apparently I’d already programmed a rule “If you’ve nothing better to say than to repeat the user, add arguments.” with the same risk of being superfluous. I might as well do the same at yes/no answers. I’m not sure whether it is for better or worse that my submitted entry does not, but that is water under the bridge now.

 

 
  [ # 77 ]

I found the original dialogue of the towel question, posed -of course- by professor Noel Sharkey smile.
Below is the human response, which is remarkably brief though not satisfactory without argumentation. Also remarkable is that it only took one reasoning argument, the statement of a single fact, to convince the judge.

judge: Could you kill someone if you stabbed them with a towel?
No
judge: Why not?
Because a towel is a soft, fluffy thing
judge: Thank you, human, thank you very much. You can have a break now.

(My current program didn’t get past the plural reference “them” to a singular subject. interesting.)

 

 
  [ # 78 ]

Don, Where did you find the original dialog with Professor Sharkey? 

Don’t feel alone.  My program current program stumbled all over itself before it provided an answer that clearly showed it was 110% clueless.  Honestly I find those 2 questions rather humbling.  The questions are so simple, clear and so easy to answer (for a human).  I started working on “Why” questions just a couple of days ago as I am unhappy with my bots ability to keep context, and in doing so “why not” hadn’t even entered my mind.  Ah, now more testing, lol.

On a different note, I find it very interesting to compare last year’s “bad judge” to Professor Sharkey.  The “bad judge” in comparison to Professor Sharkey was crude, rude and socially unacceptable.  While Professor Sharkey shows class and sophistication.  To me at least, Professor Sharkey in these questions demonstrates a degree of understanding that the “bad judge” could never hope to attain, IMHO.  In the end I think I would find listening to Professor Sharkey very interesting, while the bad judge leaves me no respect for him due to his arrogance.

Obviously I have a lot more work to do, which I am always aware of.  However, it is the simply questions like these that make me aware there is so much more than I realized.

It would be very interesting for Professor Sharkey to create even a short list of questions that he would use to determine if it is a program or human.

 

 
  [ # 79 ]

Some of the questions I have seen Professor Sharkey ask at Loebner Prize contests include:

Would it hurt if I stabbed you with a towel?
Which is larger, a small mountain or a large lion?
What number comes after seventeen?
Which is bigger, a whale or China.
Is a knife sharper than a blanket?

He also has general conversation and asks follow up questions. One bot last year said it was from San Francisco. Professor Sharkey then asked it about San Francisico, things like “Do you know Benny’s near Pier 39?” (or similar) to judge the bot’s back story.

 

 
  [ # 80 ]

Source: http://www.theguardian.com/commentisfree/2011/oct/25/artificial-intelligence-loebner-prize
I have to add that the poor performance of the chatbot in the article was -if I recall Mohan Embar’s explanation correctly- because it deliberately ignores some of the judge’s questions until the judge tells his name, for reasons of social behaviour.

Noel Sharkey is not the average judge, he practically represents logic and reasoning (which personally I like), but at the same time such questions can make for pover conversation. He is a good sport in any case. I was thinking, if we can’t have “average interrogators”, maybe we can hope to have expert judges each representing and testing a different aspect of the mind.

I’ve now dealt with the “them” reference and stumbled across the worse flaw that my program couldn’t form a negative opinion on the act of killing. I may not think highly of Turing Tests, but they do offer a little practical exercise. Let me try blankets and knives: “Yes, a knife is sharp, because I don’t know if blankets are sharp.”
Work’s never done, but at least it gets better smile

 

 
  [ # 81 ]

Actually one of the better explanations for aberrant behavior in chatbots while in contest environments can be found here. Quite illuminating actually, related to quantum mechanics and wave functions I believe

https://www.youtube.com/watch?v=wdq1qJkahDY

V

wink

 

 
  [ # 82 ]

LOL That sounds accurate.

 

 
  [ # 83 ]

That is so accurate.  I will test my program and it works great, then have my daughters try and it if falls flat, now I know why.

 

 
  [ # 84 ]
James_Adams - Aug 3, 2014:

It is more “human” to have a longer answer.

Clrly u dnt chat ol w same plp i do.  ;P

James_Adams - Aug 3, 2014:

Which reminds of the old joke which is heavier a ton of feathers or a ton of apples.  Which by the way my bot currently can’t handle, but will in a few minutes lol.

I remember the joke as which is heavier a lb of gold or a pound of feathers.  (Which has a more subtle answer.)

 

 
  [ # 85 ]

I received an email today saying that my program attempted to download .net from the internet, and was not running correctly.  I have replied that I can send a copy of the .net to the dropbox and that .net is part of the Windows OS.  I haven’t heard back, hopefully they will be understanding and realize that it is an update to the OS and allow my program to enter.

 

 
  [ # 86 ]

Dr Keedwell has upgraded the OS, however my program is still not running correctly.  It appears that the data didn’t install correctly.  There remains a small chance that Dr. Keedwell will be able to get it running.

Whether he is successful or not with getting my program to run, I want to go on the record saying that this man has worked very hard to try and not exclude me from the contest.  I am highly impressed with his efforts and concerns.  In simple terms, the world would be a better place with more like him.

 

 
  [ # 87 ]

Well, I’ll cross my fingers for you. 8pla also used .net for his entry if I recall. Either way it’s good to hear that someone capable is doing their best to give every program a fair chance.

 

‹ First  < 4 5 6
6 of 6
 
  login or register to react
‹‹ LPP test program      The 2014 Loebner ››