AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

78% “pass” rate in a modified version of the Turing Test in a Virtual World
 
 

Thought you guys might like this paper on a recent modified Turing Test run in Second Life with a 78% (!) pass rate.

http://www.sciencedirect.com/science/article/pii/S1071581914001049

Our blog post gives more background, context and a link to a possibly free copy of the article.

http://daden.co.uk/conc/blog/2015/01/daden-chatbot-significant-passes-modified-turing-test-virtual-world

Disclosure: We (Daden) came up with the idea of the test (in 2009), and supplied the chatbot engine, but Loyola Marymount University designed the test, authored the bot, ran the test and analysed and published the results.

David

 

 
  [ # 1 ]

Nice work. Do you guys still use AIML for your Discourse engine?

 

 
  [ # 2 ]

A heavily modified version of it. Not checked out AIML2 yet to see if its worth switching, ditto Chatscript, but we have done some work a more semantic/grammatic parser based engine. Realistically though its probably horses for courses, Discourse version of AIML being great for very fixed domain bots, but something else being needed for a “generalist”, and learning, chatbot.

David

 

 
  [ # 3 ]

Not bad. I think this percentage should always go along with mention that people weren’t aware of the possibility that there might be a bot. It disables the “interrogation” aspect, so it’s difficult to compare to other Turing Tests or the benchmark % that Turing described under different conditions. “Covert Turing Test” sounds good to me.

 

 
  login or register to react