AI Zone Admin Forum Add your forum

NEWS: survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

AISB Loebner Prize 2018 Finalist selection
  [ # 16 ]

Thanks for clearing up the mystery, José smile . This makes it extra ironic that guesswork really can work better. From your university background I suspected you were at least using some grammar parsing to pinpoint the nouns, but now I see you could also look at whichever word comes after “a” or “the”. Like most people I’m just not used to thinking that way.

For the Loebner Prize I always wondered if one couldn’t put Google Translate inbetween the bots and judges so that non-English bots could also participate without having to rewrite their scripts.

Maybe you could open source your chatbot, occasionally people are asking for AI/chatbot projects that they can help with on


  [ # 17 ]

As usual, it seems that truth is uglier than fiction!

Anyway, I agree with you that the most effective method of solving winograds would be using statistics. In fact, I made some experiments seraching in Google some pairs formed with the key elements from the Winograd scheme. I did not investigate further since I do not know the technology for automating Google searches and, anyway, the Loebner contest does not allow using an Internet connection. I really think that, if not solved completely, that could definitely improve the random baseline results (apart from integrating the correct answers for all current Winograd schemes, as they said: “don’t guess if you know!”).

As you said, Momo includes a part-of-speech tagger/lemmatizer. However, it did not use it for the Winograd schemes. For instance, the pattern used to answer the chicken question was simply:

a&an;&the; <wildcard length=“1” name=”$text1$” > * a&an;&the; <wildcard length=“1” name=”$text2$” > * what * ?

with two possible answers (randomly selected):

The $text1$.
The $text2$.

Very Loebner-targeted, isn’t it?

It would be funny to have a contest for the Winograd schemes alone, but as it is a relatively small part of the Loebner contest, it is not worth it investing much effort in it.

I also like your suggestion about opensourcing the bot. I will take a look at reddit to see if anyone is interested!


  [ # 18 ]

I would be worried about using that pattern outside of the Loebner Prize, yes smile

Officially Windograd schemas are supposed to be Google-proof, but the ones in the Loebner Prize weren’t. Universities that tackled unofficial Winograd schemas did 10% better by using Google search on top of other methods, and it would definitely improve results in normal conversations. Unfortunately the Google search API is now a paid service, so it’s become a less attractive option.

There was a Winograd Schema contest scheduled for last February, but it was cancelled again, I don’t expect a rerun any time soon.


 < 1 2
2 of 2
  login or register to react