AI Zone Admin Forum Add your forum

NEWS: survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Loebner Prize Report 2016
  [ # 16 ]

OK. Judge1 makes sense as organizers testing the system works.  Judge2 is a total bust as far as log files go since it only is a start command.

The only other log file which is labelled log-.txt starts its confused “hello” at 3:08, which could well be round 3, ending at 3:39. And somehow you say it has multiple rounds embedded into it so all 3 rounds appear to be merged.  Maybe something crashed in CS such that it dropped using Judge2 log and ended up in default log. That would give us log-.txt. And if it crashed on startup each time, that could merge the logs. I don’t know about that, but its almost a plausible speculation.


  [ # 17 ]

So who was Round 3 judge who walked away!


  [ # 18 ]

Tom Cheshire, technology correspondent for Sky News. From what I could gather, he said he had something to do and walked out. I voiced my concern, as he should still have stayed to rank the bots in order and suggested the contest wait for him to come back. The fact that he had already decided which was human and which was bot was immaterial. He reappeared 5 minutes before the end of round 3.

I later found out that Rose wasn’t responding to judge 3, which is probably why he left.

Approximate times for the start of rounds:

Round 1 - 13:36
Round 2 - 14:23
Round 3 - 14:54
Round 4 - 15:25

These times were when I activated Mitsuku and so each round started anywhere between 10 and 15 minutes later.


  [ # 19 ]

I will defend against the bug for next year, but the issue arises because the readme instructions for Rose were not followed. They state that it is safe to start the judge program AFTER Rose’s screen has said “initialization complete”, but the screen shot sent me clearly shows the judge’s “hello” message preceeding even the “chatbot beginning initialization” message. Normally this would not cause a problem regardless, but machine speed could impact the behavior.


  [ # 20 ]

An article of David Boyle


  [ # 21 ]

I read this interesting bit about the old protocol used in 2016. This judge found the layout difficult to work with and actually thought there was no way to correct typos.


 < 1 2
2 of 2
  login or register to react