AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Loebner Prize 2017 (inc. important dates)
 
 

The news of Hugh Loebner’s passing was sad and upsetting, but the AISB had agreed to continue to run the competition, so that’s what we intend to do.

The important dates for the Loebner Prize 2017 are as follows:

Entry Submission Deadline: Monday 10 July 2017

Announcement of Finalists: Tuesday 1 August 2017

Finals Day: Loebner Prize 2017 Finals on Saturday 16 September 2017 at Bletchley Park

Be aware that the new protocol will be in use: https://github.com/jhudsy/LoebnerPrizeProtocol

Cheers, good luck, and I hope to see many of you at Bletchley Park!

 

 
  [ # 1 ]

Andrew, may I suggest that such a major change to the LPP, which will prevent many of us from entering, is indicated on the AISB page rather than

1. Entries must work with the published Loebner Prize Protocol

As many of the usual entrants are not members of chatbots.org and will assume the original, much simpler protocol is still in place.

 

 
  [ # 2 ]

This is not a major change… It is a full replacement (for a level playing field).  A brand new chatbot built in native JavaScript is quickly becoming a foregone conclusion. So, get ready to take a break from a decade of work, and get started on a brand new chatbot in native JavaScript for this year. Not entering the contest is really not an option, for obvious reasons.  Maybe we can all help each other.

@AISB: This post is entirely polite.

 

 
  [ # 3 ]

hey Andrew, how cool to hear that the Loebner Prize is being continued!!! Many thanks for posting this here!

 

 
  [ # 4 ]

I’m glad to see that AISB is working to improve the contest vis-a-vis the interface. Even though Hugh liked his “simpler” idea, it is in the contest’s best interest to level the field of entrants so that the same people don’t keep coming back. The dichotomy of Hugh’s pride in his LPP design and his desire to invigorate the contest were at odds with each other.

It is better to embrace the 21st century now than to make a return to teletype machines and handwritten messages.

Robby.

 

 
  [ # 5 ]

Hugh created his beloved protocol to secure the Loebner Prize Competition from remote cheating.

...

From the remote web server at URL: http://www.chatbot.tk/newLPP    (Click the CHEAT button)

This message:

{"contents":"You asked: Is this working?","id":"ai0","secret":"ai0secret"

was sent remotely from the web to “server.js” listening on a local PC: 127.0.0.1:8080

...

Read (over 300 lines) more at URL: http://chatbot.tk/newLPP/remote_access.php

 

 

 

 
  [ # 6 ]

I think you earned a bronze cluster on your Guru status. smile

 

 
  [ # 7 ]

8man - Fortunately, the PCs are all connected via a local switch in a room at Bletchley Park. There’s no outside influence.

...it is in the contest’s best interest to level the field of entrants so that the same people don’t keep coming back…

Interesting thought Robby. If the only way to find a new winner is to make it too complicated for the usual suspects to enter, that seems to be a step backwards to me. Kind of like making Brazil play soccer with only 8 players instead of 11, as the other teams were finding it hard to win. wink

 

 
  [ # 8 ]

I recently acquired the book “Parsing the Turing Test”, with a relevant passage from Hugh:

There is another way to acquire communications programs. Require that all entrants submit identical appearing communications programs in addition to their entries. I used this technique in 2004. - This raises the technical bar, and thus may reduce the number of entrants and ensure a minimum level of competence among the entrants. Any applicant who cannot develop a communications program should not be an entrant.

But he also warns to keep it simple and to “conquer any urge to develop a LAN or have the programs communicate with a central server.

In my opinion a communications protocol should accommodate a majority of (good) entries. This could be achieved if the new LPP’s installation and tie-in with the old protocol are streamlined more. Speaking of which, the AISB has now updated the website for clarity:

The Loebner Prize Protocol (LPP) for communication between judges, AIs and confederates has been updated for 2017. Please click here for the updated protocol. There is also a discussion of the new LPP here. Issues raised will be addressed by the LPP developer around the end of May.

 

 
  [ # 9 ]
Steve Worswick - May 28, 2017:

8man - Fortunately, the PCs are all connected via a local switch in a room at Bletchley Park. There’s no outside influence.

...it is in the contest’s best interest to level the field of entrants so that the same people don’t keep coming back…

Interesting thought Robby. If the only way to find a new winner is to make it too complicated for the usual suspects to enter, that seems to be a step backwards to me. Kind of like making Brazil play soccer with only 8 players instead of 11, as the other teams were finding it hard to win. wink

Hugh wanted the contest to be hard. Hugh’s commitment to this was cemented early on by Joseph Weintraub, who won the first few contests with his “PC Therapist.”

Hugh’s LPP design made the entrant’s task harder by introducing fake typing as a prerequisite for winning. There was a long running discussion about “message by message” interactions, as used by most bots on the web, so that the emphasis on language and conversation would prevail against superficial typing algorithms that more easily “out” the computer program.

If the contest is to be designed for non-programmers, it becomes a bot pageant like the Chatterbox Challenge and other popularity based competitions.

Most programmers know at least a little bit about sockets just from putting their bot on the internet, with the exception of companies that do the computer programming for you. Hugh didn’t want some guy in his basement to cheat on his contest, but as you’ve said, the contest is now held on a local network that is not connected to the internet, yet many bot builders have spent time to develop web based programs. Why should these people have to interpret a protocol that is non-intuitive, illogical, and a CPU hog written in perl, which forces every entrant to have to learn perl?  The new protocol is at least generic enough that people who studied programming can use their existing toolbox to interact with it.  I know it is inconvenient to some, who expect to be one of the four finalists every year, to be called on to make some changes. This is the sort of thing that has happened several times in the history of the contest.

1994 restricted topics
1995 unlimited topics
2000 GUI interface required
2003 web based interfaces encouraged
200? LPP #1 introduced
2006 command line installation banned
201? 20 questions finalist screening added

When Hugh was running the contest out of his apartment, by himself, some of these things made sense. Now that AISB runs the contest, hopefully there will be a return to science and progress as a goal, and not just a way for a few people to win prize money at their own convenience.

 

 

 
  [ # 10 ]

I am just trying to help the AISB with good feedback, and hopefully all of us learning about this.

Steve said, “Fortunately, the PCs are all connected via a local switch in a room at Bletchley Park.” 

Hopefully there is no gateway (router) and the room is locked at Bletchley Park, for starters. 

Just pointing out that It says “false” the socket is “secure”.  It is wide open to any Internet connection.

referer'http://chatbot.tk/newLPP/',
        
origin'http://chatbot.tk',
        
cookie'io=yolws3nX7MU-_8POAAAA',
        
connection'keep-alive' },
     
time'Sat May 27 2017 15:04:41 GMT-0400 (EDT)',
     
address'::ffff:127.0.0.1',
     
xdomaintrue,
     
securefalse

This is fun though!  My website is chatting with my laptop.

 

 

 

 
  [ # 11 ]

There’s no router. The switch is in the same room as the contest is being held and is secure.

Not sure why anyone would need to learn PERL in order to enter previous years’ contests though? The only thing you had to use were the mkdir and rmdir commands.

I was under the impression that the aim of the Loebner Prize was to find the most humanlike conversational AI but I maybe misunderstood the contest’s goals.

However, there’s certainly no sour grapes on my part, as perhaps hinted in the thread. I wish everyone who manages to enter, the very best of luck. I will still probably attend the finals day as an interested visitor and will report back as usual but given the current state of the new LPP and just over 5 weeks until the entry cutoff date, I can’t see how I will be able to enter.

 

 
  [ # 12 ]

Hey Steve,

Same kind of thing happened to me in 2000. I probably should have put <sarcasm>science and progress</sarcasm>  I didn’t mean to come off all preachy.

Hugh’s long running feud with Marvin Minsky involved Minsky saying the contest had no purpose and Hugh apparently saying it did.  The audio/visual threat existed pretty much from the beginning, but may never come to pass.

Good luck in whatever you decide to do.

Robby.

 

 
  [ # 13 ]

There have been some changes to the dates published at the top of this thread.

Finals Day: Loebner Prize 2017 Finals on Saturday 16 September 2017 at Bletchley Park

- The deadline for submission for the 2017 competition has been pushed back to Monday 24 July 2017.

- The date of announcement of finalists has been pushed back to Tuesday 15 August 2017.

- The date of the final has not changed, and is still Saturday 16 September 2017 at Bletchley Park.

 

 
  [ # 14 ]

Just a courtesy update for readers,

The link in post #5 has since been

updated to: http://chatbot.ml

 

 
  login or register to react