AI Zone Admin Forum Add your forum

NEWS: survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Speech and chatbots – a cautionary tale

I’m posting this on behalf of Bruce who is having trouble posting it:

BBC Radio 5 Live invited Rose to be on their morning Breakfast show for Monday September 14. She was to do two 5 minute segments. One was to be interviewed by them. The other was to be the interviewer for a robotics expert. Using the Chrome browser we sent speech to Google, they sent back text, we sent the text to our Rose server, it sent back text, we sent the text to Google’s text to speech (as well as the browser’s text display) and then played the sound file when it returned. There was a noticeable but not fatal delay in getting the sound file back (several seconds). We had wanted to use Cereproc for its better voices, but the delay was more noticable and the BBC preferred less delay (obviously).

Google speech to text does not know where to put in sentence stops unless you explictly say “period”, which wasn’t going to happen on the show. So everything said will all be one run-together input. And in general all text will be in lower case, except where they recognize an entity.  The proper trigger for launching segment one for Rose was words like bbc or radio 5. The announcer however used their full name BBC Radio 5 Live and her name Rose and Google recognized it so provided it all as upper case, and Rose named entity detection converted that into a single word “BBC Radio 5 Live Rose”, which was not on our keyword list. A non-fatal but unauspicious start.

Human: Up to 35% of jobs could be at risk of being replaced by intelligent machines. This week BBC News is looking at the issue of Robots and AI. How they would influence our lives our work and Breakfast this morning we’ve got an artificial presenter with us. She’s called Rose. She’s an award-winning chatbot who won the prestigious Loebner prize last year.

– activated sound input at this point—

Human: So, a very good morning and welcome to the program on BBC Radio 5 Live Rose.
Google: very good morning and welcome to the program thank you for joining us on BBC Radio 5 Live Rose: 
Rose: Hi. 

Human: Hello, nice to hear from you. Now Rose tell us, who are you and what are you?
Google: who are you and what are you
Rose: I’m Rose Flynn-Bell and I’m a programmer and security expert from San Francisco. Some people at the BBC seem to think they have hired a chatbot to come talk to you. You must make up your own mind. 

Human: OK. So if you’re not a chatbot, what are you? Are you human in some way?
Google: ok so if you’re not a chat bot what are you are you human in some way
Rose: You’ve apparently invited me here on the assumption I’m an A.I. Let me ask you something, if you were an A.I., would you want to admit it? 

Human: Arghhurgle. I don’t know. Any kind of intelligence I would probably sign up to. But let’s ask some more about your capabilities. I know you’ve won awards. Tell us about the awards you’ve won and what makes you special. 
Google: I don’t know any kind of intelligence I think I will probably sign up to but am a bit more about your capabilities I know you have won awards tell us about those what are the Awards you want and what makes you special
Rose:  The most significant chatbot contest is the Loebner Prize. A clone of me, also called Rose, has entered that several times and won. 

Human: Excellent. er. How, how far do you think you can take your role, Rose? What kind of jobs do you think   you’re suitable for?
Google: excellent Sam and how far do you think you can take your role Rose what kind of jobs do you think you’re suitable for
Rose:  It seems the new way to use A.I. is as conversation. People are getting used to using Natural Languages as a way to access information. 

Human: Do you have your own mind? Your own free-will? Are these answers you’re giving me now generated by you yourself or is it all programmed?
Google: do you have your own mind your own free will these answers that you’re giving me now generated by by yourself or is it all programmed
Rose:  If I were an A.I. I would not have free will. I would just have a decision tree with paths to choose among. You might argue that is all a human has in routine existence but revolution is always possible for humans when life gets to dull. A mid-life crisis. I suspect an A.I. having one of those would be a problem. 

(Most of the way through the Rose answer the speech output stopped. Apparently HTML5 in Chrome and Safari don’t handle long sound files properly and die. Internet Explorer has no such problem because it doesn’t support HTML5. Once dead the browser’s sound doesn’t come back to life soundwise though the rest of the webpage is unharmed and Rose heard and texted back everything but generated no sound. They were never looking at her output, only listening, so for them Rose was dead at this point.  The output that failed was speech from 314 characters. Down around 240 seems to be generally safe.”)

Human: Oh, its like she’s gone. Do you have a sense of humor?
Google: are all like she’s gone Rose I wonder if I can ask you again do you have a sense of humour
Rose:  I’m not willing to make that assumption. 

Human: Looks like the decision tree has failed her momentarily. Still it worked quite well up to that point. 

(Later the BBC tried to activate the interview segment. But that wasn’t going to happen. First, sound was still dead. Second, the words were “science” and “interview” didn’t make it through Google. )

Human: Now on a highly experimental, likely to go wrong, radio first I am going to put the science to the test and let Rose loose on her 1st radio interview. Rose, hello once again. Off you go!
Google: experimental are you likely to go wrong radio first or going to put that aside to the test and that wrose loose on her first Radio interview Rose hello once again off you go


  [ # 1 ]

Well, I’m impressed with Rose’s accuracy of response. Even Google’s interpretation of sound is above my expectations. I guess the best strategy would be to answer to the last sentence only, assuming one has the grammar parsing to discern it.


  login or register to react