AI Zone Admin Forum Add your forum

NEWS: survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Natural Language Understanding Services – a comparison

At this years’ SIGdial conference, researchers from the Technical University of Munich, department of informatics published a paper entitled “Evaluating Natural Language Understanding Services for Conversational Question Answering Systems”.

The team compared performance between four services – LUIS, Watson, and RASA, by provisioning each service using large data sets of questions. The data was gathered from multiple sources, a Telegram production chatbot, and two StackExchange platforms – ask ubuntu, and Web Applications. All of the services were provisioned and trained with exactly the same data.

Check it out:


  [ # 1 ]

Reading “using large data sets of questions” suggests everyone is still on the same playing field, I think.  And this, to me, makes Artificial Intelligence very exciting. For example, The Carnegie Mellon Pronouncing Dictionary, is also “using large data sets”. This may provide an easier method of programming a robot to process visemes (mouth postures) to accurately match voice synthesis.

Artificial:  AA2 R T AH0 F IH1 SH AH0 L
:  IH2 N T EH1 L AH0 JH AH0 N S
:  IH1 Z
:  AH0 N
:  IH1 T S
:  IH1 N F AH0 N S IY0 

## The Carnegie Mellon Pronouncing Dictionary [cmudict.0.6] is Copyright 1998
## by Carnegie Mellon University. Use of this dictionary, for any research or
## commercial purpose, is completely unrestricted.  If you make use of or
## redistribute this material, we would appreciate acknowledgement of its
## origin.



  [ # 2 ]

Thank you Mr. Walker. Was this on the NLP People group on LinkedIn?  I was pleased to see Lex mentioned here as my attention was drawn to it recently by a client.

I am struck by the consistency with which this wave of chat bots have the same behavioral structures that are built around the Intent of the conversation, but I also wonder what is under the hood with some of these bots. One hears the same terminology being bantered around, but whatever deeper learning there is, it is buried beneath a veneer of old skool pattern matching.



  [ # 3 ]

Looked at Microsoft Louis, it is a nice development but could go further with sentiment analysis and narrative nesting.


  [ # 4 ]

It would be exciting seeing Watson playing jeopardy against others with the same dataset&training; together smile


  login or register to react