AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Statistical Pattern Matching
 
 

ChatScript has a very strong pattern-matcher, but it still requires the bot author to manually devise sophisticated patterns to match users’ intentions.

I would like to offer an alternative, which I now use in a private bot, and looks very promising - use a statistical text categorizer.

In the training phase, the author should create several sentences that either belong or not-belong to a certain subject (i.e. “movies”, “pets”, etc.).

Then, he/she uses a classifier, such as SVM-Perf or another open-source classifier. The classifier creates a model for each subject.

Then, given an input sentence, the program can use the model for each subject on the input sentence, find the subject with the best fit, and act accordingly. For example, if the subject is “pets”, say something about pets / change the topic to pets / etc.

The sentences for training can be collected from actual user input, so it can be much easier than writing patterns.

I wonder if and how this could be incorporated into ChatScript?

 

 
  [ # 1 ]

This is already done via the <topic> tag in AIML. However, I don’t know if Chatscript has something similar.

<category
<
pattern>I HAVE A DOG</pattern>
<
template>
<
think><set name="topic">PETS</set></think>
What breed is your dog?
</
template>
</
category>

<
topic name="PETS"
<
category
<
pattern>*</pattern>
<
template>
I never had any pets as a child.
</
template>
</
category>
<
topic
 

 
  [ # 2 ]

Steve,

ChatScript has topics just like AIML, but, that was not my point -

my point was, instead of writing “I HAVE A DOG”, which would match only this exact phrase, I would like to be able to write some sample sentences such as “I own a dog”, “my wife has a dog”, etc., and let the computer automatically calculate a general pattern, which would match new unseen sentences too.

I did this with one of my private bots (not ChatScript), and it seems to work very well.

 

 
  [ # 3 ]

Could you do something like this:

<category
<
pattern>_ DOG</pattern>
<
template>
<
random>
<
li>There is a dog in the house next door to me.</li>
<
li>I used to have a dog when I was younger.</li>
<
li>A dog near me barks all day and all night.</li>
</
random>
</
template>
</
category

Then it doesn’t matter if you say, “I have a dog”, “My wife has a dog” or even “The most popular pet for families in the UK is a dog”, then answer would still be displayed.

 

 
  [ # 4 ]

OK, this will match alll sentences that end with ‘dog’, but there are more complex cases for example: a sentence about owning a dog belongs to the “pets” topic, but a sentence about dog bite probably belongs to the “health” topic, etc.

Of course you can handle this by writing more and more detailed patterns, but, alternatively, you can just take some sample sentences and let the classifier to the job for you.

 

 
  [ # 5 ]

This suggestion causes me to wonder if I might be able to create a script that takes the conversation logs from Morti (and other bots), use SVM (or something similar) to categorize the inputs into topics, then write rules based on those topics. Hmmm. I’ll have to look into this. smile

 

 
  login or register to react