AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

How to catch “it’s” ?
 
 

Hello !
I’m facing a problem in pattern matching. Today, when my user typed “It’s unknown_word_in_dictionary” I’m looking to save this unknown word in variable to query an external web API. But, in my code, it seems that the “it’s” word is not well catched by CS.

So, when I use this :

u: ( it's _*) $$myvar = ^original(_0) I got you!
u: ( "it'
s" _*) $$myvar = ^original(_0) I got you! 

It does not work.

If you try a “:prepare” on it, it will write “It’ s”. This beginning of this sentence is not interpreted as “it be”. “it” itself is not flagged as a determinant.

Do you have some hints on how to catch this ?

Thank you in advance for your help smile

 

 
  [ # 1 ]

I found a way to do this :

concept: ~testit [ it's ]

topic: ~mytest keep repeat [ test ]

u: ( ~testit ) I got you!

u: () Nop, I didn'
catch this... 

If you type “test it’s”, it works. Now, I’m just wondering why the POS is not tagging this word as two words “it” and “be”.

 

 
  [ # 2 ]

I might be way off the mark but “It’s” isn’t using the standard apostrophe ‘
Just a guess though as I’ve had to handle similar input in the past with this unusual character.

 

 
  [ # 3 ]

It’s a bit weird because the sentence “I don’t know” is correctly splitted in POS as “I do not know”.
So, the apostrophe being the same for “don’t” and “It’s”, I’m wondering why the POS is dealing this in different way.
Perhaps a missing definition in internal dictionay of CS?

 

 
  [ # 4 ]

I had no trouble having it convert itself into “it is”.  There is no missing contraction definition. Yes, your apostrophe is not std ansi but I put your sentence into a text document and saved it as utf8, after adding “a beautiful day” after it.
Do   :tokenize It’s a beautiful day   and send me what it says or post it here

 

 
  [ # 5 ]

Hello Bruce,

With your text :

:tokenize It’s a beautiful day
TokenControl
DO_SUBSTITUTE_SYSTEM DO_NUMBER_MERGE DO_DATE_MERGE DO_SPELLCHECK DO_INTERJECTION_SPLITTING DO_PARSE NO_WITHIN SPLIT_QUOTE

Original User Input
It's a beautiful day
Tokenized into: It'
s  a  beautiful  day
Spelling changed into
it's  a  beautiful  day
Actual used input: it'
s(It's) a beautiful day

After parse TokenFlags: SPELLCHECK PRESENT USERINPUT FAULTY_PARSE NOT_SENTENCE 

And with this one :

:tokenize It's a beautiful day
TokenControl: DO_SUBSTITUTE_SYSTEM DO_NUMBER_MERGE DO_DATE_MERGE DO_SPELLCHECK DO_INTERJECTION_SPLITTING DO_PARSE NO_WITHIN SPLIT_QUOTE

Original User Input: It'
s a beautiful day
Tokenized into
It's  a  beautiful  day
Spelling changed into: it'
s  a  beautiful  day
Actual used input
it's(It'sa beautiful day

After parse TokenFlags
SPELLCHECK PRESENT USERINPUT FAULTY_PARSE NOT_SENTENCE 

And with the Apple one, just to compare with different forms of apostrophe :

:tokenize It‘s a beautiful day
TokenControl
DO_SUBSTITUTE_SYSTEM DO_NUMBER_MERGE DO_DATE_MERGE DO_SPELLCHECK DO_INTERJECTION_SPLITTING DO_PARSE NO_WITHIN SPLIT_QUOTE

Original User Input
It's a beautiful day
Tokenized into: It'
s  a  beautiful  day
Spelling changed into
it's  a  beautiful  day
Actual used input: it'
s(It's) a beautiful day

After parse TokenFlags: SPELLCHECK PRESENT USERINPUT FAULTY_PARSE NOT_SENTENCE 

And if I try with the sentence “isn’t it”, with my apostrophe.

:tokenize isn't it
TokenControl: DO_SUBSTITUTE_SYSTEM DO_NUMBER_MERGE DO_DATE_MERGE DO_SPELLCHECK DO_INTERJECTION_SPLITTING DO_PARSE NO_WITHIN SPLIT_QUOTE


Original User Input: isn'
t it
Tokenized into
isn't  it
Substituted (contractions ) into: is  not  it
Actual used input: is(isn'
tnot(isn't) it

After parse TokenFlags: DO_CONTRACTIONS PRESENT QUESTIONMARK USERINPUT 

This is very weird that the engine can tokenize “isn’t it” and not “it’s” with the same apostrophe. If you need more test on my side, feel free to ask smile

 

 

 
  [ # 6 ]

I forgot to mention that I’m running the version 8.2 on my laptop and on my server.
Thank you in advance for your help smile

 

 
  login or register to react