So, Bruce, given that ChatScript can watch the conversation and modify itself,  what are your thoughts about the theoretical limits of what it COULD someday do, and what the rate-limiting factors are on it evolving to that point (or along that axis.)

You’ve put a lot of work into providing for relatively easy development (even by a separated team) of orthogonal topics,  and trying to keep the syntax sufficiently clean and lucid so that a developer can “glance” (or so) at it and see what it is supposed to do or is doing, and not get irretrievably tangled in complexity.  (unlike some earlier products…:) 

With ^export and ^import all kinds of background processing could be added, but let’s say for now that such is limited to database calls,  web-searches,  and patterns that Unix or Perl can detect and parse and hand back to the Engine, to keep most processing in one place.

It seems that such a system is in some sense Turing complete, or whatever the right term is, in that it should be theoretically able to tackle any task given infinit time.  My question is much more practical, namely,  how much could the system evolve in, say, 5 or 10 years time?

I’m trying to open a discussion about what the enzymes or accelerators are that would boost the system’s rate of evolution.  And looking for things more powerful than, say, a color-coded language-specific editor in Eclipse.

What do computational linguists think about ChatScript? What would they add, if they could?

Specifically,  say,  this forum (Chatbots.org) is one type of meta-evolver, which can allow some limited exchange of snippets of useful code and concepts between the man-machine hybrid duals of (developer + their chatbot.)

For example, I’m still considering branching to ALICE - AIML bot in the background instead of going to the “I don’t know how to respond to that.” or other default fall-through response, to deal with off-the-wall input from users.  ALICE has digested a large corpus of human conversational volleys and I don’t know (?) if all of that wisdom can be parsed out and wrapped into a core background ChatScript brain that we all share, can contribute to, and can draw on with system calls.

I could and might write a general interface to MySQL,  so that there’s one stable paved path to look up data that we can improve from.

But it really seems, in some way, that ChatScript should be able to reflect on, learn from, and improve ChatScript code.  Then, instead of asking Bruce or Dave how some snippet of code looks, I could ask the ChatScriptLintBot to read my code and suggest (or make) improvements,
as well as to tease out useful new patterns of general interest and flag those for future use.

I’ll stop here and see what kind of interest there is in this discussion or pointers to other literature I should go read.

Wade

One immediate thought I have on not reinventing the wheel, is some sort of shared way to recognize a growing number of standard English constructs and how to parse them efficiently,  which I know I am not super great at, and which is both time-consuming, rate-limiting, and more boring that the fun stuff of getting into the engrams of a conversational partner’s mind and psyche.

I mean,  why am __I__ tasked with figuring out how to parse “what is your name, rank, and serial number?” or “what is your name and address?” or other expectable queries?  I suppose, cause I get paid for it, or might someday…:)  Still…

Maybe the MotherChatScriptBot could be asked by the worker bees “How would you parse this sentence?: ( xxx )” and it would know the 100, 200,  500, 1000 most common English constructions, and keep adding new ones as it ran into them,
or as the worker-bee chatbots (developers) ran into such constructions and Phoned-home with them to see if anyone had ever heard that phrase before and registered it, yet.

Then, when ANY member of the BORG-BOT learned a new trick, we’d ALL know it.

Wade

Lordy!  You want self-evolving intelligent system do you?

My central claim on chatbots that I make in papers is: 
1. A chatbot does not really understand what you say
2. A chatbot does not actually care about you.
3. It’s all an illusion—smoke and mirrors.
http://noticias.lainformacion.com/ciencia-y-tecnologia/ciencias-general/el-programador-que-desafia-el-test-de-turing_aBVAgpouxS8kjcPcBopAK3/
 
I don’t know that computation linguists have any knowledge of or opinion about ChatScript.

Me, I’d autotranslate all the simple rules of ALICE into chatscript rules (easy enough for a skilled programmer). Calling ALICE in the background is mega expensive if you are trying to run a server. But if performance is not an issue, then whatever you want is fine.

It will be eons before I try to build chatscript lint. Before I would do that, I would build the system that can just read in text (like a wikipedia article or a backstory on a character), integrate facts learned, and be able to use them instead of hand-scripted answers.

I do, of course, for myself, write macros to handle standard kinds of questions like WHAT_IS_YOUR_FAVORITE, WHAT_IS_YOUR_JOB, etc.  A library of those would be handy for everyone, although at present I consider it part of my competitive advantage (since I have open sourced all of chatscript the engine is not an advantage).

Oh,  I did a fast search out of curiosity, and ChatScript is mentioned and compared to other systems in at least one Computational Linguistics journal

Analysing Input in Dialog Systems
Darren Scerri#1, Alexiei Dingli#2
International Journal of Computational Linguistics and Natural Language Processing
Vol 1 Issue 2 September 2012

accessed 10 Nov 2012 at
https://docs.google.com/file/d/0B306nMx7wiLyWkVxWWc1NGtIVmc/edit

Very cool!

Bruce, you’re FAMOUS!!!

Right, Darren published something… he hadn’t told me. And there is an NLP thesis student working with chatscript who loves it: “My initial plan was to have a good baseband system with ChatScript which I would integrate into a major platform (Python and NLTK), which would handle the difficult issues, but unfortunately (?!!?) so far I haven’t encountered a situation which the system can’t handle. So, if I may ask, could you share some underlying topics, methods, theory which I could research and relate to how this system works?  In the next months I will start writing about the whole system with ChatScript and I’ll need references (more on the academic side of NLP than the included documentation).”

My reply to that was: You’re screwed (academically). I don’t work from theory per se. I invent based on need and research.  I have reviewed what others have done in the field, read papers and documentation on chatbots, read tons of academic papers on postagging and parsers, etc. Then I form my opinion of what I think they did well and clever, and what they did badly or doesn’t apply to my world.  The parser world has gone primarily statistical, I go rule based (I’m not done yet) but I note that if I tell the stanford parser “I can fruit” it will label can as an aux verb and fruit as a verb. It will do that for any noun I substitute. And it can’t handle lower case “i love you” correctly because these systems presume WSJ input.  My work is not oriented for academic, its oriented for production grade servers. Hence I will choose restrictions/cheats in design necessary for that goal.  I’m happy to converse on my “theory” as such, but mostly it’s pragmatics. I say “here’s an interesting input or situation”, how can I handle it?
I find I always disagree with existing work, be it AIML, or WordNet ontology or SUMO, or LinkParser’s implementation or NLTK or RELEX, because they don’t do what I need OR because they don’t do it fast enough. In such cases I say… time to reinvent the wheel. Ergo- ChatScript and its integrated support.

I view meaning and language as all about 2 things…

1. ability to inference among facts
2. pattern matching of various flavors.

We speak by agreement on what symbols mean and what sequences of symbols mean. Idioms are specific agreement. Grammar is general agreement, but can be be degraded into ” me likee popsickle ” and still be understood.  Patterns of meanings (synonyms) are better than patterns of words, hence the concept orientation of ChatScript.  And each word comes with its own mini concept set (its normalized forms).  So I created the dual-stream of input interpretation—normal and canonical and the ability to match a concept with thousands of members at the same speed as matching a single word.  I haven’t seen that done anywhere else.

It seems to me that there is room for a developer’s tool or process that could make a developer 5 to 10 times as productive,  or cut development time from 8 months to 8 weeks for a fluent developer to complete a new bot personality—which could be sold the for same price, regardless how much time it took to build, since the price is a function of what the utility is to the customer, not how hard the work was to accomplish.

There are two orthogonal processes involved in developing a chatbot, it seems to me.  One is the chatbot or army of chatbot clones carrying on conversations with users,  building logs of the conversations and, potentially,  flagging sections of conversations that worked and sections that were a problem,  and possibly at the same time flagging rules that seemed solidly satisfactory and rules associated with confusion.

The second process involves the meta-conversation: going through the logs and tracking down rules / patterns that are wrong, or could be improved or sharpened,  proposing an improvement or change,  confirming it improved THIS conversational volley,  checking to see if it broke anything else (regression testing),  and accepting or revising the change based on that.

The second process (“development”) is currently 100% done by a “developer” with, generally, no computer assistance other than complying with demands by the developer to trace this, that, or the other thing.  Those steps are surely both labor intensive and bottlenecks,  depending on scarce and expensive humans to get done, or neglected.

So, I’m asking the question,  what can be done by other humans ( such as the many users) or by a Developer’s Assistant Chatbot,  to make that second process less rate-limiting. 

For example,  could the chatbot, during conversation with users,  upon hitting a brick wall,  break temporarily into LEARNER mode and ask the user to rephrase the last input and have a mini tangential conversation to help the chatbot understand what the user meant in different words that perhaps it does understand.    Then mark that for the later conversation between chatbot-assistant and developer to encode a new rule that the chatbot-assistant suggests.

Or,  picturing a near-future-state Integrated Chatbot Developer Environment Workbench,  could the user click “next problem”,  pull up a rule-set and conversational snippet,  click a pull-down menu to see suggested alternative structures to go in that place,  pick the right one,  click “verify”, and “accept” and move on.

Even that still involves the developer heavily.  Maybe the chatbot-assistant could search it’s library for tentative rules and structures and patterns,  find ones that match,    suggest them to the developer and say “would this be ok?” and reduce the developer’s job to “Yes” or “No and here’s why…”

Or,  again, maybe some of that conversation about enhanced pattern detection and rules could be done with the chatbot briefly putting on a different hat / (control structure),  dialoging with the user to suggest a few things, and if the user is happy,  going ahead and modifying itself on it’s own authority, pending later review by the “developer”, who could then say “What did you learn today Angela?” and Angela could show the situations it encountered and believes it and the conversational partner “worked out” and what it learned from that.

Wade

I love your thoughts. 

I don’t at present know how to implement them, particularly in the limited resource/time I have to devote to that task.

bruce