AI Zone Admin Forum Add your forum

NEWS: survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

Save/Load Precompiled Brain, Dynamically Load/Dump Modules

Hi! New to these forums.

Currently working on an application that’s using RiveScript in Python (originally started with AIML and switched—I’m liking RiveScript much, much better). I’m building a CLI tool as one of the interfaces, which also has some features for managing and customizing the application.

One thing I liked about AIML was the ability to save/load a brain file. So each time your application boots up, it didn’t have to re-compile it, and the process would be faster. Can this be achieved with the write method? And then using the parse method (if a brain file is present) to re-load the file the next time the application boots? If so, what formats do the files need to be in?

I’m also curious if there’s a good way to dynamically load/unload modules. For the sake of argument, say there was a module for “baseball”. It wouldn’t need to be loaded into memory all the time, but if that specific knowledge base was needed in the conversation, or in a particular install, it could be loaded, and then dumped when no longer necessary.

So from the CLI tool, a user could:

$ bot learn baseball
(bot adds this knowledgebase to its brain and saves the newly created brain file for future application loads)

$ bot forget baseball
(bot removes this module from its brain and saves the now smaller brain file for next application boot).

Or, from chatting:

user: “What do you know about baseball?”

+ what do you know about *
- <call>loadmodule <star></call>

+ ok lets talk about *
- <call>unloadmodule baseball</call><call>loadmodule <star></call>



  [ # 1 ]

So, the `deparse` and `write` methods were originally put in to help me write a bot hosting service where I needed to dynamically modify RiveScript code while giving the user a friendly interface to work with, so they themselves didn’t need to write any RiveScript code.

`deparse()` takes the currently loaded reply data and exports it as an object tree, which you could then go on to manipulate in your program. i.e., to insert a new trigger you’d navigate the structure to where triggers are kept and add your own version.

Then you could take that data structure and give it to `write()` and it would turn the whole structure into RiveScript source and write it to disk.

There are a couple limitations that arise due to this specific use case:

[ul][li]You can’t take the deparsed data structure and load that back in to RiveScript as an alternative to parsing RiveScript source code.[/li]
[li]The deparse() method exports ALL of the bot’s data, so it’s not very great for exporting only a single *.rive file from the bot. Similarly, if you loaded your _entire_ bot, deparse() it, and then write() that data, it would output a single *.rive file that contains the entirety of your bot’s brain all at once.[/li][/ul]

My use case was that my bot hosting service would give the user a list of their *.rive files, allow them to edit a single file at a time, and when they save changes to it (add/modify/delete a trigger), my app would:

[ul][li]In my database I only stored the JSON blobs that are `deparse()` compatible, as this is the easiest to quickly get/send to the front-end.[/li]
[li]On saving changes, I’d modify the deparsed blob and save it back to the database.[/li]
[li]When the user “publishes” their bot, that’s when I’d take all the deparsed blobs from all the files, and one at a time, I would create a new RiveScript instance and `write()` the blob to a file, so I could keep separate files for each “module” of the bot’s brain instead of one massive *.rive file of the entire bot’s personality.[/li][/ul]

Dynamically Loading/Unloading Modules

It used to be possible that you could call `loadFile()` or `stream()` over and over, and identical triggers would overwrite their reply data with the newly parsed stuff. This was because the triggers were kept in a hash map where the trigger texts were the keys, and the overwriting behavior was pretty automatic.

Lately, most of the implementations (Go, JavaScript and Python) have switched to a new model where duplicate triggers are allowed (fixes MANY bugs around %Previous and duplicate triggers), and this means re-parsing existing data would duplicate it rather than fix the existing one, and then matching would probably get unpredictable (which version of the trigger would match? Who knows!)

For unloading data, there isn’t a way to delete triggers right now.

In my bots, I implemented “reload” commands to get my bot to reload all its triggers without needing to fully shut down and reboot itself. For this, I just undefine my RiveScript instance and let it garbage collect itself, and make a new one to replace it.

$rs undef;
$rs = new RiveScript();

Send pull requests?

Some ideas for improvements:

[ul][li]Make it possible to load a RiveScript bot directly from a deparsed object. It could have its own function like `loadDeparsed()` and do the opposite logic to what `deparse()` does (which is: copy everything from the internal data structure into the exported one, so the opposite would just copy everything back in)[/li]
[li]Allow a “mode” when parsing to have it search and replace existing triggers when parsing a new file. This would possibly slow down parsing (becoming O(n^number_of_triggers)), so should only be used when you’re intending to reload data in your bot.[/li]
[li]Add a method to delete a trigger, maybe like `deleteTrigger(topic_name, trigger_pattern)` and it would dive into the internal data structure to find and remove that trigger.[/li][/ul]


  [ # 2 ]

Thanks, this was all super-helpful!

Looks like there are several ways to achieve what I’m trying to do with the current setup, but might be some ways to optimize the process down the road if the use case warrants it.


  [ # 3 ]
Steven Schlotterbeck - Feb 28, 2018:

Thanks, this was all super-helpful!

Looks like there are several ways to achieve what I’m trying to do with the current setup, but might be some ways to optimize the process down the road if the use case warrants it.

I am currently working on final qualifying work in the direction of the preparation of Linguistics, and specifically on the process of developing a chat bot in the language of ChatScript. I have a question, does the concept of gaming classes look like in MMORPG:
concept: ~ dd (blademaster assassin wizard archer)
concept: ~ healer (cleric psychic)
concept: ~ tank (barbarian)
concept: ~ class (~ dd ~ healer ~ tank)


  login or register to react