Fasten your seat belts! Here you will learn everything you need to know about setting up Natural Language Understanding (NLU). Take off in 3, 2, 1.... 🚀🚀🚀
The first thing to know, your Vizir bot can be set up with two natural language comprehension algorithms: Elastic Search and Intentions/entities. My colleague has detailed it a little here.
Elastic Search => very powerful and efficient keyword intelligence...
Intentions/entities => recognizes intentions and keywords (different from Elastic Search).
In this article, we will see how to set up these two algorithms for an ITSM chatbot (for IT support).
NLU setting Intentions / Entities
To properly set up an NLU Intentions / Entities, the first thing to do is to create the intentions => must be chosen intelligently according to your needs. It is often said that an intention is an action verb. It should also make sense in relation to your categorization in your ITSM tool. In my example, I found that the essential intentions of the future users of this chatbot were :
Connect (to the network / to wifi / to an application / to a remote computer / to vpn, etc...)
Install (an application / a software / a printer, etc...)
Update (an application / a software / a printer, etc...)
Reset (a password for example)
Order equipment (a printer, a computer, etc...)
Report a hardware problem (paper jam, out of paper, blue screen, etc...)
Report a hardware theft
Parameterize (an app, a work station, etc ...)
Declare a newcomer
Declaring a departure
It's easy to imagine other intentions your users might have => you need to identify them to add them to the model (often thanks to the analysis of your tickets in your ITSM tool).
To create these intentions, go to the NLU > Intentions tab and click on "Add an intention", add them one by one.
Then you will need to add training phrases to all these intentions. To do this, click on the "related phrases" button to the right of the name of your intention.
Then you will just have to add sentences 1 by 1 in the text box => they will be added to the list below.
To write these sentences I have a piece of advice: put yourself in the place of an end user of the bot. For example, for the intention "To connect", ask yourself the question: "what sentence could the bot end user say for which I would like the bot to understand that it is an intention to connect". You have 12 examples just above! (Some times you just have to take your ITSM tickets and copy and paste => never forget, nothing like a little bottom-up strategy ;) ).
NB: When you're done writing your training sentences, click on "train NLU" in the top right corner of your screen. Then, publish once the NLU is trained (after the green notification).
Quick tips :
1) In all your intentions, have sentences with basic phrases: "what", "where" "how to", etc... => so as not to bias the comprehension because of turns that are only found in one intention without any particular reason.
2) Build an "other FAQ" intention to classify sentences that don't fit into any other category => ideally it should remain as empty as possible, when a false positive is detected, you can requalify the sentence and assign it to this "other" intention. This intention will need to be reviewed regularly to ensure that a new need has not emerged.
3) Avoid an imbalance in the number of training phrases between your intentions => for example: if an intention A has 90 training phrases and an intention B only 15, your bot will tend to direct its comprehension much more towards intention A.
4) Never train an intention with two identical sentences => simply because it's useless and it would be harder for you to find your way through a large number of sentences.
Translated with www.DeepL.com/Translator (free version)
2) The entities
You may have wondered, why are some words highlighted in my practice sentences? It's because of entity detection! To discover and understand what an entity is, go to the "entities" tab next to "intentions".
As you can see, we have a base of 9 "natural" entities. For example, the pre-set entity "Number" will allow your chatbot to detect numbers and figures in user queries. If you write "I have three feet", the chatbot will associate the word "three" with the "Number" entity.
In your case of ITSM chatbot, what will be powerful is to be able to detect for example: applications, hardware, workstations, etc...
In my example, I set up 3 fundamental entities: Network (wifi, vpn, server, etc...) / Office apps (applications) / Device (printer, pc, smartphone, etc...).
Let's go into more detail on the "Office apps" entity by clicking on the toothed wheel:
First of all, note that one of the great utilities of being able to detect entities is to keep them in memory => therefore to save them. To do this, just check this box and choose the attribute in which to store the entity detected by the bot. Here, I chose to store the detected application in the attribute (the variable) "tracking".
Then, to obtain a chatbot which has a variable behavior according to the detected business application, you must create synonyms; 1 for each business application. In my example, I have "OneDrive / Teams / Excel".
Finally, feed each synonym with the keywords you want to associate to it. To do this, ask yourself for example "when do I want the bot to understand OneDrive". If your answer is: "when the user writes OneDrive or Drive", fill in the synonym "OneDrive" as follows:
NB: all the words you add in the synonyms will be automatically added in the "keywords" section just below.
At this point, your entity is well configured, all you have to do is tag the words associated with your entities in the drive sentences of your intentions. To do this, go back to the "intentions" tab and proceed as in the following video: https://www.loom.com/share/6f4bdfc9bb504effbcb00752bc28af0e. This example is a bit unusual since I had already trained the OneDrive entity => it was automatically detected (and highlighted), so I de-highlighted it and then re-highlighted it in the video.
WARNING: The training sentences of your intentions should never contain one of your entities without it being highlighted, NEVER ! So : check that everything is absolutely well tagged in your training phrases.
So you have trained your intentions and your entities, now you will have to configure the resources that must be accessed thanks to the NLU understanding of the bot.
3) NLU configuration
To create a configuration, it's very simple, just click on the "add configuration" button in the NLU > Configuration tab. Just like this!
You can set up your configuration by clicking on the toothed wheel:
So you have to choose a logic. In my case, I want that when the chatbot understands that this is the "Reset password" intention, it must go to my "Reset Password" procedure.
=> this is the very basic configuration (depending on the detected intention).
You can also set it according to the AND intention of an entity. For example I want the intention "Other FAQ" + the entity "office app" to lead by default to the creation of a ticket. I need to set the following parameters:
Here, I even added the configuration that if the "office app" is equal to "Teams", you can access the Teams FAQ, and if the "office app" is equal to "OneDrive", you can access the OneDrive FAQ. This is what is called logic according to the value of the entity (don't forget to check the box provided for this purpose).
The last case we are interested in here is the "Ask for clarification before searching" option. In our example, we want that when the chatbot understands the Office app intention without entity, that it asks for a precision on the application to which to associate the user's request. Here is the expected behavior:
a) we ask the bot "how to upload a file?". :
b) the bot asks for the precision and we choose Teams :
c) the bot answers the question "how to upload a file" in relation to Teams.
To set this, just create the configuration of the intention "other faq" without entity and check the box "ask for precision before searching". Finally, you need to redirect the logic to your clarification parameterized resource.
Here is my clarification resource :
You know everything about the parameterization of the Intentions / entities algorithm ! You may have asked yourself the question, how did the chatbot answer well on the question about Teams? It's thanks to Elastic Search which operated right after the intentions/entities algorithm! Read the next section ;)
Elastic Search settings
Indeed, Elastic Search is the algorithm that is used for the FAQs. In my example, I implemented two official FAQs: Teams and OneDrive. These two FAQs have more than a hundred resources (as many as there are documentation articles on the official FAQ).
My intention/entity algorithm allows the chatbot to understand if I have a question of type other FAQ, and then to know if it's a question about Teams or OneDrive (with or without a request for clarification). Then, ES will determine which of these 221 resources (if we are in the case of Teams) should be proposed to the user => which one answers his question.
This parameter setting is done using keywords and is therefore carried out at the level of each resource (you can of course integrate it directly into your setup file as seen beforehand in the course). Go to the "understanding" parameterization of the resource. In my example, I'm on the resource "Leave a team", so I added the keyword "leave team" thanks to the input bar.
That's absolutely all there is to do with Elastic Search !
Warning : the thing not to do = to train two different resources with the same keyword => how could the bot choose between the two ?
I can do:
Training Resource a = "leave Team"
Training Resource b = "leave Group"
Because it's not the same group of keywords. If the user writes "leave Team" the bot will go to resource a. If the user writes "leave Group", the bot will go to resource b.
I can't do :
Training Resource a = "leave"; "Team" (two separate lines)
Training Resource b = "leave"; ""Group" (two separate lines)
Because both resources would be trained with the word "leave" independently of the rest, the bot would not be able to choose between resource a or b if the user writes "leave".
Now you know how to set up an NLU for your ITSM chatbot!