U.S. patent application number 16/531350 was filed with the patent office on 2020-03-12 for dynamic intent classification based on environment variables.
The applicant listed for this patent is Verint Americas Inc.. Invention is credited to Ian Beaver.
Application Number | 20200082204 16/531350 |
Document ID | / |
Family ID | 67620351 |
Filed Date | 2020-03-12 |
![](/patent/app/20200082204/US20200082204A1-20200312-D00000.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00001.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00002.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00003.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00004.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00005.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00006.png)
![](/patent/app/20200082204/US20200082204A1-20200312-D00007.png)
United States Patent
Application |
20200082204 |
Kind Code |
A1 |
Beaver; Ian |
March 12, 2020 |
DYNAMIC INTENT CLASSIFICATION BASED ON ENVIRONMENT VARIABLES
Abstract
To prevent intent classifiers from potentially choosing intents
that are ineligible for the current input due to policies, dynamic
intent classification systems and methods are provided that
dynamically control the possible set of intents using environment
variables (also referred to as external variables). Associations
between environment variables and ineligible intents, referred to
as culling rules, are used.
Inventors: |
Beaver; Ian; (Spokane,
WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Verint Americas Inc. |
Alpharetta |
GA |
US |
|
|
Family ID: |
67620351 |
Appl. No.: |
16/531350 |
Filed: |
August 5, 2019 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62728144 |
Sep 7, 2018 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06K 9/626 20130101;
G06K 9/6282 20130101; G06F 40/55 20200101; G06F 40/35 20200101;
G06K 9/6256 20130101; G06F 40/253 20200101 |
International
Class: |
G06K 9/62 20060101
G06K009/62; G06F 17/27 20060101 G06F017/27; G06F 17/28 20060101
G06F017/28 |
Claims
1. A dynamic intent classification system, comprising: a database
configured to store a plurality of culling rules based on
environment variables; and a classifier configured to determine an
intent based on a user input and the plurality of culling rules,
and output the intent.
2. The system of claim 1, wherein the database and the classifier
are comprised within a computing device.
3. The system of claim 1, wherein the database and the classifier
are comprised within a chatbot.
4. The system of claim 3, wherein the chatbot is configured to
provide a processed language output based on the intent.
5. The system of claim 1, wherein the classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier.
6. The system of claim 1, wherein the environment variables
comprise at least one of time of day or input channel.
7. The system of claim 1, wherein the classifier is dynamically
alterable using the culling rules to generate a plurality of culled
intents.
8. The system of claim 1, wherein the classifier comprises a
language model, and wherein the classifier is configured to
eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
9. A method of providing a response to input data using dynamic
intent classification based on environment variables, the method
comprising: maintaining a plurality of culling rules based on a
plurality of environment variables, at a natural language
understanding (NLU) component; receiving an input data at the NLU
component; determining an intent for the input data using the
culling rules, at a classifier of the NLU component; and outputting
the intent.
10. The method of claim 9, further comprising providing a processed
language output, by a chatbot, based on the intent.
11. The method of claim 9, further comprising: receiving the
plurality of environment variables at the NLU component; and
determining the plurality of culling rules using the plurality of
environment variables.
12. The method of claim 9, further comprising dynamically altering
the classifier using the culling rules to generate a plurality of
culled intents.
13. The method of claim 9, wherein the classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
14. The method of claim 9, further comprising maintaining the
plurality of culling rules in storage external to the
classifier.
15. The method of claim 9, wherein the classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier.
16. The method of claim 9, wherein the plurality of environment
variables comprise at least one of time of day or input
channel.
17. A method comprising: receiving a plurality of culling rules at
a classifier, wherein the plurality of culling rules are based on a
plurality of environment variables; and dynamically altering the
classifier using the plurality of culling rules.
18. The method of claim 17, further comprising maintaining the
plurality of culling rules in storage external to the
classifier.
19. The method of claim 17, wherein the classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier.
20. The method of claim 17, wherein the classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of priority to U.S.
Provisional Patent Application No. 62/728,144, filed on Sep. 7,
2018, entitled "Dynamic Intent Classification Based on Environment
Variables," the contents of which are hereby incorporated by
reference in their entirety.
BACKGROUND
[0002] Intelligent Virtual Assistants (IVAs) and chatterbots can be
implemented in a multitude of ways, but common to all is a
component for natural language understanding (NLU), which is used
for the translation of user inputs into a semantic representation.
Regardless of the means in which the user is interacting with the
IVA (keyboard, speech, gestures, etc.), the user input is first
converted into text or a numerical feature representation and fed
into the NLU component for analysis. The NLU component maps user
inputs, or conversational turns, to a derived semantic
representation commonly known as the user intention or simply
intent. In the context of natural language processing, intent is
defined as an interpretation of a statement or question that allows
one to formulate the `best` response to the statement.
[0003] The collection of syntax, semantics, and grammar rules that
defines how input language maps to an intent within the NLU
component is referred to as a language model. The NLU component may
perform pre-processing steps such as part-of-speech tagging,
stemming, and entity identification (such as proper names or dates)
before feeding the text into the language model. Each of these
parsers may be trained through machine learning methods or manually
constructed by human experts.
[0004] There are two primary techniques to construct the language
model. The first technique uses rules-based (e.g., human-curated)
methods such as regular expressions or grammars to define the
structure and content of language associated to a particular
intent. By applying these patterns or grammars to the input text,
the correct intent can be determined. The intent associated with
the matching grammar or pattern is selected in cases where there is
no ambiguity. If multiple patterns match, the correct intent may be
selected by a scoring mechanism applied to all patterns that match
or other means such as ontological methods.
[0005] The second technique to construct a language model is
through statistical means. Text that is similar to the expected
input text is labeled (e.g., by humans) to be associated with
particular intentions. This labeled data is then used to train a
model of the distribution of input text over the known intentions
using various machine learning approaches. Common approaches
include support vector machines, neural networks, and decision
trees.
[0006] For either language model construction technique, the
language model can be constructed as a tree, where general
knowledge is understood at the root and specific intents at the
leaves. For example, statistical models can be trained on language
related to particular domains or tasks such as looking up the
weather or ordering food. These specialized models can then be
leaves under a higher-level model that chooses the domain or task
or learned jointly into a single hierarchical model. These trees
can be multiple levels deep depending on how many sub-domain layers
are modeled.
[0007] Regardless of language model design, its purpose is that of
selecting the most likely intention given the user input. In this
way, it performs intent classification, where each intent known by
the NLU component can be considered a class label.
[0008] Current practices are to create an intent classifier for a
pre-defined set of intents. Regardless of the method used to
construct such a classifier, once the classifier is trained, it is
static in that it will always classify a user input into one of the
known classes (intentions). As language models evolve and expand,
this classification task becomes more difficult. When the set of
known classes grows to the thousands, it becomes increasingly
difficult for the NLU component to correctly select the intent.
Additionally, business rules and regulations often require that
specific user behaviors or features be mapped to pre-defined
intents where they can be handled differently for regulatory
compliance, threat handling, or transferred to human customer
service agents. Thus, static intent classifiers are time-consuming
and expensive.
[0009] For example, in the insurance domain, if a user asks an IVA
a policy question, the NLU component must select between all known
intents, even though that particular user may not be eligible or
authorized to access content returned from a large subset of the
known intents. This content filtering and access control is
typically handled downstream of the NLU component, after the intent
is selected. Suppose that of the 2,000 known intents within the
language model, only 1,500 were actually eligible for the given
user due to their purchased policies, location, or time of day. The
NLU component must still consider and correctly eliminate 500 of
the possible intents, which, if the language model consists of
patterns to be applied, can greatly increase processing time.
Regardless of the model type, this consideration between ineligible
classes can decrease accuracy as small sets of exceptions may be
either unrepresented or poorly represented in the language model
training data, leading to false positives.
SUMMARY
[0010] To prevent intent classifiers from potentially choosing
intents that are ineligible for the current input due to policies,
dynamic intent classification systems and methods are provided that
dynamically control the possible set of intents using environment
variables (also referred to as external variables). Associations
between environment variables and ineligible intents, referred to
as culling rules, are used.
[0011] In an implementation, a dynamic intent classification system
is provided. The system includes a database configured to store a
plurality of culling rules based on environment variables; and a
classifier configured to determine an intent based on a user input
and the plurality of culling rules, and output the intent.
[0012] Implementations may include some or all of the following
features. The database and the classifier are comprised within a
computing device. The database and the classifier are comprised
within a chatbot. The chatbot is configured to provide a processed
language output based on the intent. The classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier. The environment variables
comprise at least one of time of day or input channel. The
classifier is dynamically alterable using the culling rules to
generate a plurality of culled intents. The classifier comprises a
language model, and wherein the classifier is configured to
eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
[0013] In an implementation, a method of providing a response to
input data using dynamic intent classification based on environment
variables is provided. The method includes maintaining a plurality
of culling rules based on a plurality of environment variables, at
a natural language understanding (NLU) component; receiving an
input data at the NLU component; determining an intent for the
input data using the culling rules, at a classifier of the NLU
component; and outputting the intent.
[0014] Implementations may include some or all of the following
features. Providing a processed language output, by a chatbot,
based on the intent. Receiving the plurality of environment
variables at the NLU component, and determining the plurality of
culling rules using the plurality of environment variables.
Dynamically altering the classifier using the culling rules to
generate a plurality of culled intents. The classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model. Maintaining the
plurality of culling rules in storage external to the classifier.
The classifier is a rule-based classifier, a tree-based classifier,
a grammar-based classifier, or a statistically-trained classifier.
The plurality of environment variables comprise at least one of
time of day or input channel.
[0015] In an implementation, a method is provided. The method
includes receiving a plurality of culling rules at a classifier,
wherein the plurality of culling rules are based on a plurality of
environment variables; and dynamically altering the classifier
using the plurality of culling rules.
[0016] Implementations may include some or all of the following
features. Maintaining the plurality of culling rules in storage
external to the classifier. The classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier. The classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
[0017] This summary is provided to introduce a selection of
concepts in a simplified form that are further described below in
the detailed description. This summary is not intended to identify
key features or essential features of the claimed subject matter,
nor is it intended to be used to limit the scope of the claimed
subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The foregoing summary, as well as the following detailed
description of illustrative embodiments, is better understood when
read in conjunction with the appended drawings. For the purpose of
illustrating the embodiments, there is shown in the drawings
example constructions of the embodiments; however, the embodiments
are not limited to the specific methods and instrumentalities
disclosed. In the drawings:
[0019] FIG. 1 is an illustration of an exemplary environment for
dynamic intent classification based on environment variables;
[0020] FIG. 2 is an operational flow of an implementation of a
method for providing a response to input data using dynamic intent
classification based on environment variables;
[0021] FIG. 3 is an operational flow of an implementation of a
rules-based method for providing a response to input data using
dynamic intent classification based on environment variables;
[0022] FIG. 4 is an operational flow of an implementation of a
tree-based method for providing a response to input data using
dynamic intent classification based on environment variables;
[0023] FIG. 5 is an operational flow of an implementation of a
grammar-based method for providing a response to input data using
dynamic intent classification based on environment variables;
[0024] FIG. 6 is an operational flow of an implementation of a
statistical-based method for providing a response to input data
using dynamic intent classification based on environment variables;
and
[0025] FIG. 7 shows an exemplary computing environment in which
example embodiments and aspects may be implemented.
DETAILED DESCRIPTION
[0026] FIG. 1 is an illustration of an exemplary environment 100
for dynamic intent classification based on environment variables.
The environment 100 may include a chatbot 170 and a client device
180 in communication through a network 105. The network 105 may be
a variety of network types including the public switched telephone
network (PSTN), a cellular telephone network, and a packet switched
network (e.g., the Internet). Although only one chatbot 170 and one
client device 180 are shown in FIG. 1, there is no limit to the
number of chatbots and client devices that may be supported.
[0027] The chatbot 170 and the client device 180 may each be
implemented using a variety of computing devices such as
smartphones, desktop computers, laptop computers, tablets, set top
boxes, vehicle navigation systems, and video game consoles. Other
types of computing devices may be supported. A suitable computing
device is illustrated in FIG. 7 as the computing device 700.
[0028] The chatbot 170 is in communication with the client device
180 and is capable of conducting a conversation with a user through
the client device 180. The chatbot 170 may comprise a computer
program or artificial intelligence application that conducts a
conversation with the client device 180 via textual methods, in an
implementation. Alternatively or additionally, the chatbot 170 may
conduct a conversation with the client device via auditory methods.
Depending on the implementation, the chatbot 170 may be accessed
via a virtual assistant 172, a messaging app 174, or by other apps
or websites.
[0029] The chatbot 170 comprises a natural language understanding
(NLU) component 110. The NLU component 110 comprises a classifier
126 that determines and provides an intent 129. As described
further herein, the classifier 126 outputs the intent 129. The NLU
component 110 further comprises a pre-processor 123 that receives
and pre-processes the user input 183 and provides it output to the
classifier 126. Culling rules 121, based on or associated with
environment variables 122, are also provided to the classifier 126.
The environment variables 122, and/or the culling rules 121, may be
stored in a database or other memory or storage. The database or
other memory or storage is external to the classifier 126, and is
accessed by the classifier 126 at runtime. In an implementation,
the classifier may be trained by a trainer 125. The trainer 125 may
receive and compile prior knowledge and performance information and
train the classifier 126 accordingly.
[0030] The user input 183, such as chat data, from the client
device 180 is provided to the NLU component 110. While chatbots
encourage conversational interaction, the user input 183 may also
comprise inputs that are keyword searches or non-conversational.
Depending on the implementation, the user input 183 may be provided
directly from the client device 180 (e.g., via the network 105) or
from the chatbot 170 (e.g., via the network 105). The NLU component
110 processes the user input 183 and outputs processed language
output 186 using the intent 129. The intent 129 is the user
intention which is then used to determine how to respond to that
intention. In an implementation, the processed language output 186
may comprise the input's action, object, actor, indirect object,
and/or their respective modifying details.
[0031] In some implementations, the chatbot 170 comprises a
training module 115. The training module 115 may comprise training
data 117 and may be used to train the pre-processor 123. The
training data 117 may be based on human to virtual assistant chat
data. In an implementation, user inputs are captured from a
commercial virtual agent chatbot and used for generating the
training data 117. This data is typically short inputs ranging from
two to 22 words with an average of 8.5 words. These inputs range
between keyword searches and more conversational questions. The
training data 117 may be weighted toward imperatives, statives, and
interrogatives. In an implementation, the training data 117 is hand
tagged.
[0032] The pre-processor 123 receives and processes the user input
data 183 and provides its output to the classifier 126. The
classifier 126 considers the output of the pre-processor in
conjunction with the culling rules 121, and determines an intent
129 to return (i.e., to output as the intent 129).
[0033] The classifier 126 is a dynamic intent classifier, as the
culling rules 121 comprises associations between the environment
variables 122 and ineligible intents. The classifier uses the
culling rules 121 to dynamically control the possible set of
intents. In this manner, the classifier 126 is prevented from
potentially choosing intents that are ineligible for the current
input due to policies, for example.
[0034] FIG. 2 is an operational flow of an implementation of a
method 200 for providing a response to input data using dynamic
intent classification based on environment variables. The method
200 may be implemented using the chatbot 170 in conjunction with
the environment 100, in some implementations.
[0035] At 210, environment variables 122 are received or otherwise
maintained by a natural language understanding (NLU) component,
such as the NLU component 110 of the chatbot 170. The environment
variables 122 may be stored in storage, such as a memory device or
database for example.
[0036] At 220, culling rules 121 are generated using the
environment variables 122.
[0037] At 230, input data, such as the user input 183, is received
and sent to the classifier 126. In some implementations, the input
data is pre-processed by the pre-processor 123 prior to sending to
the classifier 126.
[0038] At 240, the intent 129 of the input data is determined by
the classifier 126 in conjunction with the culling rules 121.
[0039] At 250, the chatbot 170 may generate a response to the input
data based on the determined intent 129 and output the response as
processed language output 186.
[0040] For example, with respect to Example 1, the following
culling rules could be used to prevent intents (represented by
capital letters) relating to specific retirement plans from being
considered based on the current user's ownership of a particular
plan. Such a scenario is an example of business logic being
leveraged to improve the classification accuracy.
EXAMPLE 1
[0041] : has 401k(current user):--A, B, H
[0042] : has 403b(current user):--C, D, H (i.e., remove C, D, H
from possible intents).
[0043] : has RothIRA(current user):--E, F, G, H, I
[0044] Given these culling rules, if a user were to ask an IVA for
a financial company a question about a retirement plan policy,
intents known to the classifier 126 that do not apply to retirement
plans owned by that user will not even be considered. This can
increase classification accuracy by reducing the set of intents to
be considered from. It may also speed classification time if the
language model is implemented in a rules-based method that requires
running all rules against the input and ranking the matches.
[0045] In Example 2, the environmental knowledge of which channel
the user is talking to the IVA on is used to remove any intents
that do not apply to that channel from the language model:
EXAMPLE 2
[0046] : is Web(channel):--A, B
[0047] : is Voice(channel):-- . . .
[0048] : is SMS(channel):-- . . .
[0049] In Example 2, note that some intents are not possible when
talking to the IVA over the telephone or texting. In a banking IVA,
intent A may be to open a new checking account, which is also not
available over texting due to regulatory requirements of identity
verification. Similarly, intent B may represent closing an account.
Both of those actions are available when talking to the IVA on the
company website however, and by removing them from the language
model when they are not possible, misunderstandings related to
account management when talking to the IVA over telephone or by
text can be prevented. Thus, the knowledge of the current user
environment is used to reduce the possible intents for the
classifier 126 to choose between and lower possible confusion, per
user input.
[0050] The embodiments disclosed and contemplated herein use a set
of culling rules 121 (example of which are given in Examples 1 and
2), to dynamically modify the output of the classifier 126 itself
(not invalidate the intent after it has been output by the
classifier 126). This set of culling rules 121 can be modified at
any time and will take effect immediately without requiring any
model retraining or determination of alternative intents. The
application of such culling rules 121 is dependent on the
classification method used, and different approaches are described
herein.
[0051] Some embodiments use intent disablement for rules-based
intent classifiers. For rule-based approaches, whether manually
constructed or empirically machine-learned, it is desirable to
modify the model in the classifier 126 itself in order to realize
performance gains in addition to increase the accuracy. Any pattern
is removed that only applies to the intents that are to be culled.
So those intents cannot be obtained by the classifier 126. In other
words, it will be impossible for the classifier 126 to get to
certain intents, because no pattern exists that will allow the
classifier 126 to get to those certain intents.
[0052] FIG. 3 is an operational flow of an implementation of a
rule-based method 300 for providing a response to input data using
dynamic intent classification based on environment variables. The
method 300 may be implemented using the chatbot 170 in conjunction
with the environment 100, in some implementations.
[0053] At 310, similar to 210, environment variables 122 are
received or otherwise maintained by an NLU component, such as the
NLU component 110 of the chatbot 170. The environment variables 122
may be stored in storage, such as a memory device or database for
example. At 320, similar to 220, culling rules 121 are generated
using the environment variables 122.
[0054] At 330, the classifier 126, which in this embodiment is a
rule-based classifier, is itself modified using the environment
variables 122.
[0055] At 340, input data, such as the user input 183, is received
and sent to the classifier 126. In some implementations, the input
data is pre-processed by the pre-processor 123 prior to sending to
the classifier 126.
[0056] At 350, the intent 129 of the input data is determined by
the classifier 126, which has been modified using the environment
variables, in conjunction with the culling rules 121.
[0057] At 360, the chatbot 170 may generate a response to the input
data based on the determined intent 129 and output the response as
processed language output 186.
[0058] For tree-based methods that include decision trees, for
example, perform a bottom-up pruning of the tree to remove all
nodes and edges that lead to a culled intent. FIG. 4 is an
operational flow of an implementation of a tree-based method 400
for providing a response to input data using dynamic intent
classification based on environment variables. The method 400 may
be implemented using the chatbot 170 in conjunction with the
environment 100, in some implementations.
[0059] At 410, input data, such as the user input 183, is received
and sent to the classifier 126. In some implementations, the input
data is pre-processed by the pre-processor 123 prior to sending to
the classifier 126.
[0060] At 420, environment variables 122 are received or otherwise
maintained by an NLU component, such as the NLU component 110 of
the chatbot 170. The environment variables 122 may be stored in
storage, such as a memory device or database for example.
[0061] At 430, culling rules 121 are generated using the
environment variables 122.
[0062] At 440, the classifier 126, which is a tree-based classifier
in this implementation, is modified using the environment
variables. The modification may be performed by bottom-up pruning
of the tree to remove all nodes and edges that lead to a culled
intent.
[0063] At 450, the intent 129 of the input data is determined by
the classifier 126 in conjunction with the culling rules 121.
[0064] At 460, the chatbot 170 may generate a response to the input
data based on the determined intent 129 and output the response as
processed language output 186.
[0065] At 470, the tree is restored to its original unmodified
form. In this manner, the modification performed at 440 is
removed.
[0066] As an example, using the culling set from Example 1, if the
current user conversing with the financial IVA does not have a Roth
IRA or 403B account, the leaf nodes representing the intentions C,
D, E, F, G, H, and I would be pruned from the tree, and the edges
and nodes leading to them would be recursively removed until the
remaining decision tree had no knowledge of the culled intentions.
On the next user input, the original tree is restored and the
culling would be applied again based on whatever subset of culling
rules corresponded to that input.
[0067] For grammar-based methods, the process is similar, with the
difference being all grammars associated with the culled set of
intentions would be removed. Any dependent grammars on the culled
set would also be removed until the remaining grammars were only
used for determining intentions other than the culled set. FIG. 5
is an operational flow of an implementation of a grammar-based
method 500 for providing a response to input data using dynamic
intent classification based on environment variables. The method
500 may be implemented using the chatbot 170 in conjunction with
the environment 100, in some implementations.
[0068] At 510, input data, such as the user input 183, is received
and sent to the classifier 126. In some implementations, the input
data is pre-processed by the pre-processor 123 prior to sending to
the classifier 126.
[0069] At 520, environment variables 122 are received or otherwise
maintained by an NLU component, such as the NLU component 110 of
the chatbot 170. The environment variables 122 may be stored in
storage, such as a memory device or database for example.
[0070] At 530, culling rules 121 are generated using the
environment variables 122.
[0071] At 540, the classifier 126, which is a grammar-based
classifier in this implementation, is modified using the
environment variables. The modification may be performed by
removing grammars associated with a culled set of intentions.
[0072] At 550, the intent 129 of the input data is determined by
the classifier 126 in conjunction with the culling rules 121.
[0073] At 560, the chatbot 170 may generate a response to the input
data based on the determined intent 129 and output the response as
processed language output 186.
[0074] At 570, the original grammar rules of the classifier are
restored to the original unmodified form. In this manner, the
modification performed at 540 is removed.
[0075] Thus, the language model is dynamically tailored to the
business rules, regulations, or environmental variables applicable
to each individual user input. Any change to the culling rules 121
have immediate effect, and no external language model refinement is
needed.
[0076] Some embodiments use intent disablement for statistical
intent classifiers. With statistically-trained language models such
as support vector machines or neural networks, the language model
cannot be dynamically altered without compromising the models
performance on the remaining intentions. In addition, the
classification time of such models is typically not significantly
slowed by the inclusion of more classes. Therefore, focus on
eliminating culled intentions from the output distribution of the
model. In this way, it is guaranteed that no culled intentions are
returned by the model.
[0077] FIG. 6 is an operational flow of an implementation of a
statistical-based method for providing a response to input data
using dynamic intent classification based on environment variables.
The method 600 may be implemented using the chatbot 170 in
conjunction with the environment 100, in some implementations.
[0078] At 610, environment variables 122 are received or otherwise
maintained by an NLU component, such as the NLU component 110 of
the chatbot 170. The environment variables 122 may be stored in
storage, such as a memory device or database for example.
[0079] At 620, culling rules 121 are generated using the
environment variables 122.
[0080] At 630, input data, such as the user input 183, is received
and sent to the classifier 126, which in this implementation is a
statistical-trained classifier. In some implementations, the input
data is pre-processed by the pre-processor 123 prior to sending to
the classifier 126.
[0081] At 640, the intent 129 of the input data is determined by
the classifier 126 in conjunction with the culling rules 121, by
eliminating culled intentions from the output distribution of the
classifier 126. In an implementation, to perform the culling, apply
a masking layer over the output distribution from a statistical
model. This layer performs a bitwise AND operation between the
intention scores and the culling mask values. This is very fast to
compute and as any culled intentions will have a value of 0 in the
bitmask, they will be eliminated from the model output as any value
ANDed to 0 will return 0. The downstream components to the NLU
component 110 will therefore never see any culled intention, and
will select as the winning intent the highest scoring among the
non-zero values.
[0082] At 650, the chatbot 170 may generate a response to the input
data based on the determined intent 129 and output the response as
processed language output 186.
[0083] FIG. 7 shows an exemplary computing environment in which
example embodiments and aspects may be implemented. The computing
device environment is only one example of a suitable computing
environment and is not intended to suggest any limitation as to the
scope of use or functionality.
[0084] Numerous other general purpose or special purpose computing
devices environments or configurations may be used. Examples of
well-known computing devices, environments, and/or configurations
that may be suitable for use include, but are not limited to,
personal computers, server computers, handheld or laptop devices,
multiprocessor systems, microprocessor-based systems, network
personal computers (PCs), minicomputers, mainframe computers,
embedded systems, distributed computing environments that include
any of the above systems or devices, and the like.
[0085] Computer-executable instructions, such as program modules,
being executed by a computer may be used. Generally, program
modules include routines, programs, objects, components, data
structures, etc. that perform particular tasks or implement
particular abstract data types. Distributed computing environments
may be used where tasks are performed by remote processing devices
that are linked through a communications network or other data
transmission medium. In a distributed computing environment,
program modules and other data may be located in both local and
remote computer storage media including memory storage devices.
[0086] With reference to FIG. 7, an exemplary system for
implementing aspects described herein includes a computing device,
such as computing device 700. In its most basic configuration,
computing device 700 typically includes at least one processing
unit 702 and memory 704. Depending on the exact configuration and
type of computing device, memory 704 may be volatile (such as
random access memory (RAM)), non-volatile (such as read-only memory
(ROM), flash memory, etc.), or some combination of the two. This
most basic configuration is illustrated in FIG. 7 by dashed line
706.
[0087] Computing device 700 may have additional
features/functionality. For example, computing device 700 may
include additional storage (removable and/or non-removable)
including, but not limited to, magnetic or optical disks or tape.
Such additional storage is illustrated in FIG. 7 by removable
storage 708 and non-removable storage 710.
[0088] Computing device 700 typically includes a variety of
computer readable media. Computer readable media can be any
available media that can be accessed by the device 700 and includes
both volatile and non-volatile media, removable and non-removable
media.
[0089] Computer storage media include volatile and non-volatile,
and removable and non-removable media implemented in any method or
technology for storage of information such as computer readable
instructions, data structures, program modules or other data.
Memory 704, removable storage 708, and non-removable storage 710
are all examples of computer storage media. Computer storage media
include, but are not limited to, RAM, ROM, electrically erasable
program read-only memory (EEPROM), flash memory or other memory
technology, CD-ROM, digital versatile disks (DVD) or other optical
storage, magnetic cassettes, magnetic tape, magnetic disk storage
or other magnetic storage devices, or any other medium which can be
used to store the desired information and which can be accessed by
computing device 700. Any such computer storage media may be part
of computing device 700.
[0090] Computing device 700 may contain communication connection(s)
712 that allow the device to communicate with other devices.
Computing device 700 may also have input device(s) 714 such as a
keyboard, mouse, pen, voice input device, touch input device, etc.
Output device(s) 716 such as a display, speakers, printer, etc. may
also be included. All these devices are well known in the art and
need not be discussed at length here.
[0091] It should be understood that the various techniques
described herein may be implemented in connection with hardware
components or software components or, where appropriate, with a
combination of both. Illustrative types of hardware components that
can be used include Field-programmable Gate Arrays (FPGAs),
Application-specific Integrated Circuits (ASICs),
Application-specific Standard Products (ASSPs), System-on-a-chip
systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
The methods and apparatus of the presently disclosed subject
matter, or certain aspects or portions thereof, may take the form
of program code (i.e., instructions) embodied in tangible media,
such as floppy diskettes, CD-ROMs, hard drives, or any other
machine-readable storage medium where, when the program code is
loaded into and executed by a machine, such as a computer, the
machine becomes an apparatus for practicing the presently disclosed
subject matter.
[0092] In an implementation, a dynamic intent classification system
is provided. The system includes a database configured to store a
plurality of culling rules based on environment variables; and a
classifier configured to determine an intent based on a user input
and the plurality of culling rules, and output the intent.
[0093] Implementations may include some or all of the following
features. The database and the classifier are comprised within a
computing device. The database and the classifier are comprised
within a chatbot. The chatbot is configured to provide a processed
language output based on the intent. The classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier. The environment variables
comprise at least one of time of day or input channel. The
classifier is dynamically alterable using the culling rules to
generate a plurality of culled intents. The classifier comprises a
language model, and wherein the classifier is configured to
eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
[0094] In an implementation, a method of providing a response to
input data using dynamic intent classification based on environment
variables is provided. The method includes maintaining a plurality
of culling rules based on a plurality of environment variables, at
a natural language understanding (NLU) component; receiving an
input data at the NLU component; determining an intent for the
input data using the culling rules, at a classifier of the NLU
component; and outputting the intent.
[0095] Implementations may include some or all of the following
features. Providing a processed language output, by a chatbot,
based on the intent. Receiving the plurality of environment
variables at the NLU component, and determining the plurality of
culling rules using the plurality of environment variables.
Dynamically altering the classifier using the culling rules to
generate a plurality of culled intents. The classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model. Maintaining the
plurality of culling rules in storage external to the classifier.
The classifier is a rule-based classifier, a tree-based classifier,
a grammar-based classifier, or a statistically-trained classifier.
The plurality of environment variables comprise at least one of
time of day or input channel.
[0096] In an implementation, a method is provided. The method
includes receiving a plurality of culling rules at a classifier,
wherein the plurality of culling rules are based on a plurality of
environment variables; and dynamically altering the classifier
using the plurality of culling rules.
[0097] Implementations may include some or all of the following
features. Maintaining the plurality of culling rules in storage
external to the classifier. The classifier is a rule-based
classifier, a tree-based classifier, a grammar-based classifier, or
a statistically-trained classifier. The classifier comprises a
language model, and further comprising configuring the classifier
to eliminate ineligible intents using the culling rules without
retraining or redeployment of the language model.
[0098] Although exemplary implementations may refer to utilizing
aspects of the presently disclosed subject matter in the context of
one or more stand-alone computer systems, the subject matter is not
so limited, but rather may be implemented in connection with any
computing environment, such as a network or distributed computing
environment. Still further, aspects of the presently disclosed
subject matter may be implemented in or across a plurality of
processing chips or devices, and storage may similarly be effected
across a plurality of devices. Such devices might include personal
computers, network servers, and handheld devices, for example.
[0099] Although the subject matter has been described in language
specific to structural features and/or methodological acts, it is
to be understood that the subject matter defined in the appended
claims is not necessarily limited to the specific features or acts
described above. Rather, the specific features and acts described
above are disclosed as example forms of implementing the
claims.
* * * * *