U.S. patent application number 16/515255 was filed with the patent office on 2021-01-21 for system, method, and computer program product for determining a reason for a deep learning model output.
The applicant listed for this patent is Visa International Service Association. Invention is credited to Claudia Carolina Barcenas Cardenas, Yiwei Cai, Dan Wang, Sheng Wang, Hangqi Zhao.
Application Number | 20210019753 16/515255 |
Document ID | / |
Family ID | 1000004213888 |
Filed Date | 2021-01-21 |
![](/patent/app/20210019753/US20210019753A1-20210121-D00000.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00001.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00002.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00003.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00004.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00005.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00006.png)
![](/patent/app/20210019753/US20210019753A1-20210121-D00007.png)
![](/patent/app/20210019753/US20210019753A1-20210121-M00001.png)
![](/patent/app/20210019753/US20210019753A1-20210121-M00002.png)
![](/patent/app/20210019753/US20210019753A1-20210121-M00003.png)
View All Diagrams
United States Patent
Application |
20210019753 |
Kind Code |
A1 |
Zhao; Hangqi ; et
al. |
January 21, 2021 |
System, Method, and Computer Program Product for Determining a
Reason for a Deep Learning Model Output
Abstract
A system, method, and product for determining a reason for a
deep learning model output that obtain training data associated
with training samples and first labels for the training samples;
train a first model using the training samples and the first
labels, training the first model generating predictions for the
training samples; train a second model using the training samples
and the predictions as second labels for the training samples;
extract one or more weights of the trained second model; process,
using the first model, input data including features associated
with at least one sample, to generate output data, the output data
including at least one prediction for the at least one sample; and
apply the one or more extracted weights to the features to
determine one or more contributions of one or more features of the
features to the at least one prediction for the at least one
sample.
Inventors: |
Zhao; Hangqi; (Austin,
TX) ; Wang; Sheng; (Austin, TX) ; Wang;
Dan; (Austin, TX) ; Cai; Yiwei; (Austin,
TX) ; Barcenas Cardenas; Claudia Carolina; (Austin,
TX) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Visa International Service Association |
San Francisco |
CA |
US |
|
|
Family ID: |
1000004213888 |
Appl. No.: |
16/515255 |
Filed: |
July 18, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 17/16 20130101;
G06N 3/0454 20130101; G06F 17/18 20130101; G06Q 20/4016
20130101 |
International
Class: |
G06Q 20/40 20060101
G06Q020/40; G06N 3/04 20060101 G06N003/04; G06F 17/18 20060101
G06F017/18; G06F 17/16 20060101 G06F017/16 |
Claims
1. A computer-implemented method comprising: obtaining, with at
least one processor, training data associated with a plurality of
training samples and a plurality of first labels for the plurality
of training samples; training, with at least one processor, a first
model using the plurality of training samples and the plurality of
first labels for the plurality of training samples, wherein
training the first model generates a plurality of predictions for
the plurality of training samples; training, with at least one
processor, a second model using the plurality of training samples
and the plurality of predictions as a plurality of second labels
for the plurality of training samples; extracting, with at least
one processor, one or more weights of the trained second model;
processing, with at least one processor using the first model,
input data including a plurality of features associated with at
least one sample, to generate output data, wherein the output data
includes at least one prediction for the at least one sample; and
applying, with at least one processor, the one or more extracted
weights to the plurality of features to determine one or more
contributions of one or more features of the plurality of features
to the at least one prediction for the at least one sample.
2. The computer-implemented method of claim 1, wherein the input
data includes transaction data, wherein the at least one sample is
associated with at least one transaction, and wherein the at least
one prediction includes at least one probability of whether the at
least one transaction is a fraudulent transaction.
3. The computer-implemented method of claim 2, wherein the one or
more contributions are determined in real-time during processing of
the at least one transaction in a transaction processing
network.
4. The computer-implemented method of claim 1, wherein the first
model includes a greater number of hidden layers than the second
model.
5. The computer-implemented method of claim 1, further comprising:
ranking, with at least one processor, the plurality of features
associated with the at least one sample according to the one or
more contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one
sample; and identifying, with at least one processor, at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction.
6. The computer-implemented method of claim 1, wherein the one or
more extracted weights include a plurality of extracted weights,
wherein the plurality of extracted weights is associated with a
plurality of hidden units of at least one hidden layer of the
trained second model, and wherein the one or more contributions of
the one or more features of the plurality of features to the at
least one prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
7. The computer-implemented method of claim 1, further comprising:
determining, by at least one processor, a contribution score for
each contribution of the one or more contributions of the one or
more features of the plurality of features to the at least one
prediction for the at least one sample according to the following
Equations (1) to (4): h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0
, h j ) ( 2 ) y m = j = 1 H a j * v jm ( 3 ) Contribution ( x i
-> y m ) = j = 1 H ( x i * u ij ) * ( a j * v jm ) ( 4 )
##EQU00008## wherein Contribution (x.sub.i->y.sub.m) is the
contribution score of a feature x.sub.i to a prediction y.sub.m,
wherein K is a number of the plurality of features, wherein H is a
number of hidden units in a hidden layer of the trained second
model, wherein M is a number of output units of an output layer of
the trained second model, wherein U and V are weight matrices,
wherein a number of weight matrices U is equal to K.times.H,
wherein a number of weight matrices V is equal to H.times.M,
wherein u.sub.ij is a weight matrix applied to the feature on a
path to a hidden unit h.sub.j, wherein a.sub.j is an activation
unit of an activation layer between the hidden layer and the output
layer of the trained second model, wherein v.sub.jm iis a weight
matrix applied to an output of the activation unit a.sub.j on a
path to the prediction y.sub.m.
8. A computing system comprising: one or more processors programmed
and/or configured to: obtain training data associated with a
plurality of training samples and a plurality of first labels for
the plurality of training samples; train a first model using the
plurality of training samples and the plurality of first labels for
the plurality of training samples, wherein training the first model
generates a plurality of predictions for the plurality of training
samples; train a second model using the plurality of training
samples and the plurality of predictions as a plurality of second
labels for the plurality of training samples; extract one or more
weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, wherein the output
data includes at least one prediction for the at least one sample;
and apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample.
9. The computing system of claim 8, wherein the input data includes
transaction data, wherein the at least one sample is associated
with at least one transaction, and wherein the at least one
prediction includes at least one probability of whether the at
least one transaction is a fraudulent transaction.
10. The computing system of claim 9, wherein the one or more
contributions are determined in real-time during processing of the
at least one transaction in a transaction processing network.
11. The computing system of claim 8, wherein the first model
includes a greater number of hidden layers than the second
model.
12. The computing system of claim 8, wherein the one or more
processors are further programmed and/or configured to: rank the
plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identify at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction.
13. The computing system of claim 8, wherein the one or more
extracted weights include a plurality of extracted weights, wherein
the plurality of extracted weights is associated with a plurality
of hidden units of at least one hidden layer of the trained second
model, and wherein the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
14. The computing system of claim 8, wherein the one or more
processors are further programmed and/or configured to: determine a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4): h j = i = 1 K x i
* u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j = 1 H a j * v jm (
3 ) Contribution ( x i -> y m ) = j = 1 H ( x i * u ij ) * ( a j
* v jm ) ( 4 ) ##EQU00009## wherein Contribution
(x.sub.i->y.sub.m) is the contribution score of a feature to a
prediction y.sub.m, wherein K is a number of the plurality of
features, wherein H is a number of hidden units in a hidden layer
of the trained second model, wherein M is a number of output units
of an output layer of the trained second model, wherein U and V are
weight matrices, wherein a number of weight matrices U is equal to
K.times.H, wherein a number of weight matrices V is equal to
H.times.M, wherein u.sub.ij is a weight matrix applied to the
feature on a path to a hidden unit h.sub.j, wherein a.sub.j is an
activation unit of an activation layer between the hidden layer and
the output layer of the trained second model, wherein v.sub.im is a
weight matrix applied to an output of the activation unit a.sub.j
on a path to the prediction y.sub.m.
15. A computer program product comprising at least one
non-transitory computer-readable medium including program
instructions that, when executed by at least one processor, cause
the at least one processor to: obtain training data associated with
a plurality of training samples and a plurality of first labels for
the plurality of training samples; train a first model using the
plurality of training samples and the plurality of first labels for
the plurality of training samples, wherein training the first model
generates a plurality of predictions for the plurality of training
samples; train a second model using the plurality of training
samples and the plurality of predictions as a plurality of second
labels for the plurality of training samples; extract one or more
weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, wherein the output
data includes at least one prediction for the at least one sample;
and apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample.
16. The computer program product of claim 15, wherein the input
data includes transaction data, wherein the at least one sample is
associated with at least one transaction, wherein the at least one
prediction includes at least one probability of whether the at
least one transaction is a fraudulent transaction, and wherein the
one or more contributions are determined in real-time during
processing of the at least one transaction in a transaction
processing network.
17. The computer program product of claim 15, wherein the first
model includes a greater number of hidden layers than the second
model.
18. The computer program product of claim 15, wherein the one or
more processors are further programmed and/or configured to: rank
the plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identify at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction.
19. The computer program product of claim 15, wherein the one or
more extracted weights include a plurality of extracted weights,
wherein the plurality of extracted weights is associated with a
plurality of hidden units of at least one hidden layer of the
trained second model, and wherein the one or more contributions of
the one or more features of the plurality of features to the at
least one prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
20. The computer program product of claim 15, wherein the one or
more processors are further programmed and/or configured to:
determine a contribution score for each contribution of the one or
more contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4): h j = i = 1 K x i
* u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j = 1 H a j * v jm (
3 ) Contribution ( x i -> y m ) = j = 1 H ( x i * u ij ) * ( a j
* v jm ) ( 4 ) ##EQU00010## wherein Contribution
(x.sub.i->y.sub.m) is the contribution score of a feature to a
prediction y.sub.m, wherein K is a number of the plurality of
features, wherein H is a number of hidden units in a hidden layer
of the trained second model, wherein M is a number of output units
of an output layer of the trained second model, wherein U and V are
weight matrices, wherein a number of weight matrices U is equal to
K.times.H, wherein a number of weight matrices V is equal to
H.times.M, wherein u.sub.ij is a weight matrix applied to the
feature on a path to a hidden unit h.sub.j, wherein a.sub.j is an
activation unit of an activation layer between the hidden layer and
the output layer of the trained second model, wherein v.sub.jm is a
weight matrix applied to an output of the activation unit a.sub.j
on a path to the prediction y.sub.m.
Description
BACKGROUND
1. Field
[0001] This disclosure relates generally to model interpretability
and, in some embodiments or aspects, to a system, a method, and a
computer program product for determining a reason for a deep
learning model output.
2. Technical Considerations
[0002] Deep learning models may include millions of parameters and
be trained using billions of examples, which can offer larger
capacity and better generalization performance that is difficult
for simpler models to match. However, deep learning models may also
have decreased interpretability as compared to simpler models. For
example, deep learning models may be compared to black boxes, which
can often give accurate results, but which also lack
interpretability. As an example, explaining the results of a deep
learning model, such as by identifying reason codes for decisions
made by the deep learning model, is typically very challenging.
[0003] Accordingly, with the involvement of more complex deep
learning models in decision making, it is more difficult for users
to perceive the underlying reasoning process that leads to
decisions by the more complex deep learning models. However, in
many scenarios, it is helpful and/or necessary to provide reasons
(e.g., which features contribute more to a prediction for a
particular sample, reason codes associated therewith, etc.) for
deep learning applications. For example, model interpretation is
likely to remain a vital concern in the payment industry for years
to come, because if users do not trust a model or the decisions
thereof, the users will not implement the model or the decisions
thereof. As an example, issuers may want to know a reason why a
fraud risk of a transaction is predicted to decline the
transaction. To make the task of model interpretation more
challenging, reasons for model outputs or predictions may be
requested and/or required in real-time for making decisions in a
timelier manner, such as in fraud prediction applications, and/or
the like.
[0004] Accordingly, there is a need in the art to improve
determination of a reason for a deep learning model output.
SUMMARY
[0005] Accordingly, provided are systems, methods, and computer
program products for improving determination of a reason for a deep
learning model output.
[0006] According to some non-limiting embodiments or aspects,
provided is a computer-implemented method including: obtaining,
with at least one processor, training data associated with a
plurality of training samples and a plurality of first labels for
the plurality of training samples; training, with at least one
processor, a first model using the plurality of training samples
and the plurality of first labels for the plurality of training
samples, training the first model generating a plurality of
predictions for the plurality of training samples; training, with
at least one processor, a second model using the plurality of
training samples and the plurality of predictions as a plurality of
second labels for the plurality of training samples; extracting,
with at least one processor, one or more weights of the trained
second model; processing, with at least one processor using the
first model, input data including a plurality of features
associated with at least one sample, to generate output data, the
output data including at least one prediction for the at least one
sample; and applying, with at least one processor, the one or more
extracted weights to the plurality of features to determine one or
more contributions of one or more features of the plurality of
features to the at least one prediction for the at least one
sample.
[0007] In some non-limiting embodiments or aspects, the input data
includes transaction data, the at least one sample is associated
with at least one transaction, and the at least one prediction
includes at least one probability of whether the at least one
transaction is a fraudulent transaction.
[0008] In some non-limiting embodiments or aspects, the one or more
contributions are determined in real-time during processing of the
at least one transaction in a transaction processing network.
[0009] In some non-limiting embodiments or aspects, the first model
includes a greater number of hidden layers than the second
model.
[0010] In some non-limiting embodiments or aspects, the method
further includes: ranking, with at least one processor, the
plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identifying, with at
least one processor, at least one feature of the plurality of
features with respect to at least one other feature of the
plurality of features based on the ranked plurality of features as
at least one reason code associated with the at least one
prediction.
[0011] In some non-limiting embodiments or aspects, the one or more
extracted weights include a plurality of extracted weights, the
plurality of extracted weights is associated with a plurality of
hidden units of at least one hidden layer of the trained second
model, and the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0012] In some non-limiting embodiments or aspects, the method
further includes: determining, by at least one processor, a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00001##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature x.sub.i to a prediction y.sub.m, wherein K is a
number of the plurality of features, wherein H is a number of
hidden units in a hidden layer of the trained second model, wherein
M is a number of output units of an output layer of the trained
second model, wherein U and V are weight matrices, wherein a number
of weight matrices U is equal to K.times.H, wherein a number of
weight matrices V is equal to H.times.M, wherein u.sub.ij is a
weight matrix applied to the feature on a path to a hidden unit
h.sub.j, wherein a.sub.j is an activation unit of an activation
layer between the hidden layer and the output layer of the trained
second model, wherein v.sub.jm is a weight matrix applied to an
output of the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0013] According to some non-limiting embodiments or aspects,
provided is a computing system including: one or more processors
programmed and/or configured to: obtain training data associated
with a plurality of training samples and a plurality of first
labels for the plurality of training samples; train a first model
using the plurality of training samples and the plurality of first
labels for the plurality of training samples, training the first
model generating a plurality of predictions for the plurality of
training samples; train a second model using the plurality of
training samples and the plurality of predictions as a plurality of
second labels for the plurality of training samples; extract one or
more weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, the output data
including at least one prediction for the at least one sample; and
apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample.
[0014] In some non-limiting embodiments or aspects, the input data
includes transaction data, the at least one sample is associated
with at least one transaction, and the at least one prediction
includes at least one probability of whether the at least one
transaction is a fraudulent transaction.
[0015] In some non-limiting embodiments or aspects, the one or more
contributions are determined in real-time during processing of the
at least one transaction in a transaction processing network.
[0016] In some non-limiting embodiments or aspects, the first model
includes a greater number of hidden layers than the second
model.
[0017] In some non-limiting embodiments or aspects, the one or more
processors are further programmed and/or configured to: rank the
plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identify at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction.
[0018] In some non-limiting embodiments or aspects, the one or more
extracted weights include a plurality of extracted weights, the
plurality of extracted weights is associated with a plurality of
hidden units of at least one hidden layer of the trained second
model, and the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0019] In some non-limiting embodiments or aspects, the one or more
processors are further programmed and/or configured to: determine a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00002##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature to a prediction y.sub.m, wherein K is a number
of the plurality of features, wherein H is a number of hidden units
in a hidden layer of the trained second model, wherein M is a
number of output units of an output layer of the trained second
model, wherein U and V are weight matrices, wherein a number of
weight matrices U is equal to K.times.H, wherein a number of weight
matrices V is equal to H.times.M, wherein u.sub.ij is a weight
matrix applied to the feature on a path to a hidden unit h.sub.j,
wherein a.sub.j is an activation unit of an activation layer
between the hidden layer and the output layer of the trained second
model, wherein v.sub.jmis a weight matrix applied to an output of
the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0020] According to some non-limiting embodiments or aspects,
provided is a computer program product including at least one
non-transitory computer-readable medium including program
instructions that, when executed by at least one processor, cause
the at least one processor to: obtain training data associated with
a plurality of training samples and a plurality of first labels for
the plurality of training samples; train a first model using the
plurality of training samples and the plurality of first labels for
the plurality of training samples, training the first model
generating a plurality of predictions for the plurality of training
samples; train a second model using the plurality of training
samples and the plurality of predictions as a plurality of second
labels for the plurality of training samples; extract one or more
weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, the output data
including at least one prediction for the at least one sample; and
apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample.
[0021] In some non-limiting embodiments or aspects, the input data
includes transaction data, the at least one sample is associated
with at least one transaction, the at least one prediction includes
at least one probability of whether the at least one transaction is
a fraudulent transaction, and the one or more contributions are
determined in real-time during processing of the at least one
transaction in a transaction processing network.
[0022] In some non-limiting embodiments or aspects, the first model
includes a greater number of hidden layers than the second
model.
[0023] In some non-limiting embodiments or aspects, the one or more
processors are further programmed and/or configured to: rank the
plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identify at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction.
[0024] In some non-limiting embodiments or aspects, the one or more
extracted weights include a plurality of extracted weights, the
plurality of extracted weights is associated with a plurality of
hidden units of at least one hidden layer of the trained second
model, and the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample is determined by:
calculating a score for each path of a plurality of paths of the
one or more features through the second model based on the
plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0025] In some non-limiting embodiments or aspects, the one or more
processors are further programmed and/or configured to: determine a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00003##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature to a prediction y.sub.m, wherein K is a number
of the plurality of features, wherein H is a number of hidden units
in a hidden layer of the trained second model, wherein M is a
number of output units of an output layer of the trained second
model, wherein U and V are weight matrices, wherein a number of
weight matrices U is equal to K.times.H, wherein a number of weight
matrices V is equal to H.times.M, wherein u.sub.ij is a weight
matrix applied to the feature on a path to a hidden unit h.sub.j,
wherein a.sub.j is an activation unit of an activation layer
between the hidden layer and the output layer of the trained second
model, wherein v.sub.jm is a weight matrix applied to an output of
the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0026] Further embodiments or aspects are set forth in the
following numbered clauses:
[0027] Clause 1. A computer-implemented method comprising:
obtaining, with at least one processor, training data associated
with a plurality of training samples and a plurality of first
labels for the plurality of training samples; training, with at
least one processor, a first model using the plurality of training
samples and the plurality of first labels for the plurality of
training samples, wherein training the first model generates a
plurality of predictions for the plurality of training samples;
training, with at least one processor, a second model using the
plurality of training samples and the plurality of predictions as a
plurality of second labels for the plurality of training samples;
extracting, with at least one processor, one or more weights of the
trained second model; processing, with at least one processor using
the first model, input data including a plurality of features
associated with at least one sample, to generate output data,
wherein the output data includes at least one prediction for the at
least one sample; and applying, with at least one processor, the
one or more extracted weights to the plurality of features to
determine one or more contributions of one or more features of the
plurality of features to the at least one prediction for the at
least one sample.
[0028] Clause 2. The computer-implemented method of clause 1,
wherein the input data includes transaction data, wherein the at
least one sample is associated with at least one transaction, and
wherein the at least one prediction includes at least one
probability of whether the at least one transaction is a fraudulent
transaction.
[0029] Clause 3. The computer-implemented method of any of clauses
1 and 2, wherein the one or more contributions are determined in
real-time during processing of the at least one transaction in a
transaction processing network.
[0030] Clause 4. The computer-implemented method of any of clauses
1-3, wherein the first model includes a greater number of hidden
layers than the second model.
[0031] Clause 5. The computer-implemented method of any of clauses
1-4, further comprising: ranking, with at least one processor, the
plurality of features associated with the at least one sample
according to the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample; and identifying, with at
least one processor, at least one feature of the plurality of
features with respect to at least one other feature of the
plurality of features based on the ranked plurality of features as
at least one reason code associated with the at least one
prediction.
[0032] Clause 6. The computer-implemented method of any of clauses
1-5, wherein the one or more extracted weights include a plurality
of extracted weights, wherein the plurality of extracted weights is
associated with a plurality of hidden units of at least one hidden
layer of the trained second model, and wherein the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
is determined by: calculating a score for each path of a plurality
of paths of the one or more features through the second model based
on the plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0033] Clause 7. The computer-implemented method of any of clauses
1-6, further comprising: determining, by at least one processor, a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00004##
[0034] wherein Contribution (x.sub.i-<y.sub.m) is the
contribution score of a feature to a prediction y.sub.m, wherein K
is a number of the plurality of features, wherein H is a number of
hidden units in a hidden layer of the trained second model, wherein
M is a number of output units of an output layer of the trained
second model, wherein U and V are weight matrices, wherein a number
of weight matrices U is equal to K.times.H, wherein a number of
weight matrices V is equal to H.times.M, wherein u.sub.ij is a
weight matrix applied to the feature on a path to a hidden unit
h.sub.j, wherein a.sub.j is an activation unit of an activation
layer between the hidden layer and the output layer of the trained
second model, wherein v.sub.jm is a weight matrix applied to an
output of the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0035] Clause 8. A computing system comprising: one or more
processors programmed and/or configured to: obtain training data
associated with a plurality of training samples and a plurality of
first labels for the plurality of training samples; train a first
model using the plurality of training samples and the plurality of
first labels for the plurality of training samples, wherein
training the first model generates a plurality of predictions for
the plurality of training samples; train a second model using the
plurality of training samples and the plurality of predictions as a
plurality of second labels for the plurality of training samples;
extract one or more weights of the trained second model; process,
using the first model, input data including a plurality of features
associated with at least one sample, to generate output data,
wherein the output data includes at least one prediction for the at
least one sample; and apply the one or more extracted weights to
the plurality of features to determine one or more contributions of
one or more features of the plurality of features to the at least
one prediction for the at least one sample.
[0036] Clause 9. The computing system of clause 8, wherein the
input data includes transaction data, wherein the at least one
sample is associated with at least one transaction, and wherein the
at least one prediction includes at least one probability of
whether the at least one transaction is a fraudulent
transaction.
[0037] Clause 10. The computing system of any of clauses 8 and 9,
wherein the one or more contributions are determined in real-time
during processing of the at least one transaction in a transaction
processing network.
[0038] Clause 11. The computing system of any of clauses 8-10,
wherein the first model includes a greater number of hidden layers
than the second model.
[0039] Clause 12. The computing system of any of clauses 8-11,
wherein the one or more processors are further programmed and/or
configured to: rank the plurality of features associated with the
at least one sample according to the one or more contributions of
the one or more features of the plurality of features to the at
least one prediction for the at least one sample; and identify at
least one feature of the plurality of features with respect to at
least one other feature of the plurality of features based on the
ranked plurality of features as at least one reason code associated
with the at least one prediction.
[0040] Clause 13. The computing system of any of clauses 8-12,
wherein the one or more extracted weights include a plurality of
extracted weights, wherein the plurality of extracted weights is
associated with a plurality of hidden units of at least one hidden
layer of the trained second model, and wherein the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
is determined by: calculating a score for each path of a plurality
of paths of the one or more features through the second model based
on the plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0041] Clause 14. The computing system of any of clauses 8-13,
wherein the one or more processors are further programmed and/or
configured to: determine a contribution score for each contribution
of the one or more contributions of the one or more features of the
plurality of features to the at least one prediction for the at
least one sample according to the following Equations (1) to
(4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00005##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature to a prediction y.sub.m, wherein K is a number
of the plurality of features, wherein H is a number of hidden units
in a hidden layer of the trained second model, wherein M is a
number of output units of an output layer of the trained second
model, wherein U and V are weight matrices, wherein a number of
weight matrices U is equal to K.times.H, wherein a number of weight
matrices V is equal to H.times.M, wherein u.sub.ij is a weight
matrix applied to the feature x.sub.i on a path to a hidden unit
h.sub.j, wherein a.sub.j is an activation unit of an activation
layer between the hidden layer and the output layer of the trained
second model, wherein v.sub.jm m is a weight matrix applied to an
output of the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0042] Clause 15. A computer program product comprising at least
one non-transitory computer-readable medium including program
instructions that, when executed by at least one processor, cause
the at least one processor to: obtain training data associated with
a plurality of training samples and a plurality of first labels for
the plurality of training samples; train a first model using the
plurality of training samples and the plurality of first labels for
the plurality of training samples, wherein training the first model
generates a plurality of predictions for the plurality of training
samples; train a second model using the plurality of training
samples and the plurality of predictions as a plurality of second
labels for the plurality of training samples; extract one or more
weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, wherein the output
data includes at least one prediction for the at least one sample;
and apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample.
[0043] Clause 16. The computer program product of clause 15,
wherein the input data includes transaction data, wherein the at
least one sample is associated with at least one transaction,
wherein the at least one prediction includes at least one
probability of whether the at least one transaction is a fraudulent
transaction, and wherein the one or more contributions are
determined in real-time during processing of the at least one
transaction in a transaction processing network.
[0044] Clause 17. The computer program product of any of clauses 15
and 16, wherein the first model includes a greater number of hidden
layers than the second model.
[0045] Clause 18. The computer program product of any of clauses
15-17, wherein the one or more processors are further programmed
and/or configured to: rank the plurality of features associated
with the at least one sample according to the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one
sample; and identify at least one feature of the plurality of
features with respect to at least one other feature of the
plurality of features based on the ranked plurality of features as
at least one reason code associated with the at least one
prediction.
[0046] Clause 19. The computer program product of clauses 15-18,
wherein the one or more extracted weights include a plurality of
extracted weights, wherein the plurality of extracted weights is
associated with a plurality of hidden units of at least one hidden
layer of the trained second model, and wherein the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
is determined by: calculating a score for each path of a plurality
of paths of the one or more features through the second model based
on the plurality of extracted weights and at least one activation
function; and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features.
[0047] Clause 20. The computer program product of any of clauses
15-19 wherein the one or more processors are further programmed
and/or configured to: determine a contribution score for each
contribution of the one or more contributions of the one or more
features of the plurality of features to the at least one
prediction for the at least one sample according to the following
Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00006##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature to a prediction y.sub.m, wherein K is a number
of the plurality of features, wherein H is a number of hidden units
in a hidden layer of the trained second model, wherein M is a
number of output units of an output layer of the trained second
model, wherein U and V are weight matrices, wherein a number of
weight matrices U is equal to K.times.H, wherein a number of weight
matrices V is equal to H.times.M, wherein u.sub.ij is a weight
matrix applied to the feature on a path to a hidden unit h.sub.j,
wherein a.sub.j is an activation unit of an activation layer
between the hidden layer and the output layer of the trained second
model, wherein v.sub.jm is a weight matrix applied to an output of
the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0048] These and other features and characteristics of the present
disclosure, as well as the methods of operation and functions of
the related elements of structures and the combination of parts and
economies of manufacture, will become more apparent upon
consideration of the following description and the appended claims
with reference to the accompanying drawings, all of which form a
part of this specification, wherein like reference numerals
designate corresponding parts in the various figures. It is to be
expressly understood, however, that the drawings are for the
purpose of illustration and description only and are not intended
as a definition of limits. As used in the specification and the
claims, the singular form of "a," "an," and "the" include plural
referents unless the context clearly dictates otherwise.
BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Additional advantages and details of non-limiting
embodiments or aspects are explained in greater detail below with
reference to the exemplary embodiments that are illustrated in the
accompanying schematic figures, in which:
[0050] FIG. 1 is a diagram of non-limiting embodiments or aspects
of an environment in which systems, devices, products, apparatus,
and/or methods as described herein, may be implemented;
[0051] FIG. 2 is a diagram of non-limiting embodiments or aspects
of components of one or more devices and/or one or more systems of
FIG. 1;
[0052] FIG. 3 is a flowchart illustrating non-limiting embodiments
or aspects of a process for determining a reason for a deep
learning model output;
[0053] FIG. 4 is a diagram of an implementation of non-limiting
embodiments or aspects of a neural network;
[0054] FIG. 5 is a diagram of an implementation of non-limiting
embodiments or aspects of a neural network;
[0055] FIG. 6 is a diagram of an implementation of non-limiting
embodiments or aspects of a neural network; and
[0056] FIG. 7 is a diagram of an implementation of non-limiting
embodiments or aspects of a neural network.
DESCRIPTION
[0057] It is to be understood that the present disclosure may
assume various alternative variations and step sequences, except
where expressly specified to the contrary. It is also to be
understood that the specific devices and processes illustrated in
the attached drawings, and described in the following
specification, are simply exemplary and non-limiting embodiments or
aspects. Hence, specific dimensions and other physical
characteristics related to the embodiments or aspects disclosed
herein are not to be considered as limiting.
[0058] No aspect, component, element, structure, act, step,
function, instruction, and/or the like used herein should be
construed as critical or essential unless explicitly described as
such. Also, as used herein, the articles "a" and "an" are intended
to include one or more items, and may be used interchangeably with
"one or more" and "at least one." Furthermore, as used herein, the
term "set" is intended to include one or more items (e.g., related
items, unrelated items, a combination of related and unrelated
items, etc.) and may be used interchangeably with "one or more" or
"at least one." Where only one item is intended, the term "one" or
similar language is used. Also, as used herein, the terms "has,"
"have," "having," or the like are intended to be open-ended terms.
Further, the phrase "based on" is intended to mean "based at least
partially on" unless explicitly stated otherwise.
[0059] For purposes of the description hereinafter, the terms
"end," "upper," "lower," "right," "left," "vertical," "horizontal,"
"top," "bottom," "lateral," "longitudinal," and derivatives thereof
shall relate to non-limiting embodiments or aspects as they are
oriented in the drawing figures. However, it is to be understood
that non-limiting embodiments or aspects may assume various
alternative variations and step sequences, except where expressly
specified to the contrary. It is also to be understood that the
specific devices and processes illustrated in the attached
drawings, and described in the following specification, are simply
exemplary embodiments or aspects. Hence, specific dimensions and
other physical characteristics related to the embodiments or
aspects of the embodiments disclosed herein are not to be
considered as limiting unless otherwise indicated.
[0060] As used herein, the terms "communication" and "communicate"
may refer to the reception, receipt, transmission, transfer,
provision, and/or the like of information (e.g., data, signals,
messages, instructions, commands, and/or the like). For one unit
(e.g., a device, a system, a component of a device or system,
combinations thereof, and/or the like) to be in communication with
another unit means that the one unit is able to directly or
indirectly receive information from and/or transmit information to
the other unit. This may refer to a direct or indirect connection
(e.g., a direct communication connection, an indirect communication
connection, and/or the like) that is wired and/or wireless in
nature. Additionally, two units may be in communication with each
other even though the information transmitted may be modified,
processed, relayed, and/or routed between the first and second
unit. For example, a first unit may be in communication with a
second unit even though the first unit passively receives
information and does not actively transmit information to the
second unit. As another example, a first unit may be in
communication with a second unit if at least one intermediary unit
(e.g., a third unit located between the first unit and the second
unit) processes information received from the first unit and
communicates the processed information to the second unit. In some
non-limiting embodiments, a message may refer to a network packet
(e.g., a data packet and/or the like) that includes data. It will
be appreciated that numerous other arrangements are possible.
[0061] Some non-limiting embodiments or aspects are described
herein in connection with thresholds. As used herein, satisfying a
threshold may refer to a value being greater than the threshold,
more than the threshold, higher than the threshold, greater than or
equal to the threshold, less than the threshold, fewer than the
threshold, lower than the threshold, less than or equal to the
threshold, equal to the threshold, etc.
[0062] As used herein, the term "transaction service provider" may
refer to an entity that receives transaction authorization requests
from merchants or other entities and provides guarantees of
payment, in some cases through an agreement between the transaction
service provider and an issuer institution. For example, a
transaction service provider may include a payment network such as
Visa.RTM. or any other entity that processes transactions. The term
"transaction processing system" may refer to one or more computer
systems operated by or on behalf of a transaction service provider,
such as a transaction processing server executing one or more
software applications. A transaction processing server may include
one or more processors and, in some non-limiting embodiments, may
be operated by or on behalf of a transaction service provider.
[0063] As used herein, the term "issuer" or "issuer institution"
may refer to one or more entities, such as a bank, that provide
accounts to customers for conducting transactions (e.g., payment
transactions), such as initiating credit and/or debit payments. For
example, an issuer institution may provide an account identifier,
such as a primary account number (PAN), to a customer that uniquely
identifies one or more accounts associated with that customer. The
account identifier may be embodied on a portable financial device,
such as a physical financial instrument, e.g., a payment card,
and/or may be electronic and used for electronic payments. The term
"issuer system" refers to one or more computer systems operated by
or on behalf of an issuer institution, such as a server computer
executing one or more software applications. For example, an issuer
system may include one or more authorization servers for
authorizing a transaction.
[0064] As used herein, the term "acquirer" or "acquirer
institution" may refer to an entity licensed and/or approved by the
transaction service provider to originate transactions (e.g.,
payment transactions) using a portable financial device associated
with the transaction service provider. The transactions the
acquirer institution may originate may include payment transactions
(e.g., purchases, original credit transactions (OCTs), account
funding transactions (AFTs), and/or the like). In some non-limiting
embodiments, an acquirer institution may be a financial
institution, such as a bank. As used herein, the term "acquirer
system" may refer to one or more computer systems, computer
devices, software applications, and/or the like operated by or on
behalf of an acquirer institution.
[0065] As used herein, the term "account identifier" may include
one or more PANs, tokens, or other identifiers associated with a
customer account. The term "token" may refer to an identifier that
is used as a substitute or replacement identifier for an original
account identifier, such as a PAN. Account identifiers may be
alphanumeric or any combination of characters and/or symbols.
Tokens may be associated with a PAN or other original account
identifier in one or more data structures (e.g., one or more
databases and/or the like) such that they may be used to conduct a
transaction without directly using the original account identifier.
In some examples, an original account identifier, such as a PAN,
may be associated with a plurality of tokens for different
individuals or purposes. The term "account data," as used herein,
refers to any data concerning one or more accounts for one or more
users. Account data may include, for example, one or more account
identifiers, user identifiers, transaction histories, balances,
credit limits, issuer institution identifiers, and/or the like.
[0066] As used herein, the term "merchant" may refer to an
individual or entity that provides goods and/or services, or access
to goods and/or services, to customers based on a transaction, such
as a payment transaction. The term "merchant" or "merchant system"
may also refer to one or more computer systems operated by or on
behalf of a merchant, such as a server computer executing one or
more software applications. A "point-of-sale (POS) system," as used
herein, may refer to one or more computers and/or peripheral
devices used by a merchant to engage in payment transactions with
customers, including one or more card readers, near-field
communication (NFC) receivers, RFID receivers, and/or other
contactless transceivers or receivers, contact-based receivers,
payment terminals, computers, servers, input devices, and/or other
like devices that can be used to initiate a payment
transaction.
[0067] As used herein, the term "payment gateway" may refer to an
entity and/or a payment processing system operated by or on behalf
of such an entity (e.g., a merchant service provider, a payment
service provider, a payment facilitator, a payment facilitator that
contracts with an acquirer, a payment aggregator, and/or the like),
which provides payment services (e.g., transaction service provider
payment services, payment processing services, and/or the like) to
one or more merchants. The payment services may be associated with
the use of portable financial devices managed by a transaction
service provider. As used herein, the term "payment gateway system"
may refer to one or more computer systems, computer devices,
servers, groups of servers, and/or the like operated by or on
behalf of a payment gateway.
[0068] As used herein, the term "mobile device" may refer to one or
more portable electronic devices configured to communicate with one
or more networks. As an example, a mobile device may include a
cellular phone (e.g., a smartphone or standard cellular phone), a
portable computer (e.g., a tablet computer, a laptop computer,
etc.), a wearable device (e.g., a watch, pair of glasses, lens,
clothing, and/or the like), a personal digital assistant (PDA),
and/or other like devices. The terms "client device" and "user
device," as used herein, refer to any electronic device that is
configured to communicate with one or more servers or remote
devices and/or systems. A client device or user device may include
a mobile device, a network-enabled appliance (e.g., a
network-enabled television, refrigerator, thermostat, and/or the
like), a computer, a POS system, and/or any other device or system
capable of communicating with a network.
[0069] As used herein, the term "computing device" or "computing
system" may refer to one or more electronic devices that are
configured to directly or indirectly communicate with or over one
or more networks. The computing device may be a mobile device, a
desktop computer, or the like. Furthermore, the term "computer" may
refer to any computing device that includes the necessary
components to receive, process, and output data, and normally
includes a display, a processor, a memory, an input device, and a
network interface. An "application" or "application program
interface" (API) refers to computer code or other data sorted on a
computer-readable medium that may be executed by a processor to
facilitate the interaction between software components, such as a
client-side front-end and/or server-side back-end for receiving
data from the client. An "interface" refers to a generated display,
such as one or more graphical user interfaces (GUIs) with which a
user may interact, either directly or indirectly (e.g., through a
keyboard, mouse, touchscreen, etc.).
[0070] As used herein, the term "server" may refer to or include
one or more processors or computers, storage devices, or similar
computer arrangements that are operated by or facilitate
communication and processing for multiple parties in a network
environment, such as the Internet, although it will be appreciated
that communication may be facilitated over one or more public or
private network environments and that various other arrangements
are possible. Further, multiple computers, e.g., servers, or other
computerized devices, such as POS devices, directly or indirectly
communicating in the network environment may constitute a "system,"
such as a merchant's POS system.
[0071] As used herein, the terms "electronic wallet" and
"electronic wallet application" refer to one or more electronic
devices and/or software applications configured to initiate and/or
conduct payment transactions. For example, an electronic wallet may
include a mobile device executing an electronic wallet application,
and may further include server-side software and/or databases for
maintaining and providing transaction data to the mobile device. An
"electronic wallet provider" may include an entity that provides
and/or maintains an electronic wallet for a customer, such as
Google Wallet.TM., Android Pay.RTM., Apple Pay.RTM., Samsung
Pay.RTM., and/or other like electronic payment systems. In some
non-limiting examples, an issuer bank may be an electronic wallet
provider.
[0072] As used herein, the term "portable financial device" may
refer to a payment card (e.g., a credit or debit card), a gift
card, a smartcard, smart media, a payroll card, a healthcare card,
a wristband, a machine-readable medium containing account
information, a keychain device or fob, an RFID transponder, a
retailer discount or loyalty card, a cellular phone, an electronic
wallet mobile application, a personal digital assistant (PDA), a
security card, a computer, an access card, a wireless terminal,
and/or a transponder, and/or the like. In some non-limiting
embodiments, the portable financial device may include a volatile
or a non-volatile memory to store information, (e.g., an account
identifier, a name of the account holder, and/or the like).
[0073] Provided are improved systems, methods, and computer program
products for determining a reason for a deep learning model
output.
[0074] Existing techniques for determining reasons for model
outputs include model approximation techniques and sensitivity
analysis techniques. An existing model approximation technique may
obtain each of the training samples and labels given by a complex
deep learning model and, to determine a reason for a model output
of the deep learning model for each new sample, determine a subset
of the training samples that are similar to the new sample, train a
linear regression model using the corresponding subset of the
training samples and the results of the deep learning model for
that subset, the linear regression model being an approximation of
the deep learning model for that subset, and directly use the
weights of the corresponding linear regression model to rank each
feature and determine the top-ranked features as reason codes for
an output of the deep learning model corresponding to the new
sample. Although this existing model approximation technique may
provide relatively high interpretability due to the weights of the
linear regression being and/or corresponding to the importance
measurements, this existing technique may have (i) relatively
mediocre accuracy (e.g., as compared to sensitivity analysis
techniques, etc.), because linear regression is a relatively simple
modeling approach that provides a local approximation, and (ii)
relatively low efficiency due to the processing time and resources
used to determine the similar samples and train a model for each
similar sample, which may not be performed in real-time (e.g.,
during processing of a transaction in a transaction processing
network , etc.).
[0075] An existing sensitivity analysis technique may obtain each
of the training samples and labels given by a complex deep learning
model and, to determine a reason for a model output of the deep
learning model for each new sample, perturbate, for each feature of
that new sample, a value of the feature by changing it to some
other value (e.g., empty, null, mean, etc.), determine a prediction
for the perturbated sample using the original complex deep learning
model, and calculate a performance drop between a prediction for
the original sample from the deep learning model and the prediction
for the perturbated sample. The features in each sample may be
ranked according to the corresponding performance drops of the
features to determine the top-ranked features as reason codes for
the prediction for the new sample. Although this existing
sensitivity analysis technique may provide relatively high accuracy
due to directly measuring an impact of features on performance of
the model and a relatively high interpretability due to calculation
of the performance drop being a transparent process, this existing
technique may have relatively low efficiency due to the processing
time and resources used to execute or run the deep learning model n
number of feature times for each new sample, where the number n of
the features in each sample can be relatively large (e.g., 100
features, 1000 features, etc.), which may not be performed in
real-time (e.g., during processing of a transaction in a
transaction processing network , etc.).
[0076] In this way, existing techniques for determining reasons for
deep learning model outputs do not provide a mechanism for
determining a reason for a deep learning model output in an
efficient (e.g., real-time, etc.) manner while also providing an
acceptable level of accuracy and interpretability.
[0077] Non-limiting embodiments or aspects of the present
disclosure are directed to systems, methods, and computer program
products that obtain training data associated with a plurality of
training samples and a plurality of first labels for the plurality
of training samples; train a first model using the plurality of
training samples and the plurality of first labels for the
plurality of training samples, wherein training the first model
generates a plurality of predictions for the plurality of training
samples; train a second model using the plurality of training
samples and the plurality of predictions as a plurality of second
labels for the plurality of training samples; extract one or more
weights of the trained second model; process, using the first
model, input data including a plurality of features associated with
at least one sample, to generate output data, wherein the output
data includes at least one prediction for the at least one sample;
and apply the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample. For example, non-limiting
embodiments or aspects of the present disclosure may obtain each of
the training samples and labels given by a complex deep learning
model, train (e.g., train offline, etc.) a simpler neural network
(e.g., a two layer neural network, etc.) using the training samples
and the results from the complex deep learning model, the simpler
neural network being global approximation of the deep learning
model, extract and/or store the weights of the simpler neural
network, and, to determine a reason for a model output of the deep
learning model for each new sample, evaluate the new sample using
the simpler model and identify the contribution score of each
feature using simple calculations according to feature values and
the model weights of the simpler model, and ranking features by
their contribution score to determine certain (e.g., top-ranked,
etc.) features as reason codes.
[0078] In this way, non-limiting embodiments or aspects of the
present disclosure may provide a reason for a deep learning model
output in a more efficient (e.g., real-time, etc.) manner while
also providing an acceptable level of accuracy and
interpretability. Accordingly, non-limiting embodiments or aspects
of the present disclosure may provide a better balance between an
accuracy, an interpretability, and an efficiency of a machine
learning model by approximating an original complex deep learning
model with a simpler but more interpretable neural network model
with sufficient accuracy, and generating reason codes from the
simpler model utilizing weights of the simpler model, which enables
the provision of real time reason codes for deep learning based
models and applications, such as those in the financial industry
(e.g., fraud detection models, stand-in processing models, etc.).
For example, non-limiting embodiments or aspects of the present
disclosure may have (i) relatively high accuracy (e.g., due to a
simpler neural network being more accurate than linear regression
because a neural network considers non-linearity, etc.), (ii)
relatively high interpretability (e.g., due to a simpler neural
network being relatively easy to interpret, etc.), and (iii)
relatively high efficiency (e.g., due to the simpler neural network
being trained only once and/or offline, which enables relatively
simple calculations to be executed in real-time to determine
contributions of features in new samples, etc.)
[0079] Referring now to FIG. 1, FIG. 1 is a diagram of an example
environment 100 in which devices, systems, methods, and/or products
described herein, may be implemented. As shown in FIG. 1,
environment 100 includes transaction processing network 101, which
can include merchant system 102, payment gateway 104, acquirer
system 106, transaction service provider system 108, and/or issuer
system 110, user device 112, and/or communication network 114.
Transaction processing network 101, merchant system 102, payment
gateway 104, acquirer system 106, transaction service provider
system 108, issuer system 110, and/or user device 112 may
interconnect (e.g., establish a connection to communicate) via
wired connections, wireless connections, or a combination of wired
and wireless connections.
[0080] Merchant system 102 may include one or more devices capable
of receiving information from payment gateway 104, acquirer system
106, transaction service provider system 108, issuer system 110,
and/or user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to payment gateway 104, acquirer system 106,
transaction service provider system 108, issuer system 110, and/or
user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.). Merchant system 102 may
include a device capable of receiving information from user device
112 via a communication connection (e.g., an NFC communication
connection, an RFID communication connection, a Bluetooth.RTM.
communication connection, and/or the like) with user device 112,
and/or communicating information to user device 112 via the
communication connection. For example, merchant system 102 may
include a computing device, such as a server, a group of servers, a
client device, a group of client devices, and/or other like
devices. In some non-limiting embodiments or aspects, merchant
system 102 may be associated with a merchant as described herein.
In some non-limiting embodiments or aspects, merchant system 102
may include one or more devices, such as computers, computer
systems, and/or peripheral devices capable of being used by a
merchant to conduct a payment transaction with a user. For example,
merchant system 102 may include a POS device and/or a POS
system.
[0081] Payment gateway 104 may include one or more devices capable
of receiving information from merchant system 102, acquirer system
106, transaction service provider system 108, issuer system 110,
and/or user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to merchant system 102, acquirer system 106,
transaction service provider system 108, issuer system 110, and/or
user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.). For example, payment
gateway 104 may include a computing device, such as a server, a
group of servers, and/or other like devices. In some non-limiting
embodiments or aspects, payment gateway 104 is associated with a
payment gateway as described herein.
[0082] Acquirer system 106 may include one or more devices capable
of receiving information from merchant system 102, payment gateway
104, transaction service provider system 108, issuer system 110,
and/or user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to merchant system 102, payment gateway 104,
transaction service provider system 108, issuer system 110, and/or
user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.). For example, acquirer
system 106 may include a computing device, such as a server, a
group of servers, and/or other like devices. In some non-limiting
embodiments or aspects, acquirer system 106 may be associated with
an acquirer as described herein.
[0083] Transaction service provider system 108 may include one or
more devices capable of receiving information from merchant system
102, payment gateway 104, acquirer system 106, issuer system 110,
and/or user device 112, (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to merchant system 102, payment gateway 104, acquirer
system 106, issuer system 110, and/or user device 112 (e.g., via
communication network 114, via transaction processing network 101,
etc.). For example, transaction service provider system 108 may
include a computing device, such as a server (e.g., a transaction
processing server), a group of servers, and/or other like devices.
In some non-limiting embodiments or aspects, transaction service
provider system 108 may be associated with a transaction service
provider as described herein.
[0084] Issuer system 110 may include one or more devices capable of
receiving information from merchant system 102, payment gateway
104, acquirer system 106, transaction service provider system 108,
and/or user device 112 (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to merchant system 102, payment gateway 104, acquirer
system 106, transaction service provider system 108, and/or user
device 112 (e.g., via communication network 114, via transaction
processing network 101, etc.). For example, issuer system 110 may
include a computing device, such as a server, a group of servers,
and/or other like devices. In some non-limiting embodiments or
aspects, issuer system 110 may be associated with an issuer
institution as described herein. For example, issuer system 110 may
be associated with an issuer institution that issued a payment
account or instrument (e.g., a credit account, a debit account, a
credit card, a debit card, etc.) to a user (e.g., a user associated
with user device 112, etc.).
[0085] In some non-limiting embodiments or aspects, transaction
processing network 101 includes a plurality of systems in a
communication path for processing a transaction. For example,
transaction processing network 101 can include merchant system 102,
payment gateway 104, acquirer system 106, transaction service
provider system 108, and/or issuer system 110 in a communication
path (e.g., a communication path, a communication channel, a
communication network, etc.) for processing an electronic payment
transaction. As an example, transaction processing network 101 can
process (e.g., initiate, conduct, authorize, etc.) an electronic
payment transaction via the communication path between merchant
system 102, payment gateway 104, acquirer system 106, transaction
service provider system 108, and/or issuer system 110.
[0086] User device 112 may include one or more devices capable of
receiving information from merchant system 102, payment gateway
104, acquirer system 106, transaction service provider system 108,
and/or issuer system 110 (e.g., via communication network 114, via
transaction processing network 101, etc.) and/or communicating
information to merchant system 102, payment gateway 104, acquirer
system 106, transaction service provider system 108, and/or issuer
system 110 (e.g., via communication network 114, via transaction
processing network 101, etc.). For example, user device 112 may
include a client device and/or the like. In some non-limiting
embodiments or aspects, user device 112 may be capable of receiving
information (e.g., from merchant system 102) via a short range
wireless communication connection (e.g., an NFC communication
connection, an RFID communication connection, a Bluetooth.RTM.
communication connection, and/or the like), and/or communicating
information (e.g., to merchant system 102) via a short range
wireless communication connection. In some non-limiting embodiments
or aspects, user device 112 may include an application associated
with user device 112, such as an application stored on user device
112, a mobile application (e.g., a mobile device application, a
native application for a mobile device, a mobile cloud application
for a mobile device, an electronic wallet application, and/or the
like) stored and/or executed on user device 112.
[0087] Communication network 114 may include one or more wired
and/or wireless networks. For example, communication network 114
may include a cellular network (e.g., a long-term evolution (LTE)
network, a third generation (3G) network, a fourth generation (4G)
network, a code division multiple access (CDMA) network, etc.), a
public land mobile network (PLMN), a local area network (LAN), a
wide area network (WAN), a metropolitan area network (MAN), a
telephone network (e.g., the public switched telephone network
(PSTN)), a private network, an ad hoc network, an intranet, the
Internet, a fiber optic-based network, a cloud computing network,
and/or the like, and/or a combination of these or other types of
networks.
[0088] The number and arrangement of devices and systems shown in
FIG. 1 are provided as an example. There may be additional devices
and/or systems, fewer devices and/or systems, different devices
and/or systems, or differently arranged devices and/or systems than
those shown in FIG. 1. Furthermore, two or more devices and/or
systems shown in FIG. 1 may be implemented within a single device
and/or system, or a single device and/or system shown in FIG. 1 may
be implemented as multiple, distributed devices and/or systems.
Additionally, or alternatively, a set of devices and/or systems
(e.g., one or more devices or systems) of environment 100 may
perform one or more functions described as being performed by
another set of devices or systems of environment 100.
[0089] Referring now to FIG. 2, FIG. 2 is a diagram of example
components of a device 200. Device 200 may correspond to one or
more devices of transaction processing network 101, one or more
devices of merchant system 102, one or more devices of payment
gateway 104, one or more devices of acquirer system 106, one or
more devices of transaction service provider system 108, one or
more devices of issuer system 110, and/or user device 112. In some
non-limiting embodiments or aspects, one or more devices of
transaction processing network 101, one or more devices of merchant
system 102, one or more devices of payment gateway 104, one or more
devices of acquirer system 106, one or more devices of transaction
service provider system 108, one or more devices of issuer system
110, and/or user device 112 can include at least one device 200
and/or at least one component of device 200. As shown in FIG. 2,
device 200 may include a bus 202, a processor 204, memory 206, a
storage component 208, an input component 210, an output component
212, and a communication interface 214.
[0090] Bus 202 may include a component that permits communication
among the components of device 200. In some non-limiting
embodiments or aspects, processor 204 may be implemented in
hardware, firmware, or a combination of hardware and software. For
example, processor 204 may include a processor (e.g., a central
processing unit (CPU), a graphics processing unit (GPU), an
accelerated processing unit (APU), etc.), a microprocessor, a
digital signal processor (DSP), and/or any processing component
(e.g., a field-programmable gate array (FPGA), an
application-specific integrated circuit (ASIC), etc.) that can be
programmed to perform a function. Memory 206 may include random
access memory (RAM), read only memory (ROM), and/or another type of
dynamic or static storage device (e.g., flash memory, magnetic
memory, optical memory, etc.) that stores information and/or
instructions for use by processor 204.
[0091] Storage component 208 may store information and/or software
related to the operation and use of device 200. For example,
storage component 208 may include a hard disk (e.g., a magnetic
disk, an optical disk, a magneto-optic disk, a solid state disk,
etc.), a compact disc (CD), a digital versatile disc (DVD), a
floppy disk, a cartridge, a magnetic tape, and/or another type of
computer-readable medium, along with a corresponding drive.
[0092] Input component 210 may include a component that permits
device 200 to receive information, such as via user input (e.g., a
touch screen display, a keyboard, a keypad, a mouse, a button, a
switch, a microphone, a camera, an electroencephalogram (EEG)
monitor, etc.). Additionally, or alternatively, input component 210
may include a sensor for sensing information (e.g., a global
positioning system (GPS) component, an accelerometer, a gyroscope,
an actuator, etc.). Output component 212 may include a component
that provides output information from device 200 (e.g., a display,
a speaker, one or more light-emitting diodes (LEDs), and/or the
like).
[0093] Communication interface 214 may include a transceiver-like
component (e.g., a transceiver, a separate receiver and
transmitter, etc.) that enables device 200 to communicate with
other devices, such as via a wired connection, a wireless
connection, or a combination of wired and wireless connections.
Communication interface 214 may permit device 200 to receive
information from another device and/or provide information to
another device. For example, communication interface 214 may
include an Ethernet interface, an optical interface, a coaxial
interface, an infrared interface, a radio frequency (RF) interface,
a universal serial bus (USB) interface, a Wi-Fi.RTM. interface, a
cellular network interface, and/or the like.
[0094] Device 200 may perform one or more processes described
herein. Device 200 may perform these processes based on processor
204 executing software instructions stored by a computer-readable
medium, such as memory 206 and/or storage component 208. A
computer-readable medium (e.g., a non-transitory computer-readable
medium) is defined herein as a non-transitory memory device. A
non-transitory memory device includes memory space located inside
of a single physical storage device or memory space spread across
multiple physical storage devices.
[0095] Software instructions may be read into memory 206 and/or
storage component 208 from another computer-readable medium or from
another device via communication interface 214. When executed,
software instructions stored in memory 206 and/or storage component
208 may cause processor 204 to perform one or more processes
described herein. Additionally, or alternatively, hardwired
circuitry may be used in place of or in combination with software
instructions to perform one or more processes described herein.
Thus, embodiments or aspects described herein are not limited to
any specific combination of hardware circuitry and software.
[0096] Memory 206 and/or storage component 208 may include data
storage or one or more data structures (e.g., a database, etc.).
Device 200 may be capable of receiving information from, storing
information in, communicating information to, or searching
information stored in the data storage or one or more data
structures in memory 206 and/or storage component 208. For example,
the information may include training data, input data, output data,
transaction data, account data, or any combination thereof.
[0097] The number and arrangement of components shown in FIG. 2 are
provided as an example. In some non-limiting embodiments or
aspects, device 200 may include additional components, fewer
components, different components, or differently arranged
components than those shown in FIG. 2. Additionally, or
alternatively, a set of components (e.g., one or more components)
of device 200 may perform one or more functions described as being
performed by another set of components of device 200.
[0098] Referring now to FIG. 3, FIG. 3 is a flowchart of
non-limiting embodiments or aspects of a process 300 for
determining a reason for a deep learning model output. In some
non-limiting embodiments or aspects, one or more of the steps of
process 300 may be performed (e.g., completely, partially, etc.) by
transaction service provider system 108 (e.g., one or more devices
of transaction service provider system 108, etc.). In some
non-limiting embodiments or aspects, one or more of the steps of
process 300 may be performed (e.g., completely, partially, etc.) by
another device or a group of devices separate from or including
transaction service provider system 108, such as merchant system
102 (e.g., one or more devices of merchant system 102, etc.),
payment gateway 104 (e.g., one or more devices of payment gateway
104, etc.), acquirer system 106 (e.g., one or more devices of
acquirer system 106, etc.), issuer system 110 (e.g., one or more
devices of issuer system 110, etc.), and/or user device 112.
[0099] As shown in FIG. 3, at step 302, process 300 includes
obtaining training data associated with a plurality of training
samples and a plurality of first labels for the plurality of
training samples. For example, transaction service provider system
108 may obtain training data associated with a plurality of
training samples and a plurality of first labels for the plurality
of training samples.
[0100] In some non-limiting embodiments or aspects, training data
includes transaction data. For example, a training sample may be
associated with and/or correspond to a transaction (e.g., a prior
transaction, a previously processed transaction, etc.) and/or
include transaction data associated with the transaction (e.g.,
transaction parameters associated with the transaction, etc.), and
a first label associated with the training sample may include an
indication of whether the transaction is a fraudulent transaction
(e.g., a true or fraud label, a false or non-fraud label, etc.). As
an example, transaction data may include a classification label
associated with a transaction (e.g., a prior transaction, a
previously processed transaction, etc.), such as a true
classification label (e.g., a label that indicates that the
transaction is associated with fraud, a label that indicates that
the transaction is not associated with fraud, labels for sets of
positive and negative training examples or samples, etc.). However,
non-limiting embodiments or aspects are not limited thereto, and
training data may include any data including any type of parameters
associated with a training sample, and a first label associated
with the training sample may include any type of classification
label and/or number of classification labels.
[0101] In some non-limiting embodiments or aspects, transaction
data includes transaction parameters associated with transactions,
such as payment transactions initiated and/or conducted with an
electronic wallet application, and/or the like. Non-limiting
examples of transaction parameters include: electronic wallet card
data, decision data, authorization data, account identifier (e.g.,
PAN, etc.), transaction amount, transaction date and time,
conversion rate of currency, merchant type, acquiring institution
country, PAN country, response code, merchant name/location, type
of currency, and/or the like. Response code may refer to a
successful approval/completion of a transaction, denial because
card reported as lost or stolen, do not honor, partial approval,
VIP approval (VIP program), amount exceeds maximum, insufficient
funds, incorrect PIN, suspected fraud, activity amount exceeded,
allowable number of PIN-entry tries exceeded, and/or the like. In
some non-limiting embodiments or aspects, electronic wallet card
data includes one or more of data associated with an identifier
regarding a portable financial device to be provided to an
electronic wallet application, data associated with an identifier
of an issuer associated with the portable financial device to be
provided to an electronic wallet application, data associated with
an identifier of a transaction service provider associated with the
portable financial device to be provided to an electronic wallet
application, data associated with a name of a user associated with
the portable financial device to be provided to an electronic
wallet application, data associated with an account identifier of
an account associated with the portable financial device to be
provided to an electronic wallet application, and/or the like.
[0102] As shown in FIG. 3, at step 304, process 300 includes
training a first model using the plurality of training samples and
the plurality of first labels for the plurality of training
samples. For example, transaction service provider system 108 may
train a first model using the plurality of training samples and the
plurality of first labels for the plurality of training samples. In
some non-limiting embodiments or aspects, training the first model
generates a plurality of predictions (e.g., a plurality of first
predictions, etc.) for the plurality of training samples. For
example, transaction service provider system 108 may provide, as
input, a training sample to the first model, receive, as output, a
first prediction for the training sample, and modify one or more
parameters and/or weights of the first model according to an
objective function that depends on a first label for the training
sample and the first prediction for the training sample.
[0103] In some non-limiting embodiments or aspects, a prediction
for a training sample includes a prediction of whether a
transaction associated with and/or corresponding to the training
sample is a fraudulent transaction (e.g., a true or fraud
prediction, a false or non-fraud prediction, a probability
associated with the transaction being fraudulent, etc.). However,
non-limiting embodiments or aspects are not limited thereto, and a
prediction for a training sample may include any type of prediction
associated with a training sample.
[0104] As shown in FIG. 3, at step 306, process 300 includes
training a second model using the plurality of training samples and
the plurality of predictions as a plurality of second labels for
the plurality of training samples. For example, transaction service
provider system 108 may train a second model using the plurality of
training samples and the plurality of predictions as a plurality of
second labels for the plurality of training samples. As an example,
transaction service provider system 108 may provide, as input, a
training sample to the first model, receive, as output, a second
prediction for the training sample, and modify one or more
parameters and/or weights of the second model according to an
objective function that depends on the first prediction for the
training sample (e.g., as a second label for the training sample,
etc.) and the second prediction for the training sample.
[0105] In some non-limiting embodiments or aspects, the first model
includes a greater number of hidden layers than the second model.
For example, the first model may include a more complex deep
learning model (e.g., a deep neural network (DNN), a convolutional
neural network (CNN), a recurrent neural network (RNN), a three or
more layer neural network, etc.), and the second model may include
a simpler neural network model (e.g., a two layer neural network,
etc.). As an example, the second model may include an approximation
of the first model.
[0106] As shown in FIG. 3, at step 308, process 300 includes
extracting one or more weights of the trained second model. For
example, transaction service provider system 108 may extract one or
more weights of the trained second model. As an example,
transaction service provider system 108 may store the one or more
extracted weights (e.g., in memory, in a database, in an in-memory
database (IMDB), etc.). In such an example, connections or paths
between units or neurons in different layers of a trained neural
network may be associated with weight values, and the weight values
may be extracted from the neural network to determine contributions
of features as described in more detail herein below with respect
to FIGS. 4-7.
[0107] As shown in FIG. 3, at step 310, process 300 includes
processing, using the first model, input data including a plurality
of features associated with at least one sample, to generate output
data. For example, transaction service provider system 108 may
process, using the first model, input data including a plurality of
features associated with at least one sample, to generate output
data. In some non-limiting embodiments or aspects, the output data
includes at least one prediction for the at least one sample. For
example, transaction service provider system 108 may provide, as
input, features associated with a sample (e.g., a non-training
sample, etc.) to the first model, and receive, as output, a
prediction for the sample.
[0108] In some non-limiting embodiments or aspects, a feature is
associated with and/or corresponds to a transaction parameter. As
an example, and referring also to FIGS. 4-7, a feature may include
the transaction parameter of an AA_score, a feature may include the
transaction parameter of a transaction amount, and/or the like.
However, non-limiting embodiments or aspects are not limited
thereto, and a feature may include any transaction parameter
(and/or any other parameter), and the input data may include any
number of transaction parameters (and/or other parameters) as
feature data. In some non-limiting embodiments or aspects, a
feature is normalized before the feature is input to the first
model.
[0109] In some non-limiting embodiments or aspects, input data
includes transaction data. For example, the at least one sample
processed using the first model may correspond to a transaction
(e.g., a new transaction, a current transaction, a transaction
currently being processed in transaction processing network 101,
etc.) and/or include transaction data associated with the
transaction (e.g., transaction parameters associated with the
transaction, etc.). As an example, the at least one prediction
included in the output data generated by the first model may
include at least one probability of whether the at least one
transaction is a fraudulent transaction. However, non-limiting
embodiments or aspects are not limited thereto, and input data may
include any data including any type of parameters associated with a
sample, and a prediction for the sample may include any type of
prediction for the sample.
[0110] As shown in FIG. 3, at step 312, process 300 includes
applying the one or more extracted weights to the plurality of
features to determine one or more contributions of one or more
features of the plurality of features to the at least one
prediction for the at least one sample. For example, transaction
service provider system 108 may apply the one or more extracted
weights to the plurality of features to determine one or more
contributions of one or more features of the plurality of features
to the at least one prediction for the at least one sample. As an
example, transaction service provider system 108 may evaluate the
sample using the second model and identify the contribution score
of each feature of the sample according to simple, real-time
calculations from feature values and the extracted model
weights.
[0111] In some non-limiting embodiments or aspects, the one or more
contributions are determined in real-time during processing of at
least one transaction in transaction processing network 101. For
example, transaction service provider system 108 may receive
transaction data associated with a current transaction in
transaction processing network 101 and apply the one or more
extracted weights to features of the transaction data to determine
contributions of the features to a prediction (e.g., a first
prediction, a prediction by the first model, etc.) of whether the
transaction is fraudulent. As an example, steps 302-310 of process
300 may be performed offline (e.g., before receipt and/or
initiation of a new or current transaction in transaction
processing network 101, etc.), and steps 312-316 may be performed
online (e.g., in real-time response to receiving, initiating,
and/or conducting a transaction in transaction processing network
101, etc.).
[0112] In some non-limiting embodiments or aspects, the one or more
extracted weights include a plurality of extracted weights, the
plurality of extracted weights is associated with a plurality of
hidden units of at least one hidden layer of the trained second
model (e.g., a trained neural network, etc.), and the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
is determined by: calculating a score for each path of a plurality
of paths of the one or more features through the second model based
on the plurality of extracted weights and at least one activation
function, and summing each score for each path of the plurality of
paths to determine a contribution score for the one or more
contributions of the one or more features. For example, and
referring also to FIG. 4, in an implementation 400 of a trained
neural network model according to non-limiting embodiments or
aspects, transaction service provider system 108 may determine a
contribution score for each contribution of the one or more
contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one sample
according to the following Equations (1) to (4):
h j = i = 1 K x i * u ij ( 1 ) a j = max ( 0 , h j ) ( 2 ) y m = j
= 1 H a j * v jm ( 3 ) Contribution ( x i -> y m ) = j = 1 H ( x
i * u ij ) * ( a j * v jm ) ( 4 ) ##EQU00007##
wherein Contribution (x.sub.i->y.sub.m) is the contribution
score of a feature to a prediction y.sub.m, wherein K is a number
of the plurality of features, wherein H is a number of hidden units
in a hidden layer of the trained second model, wherein M is a
number of output units of an output layer of the trained second
model, wherein U and V are weight matrices, wherein a number of
weight matrices U is equal to K.times.H, wherein a number of weight
matrices V is equal to H.times.M, wherein u.sub.ij is a weight
matrix applied to the feature on a path to a hidden unit h.sub.j,
wherein a.sub.j is an activation unit of an activation layer
between the hidden layer and the output layer of the trained second
model, and wherein v.sub.jm is a weight matrix applied to an output
of the activation unit a.sub.j on a path to the prediction
y.sub.m.
[0113] FIGS. 5-7 are diagrams of implementations 500, 600, and 700
of non-limiting embodiments or aspects of a neural network. For
example, FIG. 5 shows an example for calculating a contribution of
a feature AA_score and a feature tran_amt from extracted weights of
a trained 2-layer neural network. As an example, features input to
the trained neural network may be normalized. In such an example,
after normalization, a feature AA_score of a value 20 may be
normalized to 0.4 and a feature tran_amt of a value 30 may be
normalized to 0.5. FIGS. 5-7 show a neural network including a
single hidden layer including three neurons or hidden units h1, h2,
and h3, a single activation function activation(x), and an output
layer including output units y1 and y2; however, non-limiting
embodiments or aspects are not limited thereto, and a neural
network (e.g., a second model, an approximation of a deep neural
network, etc.) may include any number of hidden layers, any number
of hidden units in the hidden layers, any type of activation
functions, and/or any number of output units as an approximation of
the first model or a deep neural network. As shown in FIGS. 5-7,
weights of the neural network, which are shown between the neurons
or nodes thereof, may be used to determine values of the neurons or
units thereof. For example, a value for the neuron or hidden unit
h1 may be calculated as h1=0.4*0.8+0.5*0.6=0.62, and the activation
function, which determines whether the neuron or hidden unit makes
a contribution to the output may be defined as equal to x if
(x>=0) and as equal to 0 if (x<0), which results in
a1=h1=0.62. A value for the neuron or hidden unit h2 may be
calculated as h2=0.4*(-0.5)+0.5*(-0.4)=-0.4, and the activation
function, which determines whether the neuron or hidden unit makes
a contribution to the output may be defined as equal to x if
(x>=0) and as equal to 0 if (x<0), which results in
a2=0(h2<0). A value for the neuron or hidden unit h3 may be
calculated as h3=0.4*0.7+0.5*(-0.1)=0.23, and the activation
function, which determines whether the neuron or hidden unit makes
a contribution to the output may be defined as equal to x if
(x>=0) and as equal to 0 if (x<0), which results in a3=h3
=0.23. A value for the neuron or output unit y1 may be calculated
as y1=0.62*0.9+0* 0.4+0.23*0.7=0.55. For example, y1 may indicate a
55% probability of a transaction associated with the example input
features of FIG. 5 being a fraudulent transaction. A value for the
neuron or output unit y2 may be calculated as
y2=0.62*0.1+0*0.6+0.23*(-0.1)=0.04. For example, y2 may indicate a
4% probability of the transaction associated with the example input
features of FIG. 5 being a non-fraudulent transaction.
[0114] FIG. 6 shows an implementation 600 in which paths through
the neural network for the feature AA_score are bolded and enlarged
compared to other paths through the neural network. As shown in
FIG. 6, a contribution of the feature AA_score to the output y1 may
be determined by considering only the weights associated with the
path of that feature through the neural network (e.g., Contribution
(AA, path_x)=Contribution (input->hidden)*Contribution
(hidden->output)). For example, (Contribution (AA,
path_1)=(0.4*0.8)*(0.62*0.9)=0.878, Contribution (AA,
path_2)=(0.4*-0.5)*(0*0.4)=0, and Contribution (AA,
path_3)=(0.4*0.7)*(0.23*0.7)=0.441, which results in
Contribution_score (AA)=0.878+0+0.441=1.319. For example, a score
for each path of a feature through the neural network to an output
unit is calculated, and each score for each path is added together
to determine the contribution of that feature. As an example, a
value at the input unit (e.g., after normalization) is multiplied
by the extracted weight between that input unit and the hidden unit
(e.g., h1, h2, or h3) on the path, the activation function is
applied to the value at the hidden unit, and the value at the
hidden unit after application of the activation function is
multiplied by the extracted weight between that hidden unit and the
output unit (e.g., y1).
[0115] FIG. 7 shows an implementation 700 in which paths through
the neural network for the feature tran_amt are bolded and enlarged
compared to other paths through the neural network. As shown in
FIG. 7, a contribution of the feature tran_amt to the output y1 may
be determined by considering only the weights associated with the
path of that feature through the neural network (e.g., Contribution
(Amt, path_x)=Contribution (input->hidden)*Contribution
(hidden->output)). For example, Contribution (Amt,
path_1)=(0.5*0.6)*(0.62*0.9)=0.167, Contribution (Amt,
path_2)=(0.5*-0.4)*(0*0.4)=0, and Contribution (Amt,
path_3)=(0.5*-0.1)*(0.23*0.7)=-0.008, which results in
Contribution_score (Amt)=0.167+0-0.008=0.159. For example, a score
for each path of a feature through the neural network to an output
unit is calculated, and each score for each path is added together
to determine the contribution of that feature. As an example, a
value at the input unit (e.g., after normalization) is multiplied
by the extracted weight between that input unit and the hidden unit
(e.g., h1, h2, or h3) on the path, the activation function is
applied to the value at the hidden unit, and the value at the
hidden unit after application of the activation function is
multiplied by the extracted weight between that hidden unit and the
output unit (e.g., y1). In such an example, the Contribution_score
(AA)=1.319 and the Contribution_score (Amt)=0.159 may indicate that
the feature AA_score contributes significantly more to the output
y1 and, thus, the prediction of the first model associated with the
sample including this feature, than the feature tran_amt also
included in that sample.
[0116] As shown in FIG. 3, at step 314, process 300 includes
ranking the plurality of features associated with the at least one
sample according to the one or more contributions of the one or
more features of the plurality of features to the at least one
prediction for the at least one sample. For example, transaction
service provider system 108 may rank the plurality of features
associated with the at least one sample according to the one or
more contributions of the one or more features of the plurality of
features to the at least one prediction for the at least one
sample. As an example, and referring again to FIGS. 5-7,
transaction service provider system 108 may rank the feature
AA_score as having a more significant impact than the feature
tran_amt on the prediction of the first model associated with the
sample including these features.
[0117] As shown in FIG. 3, at step 316, process 300 includes
identifying at least one feature of the plurality of features with
respect to at least one other feature of the plurality of features
based on the ranked plurality of features as at least one reason
code associated with the at least one prediction. For example,
transaction service provider system 108 may identify at least one
feature of the plurality of features with respect to at least one
other feature of the plurality of features based on the ranked
plurality of features as at least one reason code associated with
the at least one prediction. As an example, model outputs or scores
may be obtained from the first model (e.g., a deep learning model,
etc.) and reason codes for the model outputs or scores may be
obtained from the ranked features. In such an example, and
referring again to FIGS. 5-7, transaction service provider system
108 may identify the feature AA_score as a reason code associated
with a prediction for the sample including the features AA_score
and tran_amt.
[0118] In some non-limiting embodiments or aspects, a reason code
provides an indication (e.g., a numerical value, an amount, a
percentage, a statistical impact, etc.) of a contribution of a
feature to a prediction for a particular sample including that
feature. For example, feature importance may provide an indication
of a contribution of a feature at a model level, and a reason code
may provide an indication of a contribution of a feature at a
sample level (e.g., for a particular sample, which features
contribute the most for the prediction of that sample made by the
model, etc.).
[0119] Although the above methods, systems, and computer program
products have been described in detail for the purpose of
illustration based on what is currently considered to be the most
practical and preferred embodiments or aspects, it is to be
understood that such detail is solely for that purpose and that the
present disclosure is not limited to the described embodiments or
aspects; but, on the contrary, is intended to cover modifications
and equivalent arrangements that are within the spirit and scope of
the appended claims. For example, it is to be understood that the
present disclosure contemplates that, to the extent possible, one
or more features of any embodiment or aspect can be combined with
one or more features of any other embodiment or aspect.
* * * * *