U.S. patent application number 17/468846 was filed with the patent office on 2022-05-05 for methods and systems for automatic determination of a device-specific configuration for a software application operating on a user device.
This patent application is currently assigned to Taplytics Inc.. The applicant listed for this patent is Taplytics Inc.. Invention is credited to Aaron Mosha Glazer, Jonathan Taylor Norris, Imaad Ahmad Umar, Victor Nikola Vucicevich, Adam James Wootton.
Application Number | 20220137958 17/468846 |
Document ID | / |
Family ID | 1000006082536 |
Filed Date | 2022-05-05 |
United States Patent
Application |
20220137958 |
Kind Code |
A1 |
Glazer; Aaron Mosha ; et
al. |
May 5, 2022 |
METHODS AND SYSTEMS FOR AUTOMATIC DETERMINATION OF A
DEVICE-SPECIFIC CONFIGURATION FOR A SOFTWARE APPLICATION OPERATING
ON A USER DEVICE
Abstract
A method and system for automatically determining a
device-specific configuration for a software application operating
on a user device. A configuration monitoring program monitors local
user data stored on a user device and generates a device-specific
prediction model using a machine learning algorithm applied to the
monitored local data. The configuration monitoring program also
receives a global prediction model generated remotely using global
user data collected from a plurality of user devices. The
configuration monitoring program generates a predicted
device-specific configuration of the application operating on the
user device using prediction data from both the device-specific
prediction model and the global prediction model and updates the
configuration of the given application using the predicted
device-specific configuration.
Inventors: |
Glazer; Aaron Mosha;
(Toronto, CA) ; Norris; Jonathan Taylor; (Toronto,
CA) ; Wootton; Adam James; (Toronto, CA) ;
Umar; Imaad Ahmad; (West Vancouver, CA) ; Vucicevich;
Victor Nikola; (Waterloo, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Taplytics Inc. |
Toronto |
|
CA |
|
|
Assignee: |
Taplytics Inc.
Toronto
CA
|
Family ID: |
1000006082536 |
Appl. No.: |
17/468846 |
Filed: |
September 8, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16837341 |
Apr 1, 2020 |
11138003 |
|
|
17468846 |
|
|
|
|
62827885 |
Apr 2, 2019 |
|
|
|
Current U.S.
Class: |
717/121 |
Current CPC
Class: |
G06F 8/71 20130101; G06F
8/65 20130101; G06N 5/04 20130101; G06F 11/3051 20130101; G06N
20/00 20190101; G06F 21/6254 20130101; G06F 8/61 20130101; G06F
11/302 20130101 |
International
Class: |
G06F 8/71 20060101
G06F008/71; G06N 5/04 20060101 G06N005/04; G06N 20/00 20060101
G06N020/00; G06F 21/62 20060101 G06F021/62; G06F 11/30 20060101
G06F011/30; G06F 8/65 20060101 G06F008/65 |
Claims
1-28. (canceled)
29. A method for automatically determining a device-specific
configuration for a given application operating on a user device,
the user device having a processor and a non-transitory memory, the
method comprising: operating a configuration monitoring program on
the user device; monitoring, by the configuration monitoring
program, local user data stored in the non-transitory memory;
generating, by the configuration monitoring program, a local
prediction of the device-specific configuration based on applying a
machine learning algorithm to the monitored local data; generating,
by the configuration monitoring program, a global prediction of the
device-specific configuration based on a remotely-generated global
configuration prediction model; defining, by the configuration
monitoring program, a predicted device-specific configuration of
the given application operating on the user device using the local
prediction of the device-specific configuration and the global
prediction of the device-specific configuration; and updating the
configuration of the given application using the predicted
device-specific configuration.
30. The method of claim 29, further comprising: monitoring, by the
configuration monitoring program, device activity of the user
device; determining, by the configuration monitoring program, that
configuration update criteria have been satisfied based on the
device activity; and in response to determining that the
configuration update criteria have been satisfied, updating the
configuration of the given application using the predicted
device-specific configuration.
31. The method of claim 30, further comprising: defining, by the
configuration monitoring program, the predicted device-specific
configuration in response to determining that the configuration
update criteria have been satisfied.
32. The method of claim 29, wherein generating the local prediction
of the device-specific configuration comprises: generating a
device-specific configuration prediction model by applying the
machine learning algorithm to the monitored local data; and
generating the local prediction using the device-specific
configuration prediction model.
33. The method of claim 32, wherein the configuration monitoring
program is configured to generate model weights for the
device-specific configuration prediction model using aggregate
local user data corresponding to a plurality of additional user
devices.
34. The method of claim 32, further comprising: monitoring, by the
configuration monitoring program, a user feedback input
corresponding to the configuration of the given application; and
updating the device-specific configuration prediction model using
data corresponding to the user feedback input.
35. The method of claim 29, further comprising: monitoring, by the
configuration monitoring program, a user feedback input
corresponding to the configuration of the given application;
determining, by the configuration monitoring program, a local
prediction metric using the local prediction of the device-specific
configuration and the user feedback input, wherein the local
prediction metric is indicative of an accuracy of the local
prediction of the device-specific configuration; and defining the
predicted device-specific configuration using the local prediction
metric.
36. The method of claim 35, wherein defining the predicted
device-specific configuration of the given application comprises:
determining whether the local prediction metric is above a
predetermined performance threshold; and when the local prediction
metric is above the predetermined performance threshold, defining
the predicted device-specific configuration of the given
application entirely from the local prediction, otherwise defining
the predicted device-specific configuration of the given
application entirely from the global prediction.
37. The method of claim 35, wherein generating the predicted
device-specific configuration of the given application comprises:
assigning a first bias weight to the local prediction, wherein the
first bias weight is based on the local prediction metric;
assigning a second bias weight to the global prediction; and
defining the predicted device-specific configuration by combining
the local prediction and the global prediction according to the
first and second bias weights, respectively.
38. The method of claim 37, further comprising: determining whether
the local prediction metric is above a predetermined performance
threshold; and defining the predicted device-specific configuration
using only the global prediction if the local prediction metric is
below the predetermined performance threshold, otherwise defining
the predicted device-specific configuration by combining the local
prediction and the global prediction weighted according to the
first and second bias weights respectively.
39. A device configured to determine a device-specific
configuration for a given application operating on the device, the
device comprising: a non-transitory memory storing local user data;
and a processor configured to: monitor the local user data;
generate a local prediction of the device-specific configuration
based on applying a machine learning algorithm to the monitored
local data; generate a global prediction of the device-specific
configuration based on a remotely-generated global configuration
prediction model; define a predicted device-specific configuration
of the given application operating on the user device using the
local prediction of the device-specific configuration and the
global prediction of the device-specific configuration; and update
the configuration of the given application using the predicted
device-specific configuration.
40. The device of claim 39, wherein the processor is configured to:
monitor device activity of the device; determine that configuration
update criteria have been satisfied based on the device activity;
and in response to determining that the configuration update
criteria have been satisfied, update the configuration of the given
application using the predicted device-specific configuration.
41. The device of claim 40, wherein the processor is configured to:
define the predicted device-specific configuration in response to
determining that the configuration update criteria have been
satisfied.
42. The device of claim 39, wherein the processor is configured to
generate the local prediction of the device-specific configuration
by: generating a device-specific configuration prediction model by
applying the machine learning algorithm to the monitored local
data; and generating the local prediction using the device-specific
configuration prediction model.
43. The device of claim 42, wherein the processor is configured to
generate model weights for the device-specific configuration
prediction model using aggregate local user data corresponding to a
plurality of additional user devices.
44. The device of claim 42, wherein the processor is configured to:
monitor a user feedback input corresponding to the configuration of
the given application; and update the device-specific configuration
prediction model using data corresponding to the user feedback
input.
45. The device of claim 39, wherein the processor is configured to:
monitor a user feedback input corresponding to the configuration of
the given application; determine a local prediction metric using
the local prediction of the device-specific configuration and the
user feedback input, wherein the local prediction metric is
indicative of an accuracy of the local prediction of the
device-specific configuration; and define the predicted
device-specific configuration using the local prediction
metric.
46. The device of claim 45, wherein the processor is configured to
define the predicted device-specific configuration of the given
application by: determining whether the local prediction metric is
above a predetermined performance threshold; and when the local
prediction metric is above the predetermined performance threshold,
defining the predicted device-specific configuration of the given
application entirely from the local prediction, otherwise defining
the predicted device-specific configuration of the given
application entirely from the global prediction.
47. The device of claim 45, wherein the processor is configured to
define the predicted device-specific configuration of the given
application by: assigning a first bias weight to the local
prediction, wherein the first bias weight is based on the local
prediction metric; assigning a second bias weight to the global
prediction; and defining the predicted device-specific
configuration by combining the local prediction and the global
prediction according to the first and second bias weights,
respectively.
48. The device of claim 47, wherein the processor is configured to:
determine whether the local prediction metric is above a
predetermined performance threshold; and define the predicted
device-specific configuration using only the global prediction if
the local prediction metric is below the predetermined performance
threshold, otherwise define the predicted device-specific
configuration by combining the local prediction and the global
prediction weighted according to the first and second bias weights
respectively.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a Continuation of U.S. application Ser.
No. 16/837,341 filed on Apr. 1, 2020, which claims the benefit of
U.S. Provisional Application No. 62/827,885 filed on Apr. 2, 2019,
the entire contents of which are incorporated herein by
reference.
FIELD
[0002] The described embodiments relate to managing configurations
for software applications, and in particular, to methods and
systems for automatically selecting the configuration of a software
application operating on a user device.
BACKGROUND
[0003] The following is not an admission that anything discussed
below is part of the prior art or part of the common general
knowledge of a person skilled in the art.
[0004] United States Publication No. 2017/0109322A1 of McMahan et
al. purports to disclose systems and methods of determining a
global model. In particular, one or more local updates can be
received from a plurality of user devices. Each local update can be
determined by the respective user device based at least in part on
one or more data examples stored on the user device. The one or
more data examples stored on the plurality of user devices are
distributed on an uneven basis, such that no user device includes a
representative sample of the overall distribution of data examples.
The local updates can then be aggregated to determine a global
model.
[0005] United States Publication No. 2018/0018590A1 of Szeto et al.
purports to disclose a distributed, online machine learning system.
Contemplated systems include many private data servers, each having
local private data. Researchers can request that relevant private
data servers train implementations of machine learning algorithms
on their local private data without requiring de-identification of
the private data or without exposing the private data to
unauthorized computing systems. The private data servers also
generate synthetic or proxy data according to the data
distributions of the actual data. The servers then use the proxy
data to train proxy models. When the proxy models are sufficiently
similar to the trained actual models, the proxy data, proxy model
parameters, or other learned knowledge can be transmitted to one or
more non-private computing devices. The learned knowledge from many
private data servers can then be aggregated into one or more
trained global models without exposing private data.
[0006] United States Publication No. 2015/0242760A1 of Miao et al.
purports to disclose machine learning that may be personalized to
individual users of computing devices, and can be used to increase
machine learning prediction accuracy and speed, and/or reduce
memory footprint. Personalizing machine learning can include
hosting, by a computing device, a consensus machine learning model
and collecting information, locally by the computing device,
associated with an application executed by the client device.
Personalizing machine learning can also include modifying the
consensus machine learning model accessible by the application
based, at least in part, on the information collected locally by
the client device. Modifying the consensus machine learning model
can generate a personalized machine learning model. Personalizing
machine learning can also include transmitting the personalized
machine learning model to a server that updates the consensus
machine learning model.
SUMMARY
[0007] The following introduction is provided to introduce the
reader to the more detailed discussion to follow. The introduction
is not intended to limit or define any claimed or as yet unclaimed
invention. One or more inventions may reside in any combination or
sub-combination of the elements or process steps disclosed in any
part of this document including its claims and figures.
[0008] In accordance with one aspect of this disclosure, which may
be used alone or in combination with any other aspect, there is
provided a method for automatically determining a device-specific
configuration for a given application operating on a user device,
the user device having a processor, a non-transitory memory and a
configuration monitoring program stored in the non-transitory
memory. The method can include: installing a configuration
monitoring program on the user device, the configuration monitoring
program including a device-specific configuration prediction model;
monitoring, by the configuration monitoring program, local user
data stored in the non-transitory memory of the user device;
generating, by the configuration monitoring program on the user
device, a device-specific configuration prediction model, where the
device-specific configuration prediction model is generated using a
machine learning algorithm applied to the monitored local data;
receiving, by the configuration monitoring program on the user
device, a global configuration prediction model, where the global
configuration prediction model is generated remotely using global
user data collected from a plurality of user devices; defining, by
the configuration monitoring program, a predicted device-specific
configuration of the given application operating on the user device
by: generating, by the configuration monitoring program using the
device-specific configuration prediction model, a local prediction
of the device-specific configuration; generating, by the
configuration monitoring program using the global configuration
prediction model, a global prediction of the device-specific
configuration; and generating, by the configuration monitoring
program, the predicted device-specific configuration of the given
application using the local prediction of the device-specific
configuration and the global prediction of the device-specific
configuration; and updating the configuration of the given
application using the predicted device-specific configuration.
[0009] In some embodiments, the method may include: monitoring, by
the configuration monitoring program, device activity of the user
device; identifying, by the configuration monitoring program, that
configuration update criteria have been satisfied based on the
device activity; and updating the configuration of the given
application using the predicted device-specific configuration, in
response to identifying that the configuration update criteria have
been satisfied.
[0010] In some embodiments, the method may include: defining, by
the configuration monitoring program, the predicted device-specific
configuration in response to identifying that the configuration
update criteria have been satisfied.
[0011] In some embodiments, the global user data includes
anonymized aggregate data collected from a plurality of user
devices while omitting personally identifiable information.
[0012] In some embodiments, the configuration monitoring program
can be installed on a plurality of user devices, and the method
includes: transmitting a query to the plurality of user devices,
where the query includes a request for aggregate local user data;
receiving, from the configuration monitoring program operating on
each of the plurality of user devices, respective aggregate local
user data generated locally by the configuration monitoring program
on each user device from the local data stored on that user device;
and combining the aggregate local user data received from the
plurality of user devices to generate global aggregate user
data.
[0013] In some embodiments, the configuration monitoring program on
each user device is configured to generate model weights for the
device-specific configuration prediction model, and the aggregate
local user data received from the plurality of user devices
includes the model weights.
[0014] In some embodiments, the method may include: monitoring, by
the configuration monitoring program, a user feedback input
corresponding to the configuration of the given application; and
updating the device-specific configuration prediction model using
data corresponding to the user feedback input.
[0015] In some embodiments, the method may include: determining, by
the configuration monitoring program, a local model prediction
metric using the local prediction of the device-specific
configuration and the user feedback input.
[0016] In some embodiments, the local model prediction metric is
determined by the configuration monitoring program using historical
metric results.
[0017] In some embodiments, generating the predicted
device-specific configuration of the given application, includes:
determining whether the local prediction metric is above a
predetermined performance threshold; and when the local prediction
metric is above the predetermined performance threshold, generating
the predicted device-specific configuration of the given
application entirely from the local prediction and giving the
global prediction no weight, otherwise generating the predicted
device-specific configuration of the given application entirely
from the global prediction and giving the local prediction no
weight.
[0018] In some embodiments, generating the predicted
device-specific configuration of the given application includes:
assigning a first bias weight to the local prediction, where the
first bias weight is based on the local prediction metric;
assigning a second bias weight to the global prediction; and
generating the predicted device-specific configuration by combining
the local prediction and the global prediction according to the
first and second bias weights, respectively.
[0019] In some embodiments, the method may include: determining
whether the local prediction metric is above a predetermined
performance threshold; and generating the predicted device-specific
configuration using only the global prediction if the local
prediction metric is below the predetermined performance threshold,
otherwise generating the predicted device-specific configuration by
combining the local prediction and the global prediction weighted
according to the first and second bias weights respectively.
[0020] In some embodiments, the global data includes a plurality of
default configured outcomes.
[0021] In some embodiments, the configuration monitoring program
automatically updates the configuration of the given application
using the predicted device-specific configuration.
[0022] In accordance with one aspect of this disclosure, which may
be used alone or in combination with any other aspect, there is
provided a system for automatically determining a device-specific
configuration for a given application operating on a first user
device, the system including: a plurality of user devices including
the first user device, each user device having a processor, a
non-transitory memory and a configuration monitoring program stored
in the non-transitory memory; a server in communication with the
plurality of user devices, where the server is configured to:
collect global user data from the plurality of user devices; and
generate a global configuration prediction model using the global
user data; and the first user device is configured by the
configuration monitoring program to: monitor local user data stored
in the non-transitory memory of the first user device; generate a
device-specific configuration prediction model, where the
device-specific configuration prediction model is generated using a
machine learning algorithm using the monitored local data; receive
the global configuration prediction model from the server; define a
predicted device-specific configuration of the given application
operating on the first user device by: generating, using the
device-specific configuration prediction model, a local prediction
of the device-specific configuration; generating, using the global
configuration prediction model, a global prediction of the
device-specific configuration; and generating the predicted
device-specific configuration of the given application using the
local prediction of the device-specific configuration and the
global prediction of the device-specific configuration; and update
the configuration of the given application using the predicted
device-specific configuration.
[0023] In some embodiments, the first user device is configured by
the configuration monitoring program to: monitor device activity of
the first user device; identify that configuration update criteria
have been satisfied based on the device activity; and update the
configuration of the given application using the predicted
device-specific configuration, in response to identifying that the
configuration update criteria have been satisfied.
[0024] In some embodiments, the first user device is configured by
the configuration monitoring program to: define the predicted
device-specific configuration in response to identifying that the
configuration update criteria have been satisfied.
[0025] In some embodiments, the server is configured to collect the
global user data as anonymized aggregate data that omits personally
identifiable information.
[0026] In some embodiments, the server is configured to: transmit a
query to the plurality of user devices, where the query includes a
request for aggregate local user data; receive, from the
configuration monitoring program operating on each of the plurality
of user devices, respective aggregate local user data generated
locally by the configuration monitoring program on each user device
from the local data stored on that user device; and combine the
aggregate local user data received from the plurality of user
devices to generate the global user data as global aggregate user
data.
[0027] In some embodiments, the configuration monitoring program on
each user device is configured to generate training weights for the
device-specific configuration prediction model, and the aggregate
local user data received from the plurality of user devices
includes the training weights.
[0028] In some embodiments, the first user device is configured by
the configuration monitoring program to: monitor a user feedback
input corresponding to the configuration of the given application;
and update the device-specific configuration prediction model using
data corresponding to the user feedback input.
[0029] In some embodiments, the first user device is configured by
the configuration monitoring program to: determine a local model
prediction accuracy using the local prediction of the
device-specific configuration and the user feedback input.
[0030] In some embodiments, the local model prediction accuracy is
determined by the configuration monitoring program on a rolling
average basis.
[0031] In some embodiments, generating the predicted
device-specific configuration of the given application includes:
determining whether the local prediction accuracy is above a
predetermined performance threshold; and when the local prediction
accuracy is above the predetermined performance threshold,
generating the predicted device-specific configuration of the given
application entirely from the local prediction and giving the
global prediction no weight, otherwise generating the predicted
device-specific configuration of the given application entirely
from the global prediction and giving the local prediction no
weight.
[0032] In some embodiments, generating the predicted
device-specific configuration of the given application includes:
assigning a first bias weight to the local prediction, where the
first bias weight is based on the local prediction accuracy;
assigning a second bias weight to the global prediction; and
generating the predicted device-specific configuration by combining
the local prediction and the global prediction according to the
first and second bias weights, respectively.
[0033] In some embodiments, the first user device is configured by
the configuration monitoring program to: determine whether the
local prediction accuracy is above a predetermined performance
threshold; and generate the predicted device-specific configuration
using only the global prediction if the local prediction accuracy
is below the predetermined performance threshold, otherwise
generating the predicted device-specific configuration by combining
the local prediction and the global prediction weighted according
to the first and second bias weights respectively.
[0034] In some embodiments, the global data includes a plurality of
default data outcomes.
[0035] In some embodiments, the configuration monitoring program
automatically updates the configuration of the given application
using the predicted device-specific configuration.
[0036] It will be appreciated by a person skilled in the art that a
system or method disclosed herein may embody any one or more of the
features contained herein and that the features may be used in any
particular combination or sub-combination.
[0037] These and other aspects and features of various embodiments
will be described in greater detail below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0038] For a better understanding of the described embodiments and
to show more clearly how they may be carried into effect, reference
will now be made, by way of example, to the accompanying drawings
in which:
[0039] FIG. 1 is a block diagram of a computer network system in
accordance with an example embodiment;
[0040] FIG. 2 is a block diagram of a system for automatically
determining a device-specific configuration for a software
application in accordance with an example embodiment;
[0041] FIG. 3A is a flowchart illustrating a method for
automatically determining a device-specific configuration for a
software application in accordance with an example embodiment;
[0042] FIG. 3B is a flowchart illustrating a method for updating a
configuration prediction model using user feedback input in
accordance with an example embodiment;
[0043] FIG. 4A is a flowchart illustrating a method for generating
a predicted device-specific configuration for a software
application in accordance with an example embodiment;
[0044] FIG. 4B is a flowchart illustrating a method for generating
a predicted device-specific configuration for a software
application using a local prediction metric in accordance with an
example embodiment;
[0045] FIG. 4C is a flowchart illustrating a method for generating
a predicted device-specific configuration for a software
application by combining weighted local and global predictions in
accordance with an example embodiment;
[0046] FIG. 4D is a flowchart illustrating a method for generating
a predicted device-specific configuration for a software
application using a local prediction metric in accordance with an
example embodiment; and
[0047] FIG. 5 is a flowchart illustrating a method for acquiring
aggregate user data from one or more user devices in accordance
with an example embodiment.
[0048] The drawings, described below, are provided for purposes of
illustration, and not of limitation, of the aspects and features of
various examples of embodiments described herein. For simplicity
and clarity of illustration, elements shown in the drawings have
not necessarily been drawn to scale. The dimensions of some of the
elements may be exaggerated relative to other elements for
clarity.
DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0049] Various systems or methods will be described below to
provide an example of an embodiment of the claimed subject matter.
No embodiment described below limits any claimed subject matter and
any claimed subject matter may cover methods or systems that differ
from those described below. The claimed subject matter is not
limited to systems or methods having all of the features of any one
system or method described below or to features common to multiple
or all of the apparatuses or methods described below. It is
possible that a system or method described below is not an
embodiment that is recited in any claimed subject matter. Any
subject matter disclosed in a system or method described below that
is not claimed in this document may be the subject matter of
another protective instrument, for example, a continuing patent
application, and the applicants, inventors or owners do not intend
to abandon, disclaim or dedicate to the public any such subject
matter by its disclosure in this document.
[0050] Furthermore, it will be appreciated that for simplicity and
clarity of illustration, where considered appropriate, reference
numerals may be repeated among the figures to indicate
corresponding or analogous elements. In addition, numerous specific
details are set forth in order to provide a thorough understanding
of the embodiments described herein. However, it will be understood
by those of ordinary skill in the art that the embodiments
described herein may be practiced without these specific details.
In other instances, well-known methods, procedures and components
have not been described in detail so as not to obscure the
embodiments described herein. Also, the description is not to be
considered as limiting the scope of the embodiments described
herein.
[0051] It should also be noted that the terms "coupled" or
"coupling" as used herein can have several different meanings
depending in the context in which these terms are used. For
example, the terms coupled or coupling may be used to indicate that
an element or device can electrically, optically, or wirelessly
send data to another element or device as well as receive data from
another element or device. As used herein, two or more components
are said to be "coupled", or "connected" where the parts are joined
or operate together either directly or indirectly (i.e., through
one or more intermediate components), so long as a link occurs. As
used herein and in the claims, two or more parts are said to be
"directly coupled", or "directly connected", where the parts are
joined or operate together without intervening intermediate
components.
[0052] It should be noted that terms of degree such as
"substantially", "about" and "approximately" as used herein mean a
reasonable amount of deviation of the modified term such that the
end result is not significantly changed. These terms of degree may
also be construed as including a deviation of the modified term if
this deviation would not negate the meaning of the term it
modifies.
[0053] Furthermore, any recitation of numerical ranges by endpoints
herein includes all numbers and fractions subsumed within that
range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, and 5). It
is also to be understood that all numbers and fractions thereof are
presumed to be modified by the term "about" which means a variation
of up to a certain amount of the number to which reference is being
made if the end result is not significantly changed.
[0054] The example embodiments of the systems and methods described
herein may be implemented as a combination of hardware or software.
In some cases, the example embodiments described herein may be
implemented, at least in part, by using one or more computer
programs, executing on one or more programmable devices comprising
at least one processing element, and a data storage element
(including volatile memory, non-volatile memory, storage elements,
or any combination thereof). These devices may also have at least
one input device (e.g. a pushbutton keyboard, mouse, a touchscreen,
and the like), and at least one output device (e.g. a display
screen, a printer, a wireless radio, and the like) depending on the
nature of the device.
[0055] It should also be noted that there may be some elements that
are used to implement at least part of one of the embodiments
described herein that may be implemented via software that is
written in a high-level computer programming language such as
object oriented programming or script-based programming.
Accordingly, the program code may be written in Java,
Swift/Objective-C, C, C++, Javascript, Python, SQL or any other
suitable programming language and may comprise modules or classes,
as is known to those skilled in object oriented programming.
Alternatively, or in addition thereto, some of these elements
implemented via software may be written in assembly language,
machine language or firmware as needed. In either case, the
language may be a compiled or interpreted language.
[0056] At least some of these software programs may be stored on a
storage media (e.g. a computer readable medium such as, but not
limited to, ROM, magnetic disk, optical disc) or a device that is
readable by a general or special purpose programmable device. The
software program code, when read by the programmable device,
configures the programmable device to operate in a new, specific
and predefined manner in order to perform at least one of the
methods described herein.
[0057] Furthermore, at least some of the programs associated with
the systems and methods of the embodiments described herein may be
capable of being distributed in a computer program product
comprising a computer readable medium that bears computer usable
instructions for one or more processors. The medium may be provided
in various forms, including non-transitory forms such as, but not
limited to, one or more diskettes, compact disks, tapes, chips, and
magnetic and electronic storage. The computer program product may
also be distributed in an over-the-air or wireless manner, using a
wireless data connection.
[0058] The term "software application" or "application" refers to
computer-executable instructions, particularly computer-executable
instructions stored in a non-transitory medium, such as a
non-volatile memory, and executed by a computer processor. The
computer processor, when executing the instructions, may receive
inputs and transmit outputs to any of a variety of input or output
devices to which it is coupled. Software applications may include
mobile applications or "apps" for use on mobile devices such as
smartphones and tablets or other "smart" devices.
[0059] A software application can be, for example, a monolithic
software application, built in-house by the organization and
possibly running on custom hardware; a set of interconnected
modular subsystems running on similar or diverse hardware; a
software-as-a-service application operated remotely by a third
party; third party software running on outsourced infrastructure,
etc. In some cases, a software application also may be less formal,
or constructed in ad hoc fashion, such as a programmable
spreadsheet document that has been modified to perform computations
for the organization's needs.
[0060] Software applications may be deployed to and installed on a
computing device on which it is to operate. Depending on the nature
of the operating system and/or platform of the computing device, an
application may be deployed directly to the computing device,
and/or the application may be downloaded from an application
marketplace. For example, user of the user device may download the
application through an app store such as the Apple App Store.TM. or
Google.TM. Play.TM..
[0061] A software application may have a plurality of different
configurations that may be used for the application user interface.
The different configurations may be designed to encourage user
engagement with the application and/or to facilitate user
interactions. Different configurations may include variations of
user interface elements and user interface parameters that may
facilitate user interactions by different users. Modifying the
configuration of a software application to be more tailored to
individual users can increase user engagement and facilitate user
interactions with a software application.
[0062] Tailoring application configurations may encourage a user to
make additional use of the software application as this may provide
the user with an easier and more enjoyable experience using the
application. However, determining the application configuration
that is suited to an individual user is often difficult. Monitoring
application use data and performing analytics on that data can
provide insights into a user's preferred application configuration.
However, the analysis of this user interaction data is often
performed remotely from the user's device, for instance on a remote
server. Providing the server with individual interaction data may
be problematic or risky, as this interaction data often includes
personally identifiable and/or private data. This may result in the
server being a target for hacking operations and may significantly
increase the potential liability for the administrator of the
server in storing this data. As a result, the administrator may be
forced to implement significant and possibly expensive data
protection systems and policies while still running the risk of a
data breach.
[0063] In some cases, analyzing an individual user's interactions
with an application may provide insufficient data to evaluate
configurations that are likely to be preferred by the user.
Particularly where the user has just begun using the application,
or use of the application is minimal or inconsistent, it may be
difficult to acquire sufficient data to determine accurately from
that user's own interactions what application configurations are
likely to lead to increased user engagement. In such cases, it may
be desirable to use data from other users of the application to
provide a larger data set of user interactions that allow for
improved analysis of preferred application configurations.
[0064] Embodiments described herein provide systems and methods for
automatically determining an expected preferred configuration of a
software application operating on a computing device. The
embodiments described herein may determine a device-specific
configuration for the software application operating on a user
device. The device-specific configuration may be determined as the
configuration of user interface elements of the software
application that is likely to be preferred on that user device
(e.g. by the user or users of that device).
[0065] A configuration monitoring program can operate on a user's
device as part of, or in addition to, the software application
operating on the user device. The configuration monitoring program
can be configured to determine the device-specific configuration
for the software application.
[0066] The configuration monitoring program can include both a
device-specific configuration prediction model and a global
configuration prediction model. The device-specific configuration
prediction model and the global configuration prediction model can
operate contemporaneously on the user device (i.e. within the
configuration monitoring program). Each of the device-specific
configuration prediction model and the global configuration
prediction model can generate separate, and independent,
predictions for the preferred configuration of the software
application. The configuration monitoring program can then
determine a predicted device-specific configuration using the
predictions from the device-specific configuration prediction model
and the global configuration prediction model.
[0067] In some cases, the local and global predictions may be
combined in some manner to define the predicted device-specific
configuration. In other cases, the configuration monitoring program
may apply one or more prediction criteria to determine which of the
local and global predictions should be selected as the predicted
device-specific configuration.
[0068] The device-specific configuration prediction model can be
defined using local user data from the device on which the
configuration monitoring program operates. The device-specific
prediction model can thus be trained locally on the user device.
The configuration monitoring program can generate, and modify, the
device-specific configuration prediction model using a machine
learning algorithm that operates on the user device using the local
data. The device-specific prediction model may be trained locally
on the device using local interaction data from the user device.
This may reduce concerns regarding data sensitivity that might
arise if the user data is transmitted off the user device.
[0069] In some cases, the configuration monitoring program can
define the device-specific configuration prediction model as a
purely local model. That is, the device-specific configuration
prediction model may not be modified (e.g. mixed or retrained)
using data from other devices or from the global configuration
prediction model. This may ensure that the predictions generated by
device-specific configuration prediction model are targeted
specifically to that device (and in turn the user(s) of that
device).
[0070] The global prediction model can be generated remotely at a
central server. The global prediction model can be trained using
user data from a plurality of different users and different
devices. In some cases, the global prediction model may be trained
using aggregate user data compiled from a plurality of user
devices. In embodiments described herein, the aggregate user data
may be collected from the user devices while omitting any
personally identifiable information.
[0071] The configuration monitoring program may be configured to
predict or estimate the configuration of the software application
at defined intervals and/or in response to device activity (e.g.
user interactions and/or a change in the device environment or
location). The configuration monitoring program may then use the
local and global prediction models to determine a predicted
configuration. The device-specific prediction model can generate a
local prediction and the global model can generate a global
prediction for the configuration of the software application. The
configuration monitoring program can then determine the predicted
device-specific configuration or variant using the local prediction
and the global prediction. In some cases, the configuration
monitoring program may apply prediction weights to both local and
global predictions to determine a predicted device-specific variant
of the application configuration.
[0072] In some cases, the configuration monitoring program may be
configured to automatically implement the predicted device-specific
configuration in the software application operating on the user
device. In other cases, the configuration monitoring program may
provide the predicted device-specific configuration as a selectable
option for the user, for instance through a prompt on the user
device. The user may then select whether to implement the predicted
device-specific configuration or another configuration for the
application.
[0073] Additionally or alternatively, the configuration monitoring
program may use the predicted device-specific configuration to
evaluate the success/accuracy of the local and/or global prediction
models. For example, the configuration monitoring program may
monitor user input to the device, including user feedback data. The
feedback data may be generated based on user actions such as
accepting or using the prediction device-specific variant, or
manually modifying the application configuration. The feedback data
may also include various user engagement and user interaction data
related to user of the application. The accuracy of the
device-specific prediction model can then be assessed using the
feedback data. The configuration monitoring program may then update
or modify the device-specific prediction model based on model
accuracy. As the device is used, both local and global models can
be continually retrained using both local and aggregate data,
respectively.
[0074] As used herein, the configuration of a software application
generally refers to any configurable variant of a software
application that contributes to the user experience offered by that
application. By way of non-limiting examples, a configuration
variant may relate to aspects of the application user interface,
such as the visual configuration of the user interface (e.g. layout
or format), the configuration of audible or tactile outputs from
the application, and in some cases deployment of, or access to,
specific application features. It will be appreciated that
different applications will include different configuration
parameters, and in particular, these configuration may dynamically
change in response to changes in device activity, user-device
interaction, and changes in user device attributes.
[0075] For example, the configuration monitoring program may
monitor the configuration of various parameters of user interface
elements of the application as well as the relationship between the
user interface elements within the application user interface.
Examples of user interface element parameters that may be modified
can include color, user interaction responses, element position
(e.g., X and Y coordinates) within the application user interface,
element arrangement within the foreground and/or background of the
user interface, textual content, justification of the text, font,
font size. Various examples of application parameters such as user
interface element parameters that may be modified by the
configuration monitoring program, and how those modifications may
be implemented, are described in further detail in Applicant's U.S.
Pat. No. 9,507,609B2 the entirety of which is incorporated herein
by reference.
[0076] As one example, the configuration monitoring program may be
configured to monitor a messaging application operating on a user
device (e.g. an application for e-mail, text, or Short Messaging
Service (SMS)). The configuration of the messaging application that
is being monitored may include the visual layout of the application
and how specific content elements are arranged within that layout.
For instance, messages can be displayed in a particular manner
(e.g. in line or in bubbles), or otherwise sorted or classified in
a particular manner (e.g. a color-coded classification scheme). As
another example, the way in which certain user interface elements
are displayed (e.g. the text chosen for specific buttons, or the
color chosen for certain buttons) may be modified.
[0077] As another example, the configuration monitoring program may
be configured to monitor a media-playback application. The
media-playback application may be configured to define how the user
interface elements of available audio or video files are arranged
within the user interface of the application. In some cases, the
configuration monitoring program may arrange the user interface
elements in one of a number of pre-defined layout
configurations.
[0078] As a further example, the configuration monitoring program
may be configured to monitor an ordering or shopping application.
The configuration monitoring program may configure various user
interface elements within the application to define how those user
interfaces elements are displayed. In some cases, the configuration
monitoring program may select the text that may appear on
interactive user interface elements (e.g. selecting whether an
interactive element includes the text `cart` and `basket`, or
whether a different interactive element includes the text `order`,
`buy`, or `purchase`).
[0079] A "device-specific" configuration refers to a preferred or
desired configuration for a software application operating on a
user-device that is specific to that user device. In general, the
device-specific configuration refers to the configuration of the
software application being determined or selected specifically for
the device in question, whether or not it is different from the
configuration on any other given device. The device-specific
configuration may not be a static, unchanging, configuration but
may vary over time based on manually selected configurations and
even changes in the available configurations for the software
application (e.g. if the user interface for the application is
updated). The device-specific configuration for a software
application on a given user device may change dynamically in
response to monitored activity on the device, such as user
interactions and inputs, as well as changes to the device
characteristics (e.g. updating the operating system) and device
environment.
[0080] In embodiments described herein, the configuration
monitoring program may use one or more machine learning algorithms
in order to automatically determine, or predict, the
device-specific configuration for a software application operating
on a user device. Various types of machine learning algorithms may
be used to generate the prediction models that, in turn, are used
to generate the local and global configuration predictions. For
instance, the machine learning algorithms may be configured to
perform supervised learning processes in order to determine the
predicted configuration models.
[0081] Generally, during "supervised learning" processes, a machine
learning algorithm can be provided with a set of "training data"
which is composed from a set of example input data, and set of
example resultant output data. The machine learning algorithm can
iterate over the "training data" with a view to mapping (or
correlating) the input data to the desired output data. Once the
mapping is complete, each input variable can be assigned a "model
weight" based on how strongly that input variable is connected (or
correlated) to the resultant output data. The mapping can then
generate a "predictive model" which may be used to predict, or
extrapolate, new output data results using new input data.
[0082] In embodiments described herein, the output variables for
machine learning algorithms used by the configuration monitoring
program can be defined as the preferred configurations for a given
software application. The input features (or input variables) may
be defined as the classes of data (or types of data) determined to
be connected to a user's selection of a particular configuration
(e.g., weather, user activity, time of day etc.). The input
variables and the output variables will accordingly vary depending
on the particular software application at issue, as well as the
available "training data". In some cases, the input features for a
given software application may be defined by a developer of the
application, and may even be modified after the application has
been deployed.
[0083] Any suitable machine learning algorithm configured for
supervised learning and/or reinforcement learning may be used for
determining the device-specific configuration of an application.
For example, in various embodiments, the machine learning
algorithms may be classification or regression algorithms. Examples
of classification and regression algorithms include neural
networks, deep learning neutral networks, support vector machines
(SVM), decision trees, random forests, k-nearest neighbor
algorithms (KNN), logistic regression algorithms, and naive Bayes
algorithms among others.
[0084] In accordance with the some embodiments provided herein, the
global prediction model and local prediction model can be generated
using separate processes. In some cases, the global prediction
model and local prediction model may be generated using two
separate machine learning algorithms. The configuration monitoring
program may include a first machine learning algorithm operable to
generate the local prediction model for the configuration of the
software application. A global configuration program that may be
provided on a server can be used to generate the global prediction
model. The global configuration program may include a second
machine learning algorithm that can be configured to generate a
global prediction model for the software application
configuration.
[0085] The first machine learning algorithm (also referred to
herein as a "local machine learning algorithm") can operate (or be
hosted) on the user device and can be trained using local data
available on the user device. In at least some embodiments, the
local user data can include both public data and private data
(i.e., data that includes personally identifiable information (PII)
in respect of particular users). The local machine learning
algorithm can iterate over the local user data in order to generate
a device-specific configuration prediction model (also referred to
herein as a "local prediction model", or simply, a "local model").
The device-specific configuration prediction model can be used by
configuration monitoring program on the user device to produce
local predictions of a device-specific configuration for a given
software application operating on the user device. In some
embodiments, the local prediction model may remain stored solely on
the user device and may not transmitted be from the user device to
any third-party devices (i.e., a remote server). This may avoid
disclosing individual user preferences on an identifiable
basis.
[0086] The second machine learning algorithm (also referred to
herein as a "global machine learning algorithm") can operate (or be
hosted) on a remote server. The global machine learning algorithm
may be trained using an aggregated data that includes anonymized
aggregate data received from one or more user devices. As explained
herein, the aggregate data may include cumulative data that does
not otherwise include personal or private information about
particular users. The global machine learning algorithm can iterate
over the aggregate data to generate a global configuration
prediction model (also referred to herein as a "global prediction
model", or simply, a "global model"). The global prediction model
can be distributed to the user devices. The global prediction model
can be used by the configuration monitoring program on each of the
user devices to generate "generalized" global predictions of
device-specific configurations for given software applications. The
user devices may then use the predictions from the local and global
models to generate a predicted device-specific configuration for
the software application.
[0087] In some embodiments described herein, the global prediction
model may be generated based on data from local prediction models
on a plurality of user devices, rather than using machine learning
algorithm stored on the server. For example, in some cases, the
server may be configured to receive "training weights" generating
by partially-trained local prediction models hosted on the one or
more user devices. The "training weights" can be combined on the
remote server to generate the global prediction model. The global
prediction model may then distributed to the one or more user
devices for use in generating a global prediction of a
device-specific configuration for an application. In still other
cases, the global prediction model can be defined to include a set
of pre-defined baseline or default configuration options for a
particular software application.
[0088] Referring now to FIG. 1, there is shown a block diagram of a
computer network system 100 in accordance with an example
embodiment. As shown, the computer network system 100 generally
includes a server 105, a network 110 and one or more user devices
115A-115N connected via network 110.
[0089] In at least some embodiments, network 110 may be connected
to the internet. Typically, the connection between network 110 and
the Internet may be made via a firewall server (not shown). In some
cases, there may be multiple links or firewalls, or both, between
network 110 and the Internet. Some organizations may operate
multiple networks 110 or virtual networks 110, which can be
internetworked or isolated. These have been omitted for ease of
illustration, however it will be understood that the teachings
herein can be applied to such systems. Network 110 may be
constructed from one or more computer network technologies, such as
IEEE 802.3 (Ethernet), IEEE 802.11 and similar technologies.
[0090] Server 105 is a computer server that is connected to network
110. Server 105 has a processor, volatile and non-volatile memory,
at least one network interface, and may have various other
input/output devices. As with all devices shown in the system 100,
there may be multiple servers 105, although not all are shown.
[0091] User devices 115 generally refer to desktop or laptop
computers, but may also refer to smartphones, tablet computers, as
well as a wide variety of "smart" devices capable of data
communication. Like server 105, user devices 115 each include a
processor, a volatile and non-volatile memory, at least one network
interface, and input/output devices. User devices 115 may be
portable, and may at times be connected to network 110 or a portion
thereof.
[0092] As will be understood, a user device 115 may be any suitable
computing device 115 capable of executing an application. For
example, in various embodiments, the computing device 115 may
include mobile devices such as smartphones, tablets or laptops, as
well as less conventional computing devices such as: smart
appliances (e.g., computers embedded within refrigerators,
thermostats, stoves, microwaves, televisions); wearable computing
devices such as smartwatches, smart glasses, and/or smart clothing;
computers embedded into automobiles, cars or vehicles (e.g., as may
have been provided for navigation and/or entertainment purposes).
Further examples of computing devices 115 include digital media
receivers, programmable billboards, advertising displays, and/or
medical diagnostic equipment.
[0093] Each of the user devices 115 can have a plurality of
software applications operating thereon. The user devices 115 can
also include a configuration monitoring program associated with one
or more of those software applications. In some embodiments, the
configuration monitoring program may be integrated with a given
software application. Alternatively, the configuration monitoring
program may be separate application operating on the user device
115. The configuration monitoring program may be configured to
monitor the configuration of the given application on the user
device 115 and to generate device-specific predictions for the
configuration of the application that would be preferred on that
user device 115.
[0094] The configuration monitoring program can use a local
prediction model and a global prediction model to determine the
predicted configuration for the application on that user device
115. The local prediction model may be defined and trained using
local data from the user device 115. The global prediction model
may be provided by the server 105. The server 105, in turn, may
generate the global prediction model using data from a plurality of
user devices 115.
[0095] In embodiments of the system 100, the server 105 can be
configured to communicate with the user devices 115. The server 105
may request device data from the user devices 115 relating to user
interactions on those devices 115. The server 105 may generate
requests for aggregated data from each of the devices 115, rather
than any individually identifiable data. The user devices 115 may
transmit to the server 105 corresponding data in an aggregated
form, to avoid the transmitted data being individually or
personally identifiable.
[0096] The server 105 can generate and train the global prediction
model using the data received from the plurality of user devices.
The server 105 can also provide each of the user devices 115 with
the global prediction model, as well as any subsequent updates to
the global prediction model that may be generated.
[0097] Referring now to FIG. 2, there is shown a block diagram of a
system 200 for automatically determining a device-specific
configuration for a given application operating on a user device,
in accordance with an example embodiment. System 200 illustrates a
more detailed example of components of the server 105 and a user
device 115.
[0098] As shown in FIG. 1 server 105 is a computer server that is
connected to network 110. As shown in FIG. 2, server 105 can
include a processor 232, a display 234, a memory 236, a
communication interface 240 and a global configuration program 238.
Although shown as separate elements, it will be understood that the
global configuration program 238 may be stored in memory 236.
[0099] It will be understood that the server 105 need not be a
dedicated physical computer for executing the global configuration
program 238. For example, in various embodiments, the various
logical components that are shown as being provided on server 105
may be hosted by a third party "cloud" hosting service such as
Amazon.TM. Web Services.TM. Elastic Compute Cloud (Amazon EC2).
[0100] Processor 232 is a computer processor, such as a general
purpose microprocessor. In some other cases, processor 232 may be a
field programmable gate array, application specific integrated
circuit, microcontroller, or other suitable computer processor.
[0101] Processor 232 is also coupled to display 234, which is a
suitable display for outputting information and data as needed by
various computer programs. In particular, display 234 may display a
graphical user interface (GUI). In some cases, the display 234 may
be omitted from server 105, for instance where the server 105 is
configured to operate autonomously.
[0102] Communication interface 240 is one or more data network
interface, such as an IEEE 802.3 or IEEE 802.11 interface, for
communication over a network.
[0103] Processor 232 is coupled, via a computer data bus, to memory
236. Memory 236 may include both volatile and non-volatile memory.
Non-volatile memory stores computer programs consisting of
computer-executable instructions, which may be loaded into volatile
memory for execution by processor 232 as needed. It will be
understood by those of skill in the art that references herein to
server 105 as carrying out a function or acting in a particular way
imply that processor 232 is executing instructions (e.g., a
software program) stored in memory 236 and possibly transmitting or
receiving inputs and outputs via one or more interface. Memory 236
may also store data input to, or output from, processor 232 in the
course of executing the computer-executable instructions. As noted
above, memory 236 may also store the global configuration program
238.
[0104] User device 115 is generally a mobile computer such as a
smartphone or tablet or other "smart" device that may be networked
through the "Internet of Things". However, user devices 115 may
also be a non-mobile computer device, such as desktop computer. As
shown, user device 115 has a processor 212, a communication
interface 214 for data communication with communication interfaces
240 and 254, a display 220 for displaying a GUI, and a memory 216
that may include both volatile and non-volatile elements. As with
server 105, references to acts or functions by mobile device 115
imply that processor 212 is executing computer-executable
instructions (e.g., a software program) stored in memory 216.
[0105] The user device 115 also includes a configuration monitoring
program 218. While the configuration monitoring program 218 is
shown separately from memory 216, it will be understood that the
configuration monitoring program 218 may be stored in memory 216.
The user device 115 may also store a plurality of other software
applications, such as a user application 222, in memory 216. The
device memory 216 may generally store instructions which, when
executed by the device processor 212 causes, the device processor
212 to provide functionality of the various applications stored
thereon.
[0106] In various embodiments, the configuration monitoring program
218 may be a standalone program (or software application) that is
downloaded and installed on the user device 115. In other
embodiments, the configuration monitoring program 218 may be
integrated into a third-party software application, which itself,
is downloaded and installed on the user device 115 (e.g., through
an app store such as the Apple App Store or Google Play).
[0107] An application developer may use a software development kit
(SDK) associated with the present system 200 when writing source
code for the user application 222. The SDK may include programming
language units (e.g., class definitions, library functions, etc.)
that are usable by a software developer to create user interface
elements within the application 222 that may be able to communicate
with the configuration monitoring program 218 and/or logical
elements on the server 105. In some cases, a developer may use the
SDK when developing or modifying the software application 222 to
integrate the configuration monitoring program 218 into a new
application or into an existing application.
[0108] In various embodiments, the application 222 on the computing
device 115 may include executable code corresponding to the
programming language unit for a user interface element.
Configuration parameters for this user interface element may be
received by this executable code. This may allow the user interface
elements to be updated and/or modified without requiring a
re-compilation of the application code. The user interface elements
may correspond to compiled code from UI element classes provided in
the SDK, with this compiled code configuring the computing device
115 to receive the parameters. In some cases, the configuration
monitoring program 218 may be configured to define and/or modify
the configuration of application 222 by defining interface element
parameters corresponding to user interface elements defined in that
application 222.
[0109] As explained in further detail herein, the configuration
monitoring program 218 may be configured to host a local machine
learning algorithm. The local machine learning algorithm may be
configured to use local user data to generate a local prediction
model of the device-specific configuration of a software
application.
[0110] The configuration monitoring program 218 can also be
configured to receive a global prediction model generated by the
global configuration program 238 on server 105. The configuration
monitoring program 218 may then the local and global prediction
models to generate a predicted device-specific configuration for
software application 222 operating on user device 115.
[0111] In some embodiments, the configuration monitoring program
218 may also be configured to receive queries for aggregate data
from the global configuration program 238. In response to these
queries, the configuration monitoring program 218 may collect the
request data on an aggregate basis. The configuration monitoring
program 218 can then transmit the requested aggregated data back to
the global configuration program 238.
[0112] The server 105 may store device interaction data such as
application use data received from the user devices 115. The device
interaction data may be stored in a database on memory 236. The
device interaction data may reflect interactions with a given
application on the user devices 115. The device interaction data
may be stored on the server 105 in an aggregate form. That is, the
device interaction data may not provide any identifiable data
corresponding to a specific user device. This anonymized aggregate
data may be received from the configuration monitoring program 218
in response to a request from the global configuration program
238.
[0113] The device interaction data may be received from each of the
user devices 115 as aggregated device data. In response to a
request from the server 105, the configuration monitoring program
218 on user device 115 may generate aggregate device data based on
the local user data stored in the memory of user device 115. The
aggregate device data may reflect aggregate user interactions with
the user device 115. This may prevent the user interaction data
from providing personally identifiable information regarding a user
of the user device 115.
[0114] In some embodiments, the global configuration program 238
may define and store a global machine learning algorithm. The
global machine learning algorithm may be used to generate a global
configuration prediction model for a given software application,
such as user application 222. This global configuration prediction
model can then be provided to user device 115, for use by
configuration monitoring program 218. The global machine learning
algorithm may use data stored on the server 105 as "training data"
in order to generate the global configuration prediction model.
[0115] In some cases, the global configuration prediction model may
be generated by combining local prediction model "training weights"
received from the one or more user devices 115. The global
configuration prediction model may then be distributed by the
global configuration program 238 to the user devices 115. The user
devices 115 can then use the global configuration prediction model
to generate generalized predictions of a device-specific
configuration of a particular software application. In at least
some embodiments described herein, the global configuration program
238 may also be configured to distribute a default configuration
prediction model which generates default configuration predictions
for a given software application.
[0116] In addition to the components described above, the server
105 and the user device 115 may have various other additional
components not shown in FIG. 2. For example, additional input or
output devices (e.g., keyboard, pointing device, etc.) may be
included beyond those shown in FIG. 2.
[0117] Referring now to FIG. 3A, there is shown a flowchart
illustrating a method 300 for automatically determining a
device-specific configuration for an application operating on a
user device 115, in accordance with an example embodiment. The
method 300 may be carried out by various components of the systems
100 and 220, such as the configuration monitoring program 218
operating on a user device 115.
[0118] Optionally, at 302, the configuration monitoring program 218
may receive, from the server 105, a prediction model format for a
local configuration prediction model. The prediction model format
may identify input features and target variables for a local
configuration prediction model to be generated by the configuration
monitoring program 218.
[0119] For example, the prediction model format may be a dataset
schema that specifies the input variables (i.e. input data
features) and output variables for a local machine learning
algorithm defined in the configuration monitoring program 218. The
dataset schema may specify that the output variable(s) is/are a
configuration for a given application operating on the user device
115. The dataset schema may further specify the input features as
specific variables that may or may not be determinative of that
configuration (e.g., weather, user's activity, user's location,
etc.). In general, the prediction model can map the input data to
the output variable(s). This mapping can determined through a
machine learning process where the mapping is learned (at least
initially) using historical user data and/or live using real-time
data and feedback.
[0120] In some embodiments, an administrator (e.g. a server
administrator and/or an application developer) may independently
define the dataset schema. For example, the administrator user may
use the display 234 on the server 105 to define a model with input
features and output variables. The administrator may also specify
the dataset schema using an administrator computer connected to the
server 105. This model may then be distributed by the server 105 to
the user devices 115 in the form of a dataset schema. In other
embodiments, the dataset schema may be automatically generated by
the global configuration program 238 and distributed to the user
devices 115. In still other embodiments, the configuration
monitoring program 218 may be pre-configured with a dataset
schema.
[0121] At 304, the configuration monitoring program 218 can monitor
local device data on the user device 216. The local device data may
be stored in memory 216. The configuration monitoring program 218
can be configured to monitor the local device data stored in memory
216 on a continual or ongoing basis. The configuration monitoring
program 218 can use the monitored local data to train the local
machine learning algorithm hosted by the configuration monitoring
program 218.
[0122] The local data stored in memory 216 can include user
interaction data generated based on user interactions with the user
device 115. For example, the user interaction data may include
user-specified configuration data generated when a user manually
sets or modifies the configuration of an application, such as user
application 222, operating on the user device 115.
[0123] The local data stored in memory 216 can also include device
attribute data. The device attribute data may include data
indicating the status and settings of the user device 115, such as
the device type, active operating system (which may include
operating system version data), battery charge status, connectivity
status (e.g. Wi-Fi connectivity status, Telecommunications network
connectivity status, Bluetooth connectivity status, wired
connectivity status), and other settings relating to the operation
of the user device 115 generally.
[0124] The local data stored in memory 216 can also include
application attribute data. The application attribute data may
include data indicating the status and settings of individual
applications such as user application 222 operating on user device
115, such as the application identifier, application version,
application permissions, and other settings relating to the
operation of individual applications on the user device 115.
[0125] The local data stored in memory 216 can also include device
environment data. The device environment data may include data
relating to the environment in which the device 115 is present,
such as the location, date, time, weather and so forth.
[0126] As should be apparent, the local data stored in memory 216
can include data that may be considered private to a user of the
device 115. A user may wish to protect this data, and prevent
unauthorized access to that data. This private data can include
both personally identifiable information (PII) which identifies
users of the user device 115, as well as information about discrete
user events (e.g., item purchase history, click event history,
time-stamped events, etc.). The local data stored in memory 216 may
also include data relating to the devices or applications (e.g.
application identifiers, versions and so forth) that may be
unrelated to the user's personally identifiable information.
[0127] In some cases, the configuration monitoring program 218
operating on the user device 115 can be configured to monitor any
and all of the local data stored in memory 218. In some cases, the
data to be monitored by configuration monitoring program 218 may be
limited by the permissions granted to the configuration monitoring
program 218 (or the application with which it is integrated).
Nonetheless, the configuration monitoring program 218 can be
configured to monitor and evaluate private user data stored locally
on user device 115.
[0128] In some embodiments, the scope of local user data monitored
by configuration monitoring program 218 may be limited to specific
classes of data defined by the dataset schema received at 302. The
configuration monitoring program 218 may thus monitor only data
that is relevant to the input and output variables specified by the
dataset schema that is being used to generate a prediction model.
This may ensure that the local prediction model is generated using
only data likely to be relevant to predicting the device-specific
configuration.
[0129] At 306, the configuration monitoring program 218 can use the
monitored local user data to generate a device-specific
configuration prediction model (also referred to herein as a "local
prediction model", or a "local model").
[0130] The local prediction model can be generated by iteratively
training a local machine learning algorithm over the local user
data monitored at 304. In some embodiments, the local machine
learning algorithm can be iteratively trained over only the classes
of data specified by the dataset schema in 302 (i.e. only data
which relates to the input and output variables specified by the
dataset schema).
[0131] Once the training is complete, the local machine learning
algorithm can generate a local prediction model. The local
prediction model can be used to predict a device-specific
configuration of a given software application. In particular, the
local prediction model may be adapted to use local user data to
predict the configuration that would be desired by a user of that
particular user device 115.
[0132] In some embodiments, once the local prediction model is
generated, the model can be stored exclusively on the user device
115. The local prediction model may not be transmitted or
distributed to other third-party devices (i.e., server 105 or other
user device 115). This may ensure that private and sensitive local
user data, which is used to generate the local prediction model, is
not compromised.
[0133] At 308, the configuration monitoring program 218 can receive
a global configuration prediction model from the server 105. The
global configuration prediction model may also be referred to
herein as a "global prediction model", or a "global model". The
global prediction model can be generated by the global
configuration program 238 on server 105.
[0134] In some embodiments, the global configuration program 238
may generate the global prediction model using a global machine
learning algorithm. In contrast to the local machine learning
algorithm, the global machine learning algorithm is not trained
using local user data. Rather, the global machine learning
algorithm can be trained using data from a plurality of user
devices 115. The data from the user devices 115 may be combined
into an aggregate data source that may be stored, for example, on
memory 236 of server 105. An example process for generating
aggregate device data is explained further detail herein with
reference to FIG. 5.
[0135] In some cases, the aggregate data may be defined by the
configuration monitoring program 218 on the user devices 115 to
exclude sensitive or personally identifiably information (PII)
specific to users of user device 115. For example, the
configuration monitoring program 218 may generate aggregate device
data on an anonymous based by excluding discrete event data
generated through monitored device activity (i.e. data associated
with individual user interactions and related settings and/or
environmental data).
[0136] In other cases, the aggregate data can be defined to include
only cumulative statistical data (e.g. "total counts", or
"probabilities") regarding device interactions. For example, the
data may restricted to the number of user interactions at a given
time (e.g. evening) over the course of a longer time period.
Additionally or alternatively user data may be restricted to
include only the probabilities of certain events occurring (e.g.
the likelihood of a user accessing a given interface within the
application when the weather is sunny).
[0137] The input features for the global machine learning algorithm
may differ from the input features for the local machine learning
algorithm. In particular, global machine learning algorithm may be
modified to accommodate the more generalized nature of data used by
the global machine learning algorithm.
[0138] In some cases, the local model may be defined to use
variables not available to the global model, for instance variables
based on data that is not available to the global model e.g. due to
privacy concerns. In some cases, the global mode and local model
may be defined using the same variables while the values of some
variables differ between the models due to aggregation and
collection from all devices.
[0139] In other embodiments, the global prediction model may not be
generated by training a global machine learning algorithm. For
example, in at least some embodiments, the global configuration
program 238 can receive, from the configuration monitoring program
218 on a plurality of user devices 115, "training weights"
generated from partially trained local prediction models. The
"training weights" can be generated after a pre-defined number of
training iterations of the local machine learning algorithm on the
local user data. In some cases, the "training weights" can be
generated once the local machine learning algorithm satisfies one
or more training criteria, such as various machine leaning metrics
(e.g. accuracy, precision, recall etc.). The "training weights",
similar to "model weights", may provide a preliminary prediction of
the extent to which the input variables, of the local machine
learning algorithm, are connected to the target output variables
(i.e., a specific application configuration). The global
configuration program 238 can receive these "training weights" and
merge the weights from the plurality of user devices 115 to
generate the global prediction model. As the training weights do
not include personal or sensitive data, user data privacy can still
be preserved when the training weights are provided to server
105.
[0140] Where the global configuration program 238 is generated by
combining "training weights", the combining of "training weights"
can be performed using any one of a number of suitable combination
algorithms that yield high performance. For example, where the
"training weights" are generated by local machine learning
algorithms that use classification functions, the "training
weights" can be combined using a modified softmax function. In
other cases, where the "training weights" are generated by local
machine learning algorithms that use regression functions, the
"training weights" can be combined via a weighted combination.
[0141] In still other embodiments, the global prediction model may
be defined to include a set of baseline or default device-specific
configurations. The default or baseline configurations can be
specified, for example, by an administrator, such as the developer
of the application being monitored.
[0142] At 310, the configuration monitoring program 218 can monitor
user device activity. The configuration monitoring program 218 may
monitor device activity data to determine whether configuration
update criteria for a given software application have been
satisfied.
[0143] Configuration update criteria may be used to determine if
and when a change in the configuration of an application is
required. The configuration update criteria may be related to the
input variables for the local and global prediction models. For
instance, the update criteria for a given software application may
include a change in the user's activity, a change in the weather,
or a change in the user's location. To this end, the specifics of
the configuration update criteria may vary depending on the
application, and the corresponding design of the local and global
models (i.e. the defined input and output variables for these
models).
[0144] In at least some embodiments, the configuration monitoring
program 218 may monitor whether the configuration update criteria
is satisfied by monitoring user device activity, such as
user-interactions with the user device 115 (i.e., user inputs), as
well as changes to the environment of the user device or attributes
of the user device (i.e., changes in the weather, or time of
day).
[0145] At 312, the configuration monitoring program 218 can
generate a predicted device-specific configuration for a software
application operating on the user device 115. The configuration
monitoring program 218 may generate the predicted device-specific
configuration at a defined time or interval. This may allow the
configuration monitoring program to continually evaluate the
accuracy of the predicted device-specific configuration.
[0146] The configuration monitoring program 218 may also generate a
predicted device-specific configuration in response to device
activity being monitored at 308. For example, the configuration
monitoring program 218 may determine that a configuration update
criteria has been satisfied by the monitored device data. The
configuration monitoring program 218 can then generate a predicted
device-specific configuration in response to the criteria being
satisfied.
[0147] The predicted device-specific configuration indicates a
prediction or estimate of the expected configuration for the
particular application on a given user device 115. The predicted
device-specific configuration may reflect the configuration that is
expected to be selected by a user of the application. In some
cases, however, the predicted device-specific configuration may
reflect a configuration that is expected to provide the user of
device 115 with the greatest user engagement with the application.
Examples of user engagement criteria that may be considered include
click through rate on an object, a message open rate, a purchase
amount, a purchase rate etc.
[0148] For example, the configuration monitoring program 218 may
determine that the device 115 is being used on October 31.sup.st.
The configuration monitoring program 218 may then determine that
the predicted device-specific configuration for a software
application operating on the user device may include a modified
interface background that includes spider webs and witches
reflective of Halloween.
[0149] The predicted device-specific configuration can be generated
by the configuration monitoring program 218 using the predictions
from the local prediction model and the global prediction model.
The manner in which the local prediction and global predictions are
used, and may be weighted and/or combined, may vary in different
embodiments (see the various examples shown in FIGS. 4A-4D and
described further herein). However, in general the configuration
monitoring program 218 always has available both the local
prediction model and the global prediction model to provide a local
configuration prediction and a global configuration prediction
respectively for the software application being monitored.
[0150] At 314, the configuration monitoring program 218 can update
the configuration of the application using the predicted
device-specific configuration determined at 312.
[0151] In some cases, updating the configuration of the application
may involve automatically implementing the predicted
device-specific configuration in the software application. In some
cases, the predicted device-specific configuration may be limited
to defined environmental or other conditions (e.g. a defined time
period). The predicted device-specific configuration may then be
implemented only in accordance with those limited deployment
criteria. For example, the Halloween background may be
automatically deployed in the software application, but only
beginning at 6 am on October 31.sup.st and ending at 5 am on
November 1.sup.st.
[0152] In cases where the configuration monitoring program 218
automatically implements a predictive device-specific
configuration, a user may be able to manually revert to the
previous configuration or otherwise change the configuration of the
application.
[0153] In other cases, the configuration monitoring program may
prompt a user to accept or decline the predicted device-specific
configuration. The user may then accept the implemented
configuration, decline the modification, or may select a new or
different configuration for the application.
[0154] In still other cases, the configuration monitoring program
218 may merely record the predicted device-specific configuration.
The configuration monitoring program 218 may then use this
predicted device-specific configuration to evaluate the accuracy of
the combined prediction model being used based on further
monitoring of device activity.
[0155] Referring now to FIG. 3B, shown therein is a flowchart
illustrating a method 320 for assessing user feedback input. The
method 320 may be carried out by various components of the systems
100 and 220, such as the configuration monitoring program 218
operating on a user device 115.
[0156] At 322, the configuration monitoring program 218 can
identify the configuration of the application currently operating
on the user device 115. Additionally, the configuration monitoring
program 218 may also identify the predicted device-specific
configuration generated by the combined prediction model used by
configuration monitoring program 218.
[0157] At 324, the configuration monitoring program 218 can monitor
user feedback input corresponding to the configuration of the
application. This may involve monitoring whether the user selects a
suggested configuration, or otherwise selects a different
configuration for the application. User feedback input may also
include various user engagement metrics. The user feedback input
can then be stored on the memory 216 of the user device 115 (i.e.
included as part of the local device data).
[0158] At 326, the configuration monitoring program 218 can update
(or re-train) the device-specific configuration prediction model
(i.e., local prediction model) using the user feedback input
received at 324. The local prediction model can be re-trained using
the updated local user data that now includes the user feedback
input.
[0159] The configuration monitoring program 218 may be configured
to monitor historical device data that includes user interaction
data, application configuration data and predicted device-specific
configuration data. By monitoring and evaluating historical user
interaction data and application configuration data over a period
of time, the configuration monitoring program 218 may identify
changes in user interactions or user engagement with the
application. The configuration monitoring program 218 may be able
to correlate these changes with corresponding changes in the
application configuration data.
[0160] The configuration monitoring program 218 may be configured
to update the local prediction model on an ongoing and continual
basis. In some cases, the configuration monitoring program may
update the local prediction model at regular intervals based on the
device data monitored during that interval.
[0161] In some cases, the configuration monitoring program 218 may
be configured to update the local prediction model based on
accuracy metrics collected for the local prediction model. For
example, the configuration monitoring program 218 may determine
that the accuracy of the local prediction model has decreased more
than a defined re-training threshold or to an accuracy below a
training threshold. The configuration monitoring program 218 may
then re-train the local prediction model. The configuration
monitoring program 218 may continue to re-train the local
prediction model until a desired prediction accuracy is
reached.
[0162] Referring now to FIG. 4A, shown therein is a flowchart
illustrating a method 400 for generating a predicted
device-specific configuration, according to some embodiments.
Method 400 may be carried out by various components of the systems
100 and 220, such as the configuration monitoring program 218
operating on a user device 115. Method 400 is an example process
for generating a predicted device-specific configuration that may
be used at step 312 of method 300.
[0163] At 402, the configuration monitoring program 218 can use the
local prediction model to generate a local prediction for the
device-specific configuration. The local prediction may provide a
predicted configuration that may be more tailored to the user
device 115, as it is generated using a model developed using only
local data.
[0164] At 404, the configuration monitoring program 218 can use the
global prediction model received from the server to generate a
global prediction for the device-specific configuration. The global
prediction may provide a more generalized prediction of the
application configuration on the device. The global prediction may
supplement the local prediction while the local prediction model is
insufficiently trained (e.g. there is insufficient local data
available to generate an accurate local prediction model or the
local data is inconsistent).
[0165] In some embodiments, where the global prediction model
includes a set of default variations, the global prediction model
may otherwise generate a default or base-line prediction for the
configuration of the application.
[0166] At 406, the predicted device-specific configuration can be
generated using the local and global predictions of the
device-specific configuration. As discussed herein with reference
to FIGS. 4B to 4D, various methods are available for using or
combining the local and global predictions to generate the
predicted device-specific configuration. In some cases, the
device-specific predicted configuration may be generated as one of
the local prediction and the global prediction while the other of
the local prediction and the global prediction is given no weight
in the final device-specific prediction configuration. In other
cases, the device-specific predicted configuration may be generated
using a combined prediction.
[0167] Referring now to FIG. 4B, there is shown a flowchart
illustrating a method 410 for generating the predicted
device-specific configuration using the local and global
predictions in accordance with an embodiment. Method 410 may be
carried out by various components of the systems 100 and 220, such
as the configuration monitoring program 218 operating on a user
device 115. Method 410 is an example process for generating a
predicted device-specific configuration based on accuracy metrics
of the local prediction model that may be used at step 406 of
method 400.
[0168] At 412, the configuration monitoring program 218 can
determine a local prediction metric. The local prediction metric
may be a value, or set of values, that express or reflect the
historical accuracy of the local prediction model. The accuracy of
the local prediction can be determined by comparing the local
prediction with monitored device data such as the user feedback
input received at step 324. The local prediction metric may then be
compared to an accuracy threshold to determine whether the local
prediction model has been sufficiently trained.
[0169] In some embodiments, the local prediction metric can be
determined on a rolling average basis. For example, the local
prediction metric can be determined based on the performance
accuracy of the local prediction model over the last three
iterations of the method 300. In other cases, the local prediction
metric may be determined using more complex metric algorithms.
[0170] In general, the local prediction metric can be generated
using any one of a number of suitable tests for determining machine
learning algorithm accuracy. For example, where the local machine
learning algorithm is a classification algorithm, the local
prediction metric can be generated (or determined) using, for
example, precision recall, an F1 score, a true positive test, a
false positive test, or a receiver operating characteristic under
the curve test. In other cases, where the local machine learning
algorithm is a regressor algorithm, the local prediction metric may
be determined using, for example, a mean absolute error test, a
measured squared error test, or an R2 score.
[0171] In various embodiments, irrespective of how the local
prediction metric is determined, the local prediction metric may be
expressed as a scaled value. For example, a higher prediction
metric may indicate a higher performance accuracy for the local
prediction model, and a lower prediction metric may indicate a
lower performance accuracy for the local prediction model.
[0172] At 414, the configuration monitoring program 218 can
determine if the local prediction metric is above a predetermined
performance threshold (i.e., whether the local prediction model is
sufficiently accurate).
[0173] In some cases, the performance threshold may be a value
generated and transmitted by the global configuration program 238
to the configuration monitoring program 218. In other cases, the
configuration monitoring program 218 may itself be configured with
a value for the performance threshold. The value of the performance
threshold can also vary depending on the particular software
application at issue, or the type of configuration being determined
for that application. For example, the developer of a software
application may adjust the accuracy threshold within the
configuration monitoring program 218 deployed with that
application.
[0174] At 416, if the value of the local prediction metric is
determined to be below the performance threshold, then the
configuration monitoring program 218 can determine that the local
prediction model is insufficiently accurate (i.e., insufficiently
trained). In this case, the configuration monitoring program 218
can generate the predicted device-specific configuration using only
the global prediction of the device-specific configuration (i.e.
the output from the global prediction model). The local prediction
may still be recorded by the configuration monitoring program 218
and stored on memory 216. The recorded local prediction may then
assessed against the user-feedback input to re-train the local
prediction model.
[0175] Conversely, if at 414, the local prediction metric is
determined to be above the predetermined performance threshold,
then at 418, the configuration monitoring program 218 can determine
that the local prediction model is sufficiently accurate (i.e.
sufficiently trained). In this case, the configuration monitoring
program 218 can generate the predicted device-specific
configuration using only the local prediction.
[0176] In general, in method 410 the global (or default) prediction
may operate as a back-up (or an alternate option) to the local
prediction unless and until the local prediction model becomes
sufficiently accurate. In some cases, while the local prediction
metric may exceed the performance threshold in some iterations of
method 410, the configuration monitoring program 218 may revert to
using the global (or default) prediction where the local prediction
accuracy metric decreases to below the desired performance
threshold.
[0177] Referring now to FIG. 4C, there is shown a flowchart
illustrating a method 420 for generating the predicted
device-specific configuration using the local and global
predictions in accordance with an embodiment. Method 420 may be
carried out by various components of the systems 100 and 220, such
as the configuration monitoring program 218 operating on a user
device 115. Method 420 is an example process for generating a
predicted device-specific configuration by combining weighted
predictions from the local and global models that may be used at
step 406 of method 400.
[0178] At 422, the configuration monitoring program 218 can again
determine a local prediction metric. The local prediction metric
may be determined in the same manner as described above at 412.
[0179] At 424, the configuration monitoring program 218 can assign
a local bias weight to the local prediction based on the local
prediction metric. The local bias weight may be used to scale or
adjust the importance of the local prediction. For example, a
higher local prediction metric (i.e., a higher local prediction
model accuracy) may result in a higher bias weight being assigned
to the local prediction, while a lower local prediction metric
(i.e., a lower local prediction model accuracy) may result in a
lower bias weight being assigned to the local prediction.
[0180] At 426, the configuration monitoring program 218 can assign
a second bias weight to the global prediction of the
device-specific configuration. In some cases, the second bias
weight may be determined a function of the first bias weight. For
example, the first and second bias weights may be collectively
defined to provide percentage weightings that total to 100%.
[0181] At 428, the configuration monitoring program 218 can
generate the predicted device-specific configuration by combining
the local prediction and the global prediction according to their
respective bias weightings (e.g. First Bias Weight*Local
Prediction+Second Bias Weight*Global Prediction). Accordingly, the
predicted device-specific configuration is generated from a mixed
weighted combination of the local and global predictions. In this
manner, inaccuracy in the local prediction model can be mitigated
by the combining of the global and local predictions in proportion
to their bias weightings.
[0182] Referring now to FIG. 4D, there is shown a flowchart
illustrating a method 430 for generating the predicted
device-specific configuration using the local and global
predictions in accordance with an embodiment. Method 430 may be
carried out by various components of the systems 100 and 220, such
as the configuration monitoring program 218 operating on a user
device 115. Method 430 is an example process for generating a
predicted device-specific configuration using local prediction
metrics and weighted predictions that may be used at step 406 of
method 400.
[0183] At 432, 434 and 436, the configuration monitoring program
again determines the local prediction metric, and proceeds to
assign a first bias weight to the local prediction based on the
local prediction metric, and a second bias weight to the global
prediction in the same manner as described above in steps 422, 424
and 426.
[0184] At 438, the configuration monitoring program 218 can
determine if the local prediction metric is above the predetermined
performance threshold. This determination may occur in the same
manner as described above at step 414.
[0185] If at 438 it is determined that the local prediction metric
is below the predetermined performance threshold, the configuration
monitoring program 218 can determine that the local prediction is
insufficiently trained. Accordingly, at 440, the configuration
monitoring program 218 can generate the predicted device-specific
configuration using only the global (or default) prediction. In
this case, the configuration monitoring program 218 may record and
store the local prediction in order to re-train the local
prediction model. Thus, when the local prediction metric is below
the predetermined threshold, method 430 may result in the same
predicted device-specific configuration as method 410.
[0186] If at 438, it is determined that the local prediction metric
is above (or greater) than the predetermined performance threshold,
then at 442, the configuration monitoring program 218 can generate
the predicted device-specific configuration by combining the local
and global predictions according to the first and second bias
weights, respectively. Thus, when the local prediction metric is
above the predetermined threshold, method 430 can result in the
same predicted device-specific configuration as method 420.
[0187] Accordingly, in method 430, the mixed combination of the
local and global predictions is generated only when the local
prediction model is determined to be sufficiently accurate. This
may ensure that the result of the mixed weighting is not
compromised by an insufficiently trained local prediction
model.
[0188] In general, in method 430 the global (or default) prediction
may operate as a back-up (or an alternate option) to the blended
weighted local and global prediction unless and until the local
prediction model becomes sufficiently accurate. In some cases,
while the local prediction metric may exceed the performance
threshold in some iterations of method 430, the configuration
monitoring program 218 may revert to using the global (or default)
prediction where the local prediction accuracy metric decreases to
below the desired performance threshold.
[0189] Further, it will also be appreciated that the conditional
logic in methods 410 and 430 (i.e. 414 and 438 which depend on the
local prediction metric) may be, in some cases, over-ridden by a
third-party administrator (i.e., an application developer) to force
the method to adopt a desired outcome.
[0190] Referring now to FIG. 5, shown therein is a flowchart
illustrating a method 500 for acquiring data from one or more user
devices 115. Method 500 is an example process for acquiring device
data that can be used to generate an anonymized aggregate data
source. The method 500 may be carried out by various components of
the systems 100 and 220, such as the global configuration program
238 operating on the server 105.
[0191] At 502, the global configuration program 238 can generate a
data request query. The server 105 can then transmit the data
request query to one or more user devices 115 requesting local
device data. The data request query generated by the global
configuration program 238 can be modified (or otherwise filtered or
restricted) to request only anonymized aggregate local device data.
In some cases, the queries may be restricted to request only
cumulative statistics in respect of an application operating on the
user device 115 (e.g., total counts, or total probabilities). For
example, the request may be in respect of the probability that a
user selects a particular configuration based on previously
available data. In other cases, the queries may be restricted to
request only training weights generated by partially trained local
machine learning algorithms operating on user devices 115. In this
manner, the queries can exclude requests for data that include
sensitive or personally identifiable information (PII) of the user
of the user device 115 (i.e., time-stamped discrete or singular
events).
[0192] In various embodiments, the queries may generated by the
global configuration program 238 using structured query language
(SQL), or any other suitable query generating language. The queries
may be transmitted to the user devices 115 as silent pushes.
[0193] The configuration monitoring program 218 may, in response to
the query, scan the local device data (i.e., stored on memory 216)
in order to gather the queried data. In some cases, the
configuration monitoring program 218 can collect or group the local
device data in order to generate cumulative data to be transmitted
to the server 105. The configuration monitoring program 218 may
also evaluate the stored device data to determine aggregate device
data, such as the probability of certain events occurring on the
device 115. The configuration monitoring program 218 may then
transmit the collected data to the server 105.
[0194] At 504, the global configuration program 238 receives the
requested aggregate data from one or more user devices 115. The
aggregate data may then be stored in memory 236.
[0195] In some cases, the global configuration program 238 may
evaluate the received aggregate data to ensure that only anonymized
aggregate data was received. The global configuration program 238
may discard, and dispose of, any non-anonymized data received to
reduce the possibility of that data being comprised.
[0196] At 506, the global configuration monitoring program 238 can
aggregate or combine the received data from the user devices 115 to
generate an aggregate data source. The aggregate data source may
then be used to generate a global configuration model of the
device-specific configuration for a particular application.
[0197] In some cases, the aggregate data source may be specific to
a particular application. That is, data relate to an individual
application may be stored in a separate aggregate data source for
that application. In other cases, the aggregate data source may
include data related to multiple different software
application.
[0198] In some cases, the aggregate data source may group
application data based on an application type. For example,
aggregate data related to messaging applications may be stored in a
first aggregate data source while aggregate data related to a media
playback application may be stored in a second aggregate data
source. This may allow aggregate data from different applications
of the same application type to be used to define a global
prediction model for individual applications of that application
type.
[0199] As used herein, the wording "and/or" is intended to
represent an inclusive--or. That is, "X and/or Y" is intended to
mean X or Y or both, for example. As a further example, "X, Y,
and/or Z" is intended to mean X or Y or Z or any combination
thereof.
[0200] While the above description describes features of example
embodiments, it will be appreciated that some features and/or
functions of the described embodiments are susceptible to
modification without departing from the spirit and principles of
operation of the described embodiments. For example, the various
characteristics which are described by means of the represented
embodiments or examples may be selectively combined with each
other. Accordingly, what has been described above is intended to be
illustrative of the claimed concept and non-limiting. It will be
understood by persons skilled in the art that other variants and
modifications may be made without departing from the scope of the
invention as defined in the claims appended hereto. The scope of
the claims should not be limited by the preferred embodiments and
examples, but should be given the broadest interpretation
consistent with the description as a whole.
* * * * *