U.S. patent application number 17/028166 was filed with the patent office on 2021-03-11 for data driven methods and systems for what if analysis.
This patent application is currently assigned to Oracle International Corporation. The applicant listed for this patent is Oracle International Corporation. Invention is credited to Amit Ganesh, Dustin Garvey, Sumathi Gopalakrishnan, Sampanna Shahaji Salunke, Uri Shaft.
Application Number | 20210073680 17/028166 |
Document ID | / |
Family ID | 1000005227369 |
Filed Date | 2021-03-11 |
![](/patent/app/20210073680/US20210073680A1-20210311-D00000.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00001.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00002.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00003.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00004.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00005.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00006.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00007.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00008.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00009.png)
![](/patent/app/20210073680/US20210073680A1-20210311-D00010.png)
View All Diagrams
United States Patent
Application |
20210073680 |
Kind Code |
A1 |
Garvey; Dustin ; et
al. |
March 11, 2021 |
DATA DRIVEN METHODS AND SYSTEMS FOR WHAT IF ANALYSIS
Abstract
Techniques are described for applying what-f analytics to
simulate performance of computing resources in cloud and other
computing environments. In one or more embodiments, a plurality of
time-series datasets are received including time-series datasets
representing a plurality of demands on a resource and datasets
representing performance metrics for a resource. Based on the
datasets at least one demand propagation model and at least one
resource prediction model are trained. Responsive to receiving an
adjustment to a first set of one or more values associated with a
first demand: (a) a second adjustment is generated for a second set
of one or more values associated with a second demand; and (b) a
third adjustment is generated for a third set of one or more values
that is associated with the resource performance metric.
Inventors: |
Garvey; Dustin; (Oakland,
CA) ; Salunke; Sampanna Shahaji; (Dublin, CA)
; Shaft; Uri; (Fremont, CA) ; Ganesh; Amit;
(San Jose, CA) ; Gopalakrishnan; Sumathi; (Redwood
Shores, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Oracle International Corporation |
Redwood Shores |
CA |
US |
|
|
Assignee: |
Oracle International
Corporation
Redwood Shores
CA
|
Family ID: |
1000005227369 |
Appl. No.: |
17/028166 |
Filed: |
September 22, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15612999 |
Jun 2, 2017 |
10817803 |
|
|
17028166 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06N 20/00 20190101;
G06F 9/46 20130101 |
International
Class: |
G06N 20/00 20060101
G06N020/00; G06F 9/46 20060101 G06F009/46 |
Claims
1. A method comprising: receiving a request to perform a what-if
simulation for one or more performance metrics of a system based on
a first change to a first demand on one or more system resources;
responsive to receiving the request to perform the what-if
simulation, generating a first prediction of at least a second
change to a second demand on the one or more system resources; and
based at least on the first change to the first demand and the
second change to the second demand, generating a second prediction
of a third change to the one or more performance metrics of the
system.
2. The method of claim 1, wherein the first change is a change to
at least one historical value that corresponds to a measurement of
the first demand on the one or more system resources.
3. The method of claim 1, wherein generating the second prediction
of the third change to the one or more performance metrics of the
system comprises predicting a change to at least one historical
value that corresponds to a measurement of the one or more
performance metrics of the system.
4. The method of claim 1, wherein the second prediction of the
third change to the one or more performance metrics of the system
comprises a change to at least one future value that corresponds to
a forecast of the one or more performance metrics of the
system.
5. The method of claim 1, wherein the second prediction of the
third change to the one or more performance metrics of the system
comprises a change to an uncertainty interval in a forecast of the
one or more performance metrics of the system.
6. The method of claim 1, further comprising presenting a
historical or forecast simulation based on at least the third
change to the one or more performance metrics of the system.
7. The method of claim 1, wherein at least one of the first
prediction or the second prediction is generated based on one or
more seasonal patterns.
8. The method of claim 1, wherein the request is received from a
tenant of a cloud service; and wherein responsive to the request
the cloud service presents, to the tenant, or stores, at a location
accessible to the tenant, a result of the what-if simulation.
9. The method of claim 1, further comprising configuring one or
more system resources based on a result of the what-if
simulation.
10. One or more non-transitory computer-readable media storing
instructions that, when executed by one or more hardware
processors, cause: receiving a request to perform a what-if
simulation for one or more performance metrics of a system based on
a first change to a first demand on one or more system resources;
responsive to receiving the request to perform the what-if
simulation, generating a first prediction of at least a second
change to a second demand on the one or more system resources; and
based at least on the first change to the first demand and the
second change to the second demand, generating a second prediction
of a third change to the one or more performance metrics of the
system.
11. The media of claim 10, wherein the first change is a change to
at least one historical value that corresponds to a measurement of
the first demand on the one or more system resources.
12. The media of claim 10, wherein generating the second prediction
of the third change to the one or more performance metrics of the
system comprises predicting a change to at least one historical
value that corresponds to a measurement of the one or more
performance metrics of the system.
13. The media of claim 10, wherein the second prediction of the
third change to the one or more performance metrics of the system
comprises a change to at least one future value that corresponds to
a forecast of the one or more performance metrics of the
system.
14. The media of claim 10, wherein the second prediction of the
third change to the one or more performance metrics of the system
comprises a change to an uncertainty interval in a forecast of the
one or more performance metrics of the system.
15. The media of claim 10, wherein the instructions further cause
presenting a historical or forecast simulation based on at least
the third change to the one or more performance metrics of the
system.
16. The media of claim 10, wherein at least one of the first
prediction or the second prediction is generated based on one or
more seasonal patterns.
17. The media of claim 10, wherein the request is received from a
tenant of a cloud service; and wherein responsive to the request
the cloud service presents, to the tenant, or stores, at a location
accessible to the tenant, a result of the what-if simulation.
18. The media of claim 10, further comprising configuring one or
more system resources based on a result of the what-if
simulation.
19. A computing system comprising: one or more hardware processors;
one or more non-transitory computer-readable media storing
instructions that, when executed by the one or more hardware
processors, cause: receiving a request to perform a what-if
simulation for one or more performance metrics of a system based on
a first change to a first demand on one or more system resources;
responsive to receiving the request to perform the what-if
simulation, generating a first prediction of at least a second
change to a second demand on the one or more system resources; and
based at least on the first change to the first demand and the
second change to the second demand, generating a second prediction
of a third change to the one or more performance metrics of the
system.
20. The computing system of claim 19, wherein the first change is a
change to at least one historical value that corresponds to a
measurement of the first demand on the one or more system
resources.
Description
INCORPORATION BY REFERENCE; DISCLAIMER
[0001] The following application is hereby incorporated by
reference: U.S. application Ser. No. 15/612,999 filed on Jun. 2,
2017. The Applicant hereby rescinds any disclaimer of claim scope
in the parent application(s) or the prosecution history thereof and
advises the USPTO that the claims in this application may be
broader than any claim in the parent application(s).
RELATED APPLICATIONS
[0002] This application is related to U.S. application Ser. No.
15/266,971, entitled "SEASONAL AWARE METHOD FOR FORECASTING AND
CAPACITY PLANNING"; U.S. application Ser. No. 15/445,763, entitled
"METHOD FOR CREATING PERIOD PROFILE FOR TIME-SERIES DATA WITH
RECURRENT PATTERNS"; U.S. application Ser. No. 15/266,979, entitled
"SYSTEMS AND METHODS FOR DETECTING AND ACCOMMODATING STATE CHANGES
IN MODELLING"; U.S. application Ser. No. 15/140,358, entitled
"SCALABLE TRI-POINT ARBITRATION AND CLUSTERING"; U.S. application
Ser. No. 15/057,065, entitled "SYSTEM FOR DETECTING AND
CHARACTERIZING SEASONS"; U.S. application Ser. No. 15/057,060,
entitled "SUPERVISED METHOD FOR CLASSIFYING SEASONAL PATTERNS";
U.S. application Ser. No. 15/057,062, entitled "UNSUPERVISED METHOD
FOR CLASSIFYING SEASONAL PATTERNS"; and U.S. Provisional Patent
Appl. No. 62/370,880, entitled "UNSUPERVISED METHOD FOR BASELINING
AND ANOMALY DETECTION IN TIME-SERIES DATA FOR ENTERPRISE SYSTEMS",
the entire contents for each of which are incorporated by reference
herein as if set forth in their entirety.
TECHNICAL FIELD
[0003] The present disclosure relates to analytical models that
process time-series data. In particular, the present disclosure
relates to training and evaluating what-if models to analyze
performance of computing resources.
BACKGROUND
[0004] In large-scale computing environments, such as datacenters
and cloud computing platforms, many different inputs may affect the
performance of hardware and software resources. For example, the
performance of a database server may be impacted by the frequency
of database transactions, user calls, and/or query executions,
among other factors. As the number and frequency of inputs into a
resource increase, so does the likelihood that the resource's
performance will degrade.
[0005] System administrators are generally responsible for ensuring
that resources within the computing environment are meeting
performance expectations. As the demand on resources increase,
system administrators may deploy additional resources and/or
balance the load across existing resources to maintain the quality
of service. As the demands decrease, resources may be brought
offline to mitigate waste and improve efficiency. If administrators
fail to adequately understand or anticipate demands, system
resources may become overloaded, leading to performance
degradation.
[0006] The approaches described in this section are approaches that
could be pursued, but not necessarily approaches that have been
previously conceived or pursued. Therefore, unless otherwise
indicated, it should not be assumed that any of the approaches
described in this section qualify as prior art merely by virtue of
their inclusion in this section.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The embodiments are illustrated by way of example and not by
way of limitation in the figures of the accompanying drawings. It
should be noted that references to "an" or "one" embodiment in this
disclosure are not necessarily to the same embodiment, and they
mean at least one. In the drawings:
[0008] FIG. 1A illustrates an example system comprising a set of
what-if analytic services that are driven by time-series data
captured from one or more host devices, in accordance with one or
more embodiments;
[0009] FIG. 1B illustrates an example dataflow for simulating
scenarios and generating scenario outputs, in accordance with one
or more embodiments;
[0010] FIG. 2 illustrates an example set of operations for training
a set of correlation prediction models, in accordance with one or
more embodiments;
[0011] FIG. 3 illustrates an example set of demand propagation
models, in accordance with one or more embodiments;
[0012] FIG. 4 illustrates an example set of correlation prediction
models, in accordance with one or more embodiments;
[0013] FIG. 5 illustrates an example set of operations for
performing seasonal aware training of simulation models, in
accordance with one or more embodiments;
[0014] FIG. 6 illustrates an example set of seasonal-aware
correlation prediction models, in accordance with one or more
embodiments;
[0015] FIG. 7 illustrates an example set of operations for
simulating a historical scenario, in accordance with one or more
embodiments;
[0016] FIG. 8A illustrates an example presentation of propagating
adjustments in one demand to another demand, in accordance with one
or more embodiments;
[0017] FIG. 8B illustrates an example presentation of adjustments
generated for resource time-series during a historical simulation,
in accordance with one or more embodiments;
[0018] FIG. 9 illustrates an example set of operations for
simulating a forecast scenario, in accordance with one or more
embodiments;
[0019] FIG. 10A illustrates an example set of demand time-series
that were adjusted during a forecast scenario simulation, in
accordance with one or more embodiments;
[0020] FIG. 10B illustrates an example set of resource time-series
that were adjusted during a forecast scenario simulation, in
accordance with one or more embodiments; and
[0021] FIG. 11 illustrates an example computing system upon which
one or more embodiments may be implemented.
DETAILED DESCRIPTION
[0022] In the following description, for the purposes of
explanation, numerous specific details are set forth in order to
provide a thorough understanding. One or more embodiments may be
practiced without these specific details. Features described in one
embodiment may be combined with features described in a different
embodiment. In some examples, well-known structures and devices are
described with reference to a block diagram form in order to avoid
unnecessarily obscuring the present invention. [0023] 1. GENERAL
OVERVIEW [0024] 2. ARCHITECTURAL OVERVIEW [0025] 3. SIMULATION
MODELS BASED ON DEMAND AND RESOURCE CORRELATION [0026] 4.
SEASONAL-AWARE SIMULATION MODELS [0027] 5. HISTORICAL SCENARIO
SIMULATIONS [0028] 6. FORECAST SCENARIO SIMULATIONS [0029] 7.
COMPUTER NETWORKS AND CLOUD NETWORKS [0030] 8. MICROSERVICE
APPLICATIONS [0031] 9. HARDWARE OVERVIEW [0032] 10. MISCELLANEOUS;
EXTENSIONS
[0033] 1. General Overview
[0034] System administrators may rely on various tools to monitor
system performance in computing environments. For example, system
administrators may deploy performance profilers to track various
performance metrics associated with hardware and/or software
resources. Performance profilers may generate periodic reports that
track overall system performance as well as the performance of
individual resources. Performance profilers may further generate
alerts to notify the system administrator if the performance
metrics cross a threshold. The alerts allow system administrators
to address problems that may arise in the event of performance
degradation or too many idle resources.
[0035] Performance profilers are generally directed to analyzing
historical metrics. For example, a performance profiler may track
various metrics, such as central processing unit (CPU) utilization,
physical memory accesses, function calls, database transactions,
etc. While historical profiling presents an overall picture of how
the system has performed over time, the results are generally
constrained to historical observations. System administrators are
often interested in analyzing other possible scenarios, even if
these scenarios have not occurred in the past.
[0036] What-if analytical models allow system administrators to
simulate and observe the impact of changes within a system
environment. One approach to the what-if analysis is for the system
administrator to define performance capacity of a system and the
performance requirements for software resources deployed within a
system. In the context of a cloud service, for instance, a cloud
administrator may define an actual or hypothetical host capacity
and the requirements of running an instance of the cloud service on
the host. The cloud administrator may then simulate adding and
removing instances from the host to determine how system
performance is affected. This approach allows administrators to
more accurately predict how adding resources affects system
performance. However, the approach relies on the administrator's
domain knowledge. In complex system with variable inputs and
demands, the administrator may not be aware of or be able to keep
up with how changes will affect a particular resource.
[0037] According to techniques described herein, machine-learning
processes and systems may be used to train what-if analytical
models (also referred to herein as simulation models) based on
historical metrics captured from a computing environment. The
machine-learning processes may train/build the simulation models
based on a variety of inputs and machine-learning algorithms. The
what-if analytic may be implemented to learn complex relationships
between system demands and resources from historical time-series
data. Additionally or alternatively, the what-if analytic 138 may
account for trends and seasonal patterns when training the
simulation models. These techniques may provide greater accuracy
and more robust analytic capabilities when simulating scenarios in
computing environments. As a result, capacity planning operations
may be more optimally tuned for a wider variety of scenarios to
increase the efficiency with which computing resources are
allocated and deallocated within the computing environment.
[0038] In one or more embodiments, a simulation model comprises a
set of one or more underlying correlation predication models. Each
correlation prediction model may be trained using a set time-series
datasets associated with at least one demand on a resource and at
least one performance metric for the resource. A demand time-series
may comprise a sequence of data points captured of time for any
input or metric representing the use of a resource. Example demands
may include, but are not limited to, user logons to access a
resource, transactions per second (e.g., on a database or other
transactional system), executions per second, and resource calls
per second. Example performance metrics that may be tracked may
include, but are not limited to CPU performance metrics (e.g., CPU
utilization rates, thread counts, etc.), memory bandwidth metrics
(e.g., memory usage rates, cache hit rates, etc.), I/O metrics
(e.g., physical reads and writes to disk), and network metrics
(e.g., packet counts, packet flow rates, etc.).
[0039] Once trained, a correlation prediction model may be used to
project values for one or more performance metrics of a resource as
a function of learned correlation patterns. For example, a
correlation prediction model may output a projected CPU utilization
rate based on an input demand or combination of demands. Other
correlation prediction models may output other performance metrics
depending on the particular implementation.
[0040] In one or more embodiments, a simulation model accounts for
seasonal correlation patterns. For example, within a computing
environment, high and/or low demands may recur on a seasonal basis
(e.g., monthly, weekly, daily, etc.) In some cases, the seasonal
pattern may correlate strongly with a set of resource performance
values, while in other cases, the seasonal pattern may be weakly
correlated (or may not correlate at all) with resource performance.
For example, weekly high transaction loads may correlate with a
decrease in performance for a database resource. On the other hand,
weekly low transaction loads may not be correlated with database
performance. This may occur due to maintenance and batch process
scheduling during low transaction periods of time. The maintenance
and batch processing may also significantly impact the database
performance, reducing the correlation of low transaction seasonal
periods.
[0041] In one or more embodiments, a simulation model comprises
different correlation prediction models to account for different
seasonal patterns. For example, one correlation prediction model
may be trained as a function of the correlation between seasonal
high data points in transactions and CPU utilization. Another
correlation model may be trained as a function of seasonal lows
values or some other seasonal pattern. The projected performance
values may thus vary based on learned seasonal behavior.
[0042] In one or more embodiments, the simulation model may be
trained to account for patterns and interdependencies between
different demands on a resource. For example, an increase in one
demand may correlate to an increase or decrease in another demand.
In order to capture this relationship, a demand propagation model
may be trained based on the correlation patterns extracted from
historical time-series data. The demand propagation models may be
used to predict how changes one demand would affect values in other
demands.
[0043] Once trained, a simulation model may be evaluated against
one or more what-if scenarios. Each what-if scenario defines a set
of one more scenario parameters including an adjustment to one or
more resource and/or demand time-series values. For example, a user
may wish to determine how an increase in the number of database
transactions and/or one or more other demand affects CPU
utilization on a target host. To run the simulation, the user may
input, into the simulation model, the magnitude of the adjustment
of the set of resource values. In response, the simulation model
generates and present a projected adjustment to a set of one or
more demand values.
[0044] In one or more embodiments, the simulation model may run
historical simulations and/or forecast simulations. With a
historical simulation, adjustments are made to one or more
historical values to determine how the change would have affected
past performance. For example, the user may query the simulation
model to determine how one hundred more instances of a cloud server
would have affected performance of a particular target host in the
past week. A forecast simulation, on the other hand, may involve
changes to one or more historical values and/or one or more
forecasted values to determine how a change would affect future
performance. In this case, the simulation model may comprise a
forecasting model to project future values. In response to
receiving an adjustment to one or more values (e.g., to a resource
time-series), the simulation model may adjust a set of forecasted
values.
[0045] In one or more embodiments, the what-if analytic may be used
to optimize system resources and configurations. For example, one
or more simulations may be run to determine how various adjustments
would affect resource performance. Based on the output of the
simulations, one or more responsive actions may be taken. Example
responsive actions may include deploying additional resources,
bringing resources offline, and adjusting system configurations. If
a system administrator is expecting additional demands on a
resource, for instance, various scenarios may be evaluated to
determine which scenario leads to the optimal resource
configuration, which may be the configuration that maximizes
performance (or satisfies a threshold level) using the fewest of
resources.
[0046] 2. Architectural Overview
[0047] In one or more embodiments, training of a what-if analytic
is driven by a plurality of time-series signals. A time series
signal, in this context, comprises a sequence of values that are
captured over time. The source of the time series signal and the
type of information that is captured may vary from implementation
to implementation. For example, a time series may be collected from
one or more software and/or hardware resources and capture various
performance attributes of the resources from which the data was
collected. As another example, a time series may be collected using
one or more sensors that measure physical properties, such as
temperature, pressure, motion, traffic flow, or other attributes of
an object or environment.
[0048] FIG. 1A illustrates an example system comprising a set of
what-if analytic services that are driven by time-series data
captured from one or more host devices. System 100 generally
comprises hosts 110a-n, data collector 120, analytic services 130,
data repository 140, and clients 150a-k. Components of system 100
may be implemented in one or more host machines operating within
one or more clouds or other networked environments, depending on
the particular implementation. Each component may be distributed
over multiple applications and/or machines. Multiple components may
be combined into one application and/or machine. Operations
described with respect to one component may instead be performed by
another component.
[0049] Hosts 110a-n represent a set of one or more network hosts
and generally comprise targets 112a-i and agents 114a-j. A "target"
in this context refers to a resource that serves as a source of
time series data. For example, a target may be a software
deployment such as a database server instance, middleware instance,
or some other software resource executing on a network host.
Additionally or alternatively, a target may be a hardware resource,
an environmental characteristic, or some other physical resource
for which metrics may be measured and tracked.
[0050] Agents 114a-j comprise hardware and/or software logic for
capturing time-series measurements from a corresponding target (or
set of targets) and sending these metrics to data collector 120. In
one or more embodiments, an agent includes a process, such as a
service or daemon, that executes on a corresponding host machine
and monitors one or more software and/or hardware resources that
have been deployed. Additionally or alternatively, an agent may
include one or more hardware sensors, such as
microelectromechanical (MEMs) accelerometers, thermometers,
pressure sensors, etc., that capture time-series measurements of a
physical environment and/or resource. Although only one agent and
target is illustrated per host in FIG. 1A, the number of agents
and/or targets per host may vary from implementation to
implementation. Multiple agents may be installed on a given host to
monitor different target sources of time series data. In other
embodiments, an agent that resides remotely on a different host
than a target may be responsible for collecting sample time-series
data from the target.
[0051] Data collector 120 includes logic for aggregating data
captured by agents 114a-j into a set of one or more time-series.
Data collector 120 may store the time series data in data
repository 140. Additionally or alternatively, data collector 120
may provide the time-series data to time-series analytic 130. In
one or more embodiments, data collector 120 receives data from
agents 114a-j over one or more data communication networks, such as
the Internet. Example communication protocols that may be used to
transport data between the components illustrated within system 100
may include, without limitation, the hypertext transfer protocol
(HTTP), simple network management protocol (SNMP), and other
communication protocols of the internet protocol (IP) suite.
[0052] In one or more embodiments, data collector 120 collects
demand and resource metrics from agents 114a-j. As previously
indicated, example demand metrics may include, but are not limited
to, user logons to access a resource, transactions per second
(e.g., on a database or other transactional system), executions per
second, and calls per second associated with the resource. Example
performance metrics that may be tracked may include, but are not
limited to CPU performance metrics (e.g., CPU utilization rates,
thread counts, etc.), memory bandwidth metrics (e.g., memory usage
rates, cache hit rates, etc.), I/O metrics (e.g., physical reads
and writes to disk), and network metrics (e.g., packet counts,
packet flow rates, etc.).
[0053] Analytic services 130 includes correlation modelling logic
132, seasonality modelling logic 134, forecast modelling logic 136,
and what-if analytic 138. Each service may be invoked independently
or in combination to train and/or evaluate time-series models, as
described in further detail below. For example, correlation
modelling logic 132 may train/evaluate correlation prediction
models (e.g., demand propagation models and resource prediction
models), seasonality modelling logic 134 may train/evaluate
seasonal behavioral models, forecast modelling logic 136 may
train/evaluate forecast models, and what-if analytic 138 may
evaluate/simulate what-if scenarios using the underlying models.
Models may be trained/updated periodically or on-demand based on
time-series data collected from targets 112a-i.
[0054] Data repository 140 includes volatile and/or non-volatile
storage for storing data for analytic services 130, such as trained
simulation models and the results of running scenario simulations.
Data repository 140 may be implemented by any type of storage unit
and/or device (e.g., a file system, database, collection of tables,
disk, tape cartridge, random access memory, disk, or any other
storage mechanism) for storing data. Further, data repository 140
may include multiple different storage units and/or devices. The
multiple different storage units and/or devices may or may not be
of the same type or located at the same physical site. Further,
data repository 140 may be implemented or may execute on the same
computing system as one or more other components of FIG. 1A and/or
may reside remotely from one or more other components.
[0055] Clients 150a-k represent one or more clients that may access
analytic services 130 to evaluate what-if scenarios. A "client" in
this context may be a human user, such as an administrator, a
client program, or some other application instance. A client may
execute locally on the same host as time-series analytic or may
execute on a different machine. If executing on a different
machine, the client may communicate with analytic services 130 via
one or more data communication protocols according to a
client-server model, such as by submitting HTTP requests invoking
one or more of the services and receiving HTTP responses comprising
results generated by one or more of the services. Analytic services
130 may provide clients 150a-k with an interface through which one
or more of the provided services may be invoked. Example interfaces
may comprise, without limitation, a graphical user interface (GUI),
an application programming interface (API), a command-line
interface (CLI) or some other interface that allows a user to
interact with and invoke one or more of the provided services.
[0056] FIG. 1B illustrates an example dataflow for simulating
scenarios and generating scenario outputs, in accordance with one
or more embodiments. During a training phase, analytic services 130
receives, as input, demand and resource time-series data 142.
Correlation modelling logic 132, seasonality modelling logic 134,
and forecast modelling logic 136 each process demand and resource
time-series data 142 to build respective time-series models. For
example, correlation modelling logic 132 may train correlation
prediction models, seasonality modelling logic 134 may train
seasonal pattern models, and forecasting modelling logic 136 may
train forecast models. Example operations for building and training
these time-series models are described in further detail below.
[0057] In one or more embodiments, seasonality modelling logic 134
provides seasonal pattern representations to correlation modelling
logic 132 and/or forecast modelling logic 136. Based on the
seasonal patterns representations, correlation modelling logic 132
may generate seasonal-aware correlation prediction models that
account for seasonal behavior in the correlation patterns.
Additionally or alternatively, forecast modelling logic 136 may
generate seasonal-aware forecast models that account for seasonal
behavior in the forecasts.
[0058] During an evaluation phase, what-if analytic 138 receives,
as input, (a) scenario parameters 144, (b) trained correlation
prediction models (including demand propagation and resource
prediction models generated by correlation modelling logic 132),
(c) trained forecast models (from forecast modelling logic 136).
Scenario parameters 144 comprise a set of values that define a
particular scenario to simulate. In one or more embodiments,
scenario parameters 144 define at least one adjustment to a demand
or resource time-series value. For example, a scenario may be
defined as follows: "What if my system saw X % more user-demand".
Based on this scenario definition, historical and/or forecast
simulations may be evaluated by what-if analytic 138.
[0059] During the evaluation phase, what-if analytic 138 may use
the demand propagation models to project how the change in user
demand would affect other demands. For example, for a scenario
"What if my system saw X % more transactions", what-if analytic may
determine what change, if any, would occur in the number of redo
records generated per second and/or any other demand on a database
resource. What-if analytic 138 may then generate adjustments to the
other demands according to the trained demand propagation
models.
[0060] Additionally or alternatively, what-if analytic 138 may use
the resource prediction models to project how the change in user
demand would affect resource performance. The resource prediction
model may account for adjustments to other demands. For example,
physical writes to disk may be a function of both transactions per
second and redo records per second. Thus, an adjustment to the
number of transactions per second may be propagated to the redo
records per second before adjusting the physical writes to disk. In
other words, the projected change to the physical writes is not
computed solely as a function of the adjustment to a single demand
(e.g., based on a one-to-one mapping between the demand and
resource). Rather, complex relationships (e.g., many to one and
many-to-many mappings) between different demands and resources are
accounted for in the simulation models, which may increase the
accuracy of the projected changes.
[0061] Based on the evaluation, what-if analytic 138 generates
scenario outputs 146. In one or more embodiments, scenario outputs
146 capture adjustments made to demand and/or resource time-series
data for a scenario definition. In the example scenario "What if my
system saw X % more transactions", for instance, the scenario
output may reflect historical and/or forecasted changes to one or
more resource performance metrics and/or other demand metrics.
Scenario outputs may be stored in data repository 140, provided to
clients 150a-k (e.g., by sending over a network to another host
device or notifying a separate application executing on a host)
and/or presented via an interactive display. An application or
other user may process the scenario outputs to make adjustments to
targets 112a-i. For example, if a scenario simulation indicates
that predicted performance metrics for a scenario will not satisfy
a threshold, additional resources may be deployed and/or existing
resources may be reconfigured (e.g., through load balancing,
pooling etc.) to boost performance. The number of resources that
are deployed may also be selected based on scenario simulations
that satisfy the target performance threshold. In other cases,
resources may be brought offline or other responsive actions may be
taken to optimize system 100 for a particular scenario.
[0062] 3. Demand Propagation and Resource Prediction Models
[0063] Performance of a software or hardware resource may be
function of many different inputs and interactions. For example,
CPU utilization on a database host may be impacted by the frequency
of transactions--a greater the number of transactions per second
may typically correlate with a greater CPU utilization. However,
CPU utilization may also be affected by other demands, such as the
types of queries executed by the database host. On-line transaction
processing (OLTP) type queries/workloads, such as table inserts and
updates, generally do not impact CPU utilization as significantly
as online analytical processing (OLAP) queries/workloads. The
reason for the disparity is that OLAP queries generally involve
scans and analysis of a much greater number of records within the
database than an OLTP query. Other demands may also have varying
impact on the database host's performance. Other examples include,
but are not limited to, the number of redo records (i.e., a log
that stores a history of changes made to the database) generated
per second, and the number of user calls to the database.
[0064] In one or more embodiments, correlation modelling logic 132
builds a set of correlation prediction models to represent
relationships between demands and resources. A correlation
prediction model, in this context, is a data object or data
structure that is generated, in data repository 140, as a
representation of learned pattern of correlation between different
time-series. For example, one type of correlation prediction model,
also referred to herein as a demand propagation model, may map
values for a demand metric to correlated values for another demand
metric. For instance, a demand propagation model that maps values
associated with one demand (e.g., transactions per second) to
another demand (e.g., redo records generated per second). Another
type of correlation prediction model, also referred to herein as a
resource prediction, may map values for a demand metric to
correlated values for a resource performance metric. In the context
of CPU utilization, for instance, a model may map one or more
demand value ranges (e.g., transactions per second, user calls per
second, etc.) to correlated CPU utilization rates.
[0065] In one or more embodiments, correlation modelling logic 132
comprises a set of training processes implementing machine-learning
algorithms to train a set of models. The machine-learning
algorithms may perform regression, clustering, and/or
classification to train models, as described further below. The set
of training processes may initially train the set of models using a
training set of time-series data. As additional time-series data is
received, the correlation prediction models may be updated to
reflect changes and/or additional learned patterns between
resources and demands.
[0066] FIG. 2 illustrates an example set of operations for training
a set of demand propagation and correlation prediction models, in
accordance with one or more embodiments. The set of operations
include receiving time-series data for a set of demands (Operation
210). For example, correlation modelling logic 132 may receive,
from data collector 120, a sequence of data points capturing the
number of instruction executions, transactions per second, redo
records generated, function calls, and/or user calls per second for
one or more of targets 112a-i. Additionally or alternatively, other
demands may also be captured for analysis, depending on the
particular implementation. As previously indicated, time-series
data may be received on-demand, periodically, or on a
streaming/continuous basis from data collector 120.
[0067] In one or more embodiments, the training set of time-series
data is conditioned before analysis. For example, correlation
modelling logic 132 may perform an hourly rollup and/or a
time-alignment of different time-series dataset to compare values
from the same hourly time-range. In other embodiments, values may
be rolled up over a different time period (e.g., on a per minute
basis, per half-hour, daily, etc.)
[0068] Before the training set of time-series data is used to train
correlation prediction models, correlation modelling logic 132
analyzes different demands and filters out/removes demands that
contain redundant information. Demand filtering reduces processing
and storage overhead by eliminating the training and storage of
redundant models. Demand filtering is described further below with
reference to operations 220 to 260.
[0069] During the filtering phase, correlation modelling logic 132
selects a demand pair to analyze (Operation 220). For example, if
the time series includes three inputs/demand on a system, D.sub.1,
D.sub.2, and D.sub.3, there are three possible demand pairs:
{D.sub.1, D.sub.2}, {D.sub.1, D.sub.3} and {D.sub.2, D.sub.3}. The
demand pairs may be selected and processed n any order.
[0070] As part of the filtering phase, correlation modelling logic
132 next determines a correlation coefficient using the training
set of time-series data for the selected demand pair (Operation
230). For example, a Pearson's correlation coefficient may be
computed as follows:
p = .SIGMA. i = 1 n ( d i - d _ ) ( x i - x _ ) .SIGMA. i = 1 n ( d
i - d _ ) 2 .SIGMA. i = 1 n ( x i - x _ ) 2 ( 1 ) ##EQU00001##
where: [0071] d is a first demand time-series dataset including n
values {d.sub.1, . . . , d.sub.n}; [0072] x is a second demand
time-series dataset including n values {x.sub.1, . . . , x.sub.n};
[0073] d is the sample mean for the first demand time series, and
[0074] x is the sample mean for the second demand time-series
dataset. Other methods of computing a correlation coefficient, such
as spearman's rank correlation coefficient, may also be used,
depending on the particular implementation.
[0075] Correlation modelling logic 132 next compares the
correlation coefficient to a filter threshold to determine whether
to filter one of the demands from the demand pair (Operation 240).
The filter threshold may vary from implementation to
implementation. In one or more embodiments, the filter threshold
for the Pearson's correlation coefficient may be set at 0.99. A
correlation coefficient this high indicates that the demands track
each other very closely and may be treated as representing the same
behavioral patterns. In other cases. a slightly lower correlation
coefficient may be used to allow for a greater amount of deviation
between the demands, thereby increasing the likelihood of
filtering.
[0076] If the filter threshold is satisfied, then correlation
modeling logic 132 removes one of the demands from the demand pair
(Operation 250). Once removed, a demand is not used to train demand
propagation models or resource prediction models. Instead, the
demand propagation model and resource prediction model for the
remaining demand in the demand pair may be applied to the removed
demand since the demands are highly correlated.
[0077] Correlation modelling logic 132 next determines whether
there are any demand pairs remaining (Operation 260). The
determination may be made based on whether any demands were
filtered. For example, in the above example with three demand
pairs, if {D.sub.1, D.sub.2} was analyzed and no demands were
filtered, then there are two remaining demand pairs to analyzed:
{D.sub.1, D.sub.3} and {D.sub.2, D.sub.3}. On the other hand, if
D.sub.2 was filtered out, then only one demand pair is left to
analyze: {D.sub.1, D.sub.3}. The process returns to operation 220
for each reaming demand pair.
[0078] Once all demand pairs have been processed, correlation
modelling logic 132 trains a set of one or more demand propagation
models for the remaining demand pairs (Operation 270). In one or
more embodiments, correlation modelling logic 132 trains a demand
propagation model by fitting a correlation prediction model to the
observed data sets of each demand pair. Given demand pair {D.sub.1,
D.sub.2}, for instance, a predictive model may be fit using linear
regression as follows:
D.sub.1=D.sub.2.beta.+ .sub.i (2)
where [0079] .beta. is a parameter vector with elements that are
partial derivatives; and [0080] .sub.i represents an error term.
Linear regression may be used to compute a "best fit" line through
the demand time-series data points. The best fit in this case is
the line that most accurately represents the relationship between
the demand pairs. The best fit may be determined by minimizing the
sum of squared residuals, percentage of squared residuals, or
according to any other linear regression estimation method. In
other cases, nonlinear models may be trained to fit polynomials or
other functions to the data points.
[0081] In one or more embodiments, demand propagation models are
only trained for demand pairs if the correlation coefficient is
above a threshold. The training threshold in this case may be set
much lower than the filter threshold as no demand pairs remain with
a correlation coefficient above the filter threshold. For example,
a training threshold may be set at 0.25 or any other value,
depending on the particular implementation. The training threshold
may eliminate having to train/store demand propagation models for
demands that have little to no correlation.
[0082] FIG. 3 illustrates an example set of demand propagation
models, in accordance with one or more embodiments. In the example,
demand propagation models are trained for demand pairs that have a
correlation above a threshold (e.g., 0.25). Referring to the demand
pairs: [0083] Chart 302 depicts the correlation between the size of
redo data generated per second and executions per second on a
database host. The correlation coefficient is 0.13, which is below
the threshold. Thus, no demand propagation model is trained. [0084]
Chart 304 depicts the correlation between transactions per second
and executions per second on a database host. The correlation is
0.73, which is above the threshold. Thus, a demand propagation
model is fit to the data points. [0085] Chart 306 depicts the
correlation between user calls per second and executions per second
on the database host. The correlation is 0.44, which is above the
threshold. Thus, a demand propagation model is fit to the data
points. [0086] Chart 308 depicts the correlation between
transactions per second and redo data generated per second on a
database host. The correlation coefficient is 0.1, which is below
the threshold. Thus, no demand propagation model is trained. [0087]
Chart 310 depicts the correlation between the number of user calls
per second and the size of redo records generated per second on a
database host. The correlation coefficient is 0.08, which is below
the threshold. Thus, no demand propagation model is trained. [0088]
Chart 312 depicts the correlation between user calls per second and
transactions per second on a database host. The correlation
coefficient is 0.05, which is below the threshold. Thus, no demand
propagation model is trained.
[0089] Returning again to FIG. 2, the process includes receiving
time-series datasets for resource performance metrics (Operation
280). For example, correlation modelling logic 132 may receive a
sequence of samples that track CPU utilization, memory bandwidth,
I/O operations, and/or other performance metric over time. The
resource performance metrics may be received at any point in the
process on demand, periodically, or on a streaming basis. Upon
receipt, the resource performance metrics may be conditioned as
previously described.
[0090] Using the time-series data for the remaining demands and the
resource performance metrics, correlation modelling logic 132
trains a set of resource prediction models (Operation 290). In one
or more embodiments, correlation modelling logic 132 trains a
resource prediction model by fitting a correlation prediction model
to the observed data sets of a demand, resource metric pairing.
Given demand-resource pair {D.sub.1, R.sub.1}, for instance, a
predictive model may be fit using linear regression as follows:
R.sub.1=D.sub.1.beta.+ .sub.i (3)
where [0091] .beta. is a parameter vector with elements that are
partial derivatives; and [0092] .sub.i represents an error term.
Linear regression may be used to compute a "best fit" line through
the demand and resource time-series data points. The best fit in
this case is the line that most accurately represents the
relationship between the demand and resource performance. The best
fit may be determined by minimizing the sum of squared residuals,
percentage of squared residuals, or according to any other linear
regression estimation method. In other cases, nonlinear models may
be trained to fit polynomials or other functions to the data
points.
[0093] In one or more embodiments, resource prediction models are
only trained for demand-to-resource pairs if the correlation
coefficient is above a threshold. For example, a training threshold
may be set at 0.25, as above for the demand propagation models, or
any other value, depending on the particular implementation. The
training threshold may eliminate having to train/store resource
prediction models for demands that have little to no
correlation.
[0094] FIG. 4 illustrates an example set of resource prediction
models, in accordance with one or more embodiments. In the example,
resource prediction models are trained for resource-to-demand
pairings that have a correlation above a threshold (e.g., 0.4).
Referring to the pairings: [0095] Chart 402 depicts the correlation
between CPU utilization and executions per second on a database
host. The correlation coefficient is 0.28, which is below the
threshold. Thus, no resource prediction propagation model is
trained. [0096] Chart 404 depicts the correlation between CPU
utilization and redo size generated per second on a database host.
The correlation is 0.51, which is above the threshold. Thus, a
resource prediction model is fit to the data points. [0097] Chart
406 depicts the correlation between CPU utilization and
transactions per second on a database host. The correlation is
0.43, which is above the threshold. Thus, a resource prediction
model is fit to the data points. [0098] Chart 408 depicts the
correlation between CPU utilization and user calls per second on a
database host. The correlation coefficient is 0.43, which is above
the threshold. Thus, a resource prediction model is fit to the data
points.
[0099] As depicted in FIG. 4, host CPU utilization is a function of
redo size, transaction, and user calls per second. In other cases,
CPU utilization may be a function of other demands in addition or
as an alternative to those listed. Correlation modelling logic 132
may select demands during runtime to predict each resource. The
selected demands may be adjusted over time as additional
correlation patterns are detected. If no demand is correlated to a
given resource metric, then correlation modelling logic 132 may
proceed without training a resource prediction model for the given
resource performance metric.
[0100] 4. Seasonal-Aware Simulation Models
[0101] Correlation patterns may vary between different seasons. A
season in this context refers to pattern that recurs periodically.
In a database host, for instance, daily high seasons for
transactions may be observed between the hours of 9 to 5 from
Monday to Friday. Daily low seasons may occur outside of these
hours. In other cases, seasonal high, lows, and/or other patterns
may recur on an hourly, weekly, monthly, yearly, or holiday basis.
As previously described, the daily highs in transactions may
correlate with a high CPU utilization rate. However, daily lows may
not be closely correlated with the CPU utilization rate as batch
processes may be run in the evenings on the database host. In other
cases, seasonal lows may follow a different correlation pattern
than the daily highs. Therefore, training different correlation
models (e.g., demand propagation and resource prediction models)
for different resources may lead to more accurate
representations.
[0102] In one or more embodiments, seasonality modelling logic 134
builds a seasonal classification model that represents different
seasonal patterns. For example, the seasonal classification model
may cluster time-series data points that are associated with
different seasonal pattern classifications. The seasonal pattern
classifications and clusters may then be provided to correlation
modelling logic 132. In response, correlation modelling logic 132
may build demand propagation models and/or resource prediction
models, as previously described, for the data points belonging to
each cluster. In other words, separate models are trained and build
for each cluster.
[0103] Correlation prediction models for a set of demands and
resources may vary between different clusters/seasonal pattern
classifications. For example, CPU utilization may be a function of
transactions, user calls, and redo logs generated per second in the
high season. In the low season, CPU utilization may only be a
function of executions per second. In other words, the correlation
coefficients between a particular demand and resource pair may vary
from one season to the next. As a result, correlation modelling
logic 132 may select different demands to train resource prediction
models for different seasonal patterns.
[0104] FIG. 5 illustrates an example set of operations for
performing seasonal aware training of simulation models, in
accordance with one or more embodiments. The process includes
identifying seasonal patterns in a time-series (Operation 510). For
example, seasonal modelling logic 134 may classify different
sub-periods/instances of a resource and/or demand time-series as
sparse high for those instances that exhibit sharp, relatively
short bursts on a recurring basis. Other classifications may
include, but are not limited to, dense highs, sparse lows, and
dense lows. Example techniques for classifying seasonal patterns
are described in U.S. application Ser. No. 15/266,971, entitled
"SEASONAL AWARE METHOD FOR FORECASTING AND CAPACITY PLANNING"; U.S.
Provisional Patent Appl. No. 62/301,585, entitled "METHOD FOR
CREATING PERIOD PROFILE FOR TIME-SERIES DATA WITH RECURRENT
PATTERNS"; U.S. application Ser. No. 15/057,065, entitled "SYSTEM
FOR DETECTING AND CHARACTERIZING SEASONS"; U.S. application Ser.
No. 15/057,060, entitled "SUPERVISED METHOD FOR CLASSIFYING
SEASONAL PATTERNS"; U.S. application Ser. No. 15/057,062, entitled
"UNSUPERVISED METHOD FOR CLASSIFYING SEASONAL PATTERNS"; and U.S.
Provisional Patent Appl. No. 62/370,880, entitled "UNSUPERVISED
METHOD FOR BASELINING AND ANOMALY DETECTION IN TIME-SERIES DATA FOR
ENTERPRISE SYSTEMS", previously incorporated by reference.
[0105] Responsive to identifying a seasonal pattern, seasonality
modelling logic 126 extracts data points in the time series that
belong to the seasonal pattern (Operation 520). For example, if a
high season for resource utilization is identified to recur weekly
on Mondays from 9 to noon, then data points captured during this
time frame may be clustered or otherwise grouped together within
data repository 140.
[0106] Once extracted, correlation modelling logic 132 uses the
data points to train a set of one or more correlation prediction
models for the seasonal pattern (Operation 530). For example,
correlation modelling logic 132 may generated demand propagation
models and/or resource prediction models as previously
described.
[0107] The process next determines whether there are any remaining
seasonal patterns to analyze (Operation 540). If so, then the
process repeats for the next selected seasonal patterns. Thus,
correlation prediction models may be generated for each respective
seasonal pattern. For example, if sparse high, dense high, and low
seasonal patterns were identified, then the process may repeat for
each of these seasonal pattern classifications. After correlation
prediction models have been trained for each seasonal pattern, the
process ends.
[0108] As can be seen from the process in FIG. 5, different sets of
data points may be used to train correlation prediction models
associated with different respective seasonal patterns. For
example, for a high season, data points from a first set of one or
more sub-periods (e.g., Monday to Thursday 9a.m. to 4p.m.) may be
extracted to train the correlation prediction model. Data points
from a different set of one or more sub-periods may be used to
train correlation prediction models for other seasonal patterns,
such as a low season.
[0109] FIG. 6 illustrates an example set of seasonal-aware
correlation prediction models, in accordance with one or more
embodiments. Chart 602 depicts a resource prediction model the maps
values of executions per second to CPU utilization rates in the
sparse high season. A best fit line is generated using time-series
samples from the sparse high season. Chart 604 depicts a resource
prediction model that maps values of executions per second to CPU
utilization rates in the dense high season. As can be seen, the
resource prediction model that is fit to the data has a different
slope than in the sparse high season.
[0110] In one or more embodiments, data repository 140 stores a
mapping between seasonal patterns and the corresponding set of
trained correlation prediction models. During the evaluation phase,
described n further detail below, the mapping data may be accessed
to determine which correlation prediction models to use for a given
scenario. For example, a scenario may be defined as follows "What
if my system saw X times more transactions in the high season". In
response, what-in analytic 138 may determine which correlation
prediction models are associated with the high season by reading
the mapping data. What-if analytic 138 may use the associated
correlation prediction models to generate the scenario output. In
other cases, a scenario may apply to multiple seasons. For example,
a general scenario of "What if my system saw X times more
transactions" may apply across all seasons. In this case, what-if
analytic 138 may analyze each season independently and stitch
together the results.
[0111] 5. Historical Scenario Simulations
[0112] In one or more embodiments, what-if analytic 138 is
configured to perform historical what-if scenario simulations. With
historical simulations, modifications are made to historical
time-series data to see how the system would have behaved. One
benefit of adjusting historical data is that what-if analytic 138
is able to provide the user with scenario outputs that span the
distribution of known operating modes. Historical simulations may
be performed with no inferences about future conditions. In other
words, a historical simulation may be performed without having to
generate a forecast.
[0113] FIG. 7 illustrates an example set of operations for
simulating a historical scenario, in accordance with one or more
embodiments. The process includes receiving a set of historical
simulation parameters (Operation 710). In one or more embodiments,
the scenario parameters define a historical adjustment. For
example, an example historical simulation scenario may be defined
as "What if there were three times the amount of users in the last
three months?" As another example, a historical simulation scenario
may be defined as "What if there were ten percent more transactions
last week?" In both examples, the scenario defines a historical
adjustment and a period in the past to analyze. In other cases, a
default timeframe to analyze may be chosen rather than explicitly
defined. For example a user may submit a scenario definition as
follows "What if there were ten percent more transactions on
database host DBHost?" In this scenario definition, the user
identifies an adjustment/modification and a particular target
resource, but does not define the timeframe. Any default timeframe
(e.g., the past week, month, six months, etc.) may be selected for
analysis in this case.
[0114] Once the scenario parameters have been submitted, what-if
analytic 138 adjusts one or more demands by a prescribed amount
(Operation 720). For example, if the scenario requests a simulation
of a historical scenario with three times more transactions, then
what-if analytic 138 may adjust the transaction time-series data
collected from one or more of targets 112a-i to treble the values.
Adjustments may be made to values over a prescribed or default
timeframe, depending on the particular implementation.
[0115] In one or more embodiments, the adjustments that are made by
what-if analytic 138 vary depending on whether the scenario is
attempting to model a user change or not. If the scenario defines a
user change, then what-if analytic 138 models the user change by
adjusting all of the demands by a prescribed amount. For example,
if a scenario requests a simulation of four times change in users,
then what-if analytic may adjust all demands by four times. If the
scenario is more particular and adjusts a single demand, then the
process may start by adjusting the prescribed demand by the
prescribed amount. What-if analytic 138 may access the relevant
demand propagation models to predict how the adjusted demand would
propagate to other related demands, as described in the operations
below.
[0116] During the historical simulation, what-if analytic 138
determines whether to propagate changes to a prescribed demand to
other demands (Operation 730). As previously indicated, adjustments
may be propagated if the scenario is making adjustments to
individual demands rather than user changes. If the scenario is
defining a user change, then the adjustment may be applied across
all demands, and the process may skip to Operation 750. If the
adjustment is to a prescribed demand or set of demands, then
what-if analytic 138 may determine whether there are any associated
demand propagation models involving the adjusted demand. For
instance, a trained demand propagation model may be available if
the adjusted demand was correlated more than the training threshold
with another demand. Conversely, a trained demand propagation may
not be available for the adjusted demand if the demand is not
correlated with other demands.
[0117] If a trained demand propagation model is available, then
what-if analytic 138 use the demand propagation model to propagate
the adjustment to other, related demands. (Operation 740). For
example, an adjusted execution per second value may be mapped to a
corresponding value for transactions per second using the demand
propagation model depicted in chart 304. The demand propagation
model is also reversible, such that a change in transactions per
second may be mapped to a corresponding execution value. Similarly,
an adjustment in the number of executions per second may be mapped
to a corresponding adjustment in user calls per second, and vice
versa, using chart 306.
[0118] Once the adjustments to the demands are complete, what-if
analytic determines 138 whether there are any resource prediction
models associated with the adjusted demands (Operation 750). For
example, what-if analytic 138 may search data store 140 for a
trained resource prediction model that maps an adjusted demand to a
resource performance metric. In some cases, a resource prediction
model may not be available. This scenario may occur if none of the
adjusted demands are correlated with a relevant resource
performance metric. If none of the adjusted demands are correlated
to resource performance, then the process may proceed without
making any adjustments to a resource performance metric.
[0119] If a resource prediction model is available, then what-if
analytic 138 may use the resource prediction model to generate an
adjustment to at least one resource performance time-series
(Operation 760). For example, a resource prediction model may map
an adjusted value for executions per second to a corresponding CPU
utilization value. What-if analytic 138 may adjust the historical
CPU utilization rate to the value mapped to the adjusted executions
per second value.
[0120] In one or more embodiments, an adjustment to a resource
performance metric may be determined based on multiple resource
prediction models. For example, one resource prediction model may
map an adjusted value for transactions per second to an adjusted
CPU utilization rate. A second model may map an adjusted redo size
per second to an adjusted CPU utilization rate. A final adjustment
may be computed by combining (e.g., through averaging or otherwise
aggregating) the adjustments from the different resource prediction
models.
[0121] In one or more embodiments, adjustments may be performed
based on seasonal pattern classifications. For example, if a
historical simulation is being run for a prescribed seasonal
pattern (e.g., a high season, sparse high, low season, etc.), then
what-if analytic 138 may search data store 140 for a mapping
between the prescribed season seasonal pattern and a corresponding
set of resource prediction models. What-if analytic 138 may then
select/use one or more resource prediction models to map adjusted
demand values to adjusted resource performance values for the
prescribed seasonal pattern. In the event that the simulation spans
multiple seasons, the models may be isolated to regions of time
based on collective seasons. For example, the resource prediction
model depicted in chart 602 may be used to map adjustments to
executions per second in the sparse high season to corresponding
adjustments to CPU utilization rate. For adjustments in the dense
high season, the resource prediction model depicted n chart 604 may
be used.
[0122] Once the evaluation phase is complete, what-if analytic 138
presents the time-series datasets for one or more demands and one
or more resources, including any generated adjustments (Operation
770). The presentation of the adjusted time-series datasets may
vary from implementation to implementation. For example, the
time-series datasets may be displayed via an interactive display
that is coupled to a client computing system. The interactive
display may allow a user to drill down to view adjustments to
individual demands, resources, or custom groups of demands and/or
resources.
[0123] In one or more embodiments, an interactive display
presenting the results of a historical simulation may allow a user
to select responsive actions, including on or more of the
responsive actions previously described, for the system to perform.
For example, system 100 may adjust the configurations of 112a-i,
deploy additional resources, and/or perform load balancing to
replicate a simulated scenario.
[0124] In one or more embodiments, what-if analytic 138 may present
an uncertainty interval for a historical simulation. An uncertainty
interval may be a visualization or other indication of a degree of
uncertainty in the predicted adjustments. For example, during
training, correlation modelling logic 132 may calculate and cache
the uncertainties for possible combinations of evaluation. For a
model that maps demand X to resource (or demand) Y, these
possibilities include the uncertainty of predicting Y from X and
predicting X from Y.
[0125] In one or more embodiments, the uncertainty interval is a
range of values that include an upper limit and a lower limit
within which at least a certain threshold of values falls within a
given level of confidence. For example, a historical CPU
utilization rate may be observed to be 50 percent. In response to
an adjustment of three times the amount of users, the
projected/adjusted CPU utilization rate may jump to 65% (these
values are given by way of example only and may vary from system to
system). Based on historical patterns, the lower limit of the
uncertainty level may be 55%, and the upper limit may be 75% with
95% confidence. The uncertainty interval thus gives a range of
expected values within a prescribed level of confidence.
[0126] FIG. 8A illustrates an example presentation of propagating
adjustments in one demand to another demand, in accordance with one
or more embodiments. The scenario parameters for this simulation
may be defined as follows: "What if may system saw three times more
executions per seconds in the last five months?" Based on the
simulation, what-if analytic 138 may present chart 800, which
illustrates an adjustment to a historical time-series for
executions per second. Time-series plot 804 depicts historical
executions per second that were performed between May and
September. Time-series plot 802 depicts an example adjustment that
increases the number of executions by three times.
[0127] What-if analytic 138 may further present chart 810 as part
of the historical simulation. Time-series plot 818 depicts
historical transactions per second that were performed between May
and September. Using a demand propagation model, the transactions
per second are adjusted based on the three times increase in
executions per second. Time-series plot 812 depicts the adjustment.
Plot 814 depicts an upper bound for the uncertainty interval, and
plot 816 depicts a lower bound for the uncertainty interval.
[0128] FIG. 8B illustrates an example presentation of adjustments
generated for resource time-series during the historical simulation
defined with reference to FIG. 8A. Chart 820 depicts the historical
CPU utilization rate (time-series plot 830). Chart 820 further
depicts a projected/adjusted CPU utilization rate (time-series plot
824) computed based on the three time increase to executions per
second using a resource prediction model. Plot 826 depicts the
upper bound in the uncertainty interval, and plot 828 depicts the
lower bound. Time-series plot 822 depicts three times the CPU
utilization rate, which may be omitted from the scenario output but
is shown for purposes of illustration. As can be seen, the
projected CPU utilization is much lower than multiplying the CPU
utilization rate by the same amount as the increase in executions
per second. The training of time-series models as previously
described allows for a much more complex and meaningful analysis
between the various interactions between demands and resource
performance.
[0129] Chart 840 depicts the historical number of physical writes
per second (time-series plot 850). Chart 840 further depicts a
projected/adjusted number of physical writes per second
(time-series plot 844) based on the adjusted demands. Plot 846
depicts the upper bound of the uncertainty interval, and plot 848
depicts the lower bound of the interval. Time-series plot 842
depicts three times the historical physical writes per second rate,
which, as can be seen, is inaccurate in comparison to the adjusted
value, which uses the trained models to account for complex
interactions within the system.
[0130] 6. Forecast Scenario Simulations
[0131] In one or more embodiments, what-if analytic 138 is
configured to perform forecast what-if scenario simulations. With
forecast scenario simulations, modifications may be made to
historical and/or forecast time-series data to see how the system
is predicted to behave. In contrast to historical simulations,
forecast simulations include modelling changes in future demands
and/or resources. The adjustments may account for changes, trends,
and seasonal patterns as determined from historical data collected
from targets 112a-i.
[0132] FIG. 9 illustrates an example set of operations for
simulating a forecast scenario, in accordance with one or more
embodiments. The set of operations includes generating a set of
forecasts for each demand and resource time-series (Operation 910).
In one or more embodiments, forecast modelling logic 136 is
configured to generate a forecast that account for seasonal
patterns and trends in the time-series data. For example, forecast
modelling logic 136 may generate an Additive or Multiplicative
Holt-Winters forecasting model. The Holt-Winters models and other
example forecasting models that may be trained are described in
U.S. application Ser. No. 15/266,971, entitled "SEASONAL AWARE
METHOD FOR FORECASTING AND CAPACITY PLANNING", previously
incorporated by reference.
[0133] The process further includes receiving a set of forecast
simulation parameters (Operation 920). In one or more embodiments,
the scenario parameters define a future adjustment. For example, an
example forecast simulation scenario may be defined as "What if
there are three times the amount of forecasted users in the next
three months?" As another example, a historical simulation scenario
may be defined as "What if there are ten percent more transactions
than forecasted for next week?" In both examples, the scenario
defines an adjustment to a forecasted value and a period in the
future to analyze. In other cases, a default timeframe to analyze
may be chosen rather than explicitly defined. For example a user
may submit a scenario definition as follows "What if there are ten
percent more transactions on database host DBHost than forecasted?"
In this scenario definition, the user identifies an
adjustment/modification and a particular target resource, but does
not define the timeframe. Any default timeframe (e.g., the next
week, month, six months, etc.) may be selected for analysis in this
case.
[0134] Responsive to receiving the forecast scenario parameters,
what-if analytic 138 adjusts a historical and/or forecasted value
by a prescribed amount (Operation 930). For example, if the
scenario requests a simulation of a scenario with three times more
transactions than forecasted, then what-if analytic 138 may adjust
the forecast for future transactions per second projected for
targets 112a-i to treble the values. Adjustments may be made to
values over a prescribed or default timeframe, depending on the
particular implementation.
[0135] During the forecast scenario simulation, what-if analytic
138 determines whether to propagate changes to the prescribed one
or more demands to other demands (Operation 940). Similar to the
historical simulation, adjustments may be propagated if the
scenario is making adjustments to individual demands rather than
user changes. If the scenario is defining a user change in the
future, then the adjustment may be applied across all demands, and
the process may skip to Operation 960. If the adjustment is to a
prescribed demand or set of demands, then what-if analytic 138 may
determine whether there are any associated demand propagation
models involving the adjusted demand(s) as described above for in
the historical scenario simulation.
[0136] If a trained demand propagation model is available, then
what-if analytic 138 use the demand propagation model to propagate
the adjustment to other, related demands. (Operation 950). For
example, an adjusted forecast for executions per second may be
mapped to an adjusted forecast for transactions per second using
the demand propagation model depicted in chart 304. The mapped
values in this case incorporate the trend and seasonal factors from
the forecast. The demand propagation models, as well as the
resource prediction models, are also reversible as previously
described.
[0137] Once the adjustments to the demand forecasts are complete,
what-if analytic determines 138 whether there are any resource
prediction models associated with the adjusted demands (Operation
960). For example, what-if analytic 138 may search data store 140
for a trained resource prediction model that maps an adjusted
demand to a resource performance metric. In some cases, a resource
prediction model may not be available. This scenario may occur if
none of the adjusted demands are correlated with a relevant
resource performance metric. If none of the adjusted demands are
correlated to resource performance, then the process may proceed
without making any adjustments to a resource performance
metric.
[0138] If a resource prediction model is available, then what-if
analytic 138 may use the resource prediction model to generate an
adjustment to at least one resource performance time-series
(Operation 970). For example, a resource prediction model may map
an adjusted value for executions per second to a corresponding CPU
utilization value. What-if analytic 138 may adjust a forecast CPU
utilization rate to the value mapped to the adjusted executions per
second value. Similar to the demand adjustments, the resource
performance adjustments may incorporate the trend and/or seasonal
factors of the forecast.
[0139] As with the historical simulation, an adjustment to a
forecast resource performance metric may be determined based on
multiple resource prediction models. Additionally or alternatively,
adjustments may be performed based on seasonal pattern
classifications. For example, if a historical simulation is being
run for a prescribed seasonal pattern (e.g., a high season, sparse
high, low season, etc.), then what-if analytic 138 may search data
store 140 for a mapping between the prescribed season seasonal
pattern and a corresponding set of resource prediction models.
What-if analytic 138 may then select/use one or more resource
prediction models to map adjusted demand forecasts to adjusted
resource performance forecasts for the prescribed seasonal pattern.
In the event that the simulation spans multiple seasons, the models
may be isolated to regions of time based on collective seasons.
[0140] Once the evaluation phase is complete, what-if analytic 138
presents the time-series datasets for one or more demand forecasts
and one or more resource performance forecasts, including any
generated adjustments (Operation 980). The presentation of the
adjusted time-series datasets may vary from implementation to
implementation. For example, the time-series datasets may be
displayed via an interactive display that is coupled to a client
computing system. The interactive display may allow a user to drill
down to view adjustments to individual demands, resources, or
custom groups of demands and/or resources.
[0141] In one or more embodiments, an interactive display
presenting the results of a forecast scenario simulation may allow
a user to select responsive actions, including on or more of the
responsive actions previously described, for the system to perform.
For example, system 100 may adjust the configurations of 112a-i,
deploy additional resources, and/or perform load balancing to
replicate a simulated scenario.
[0142] As with the historical simulation, what-if analytic 138 may
present an uncertainty interval for a historical simulation.
However, in the case of a forecast scenario simulation, the
original forecast may already be associated with an uncertainty
interval, as opposed to historical observations. During training,
forecast modelling logic 136 and/or correlation modelling logic 132
may calculate and cache the uncertainties for possible combinations
of evaluation. For a model that maps demand X to resource (or
demand) Y, these possibilities include the uncertainty of
predicting a forecasted Y from a forecasted X, predicting a
forecasted X from a forecasted Y, predicting a forecasted Y from X,
and predicting a forecasted X from Y.
[0143] In one or more embodiments, what-if analytic 138 may modify
the uncertainty interval when adjusting the forecasted values. For
example, an original forecast may have an interval that is computed
based on the forecast model. During the forecast scenario
simulation, the resource prediction model may introduce additional
uncertainty. The uncertainty interval may be shifted around the
changed forecast values and/or expanded to reflect the uncertainty
of predicting a change in the forecasted performance metric based
on a change to the historical and/or forecasted demand metrics.
[0144] FIG. 10A illustrates an example set of demand time-series
that were adjusted during a forecast scenario simulation, in
accordance with one or more embodiments. The scenario parameters
for this simulation may be defined as follows: "What if may system
saw five times times more executions per seconds in the next two
months?" Based on the simulation, what-if analytic 138 may present
chart 1000, which illustrates an adjusted forecast (time-series
plot 1004) for executions per second. Time-series plot 1002 depicts
historical executions per second leading up to the forecast.
[0145] What-if analytic 138 may further present chart 1010 as part
of the forecast scenario simulation. Time-series plot 1012 depicts
historical size of redo data generated per second between March and
July leading up to the forecast. Using a demand propagation model,
the forecasted redo generated per second is adjusted, as depicted
in time-series plot 1014, based on the five times increase in
forecasted executions per second.
[0146] FIG. 10B illustrates an example set of resource time-series
that were adjusted during a forecast scenario simulation, in
accordance with one or more embodiments. Chart 820 depicts the
historical number of physical reads per second (time-series plot
1022) and an adjusted forecast (time-series plot 1024). Chart 1030
depicts the historical physical writes per second (time-series plot
1032) and an adjusted forecast (time-series plot 1034).
[0147] 7. Computer Networks and Cloud Networks
[0148] In one or more embodiments, a computer network provides
connectivity among a set of nodes. The nodes may be local to and/or
remote from each other. The nodes are connected by a set of links.
Examples of links include a coaxial cable, an unshielded twisted
cable, a copper cable, an optical fiber, and a virtual link.
[0149] A subset of nodes implements the computer network. Examples
of such nodes include a switch, a router, a firewall, and a network
address translator (NAT). Another subset of nodes uses the computer
network. Such nodes (also referred to as "hosts") may execute a
client process and/or a server process. A client process makes a
request for a computing service (such as, execution of a particular
application, and/or storage of a particular amount of data). A
server process responds by executing the requested service and/or
returning corresponding data.
[0150] A computer network may be a physical network, including
physical nodes connected by physical links. A physical node is any
digital device. A physical node may be a function-specific hardware
device, such as a hardware switch, a hardware router, a hardware
firewall, and a hardware NAT. Additionally or alternatively, a
physical node may be a generic machine that is configured to
execute various virtual machines and/or applications performing
respective functions. A physical link is a physical medium
connecting two or more physical nodes. Examples of links include a
coaxial cable, an unshielded twisted cable, a copper cable, and an
optical fiber.
[0151] A computer network may be an overlay network. An overlay
network is a logical network implemented on top of another network
(such as, a physical network). Each node in an overlay network
corresponds to a respective node in the underlying network. Hence,
each node in an overlay network is associated with both an overlay
address (to address to the overlay node) and an underlay address
(to address the underlay node that implements the overlay node). An
overlay node may be a digital device and/or a software process
(such as, a virtual machine, an application instance, or a thread)
A link that connects overlay nodes is implemented as a tunnel
through the underlying network. The overlay nodes at either end of
the tunnel treat the underlying multi-hop path between them as a
single logical link. Tunneling is performed through encapsulation
and decapsulation.
[0152] In an embodiment, a client may be local to and/or remote
from a computer network. The client may access the computer network
over other computer networks, such as a private network or the
Internet. The client may communicate requests to the computer
network using a communications protocol, such as Hypertext Transfer
Protocol (HTTP). The requests are communicated through an
interface, such as a client interface (such as a web browser), a
program interface, or an application programming interface
(API).
[0153] In an embodiment, a computer network provides connectivity
between clients and network resources. Network resources include
hardware and/or software configured to execute server processes.
Examples of network resources include a processor, a data storage,
a virtual machine, a container, and/or a software application.
Network resources are shared amongst multiple clients. Clients
request computing services from a computer network independently of
each other. Network resources are dynamically assigned to the
requests and/or clients on an on-demand basis. Network resources
assigned to each request and/or client may be scaled up or down
based on, for example, (a) the computing services requested by a
particular client, (b) the aggregated computing services requested
by a particular tenant, and/or (c) the aggregated computing
services requested of the computer network. Such a computer network
may be referred to as a "cloud network."
[0154] In an embodiment, a service provider provides a cloud
network to one or more end users. Various service models may be
implemented by the cloud network, including but not limited to
Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and
Infrastructure-as-a-Service (IaaS). In SaaS, a service provider
provides end users the capability to use the service provider's
applications, which are executing on the network resources. In
PaaS, the service provider provides end users the capability to
deploy custom applications onto the network resources. The custom
applications may be created using programming languages, libraries,
services, and tools supported by the service provider. In IaaS, the
service provider provides end users the capability to provision
processing, storage, networks, and other fundamental computing
resources provided by the network resources. Any arbitrary
applications, including an operating system, may be deployed on the
network resources.
[0155] In an embodiment, various deployment models may be
implemented by a computer network, including but not limited to a
private cloud, a public cloud, and a hybrid cloud. In a private
cloud, network resources are provisioned for exclusive use by a
particular group of one or more entities (the term "entity" as used
herein refers to a corporation, organization, person, or other
entity). The network resources may be local to and/or remote from
the premises of the particular group of entities. In a public
cloud, cloud resources are provisioned for multiple entities that
are independent from each other (also referred to as "tenants" or
"customers"). The computer network and the network resources
thereof are accessed by clients corresponding to different tenants.
Such a computer network may be referred to as a "multi-tenant
computer network." Several tenants may use a same particular
network resource at different times and/or at the same time. The
network resources may be local to and/or remote from the premises
of the tenants. In a hybrid cloud, a computer network comprises a
private cloud and a public cloud. An interface between the private
cloud and the public cloud allows for data and application
portability. Data stored at the private cloud and data stored at
the public cloud may be exchanged through the interface.
Applications implemented at the private cloud and applications
implemented at the public cloud may have dependencies on each
other. A call from an application at the private cloud to an
application at the public cloud (and vice versa) may be executed
through the interface.
[0156] In an embodiment, tenants of a multi-tenant computer network
are independent of each other. For example, a business or operation
of one tenant may be separate from a business or operation of
another tenant. Different tenants may demand different network
requirements for the computer network. Examples of network
requirements include processing speed, amount of data storage,
security requirements, performance requirements, throughput
requirements, latency requirements, resiliency requirements,
Quality of Service (QoS) requirements, tenant isolation, and/or
consistency. The same computer network may need to implement
different network requirements demanded by different tenants.
[0157] In one or more embodiments, in a multi-tenant computer
network, tenant isolation is implemented to ensure that the
applications and/or data of different tenants are not shared with
each other. Various tenant isolation approaches may be used.
[0158] In an embodiment, each tenant is associated with a tenant
ID. Each network resource of the multi-tenant computer network is
tagged with a tenant ID. A tenant is permitted access to a
particular network resource only if the tenant and the particular
network resources are associated with a same tenant ID.
[0159] In an embodiment, each tenant is associated with a tenant
ID. Each application, implemented by the computer network, is
tagged with a tenant ID. Additionally or alternatively, each data
structure and/or dataset, stored by the computer network, is tagged
with a tenant ID. A tenant is permitted access to a particular
application, data structure, and/or dataset only if the tenant and
the particular application, data structure, and/or dataset are
associated with a same tenant ID.
[0160] As an example, each database implemented by a multi-tenant
computer network may be tagged with a tenant ID. Only a tenant
associated with the corresponding tenant ID may access data of a
particular database. As another example, each entry in a database
implemented by a multi-tenant computer network may be tagged with a
tenant ID. Only a tenant associated with the corresponding tenant
ID may access data of a particular entry. However, the database may
be shared by multiple tenants.
[0161] In an embodiment, a subscription list indicates which
tenants have authorization to access which applications. For each
application, a list of tenant IDs of tenants authorized to access
the application is stored. A tenant is permitted access to a
particular application only if the tenant ID of the tenant is
included in the subscription list corresponding to the particular
application.
[0162] In an embodiment, network resources (such as digital
devices, virtual machines, application instances, and threads)
corresponding to different tenants are isolated to tenant-specific
overlay networks maintained by the multi-tenant computer network.
As an example, packets from any source device in a tenant overlay
network may only be transmitted to other devices within the same
tenant overlay network. Encapsulation tunnels are used to prohibit
any transmissions from a source device on a tenant overlay network
to devices in other tenant overlay networks. Specifically, the
packets, received from the source device, are encapsulated within
an outer packet. The outer packet is transmitted from a first
encapsulation tunnel endpoint (in communication with the source
device in the tenant overlay network) to a second encapsulation
tunnel endpoint (in communication with the destination device in
the tenant overlay network). The second encapsulation tunnel
endpoint decapsulates the outer packet to obtain the original
packet transmitted by the source device. The original packet is
transmitted from the second encapsulation tunnel endpoint to the
destination device in the same particular overlay network.
[0163] 8. Microservice Applications
[0164] According to one or more embodiments, the techniques
described herein are implemented in a microservice architecture. A
microservice in this context refers to software logic designed to
be independently deployable, having endpoints that may be logically
coupled to other microservices to build a variety of applications.
Applications built using microservices are distinct from monolithic
applications, which are designed as a single fixed unit and
generally comprise a single logical executable. With microservice
applications, different microservices are independently deployable
as separate executables. Microservices may communicate using
HyperText Transfer Protocol (HTTP) messages and/or according to
other communication protocols via API endpoints. Microservices may
be managed and updated separately, written in different languages,
and be executed independently from other microservices.
[0165] Microservices provide flexibility in managing and building
applications. Different applications may be built by connecting
different sets of microservices without changing the source code of
the microservices. Thus, the microservices act as logical building
blocks that may be arranged in a variety of ways to build different
applications. Microservices may provide monitoring services that
notify a microservices manager (such as If-This-Then-That (IFTTT),
Zapier, or Oracle Self-Service Automation (OSSA)) when trigger
events from a set of trigger events exposed to the microservices
manager occur. Microservices exposed for an application may
alternatively or additionally provide action services that perform
an action in the application (controllable and configurable via the
microservices manager by passing in values, connecting the actions
to other triggers and/or data passed along from other actions in
the microservices manager) based on data received from the
microservices manager. The microservice triggers and/or actions may
be chained together to form recipes of actions that occur in
optionally different applications that are otherwise unaware of or
have no control or dependency on each other. These managed
applications may be authenticated or plugged in to the
microservices manager, for example, with user-supplied application
credentials to the manager, without requiring reauthentication each
time the managed application is used alone or in combination with
other applications.
[0166] In one or more embodiments, microservices may be connected
via a GUI. For example, microservices may be displayed as logical
blocks within a window, frame, other element of a GUI. A user may
drag and drop microservices into an area of the GUI used to build
an application. The user may connect the output of one microservice
into the input of another microservice using directed arrows or any
other GUI element. The application builder may run verification
tests to confirm that the output and inputs are compatible (e.g.,
by checking the datatypes, size restrictions, etc.)
[0167] Triggers
[0168] The techniques described above may be encapsulated into a
microservice, according to one or more embodiments. In other words,
a microservice may trigger a notification (into the microservices
manager for optional use by other plugged in applications, herein
referred to as the "target" microservice) based on the above
techniques and/or may be represented as a GUI block and connected
to one or more other microservices. The trigger condition may
include absolute or relative thresholds for values, and/or absolute
or relative thresholds for the amount or duration of data to
analyze, such that the trigger to the microservices manager occurs
whenever a plugged-in microservice application detects that a
threshold is crossed. For example, a user may request a trigger
into the microservices manager when the microservice application
detects a value has crossed a triggering threshold.
[0169] In one embodiment, the trigger, when satisfied, might output
data for consumption by the target microservice. In another
embodiment, the trigger, when satisfied, outputs a binary value
indicating the trigger has been satisfied, or outputs the name of
the field or other context information for which the trigger
condition was satisfied. Additionally or alternatively, the target
microservice may be connected to one or more other microservices
such that an alert is input to the other microservices. Other
microservices may perform responsive actions based on the above
techniques, including, but not limited to, deploying additional
resources, adjusting system configurations, and/or generating
GUIs.
[0170] Actions
[0171] In one or more embodiments, a plugged-in microservice
application may expose actions to the microservices manager. The
exposed actions may receive, as input, data or an identification of
a data object or location of data, that causes data to be moved
into a data cloud.
[0172] In one or more embodiments, the exposed actions may receive,
as input, a request to increase or decrease existing alert
thresholds. The input might identify existing in-application alert
thresholds and whether to increase or decrease, or delete the
threshold. Additionally or alternatively, the input might request
the microservice application to create new in-application alert
thresholds. The in-application alerts may trigger alerts to the
user while logged into the application, or may trigger alerts to
the user using default or user-selected alert mechanisms available
within the microservice application itself, rather than through
other applications plugged into the microservices manager.
[0173] In one or more embodiments, the microservice application may
generate and provide an output based on input that identifies,
locates, or provides historical data, and defines the extent or
scope of the requested output. The action, when triggered, causes
the microservice application to provide, store, or display the
output, for example, as a data model or as aggregate data that
describes a data model.
[0174] 9. Hardware Overview
[0175] According to one or more embodiments, the techniques
described herein are implemented by one or more special-purpose
computing devices. The special-purpose computing devices may be
hard-wired to perform the techniques, or may include digital
electronic devices such as one or more application-specific
integrated circuits (ASICs) or field programmable gate arrays
(FPGAs) that are persistently programmed to perform the techniques,
or may include one or more general purpose hardware processors
programmed to perform the techniques pursuant to program
instructions in firmware, memory, other storage, or a combination.
Such special-purpose computing devices may also combine custom
hard-wired logic, ASICs, or FPGAs with custom programming to
accomplish the techniques. The special-purpose computing devices
may be desktop computer systems, portable computer systems,
handheld devices, networking devices or any other device that
incorporates hard-wired and/or program logic to implement the
techniques.
[0176] For example, FIG. 11 is a block diagram that illustrates
computer system 1100 upon which one or more embodiments may be
implemented. Computer system 1100 includes bus 1102 or other
communication mechanism for communicating information, and hardware
processor 1104 coupled with bus 1102 for processing information.
Hardware processor 1104 may be, for example, a general purpose
microprocessor.
[0177] Computer system 1100 also includes main memory 1106, such as
a random access memory (RAM) or other dynamic storage device,
coupled to bus 1102 for storing information and instructions to be
executed by processor 1104. Main memory 1106 also may be used for
storing temporary variables or other intermediate information
during execution of instructions to be executed by processor 1104.
Such instructions, when stored in non-transitory storage media
accessible to processor 1104, render computer system 1100 into a
special-purpose machine that is customized to perform the
operations specified in the instructions.
[0178] Computer system 1100 further includes read only memory (ROM)
1108 or other static storage device coupled to bus 1102 for storing
static information and instructions for processor 1104. Storage
device 1110, such as a magnetic disk or optical disk, is provided
and coupled to bus 1102 for storing information and
instructions.
[0179] Computer system 1100 may be coupled via bus 1102 to display
1112, such as a cathode ray tube (CRT), liquid crystal display
(LCD), or light-emitting diode (LED), for displaying information to
a computer user. Input device 1114, which may include physical
and/or touchscreen based alphanumeric keys, is coupled to bus 1102
for communicating information and command selections to processor
1104. Another type of user input device is cursor control 1116,
such as a mouse, a trackball, or cursor direction keys for
communicating direction information and command selections to
processor 1104 and for controlling cursor movement on display 1112.
This input device typically has two degrees of freedom in two axes,
a first axis (e.g., x) and a second axis (e.g., y), that allows the
device to specify positions in a plane.
[0180] Computer system 1100 may implement the techniques described
herein using customized hard-wired logic, one or more ASICs or
FPGAs, firmware and/or program logic which in combination with the
computer system causes or programs computer system 1100 to be a
special-purpose machine. According to one embodiment, the
techniques herein are performed by computer system 1100 in response
to processor 1104 executing one or more sequences of one or more
instructions contained in main memory 1106. Such instructions may
be read into main memory 1106 from another storage medium, such as
storage device 1110. Execution of the sequences of instructions
contained in main memory 1106 causes processor 1104 to perform the
process steps described herein. In alternative embodiments,
hard-wired circuitry may be used in place of or in combination with
software instructions.
[0181] The term "storage media" as used herein refers to any
non-transitory media that store data and/or instructions that cause
a machine to operation in a specific fashion. Such storage media
may comprise non-volatile media and/or volatile media. Non-volatile
media includes, for example, optical or magnetic disks, such as
storage device 1110. Volatile media includes dynamic memory, such
as main memory 1106. Common forms of storage media include, for
example, a floppy disk, a flexible disk, hard disk, solid state
drive, magnetic tape, or any other magnetic data storage medium, a
CD-ROM, any other optical data storage medium, any physical medium
with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM,
NVRAM, any other memory chip or cartridge.
[0182] Storage media is distinct from but may be used in
conjunction with transmission media. Transmission media
participates in transferring information between storage media. For
example, transmission media includes coaxial cables, copper wire
and fiber optics, including the wires that comprise bus 1102.
Transmission media can also take the form of acoustic or light
waves, such as those generated during radio-wave and infra-red data
communications.
[0183] Various forms of media may be involved in carrying one or
more sequences of one or more instructions to processor 1104 for
execution. For example, the instructions may initially be carried
on a magnetic disk or solid state drive of a remote computer. The
remote computer can load the instructions into its dynamic memory
and send the instructions over a telephone line using a modem. A
modem local to computer system 1100 can receive the data on the
telephone line and use an infra-red transmitter to convert the data
to an infra-red signal. An infra-red detector can receive the data
carried in the infra-red signal and appropriate circuitry can place
the data on bus 1102. Bus 1102 carries the data to main memory
1106, from which processor 1104 retrieves and executes the
instructions. The instructions received by main memory 1106 may
optionally be stored on storage device 1110 either before or after
execution by processor 1104.
[0184] Computer system 1100 also includes a communication interface
1118 coupled to bus 1102. Communication interface 1118 provides a
two-way data communication coupling to a network link 1120 that is
connected to local network 1122. For example, communication
interface 1118 may be an integrated services digital network (ISDN)
card, cable modem, satellite modem, or a modem to provide a data
communication connection to a corresponding type of telephone line.
As another example, communication interface 1118 may be a local
area network (LAN) card to provide a data communication connection
to a compatible LAN. Wireless links may also be implemented. In any
such implementation, communication interface 1118 sends and
receives electrical, electromagnetic or optical signals that carry
digital data streams representing various types of information.
[0185] Network link 1120 typically provides data communication
through one or more networks to other data devices. For example,
network link 1120 may provide a connection through local network
1122 to host computer 1124 or to data equipment operated by
Internet Service Provider (ISP) 1126. ISP 1126 in turn provides
data communication services through the world wide packet data
communication network now commonly referred to as the "Internet"
1128. Local network 1122 and Internet 1128 both use electrical,
electromagnetic or optical signals that carry digital data streams.
The signals through the various networks and the signals on network
link 1120 and through communication interface 1118, which carry the
digital data to and from computer system 1100, are example forms of
transmission media.
[0186] Computer system 1100 can send messages and receive data,
including program code, through the network(s), network link 1120
and communication interface 1118. In the Internet example, server
1130 might transmit a requested code for an application program
through Internet 1128, ISP 1126, local network 1122 and
communication interface 1118.
[0187] The received code may be executed by processor 1104 as it is
received, and/or stored in storage device 1110, or other
non-volatile storage for later execution.
[0188] 10. Miscellaneous; Extensions
[0189] Embodiments are directed to a system with one or more
devices that include a hardware processor and that are configured
to perform any of the operations described herein and/or recited in
any of the claims below.
[0190] In an embodiment, a non-transitory computer readable storage
medium comprises instructions which, when executed by one or more
hardware processors, causes performance of any of the operations
described herein and/or recited in any of the claims.
[0191] Any combination of the features and functionalities
described herein may be used in accordance with one or more
embodiments. In the foregoing specification, embodiments have been
described with reference to numerous specific details that may vary
from implementation to implementation. The specification and
drawings are, accordingly, to be regarded in an illustrative rather
than a restrictive sense. The sole and exclusive indicator of the
scope of the invention, and what is intended by the applicants to
be the scope of the invention, is the literal and equivalent scope
of the set of claims that issue from this application, in the
specific form in which such claims issue, including any subsequent
correction.
* * * * *