U.S. patent application number 14/665173 was filed with the patent office on 2015-07-16 for automatic workload repository performance baselines.
The applicant listed for this patent is Oracle International Corporation. Invention is credited to John Beresniewicz, Michael Feng, Gary Ngai, Mark Ramacher, Graham Wood.
Application Number | 20150199257 14/665173 |
Document ID | / |
Family ID | 40564807 |
Filed Date | 2015-07-16 |
United States Patent
Application |
20150199257 |
Kind Code |
A1 |
Feng; Michael ; et
al. |
July 16, 2015 |
AUTOMATIC WORKLOAD REPOSITORY PERFORMANCE BASELINES
Abstract
Techniques that improve manageability of systems. Techniques are
provided for creating different types of baselines that are more
flexible and dynamic in nature. A future-based baseline may be
created defining a period of time, wherein at least a portion of
the period of time is in the future. A baseline may be created that
is a composite of multiple baselines. In general, baselines may be
specified having one or more periods of time that are either
contiguous or non-contiguous. A template for creating a set of
baselines based on a set of time periods may also be created, where
the template can be used to create a baseline for each of the set
of time periods. A moving window baseline may be created having an
associated time window that changes with passage of time, where
accordingly the data associated with the baseline may also
dynamically change with passage of time.
Inventors: |
Feng; Michael; (San Mateo,
CA) ; Ramacher; Mark; (San Carlos, CA) ; Ngai;
Gary; (Saratoga, CA) ; Wood; Graham; (El
Granada, CA) ; Beresniewicz; John; (Half Moon Bay,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Oracle International Corporation |
Redwood Shores |
CA |
US |
|
|
Family ID: |
40564807 |
Appl. No.: |
14/665173 |
Filed: |
March 23, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12111011 |
Apr 28, 2008 |
8990811 |
|
|
14665173 |
|
|
|
|
60981373 |
Oct 19, 2007 |
|
|
|
Current U.S.
Class: |
714/37 |
Current CPC
Class: |
G06F 11/3616 20130101;
G06F 11/3409 20130101; G06F 11/3428 20130101; G06F 11/3495
20130101 |
International
Class: |
G06F 11/34 20060101
G06F011/34 |
Claims
1. A non-transitory computer-readable medium storing a plurality of
instructions for controlling a data processor to create a composite
baseline, the plurality of instructions comprising: instructions
that cause the data processor to receive information identifying a
plurality of baselines; and instructions that cause the data
processor to create the composite baseline based upon the
information.
2. The non-transitory computer-readable medium of claim 1, the
plurality of instructions further comprising instructions that
cause the data processor to determine metrics data that are
associated with the plurality of baselines and instructions that
cause the data processor to associate the metrics data with the
composite baseline.
3. The non-transitory computer-readable medium of claim 2, the
plurality of instructions further comprising: instructions that
cause the data processor to receive time information defining a
period of time for the composite baseline; and instructions that
cause the data processor to filter the metrics data based on the
time information.
4. The non-transitory computer-readable medium of claim 2, the
plurality of instructions further comprising: instructions that
cause the data processor to receive a criterion for the composite
baseline; and instructions that cause the data processor to filter
the metrics data based on the criterion.
5. The non-transitory computer-readable medium of claim 2, the
plurality of instructions further comprising: instructions that
cause the data processor to define a time window; instructions that
cause the data processor to associate the time window with a first
period of time; instructions that cause the data processor to
associate a first metrics data based on the first period of time;
instructions that cause the data processor to associate the time
window with a second period of time, the second period of time
including a period of time not included in the first period of
time; and instructions that cause the data processor to associate a
second metrics data based on the second period of time.
6. The non-transitory computer-readable medium of claim 1 wherein
the time information defines a window corresponding to a period of
time of "N" days, where the "N" days is automatically updated once
every predetermined time interval as time progresses.
7. A method for creating a composite baseline, the method
comprising: receiving, by a computer, information identifying a
plurality of baselines; and creating, by the computer, the
composite baseline based upon the information.
8. The method of claim 7, further comprising: determining metrics
data that are associated with the plurality of baselines; and
associating the metrics data with the composite baseline.
9. The method of claim 8, further comprising: receiving time
information defining a period of time for the composite baseline;
and filtering the metrics data based on the time information.
10. The method of claim 8, further comprising: receiving a
criterion for the composite baseline; and filtering the metrics
data based on the criterion.
11. The method of claim 8, further comprising: defining a time
window; associating the time window with a first period of time;
associating a first metrics data based on the first period of time;
associating the time window with a second period of time, the
second period of time including a period of time not included in
the first period of time; and associating a second metrics data
based on the second period of time.
12. The method of claim 7 wherein the time information defines a
window corresponding to a period of time of "N" days, where the "N"
days is automatically updated once every predetermined time
interval as time progresses.
13. A system for creating a composite baseline, the system
comprising: a hardware processor; and a memory storing a plurality
of instructions that cause the processor to: receive information
identifying a plurality of baselines; and create the composite
baseline based upon the information.
14. The system of claim 13, the plurality of instructions further
comprising instructions that cause the data processor to determine
metrics data that are associated with the plurality of baselines
and instructions that cause the data processor to associate the
metrics data with the composite baseline.
15. The system of claim 14, the plurality of instructions further
comprising: instructions that cause the data processor to receive
time information defining a period of time for the composite
baseline; and instructions that cause the data processor to filter
the metrics data based on the time information.
16. The system of claim 14, the plurality of instructions further
comprising: instructions that cause the data processor to receive a
criterion for the composite baseline; and instructions that cause
the data processor to filter the metrics data based on the
criterion.
17. The system of claim 14, the plurality of instructions further
comprising: instructions that cause the data processor to define a
time window; instructions that cause the data processor to
associate the time window with a first period of time; instructions
that cause the data processor to associate a first metrics data
based on the first period of time; instructions that cause the data
processor to associate the time window with a second period of
time, the second period of time including a period of time not
included in the first period of time; and instructions that cause
the data processor to associate a second metrics data based on the
second period of time.
18. The system of claim 13 wherein the time information defines a
window corresponding to a period of time of "N" days, where the "N"
days is automatically updated once every predetermined time
interval as time progresses.
Description
CROSS-REFERENCES TO RELATED APPLICATIONS
[0001] This application is a divisional of application Ser. No.
12/111,011, filed Apr. 28, 2008, which claims the benefit and
priority under 35 U.S.C. 119(e) of Application No. 60/981,373,
filed Oct. 19, 2007, the entire contents of which are herein
incorporated by reference for all purposes.
BACKGROUND OF THE INVENTION
[0002] The present invention relates to creating baselines, and
more particularly to techniques that enable creation and use of
dynamic baselines that provide enhanced capabilities.
[0003] An essential task of an administrator of any system, such as
a software system, is to be able to detect, diagnose, and tune
performance problems that may occur in the system. These systems
may include software systems, computer server systems, computer
network systems, telephony communication systems, etc. As an
example, for a product instance of a software system such as a
database system (for example, database systems provided by
Oracle.TM. Corporation of Redwood Shores, Calif.), it is very
useful to know how the product instance is performing at a certain
time compared with another time.
[0004] In order to carry out such comparisons, users need to
capture and store performance metrics data for various interesting
time periods. Having access to baselines of performance metrics
data allows users to access and compare a current set of
performance metrics data against other sets of performance metrics
data captured at other times. A baseline is a construct for
specifying a time period of interest in which performance metrics
data are captured over the time period. Conventionally, a baseline
has been restricted to specify time periods that are static in
nature. Further, baselines have been restricted to time periods in
the past. These limitations have proven to be quite restrictive.
Comparisons of current performance metrics against static and often
outdated performance baselines can dramatically reduce an
administrator's ability to detect, diagnose, and tune performance
problems that may occur in a software system.
[0005] In light of the above, it is desirable to provide a more
flexible and adaptive approach to defining, capturing, and
maintaining performance metrics baselines.
BRIEF SUMMARY OF THE INVENTION
[0006] Embodiments of the present invention provides techniques
that improve manageability of systems. Techniques are provided for
creating different types of baselines that are more flexible and
dynamic in nature. According to an embodiment, techniques for
creating a future-based baseline is provided. The techniques
provide for receiving time information defining a period of time
for a baseline where a portion of the period of time is temporally
in the future. A baseline may be created based on such time
information. Once the baseline is created, metrics data to be
associated with the baseline may then be determined based upon the
time information.
[0007] Depending on the embodiment, the time information may
further define an additional period of time, where the period of
time and the additional period of time may be temporally contiguous
or non-contiguous. The techniques may further provide for receiving
a criterion for the baseline. In such cases, the metrics data to be
associated with the baseline may further be filtered to satisfy the
criterion.
[0008] According to an embodiment, techniques for creating a
composite baseline is provided. The techniques may provide for
receiving information that identify a group of baselines. Based
upon the information a composite baseline can be created. Once the
baseline is defined, metrics data to be associated with the
plurality of baselines may then be determined based in part upon
the time information.
[0009] Depending on the specific embodiment, time information
defining a period of time can be further received. The metrics data
to be associated with the composite baseline may then be further
filtered to satisfy the time information. According to an
embodiment, a criterion for the composite baseline may also be
further received. The metrics data that are to be associated with
the composite baseline may then be further filtered to satisfy the
criterion.
[0010] According to an embodiment, techniques for creating
baselines using baseline templates are provided. As a first step,
baseline templates that can be used to define how baselines are to
be created are defined and created. The next step may involves
receiving time information relating to a baseline template. Based
on the time information, a set of time periods can be derived from
the baseline template. The template defines a method for creating
baselines. After receiving the time information from the baseline
template, a baseline may then be created for each of the set of
time periods. Next, metrics data to be associated with each
baseline may be determined for each of the plurality of time
periods.
[0011] A moving window baseline may be created having an associated
time window that changes with passage of time; accordingly the data
associated with the baseline may also dynamically change with
passage of time. A future-based baseline may be created having an
associated time window, wherein at least a portion of the time
window is in the future at the time when the baseline is defined. A
baseline may be created that is a composite of multiple baselines.
In general, baselines may have one or more time periods associated
with them. The time periods may be either contiguous or
non-contiguous. A template for creating a set of baselines based on
a set of time periods may also be defined and used.
[0012] According to an embodiment, techniques for creating a moving
windows baseline in combination with any of several techniques
described above is provided. The techniques may provide for
receiving time information defining a time window. The time window
may be defined to correspond to different periods of time as time
progresses. After the time window is specified, a baseline based
upon the time information may be created. According to an
embodiment, the time window may specify a first period of time. A
first metrics data to be associated with the baseline is then
determined based upon the first period of time.
[0013] As time progresses, the time window may specify a second
period of time. A second metrics data to be associated with the
baseline is then determined based upon the second period of time.
Depending on how much time has progressed, the second metrics data
may include metrics data that is not included in the first metrics
data. The techniques may further include receiving a criterion for
the baseline. Metrics data to be associated with the baseline may
then be further filtered to satisfy the criterion.
[0014] The time window of a baseline according to an embodiment may
be defined to correspond to a period of time of "N" days. As time
progresses, the "N" days is automatically updated. The frequency by
which the time window is updated can be predetermined and
programmed. for example, the time window may be automatically
updated once every predetermined time interval as time progresses.
In general, an additional time information may be included to
define an additional time window. According to the embodiment, the
time window and the additional time window may be temporally
contiguous or non-contiguous, depending on the specific
embodiment.
[0015] The foregoing, together with other features, embodiments,
and advantages of the embodiments of the present invention, will
become more apparent when referring to the following specification,
claims, and accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] FIG. 1 is a simplified illustration of a system for creating
a baseline according to an embodiment of the invention;
[0017] FIG. 2A is a simplified illustration of the process to
update a moving time window baseline as time progresses according
to an embodiment of the invention;
[0018] FIG. 2B is a simplified diagram of a technique for creating
a moving baseline;
[0019] FIG. 3A is a simplified diagram of a future-based baseline
according to another embodiment of the invention;
[0020] FIG. 3B is a simplified diagram of a technique for creating
a future-based baseline;
[0021] FIG. 4A is a simplified illustration of a composite baseline
adapted to group together a set of baselines according to an
embodiment of the invention;
[0022] FIG. 4B is a simplified diagram of a technique for creating
a composite baseline;
[0023] FIGS. 5A and 5B are simplified diagrams of a technique for
mapping performance metrics data to a time period of a
baseline;
[0024] FIG. 6 is a simplified diagram depicting a technique for
creating baselines using baseline templates according to an
embodiment of the invention; and
[0025] FIG. 7 is a simplified block diagram of a computer system
that may be used to practice an embodiment of the various
inventions described in this application.
DETAILED DESCRIPTION OF THE INVENTION
[0026] In the following description, for the purposes of
explanation, specific details are set forth in order to provide a
thorough understanding of the invention. However, it will be
apparent that the invention may be practiced without these specific
details.
[0027] Embodiments of the present invention provide techniques for
defining and using various novel types of baselines. A baseline is
a construct for specifying a time period of interest. A baseline is
defined by an identifier (e.g., a baseline name) and a period of
time. A set of performance metrics data corresponding to the period
of time captured for a monitored system may then be associated with
the baseline. In general, performance metrics data may comprise
performance metrics captured for a system such as a software
instance, a network, a group of servers, and the like. For purposes
of this application, the system for which performance metrics data
is captured is referred to as a "monitored system."
[0028] Various different types of performance metrics may be
captured for a monitored system. Examples of performance metrics
include CPU load, I/O traffic, available bandwidth, available
storage. etc. The performance metrics data for a monitored system
may be captured on a continual periodic basis (e.g., every second,
every minute, at a certain time every day, once a week, etc.). The
data may also be captured in response to an event, for example,
when there is a critical error in the system, when the response
time of the monitored system drops below a threshold, etc.
[0029] Baselines may be used for various different purposes such as
for detecting, diagnosing, and tuning performance problems that may
occur in any of several systems being monitored. For example,
baselines may be used as a basis for comparisons of data
corresponding to different time periods. In one embodiment, a
baseline provides a tool for a system administrator to better
monitor and manage a monitored system. Baselines may thus be used
for monitoring, tuning, and managing various software system
products.
[0030] Using baselines, performance metrics data captured at some
time may be compared with a current performance metrics measurement
to determine a current state of a system. For example, an
administrator may set up a baseline to be associated with
performance metrics data and performance metrics statistics
captured over a period of time of normal operation ("good period")
and another baseline to be associated with performance metrics data
and performance metrics statics captured over a period with
performance problems (the "trouble period"). Using baselines, the
administer may compare a current set of performance metrics data
against the normal baseline and the trouble baseline to better
monitor, detect, and diagnose a potential current problem.
[0031] In general, many uses exist for baselines according to
various embodiments of the invention. According to an embodiment,
baselines can be created for diagnosing and tuning performance
problems related to monitored systems. Baselines can be created for
understanding relative values for statistics. Baselines can also be
created to detect unexpected workload changes. Baselines can be
used for analyzing the performance impact of changes to monitored
systems such as application changes or version upgrades or
monitoring and/or dealing with normal/expected system performance
drifts due to business/demand increases. Other uses include
monitoring and dealing with normal/expected system performance peak
uses arising from load spikes such as those associated with daily
trends (e.g. 9-10 am peak e-mail checking usage) or more seasonal
variations (e.g., holiday shopping load on web server). Baselines
can be created to detect and manage pending performance problems.
Baselines can be created to more dynamically carry out capacity
planning.
[0032] FIG. 1 is a simplified block diagram of a system 1000 for
creating a baseline according to an embodiment of the invention. As
depicted in FIG. 1, system 1000 includes a monitored system 1010, a
baseline repository 1020, a metrics repository 1030, a baseline
engine 1050, a tools module 1060, and a user system 1070.
[0033] Monitored system 1010 represents a system that is monitored
and for which performance metrics data is captured. Monitored
system 1010 may be a software system, a computer server system, a
computer network system, a telephony communication system, etc.
According to an embodiment, monitored system 1010 is a product
instance of a software system such as a database system (e.g., a
data system instance provided by Oracle Corporation.TM. of Redwood
Shores, Calif.).
[0034] Metrics repository 1030 is used to store performance metrics
data gathered from monitored system 1010. Various different types
of performance metrics data may be captured for monitored system
1010. The performance metrics data gathered may be captured on a
periodic basis (e.g., every minute, every day, etc.) or may be
captured upon the occurrence of one or more events. According to an
embodiment, a group of performance metrics data captured at a time
can be defined to make up a snapshot of monitored system 1010. As
time progresses, a set of snapshots, each of which can be uniquely
defined by a unique id and be retrieved by the unique id, can be
captured and stored periodically for metrics repository 1030.
[0035] Baseline repository 1020 is a repository system adapted to
store data related to baselines. As previously indicated, a
baseline can be generally defined by a baseline name and a time
period associated with the baseline. Baseline repository 1020 may
be configured to store baseline definitions 1040 and baseline data
1045. Baseline definitions 1040 include information that defines
one or more baselines. Baseline definitions 1040 may also include
other information for a baseline such as an expiry time, if any,
associated with the baseline, information identifying a creator of
the baseline, information identifying a time when the baseline was
created, and the like. The baseline definitions stored may include
definitions for any of various types of baselines, including but
not limited to moving windows baselines, composite baselines,
future-based baselines, non-contiguous baselines, and other types
of baselines (to be described in more detail below).
[0036] Baseline data 1045 represents performance metrics data that
is associated with baselines defined by baseline definitions 1040.
Examples of performance metrics include CPU usage, network usage,
error counts, page fault counts, etc. In one embodiment, for each
baseline, performance metrics data is determined dynamically for
the baseline based upon the time period defined for the baseline.
For example, if a baseline specifies that performance metrics data
from the preceding five days are associated with the baseline, a
query for the performance metrics data will result in a dynamic
search of metrics repository 1030 for all performance metrics data
gathered in the last five days.
[0037] According to another embodiment, performance metrics data
may be stored as part of baseline data 1045. For example, if a
baseline specifies that performance metrics data from the preceding
five days are associated with the baseline, a query for the
performance metrics data will retrieve result in a direct fetching
of the performance metrics data as part of the baseline data 1045.
In an alternative embodiment, baseline data 1045 includes pointers
that identifies performance metrics data stored in metrics
repository 1030 to be associated with the baseline. For example, if
a baseline specifies that performance metrics data from the
preceding five days are associated with the baseline, a query for
the performance metrics data will retrieve in a fetching of the
performance metrics data from metrics repository 1030 through the
pointers stored as part of the baseline data 1045.
[0038] A user may define a baseline using user system 1070. User
system 1070 may be adapted to provide an interface that enables a
user to define, access, and manipulate baseline-related data and/or
performance metrics. Embodiments of various interfaces provided by
user system 1070 may include graphical user interfaces, command
line interfaces, remote access systems, web interfaces, phone
accessed systems, etc. User system 1070 may be provide an interface
to tools module 1060, for example, to perform a number of analysis
on baselines using one or more tools. In general, user system 1070
may provide an interface for imputing information to system 1000
and for displaying information from system 1000.
[0039] According to an embodiment, baseline engine 1050 is
configured to perform processing related to baselines. For example,
baseline engine 1050 may facilitate receiving baseline definitions
from user system 1070 and storing them in baseline repository 1020.
Baseline engine 1050 may also be configured to determine and
associate performance metrics data from metrics repository 1030 to
be associated with each defined baseline.
[0040] Tools module 1060 may provide one or more tools for
performing analysis related to baselines. Examples of tools include
tools for outputting information related to baselines, tools for
comparing performance metrics data associated with two or more
baselines, tools for generating reports, statistical analysis
tools, and alerting and messaging tools for generating alerts or
sending messages based upon performance metrics data stored in
repository 1030 and/or baselines-related data.
[0041] The system depicted in FIG. 1 is merely illustrative of an
embodiment of the present invention. Alternative embodiments of the
system as illustrated may differ from those depicted in FIG. 1 but
will be understood by one of ordinary skill in the art to be
encompassed by the current invention. The various components
depicted in FIG. 1 may be implemented in software (e.g., program,
code, modules, instructions executed by a processor), in hardware,
or combinations thereof.
[0042] According to an embodiment of the present invention,
techniques for creating and using a moving window baseline is
provided. A moving window is characterized by a time period that
dynamically changes as time progresses. Since the performance
metrics data associated with a baseline depends upon the time
period for the baseline, the performance metrics data associated
with a moving baseline change as time progresses. An example of a
moving window baseline is a baseline characterized by a
customizable trailing period of time (e.g., last N months, N weeks,
N days, N hours, etc.).
[0043] FIG. 2A is a simplified illustration showing how a moving
time window dynamically changes as time progresses according to an
embodiment of the present invention. An example of a moving window
baseline with a trailing 2-day time window is depicted. At Time 1,
when day 3 is the current time, the time window associated with the
moving window baseline is a 2-day time window 2010 starting on the
first day and ending on the second day. Accordingly, at Time 1,
performance metrics data corresponding to the first day and the
second day is associated with the baseline. The time window shifts
with passage of each day. For example, at a Time 2, when day 4 is
now the current time, the updated 2-day time window 2020 associated
with the baseline now starts on the second day and ends on the
third day. At Time 2, day 1 is now no longer covered by the
baseline and a new day four has been added to the baseline.
Accordingly, at Time 2, performance metrics data corresponding to
the second day and the third day is associated with the baseline.
As time further progresses, at Time 3 when day 5 is the current
time, the 2-day interval for the baseline now starts on the third
day and ends on the fourth day. Accordingly, at Time 3, performance
metrics data corresponding to the third day and the fourth day is
associated with the baseline.
[0044] In the manner described above, the time period associated
with a moving window baseline is updated with passage of time.
Similarly, the performance metrics data associated with the moving
window baseline is also continually updated as time progresses--and
as the moving time window is updated.
[0045] The above time window has been shown to increment in 1 day
intervals. However, in general, the time window can be updated in
any time increments. According to an embodiment, the time window is
updated once every day. According to other embodiments, the time
window can be updated in any of other time increments, including
once every hour, once every minute, etc., depending on the specific
embodiment.
[0046] Moving window baselines may be used for any of various
purposes. In one embodiment, moving window baselines is used for
comparing a current performance state of a system against a
baseline performance state of the system from a recent past. The
baseline depicted in FIG. 2A may be adapted to be used as a
constantly updated set of recent snapshots of a system's
performance metric. Such a set of snapshots would be useful as a
baseline to which to compare a system's current performance
metrics.
[0047] FIG. 2B is a simplified diagram of a method for creating and
associating information with a moving window baseline according to
an embodiment of the invention. The processing depicted in FIG. 2B
may be performed by hardware modules, software modules (e.g.,
program, code, instructions executed by a processor), or
combinations thereof. In one embodiment, the processing may be
performed by baseline engine 1050 depicted in FIG. 1.
[0048] As shown in FIG. 2B, information is received identifying a
name for the baseline (step 2110). Time information received for
defining a moving time window include a period of time that can be
updated as time progresses. The moving time window is associated
with the baseline (step 2120). For example, the moving time window
may define a window specifying the last "N" days. A baseline is
created based upon information received in steps 2110 and 2120
(step 2125). Based upon the time information, performance metrics
data to be associated with the baseline is determined (step 2130).
In one embodiment, as part of 2130, the performance metrics data to
be associated with the baseline may be extracted from the metrics
repository and stored in the baseline repository as part of
baseline data. In an alternative embodiment, the performance
metrics data may be associated with the baseline by storing
information (e.g., pointers to the performance metrics data in the
metrics repository) identifying the data to be associated with the
baseline.
[0049] According to an embodiment, the determination of performance
metrics data to be associated with the baseline may take place soon
after time information relating to the time period is received. In
an alternative embodiment, the determination of performance metrics
data to be associated may also take place later dynamically in
response to a query for performance metrics data associated with
the baseline.
[0050] As time progresses, a check may be made to determine if the
time window associated with the baseline needs to be updated (step
2140). Various techniques can be used to determine whether time
period associated with the baseline needs to be updated. According
to one technique, a timer may be periodically polled to determine
whether sufficient time has passed to necessitate an update of the
time window associated with the baseline. According to another
technique, the assessment can be based on an event-driven
technique, where the baseline system is notified when a certain
time has been reached and the time window is to be updated. Other
techniques may also be used.
[0051] If it is determined that the time period needs to be
updated, the time window for the baseline is updated to cover a new
period of time (step 2150). The updating in 2150 may include
updating the start time and end time of the time window associated
with the baseline. Using the baseline depicted in FIG. 2A as an
example, upon determining that it is Time 2, the time window with a
prior start time of Day 1 and a prior end time of Day 2 is updated
such that the new start time is Day 2 and the new end time is Day
3.
[0052] Performance metrics data to be associated with the baseline
may be based upon the updated time period (step 2160). In one
embodiment, as part of 2160, the performance metrics data is
associated with the baseline, where the actual data is determined
based upon the updated time window. The performance metrics data
may be extracted from the metrics repository and stored in the
baseline repository as part of baseline data. In an alternative
embodiment, the performance metrics data determined in 2160 may be
associated with the baseline by storing information (e.g., pointers
to the performance metrics data in the metrics repository)
identifying the data to be associated with the baseline.
[0053] In the manner described above, the time window associated
with a moving window baseline can be updated with passage of time.
In one example, performance metrics data associated with the
baseline is dynamically determined and updated as the time window
is updated. As a result, the performance metrics data associated
with a moving window baseline is automatically and dynamically
updated with passage of time.
[0054] According to an embodiment of the present invention, a
future-based baseline may be created that is characterized by a
time period that includes at least a portion of time in the future.
For a future-based baseline, at least the end time defining the
time period for the baseline is in the future from a present time.
The start time defining the time period may also be in the future,
in which case the entire time period associated with the baseline
is in the future. For example, if the present time is Feb. 1, 2008,
a future-based baseline may be created such that the time period
for the baseline is the month of March 2008.
[0055] FIG. 3A is a simplified diagram depicting a future-based
baseline 3000 according to another embodiment of the invention. As
depicted in FIG. 3A, at Time 1 when the present time is December 1,
future-based baseline 3000 may be created having an associated time
window defined to be December 15 through December 25 (the Christmas
shopping season). Embodiments of the present invention thus provide
the ability to create a baseline wherein at least a portion of the
time window associated with the baseline is in the future. A
baseline may thus be defined even though the performance metrics
data to be associated with the baseline has not yet been
captured.
[0056] As depicted in FIG. 3A, as time progresses, a part or all of
the time window associated with a future-based baseline may start
falling into the past. For example, at Time 2 when the present time
is sometime between December 15 and December 25, a portion of the
time window associated with baseline 3000 is in the past and a
portion is in the future. As time further progresses, at Time 3,
baseline 3000 is associated with a time window where all of the
time window is in the past.
[0057] A future-based baseline may have several uses. In one
embodiment, a future-based baseline may allow a user to specify
baseline whose associated time period is based on a future
condition or event of a monitored system. For example, an online
merchant may in advance define a baseline (e.g., baseline 3000
depicted in FIG. 3A) for Christmas shopping time when the online
merchant's system is likely to come under heavy load. A
future-based baseline may also be defined for a future time period
when a CPU load of a monitored system reaches above 90%. In this
scenario, the next time the monitored system reaches above 90%
load, a time window is defined to be associated with this time
interval. Similarly, a baseline can be programmed to define a
future time period based on other performance metrics, such as when
an I/O load exceeds a certain threshold or when the error count
exceeds a certain tolerance. Time windows may also be defined to be
based off certain events.
[0058] FIG. 3B is a simplified diagram depicting a method for
creating a future-based baseline according to an embodiment of the
invention. The processing depicted in FIG. 3B may be performed by
hardware modules, software modules (e.g., program, code,
instructions executed by a processor), or combinations thereof. In
one embodiment, the processing may be performed by baseline engine
1050 depicted in FIG. 1.
[0059] As depicted in FIG. 3B, information is received identifying
an identifier (e.g., a name) for a baseline (step 3110). Time
information that specifies a time period for the baseline is then
received, wherein at least a portion of the time period is in the
future (step 3120). According to an embodiment, the time period for
a future-based baseline may be a static time window (e.g., December
15 to December 25, with the present time some time prior to
December 15), a moving time window (e.g., a window between the
present and 5 days into the future), or even a group of
non-contiguous time windows (e.g., the first day of each week for
the next four weeks)--so long as a portion of the time period
covers a time period in the future. The information defining a
future-based baseline may be stored as part of baselines
definitions 1040 depicted in FIG. 1.
[0060] A baseline is created based upon information received in
steps 3110 and 3120 (step 3125). In one embodiment, performance
metrics data to be associated for a future-based baseline can be
determined after at least a portion of the future-based baseline
time window falls in the past (step 3130). Accordingly, in order to
determine performance metrics data to be associated with a
future-based baseline, a portion of the time window of the baseline
that is in the past is first determined. That data is then
associated with the baseline. As time progresses, the baseline is
updated as performance metrics data in the future is determined and
associated with the baseline.
[0061] In general, a baseline, including a future-based baseline,
is typically associated with one contiguous interval of time--i.e.
one time window. However, a baseline can also be associated with
multiple windows of times that are not contiguous. According to an
embodiment of the present invention, a baseline may be defined such
that several non-contiguous windows of times are associated with
the baseline. Such a baseline may be referred to as a
"non-contiguous" baseline. For example, a baseline may be created
with a time period defined as "9 AM-12 PM every weekday of the
current week." Assuming that the present time is a Wednesday
afternoon, the above-specified time period defines a group of 7
non-contiguous time windows: Sunday 9 AM-12 PM, Saturday 9 AM-12
PM, Friday 9 AM-12 PM, and Thursday 9 AM-12 PM being in the future,
and Wednesday 9 AM-12 PM, Tuesday 9 AM-12, and Monday 9 AM-12 being
the past. Another example of a non-contiguous time baseline is a
baseline associated with a time period defined by all holiday
shopping seasons from the current year. In this case, the time
period might define a period around Thanksgivings and a period
around Christmas. In summary, the non-contiguous time windows
associated with a non-contiguous baseline may lie in the past, in
the future, or lie both in the past and the future.
[0062] According to an embodiment of the present invention, a
baseline may be created that is a composite of multiple baselines.
This baseline may be referred to as a "composite" baseline. A
composite baseline is thus a baseline that comprises other
baselines. According to an embodiment, the baselines associated
together by a composite baseline could be of different types
including moving window baselines, future-based baselines,
non-contiguous time baselines, and even other composite
baselines.
[0063] FIG. 4A is a simplified illustration of a composite baseline
4000 according to an embodiment of the invention. Baseline 4000 is
a composite of a first baseline 4010, a second baseline 4020, and a
third baseline 4030. As depicted in FIG. 4A, second baseline 4020
is also a composite baseline, which is further associated with a
fourth baseline 4040 and a fifth baseline 4050.
[0064] According to an embodiment, a composite baseline such as
baseline 4000 may be used to group together a set of baselines.
Once performance data is associated with a composite baseline, the
composite baseline may be used and analyzed just like any ordinary
baseline. Analysis of the merged time periods and the combined
performance metrics data can be carried out like any other time
period and groups of performance metrics data.
[0065] According to one embodiment, an envelop time period may be
associated with the composite baseline. The time period of the
baseline is determined from the merger of the time periods
associated with the set of baselines. Before being associated with
the composite baseline, the merged period is processed where
portions that fall outside the envelop time period are deleted. The
performance metrics data associated with the set of baselines are
merged and associated with the composite baseline.
[0066] FIG. 4B is a simplified diagram of a method for creating a
composite baseline and determining performance metrics data to be
associated with the composite baseline according to an embodiment
of the present invention. The processing depicted in FIG. 4B may be
performed by hardware modules, software modules (e.g., program,
code, instructions executed by a processor), or combinations
thereof. In one embodiment, the processing may be performed by
baseline engine 1050 depicted in FIG. 1.
[0067] As depicted in FIG. 4B, name information may be received for
identifying a composite baseline (step 4110). Information
identifying a group of baselines to be associated with the
composite baseline may be received (step 4120). A baseline is
created based upon information received in steps 4110 and 4120
(step 4125). In one embodiment, user system 1070 depicted in FIG. 1
may provide an interface that allows a user to associate one or
more baselines with the composite baseline. A composite baseline
may be created based upon the information received in 4110 and 4120
(step 4130).
[0068] Processing may then be performed to determine the
performance metrics data to be associated with the composite
baseline. According to an embodiment, performance metrics data for
the composite baseline may be determined by associating performance
metrics data corresponding to each baseline with the composite
baseline (step 4140). The performance metrics data associated with
the composite baseline thus represents a collection of performance
metrics data based on a grouping of baselines. If the performance
metrics data for a baseline associated with the composite baseline
needs to be updated as time progresses (e.g., for a moving window
baseline), then the performance metrics data associated for the
composite baseline may also be updated as time progresses.
[0069] According to the embodiment, the determination of
performance metrics data to be associated with the composite
baseline may take place soon after information relating to the
group of baselines to be associated has been received. According to
another embodiment, the determination of performance metrics data
to be associated may take place later dynamically in response to a
query for performance metrics data associated with the
baseline.
[0070] In some composite baselines, no explicit time period may be
defined for the composite baseline. Instead, the time period for
the composite baseline may be derived from combining the time
periods defined for the baselines associated with the composite
baseline. In an alternative embodiment, a separate time period may
be defined for a composite baseline. For such as composite
baseline, the performance metrics data associated with the baseline
must further satisfy the specific time period.
[0071] In the case where explicit time periods may be defined for
the composite baseline, a method for creating a composite baseline
may include the following variation. Information is received
indicating a name for a composite baseline. Information is received
identifying a set of one or more baselines to be associated with
the composite baseline. In addition, information is received
identifying a time period for the composite baseline. A composite
baseline is then created based upon all the information
received.
[0072] Processing may then be performed, based on all the
information received, to determine performance metrics data to be
associated. According to an embodiment, the performance metrics
data to be associated is first determined by obtaining performance
metrics data associated with the set of one or more baselines.
[0073] The performance metrics data should satisfy the explicit
time constraints of the composite baseline. For example, if a
composite baseline is associated with or comprises two baselines,
one with a time window of between 8 AM-10 AM and the other with a
time window of between 12 PM-1 PM, and if an explicit time period
of 8:30 AM-12:30 PM is specifically defined for the composite
baseline, the performance metrics data to be associated with the
composite baseline will be a subset of the performance metrics data
associated with the time windows of the two baselines subject to
the explicit time period. More specifically, the performance
metrics data will be the performance metrics data combined from the
two baselines but with performance metrics data corresponding to 8
AM-8:30 AM and performance metrics corresponding to 12:30 PM-1 PM
removed since they fall outside the explicit time period of 8:30
AM-12:30 PM. Consequently, only performance metrics data falling
within the time period defined by the composite baseline is
associated with the composite baseline.
[0074] In general, according to an embodiment, the performance
metrics data to be associated with a baseline of any type may
determined based on one or more time windows associated with the
baseline. Different techniques may be used to map performance
metrics data to a time period. In one embodiment, a time period is
characterized by a start time t.sub.s and an end time t.sub.e. For
example, for baseline 3000 depicted in FIG. 3A, t.sub.s is December
15 and t.sub.e is December 25. The performance metrics data that is
captured between t.sub.s and t.sub.e of the time window is then
mapped to the baseline associated with time period. For baseline
3000, performance metrics data captured between December 15 and
December 25 can be mapped to and associated with the baseline with
a time period of between December 15 and December 25.
[0075] According to an embodiment, a method to mapping performance
metrics data is shown in FIG. 5A and FIG. 5B. According to the
embodiment, a series of snapshots are captured and stored on a
continual and regular basis. Each of a series of snapshots can be
uniquely identifiable by a snapshot id. When a baseline defines one
or more time periods for monitoring, a series of snapshots,
identified by their snapshot ids, may then be mapped to the one or
more time periods.
[0076] According to an embodiment, the technique for mapping
snapshots to a time period includes determining a time period for a
baseline (step S110). After the time period is determined, a
starting time T.sub.s and an ending time T.sub.e for the time
period is determined (step S120). An attempt is carried out to find
a snapshot s.sub.s within a predetermined time interval of T.sub.s
(step S130). If multiple snapshots are found within the
predetermined time interval, then one snapshot closest to the
starting may be selected. An attempt is also carried out to find a
snapshot s.sub.e within a predetermined time interval of T.sub.e
(step S140). If multiple snapshots are found within the
predetermined time interval, then one snapshot closest to the
ending time may be selected.
[0077] According to an embodiment, a determination is next made
regarding whether both the first and second snapshots are found
(step S150). If both are found, then a determination of all
snapshots between snapshot s.sub.s and snapshot s.sub.e is next
carried out (step S160). The set of snapshots including snapshot
s.sub.s and snapshot s.sub.e and all snapshots captured between
snapshot s.sub.s and snapshot s.sub.e are then associated with the
time period (step S170).
[0078] If one or both are not found, then an attempt is made to
find four versions of the snapshots around the starting time and
the ending time. A determination is made whether s.sub.s is found
(step S200). If it is found, a snapshot s.sub.1 and a snapshot
s.sub.2 are to be s.sub.s (step S210). If s.sub.s is not found, a
determination for a snapshot s.sub.1 and a snapshot s.sub.2 is
carried out to by searching forward for a snapshot closest to
T.sub.s and backward around T.sub.s a snapshot closest to T.sub.s
(step S220). A determination is next made whether s.sub.e is found
(step S230). If it is found, a snapshot e.sub.1 and a snapshot
e.sub.2 are set to be s.sub.e (step S240). If s.sub.e is not found,
a determination for a snapshot e.sub.1 and a snapshot e.sub.2 is
carried out to by searching forward a snapshot closest to T.sub.e
and backward a snapshot closest to T.sub.e (step S250).
[0079] Next, the four sets of snapshots s.sub.1, s.sub.2, e.sub.1,
and e.sub.2 and paired up four ways to select the pair that bound a
time interval that has the most overlap with the time period,
setting the pair to be (s.sub.s, s.sub.e) (step S260). After
s.sub.s, s.sub.e have been set, then a determination of all
snapshots between snapshot s.sub.s and snapshot s.sub.e is next
carried out (step S270). The set of snapshots including snapshot
s.sub.s and snapshot s.sub.e and all snapshots captured in between
are then associated with the time period (step S280).
[0080] In general, according to an embodiment, filter criteria may
also be specified for any baseline. In the embodiment, performance
metrics data associated with the baseline must further satisfy the
filter criteria specified for the baseline is associated with the
baseline. A filter criterion may be defined based on any attribute
or value of the performance metrics data captured for a monitored
system for which the baseline is defined. For example, filter
criteria may be related to CPU load (e.g., data when the CPU load
exceed a threshold), storage capacity, etc. For example, a criteria
specifying a 90% CPU Load for last Monday will associate with a
baseline a group of performance metrics data captured last Monday
when the CPU Load is 90% or higher.
[0081] According to an embodiment, an expiration criterion may be
associated with any baseline. The criteria may specify a condition
based on time or events under which a baseline may be expired for
any of several reasons. For example, an administrator may not be
interested in a baseline that has not gathered new performance
metrics data or that has not been used because such data is
considered too out of date. In such cases, the administrator may
want to remove data associated with such baselines from the system.
Since manually deleting the individual data associated with
baselines often constitute too big burden to a system
administrator, one way to automate such deletions is to set an
expiration time associated with each baseline. Similarly, after a
system has been upgraded to a new software or a new operation
system, an administrator of a system may not be interested in many
of the previously obtained baselines. One way to automate deletions
of such obsolete data is to associate an expiration event for each
baseline.
[0082] According to another embodiment, a baseline may be
automatically deleted after a fixed period of time of nonuse. For
example, if the baseline has not been used or modified for six
months, for example, the baseline may be automatically expired. A
baseline may also be automatically deleted after a occurrence or
non-occurrence of an event. For example, if a system has been
upgraded or software patch installed, a baseline with performance
metrics data associated with a prior version of software may be
deleted. As another example, if the error count of a monitored
system has not exceeded a certain threshold for three weeks, a
baseline may also be automatically expired.
[0083] In general, according to an embodiment, a baseline may be
configured to include a non-contiguous time period defining
multiple non-contiguous time periods. For a non-contiguous-based
baseline, the performance metrics data associated with the
non-contiguous baseline is the performance metrics data
corresponding to the non-contiguous time periods associated with
the baseline. Accordingly, for a non-contiguous baseline,
performance metrics data is determined corresponding to each time
period associated with the baseline. The performance metrics data
gathered for the multiple time periods is then aggregated and
associated with the non-contiguous baseline.
[0084] According to the embodiment, the determination of
performance metrics data to be associated with the baseline may
take place soon after the time period for the baseline is received.
In an alternative embodiment, the determination of performance
metrics data to be associated with the non-contiguous baseline may
take place later dynamically in response to a query for performance
metrics data associated with the baseline. A moving window baseline
and/or a future-based baseline may also be a non-contiguous
baseline.
[0085] FIG. 6 is a simplified diagram depicting a technique for
creating baselines using baseline templates according to an
embodiment of the invention. The techniques depicted in FIG. 6 may
be performed by hardware modules, software modules (e.g., program,
code, instructions executed by a processor), or combinations
thereof. In one embodiment, the processing may be performed by
baseline engine 1050 depicted in FIG. 1.
[0086] According to an embodiment, a baseline template is defined
for creating a set of baselines. A baseline template may specify a
time schedule for creating a set of baselines. It may specify the
intervals for which each of the set of baselines is to be created.
For example, a baseline template may specify that for each of the
time periods of between 9 AM to 12 PM of each Monday during the
last month a baseline should be created. If there are 4 Mondays in
the last month, there will be four baselines created each with a
non-contiguous set of time periods, the first of the baselines will
be associated with a time period of between 9 AM to 12 PM on the
first Monday, the second of the baselines will be associated with a
time period of between 9 AM to 12 PM on the second Monday, the
third of the baselines will be associated with a time period of
between 9 AM to 12 PM on the third Monday, and the fourth of the
baselines will be associated with a time period of between 9 AM to
12 PM on the second Monday.
[0087] According to an embodiment, the techniques for creating
baselines based on baseline templates can also include setting an
expiration date for the baselines that are created by the baseline
template. In an example, a baseline template might specify that a
set of baselines are to be created for each of the last 5 trailing
days. On Day 5, five baselines would have been created, one for Day
1, one for Day 2, one for Day 3, one for Day 4, and one for Day 5.
One Day 6, the baseline for Day 1 is created, and a new baseline
for Day 6 is created. This process for creating a rolling set of
baselines can go on indefinitely, according to an embodiment.
[0088] As depicted in FIG. 6, one embodiment of a technique for
creating a set of baselines using a baseline template is disclosed.
The technique includes receiving time information relating to a
baseline template (step 6010). The technique can specify, for
example, a template to be created for each of the time periods of
between 9 AM to 12 PM of each Monday of the last month. Based on
the time information associated with the baseline template, a set
of time periods is then determined. If there are 4 Mondays in the
last month, four time periods will be created, the first which is a
time period of between 9 AM to 12 PM on the first Monday, the
second of which is a time period of between 9 AM to 12 PM on the
second Monday, the third of which is a time period of between 9 AM
to 12 PM on the third Monday, and the fourth of which is time
period of between 9 AM to 12 PM on the second Monday.
[0089] Other types of time information may exist. A slight
variation may specify a time period of 9 AM to 12 PM every Monday
of the next month. If there are 4 Mondays in the next month, there
will be four non-contiguous set of time periods created. Another
variation of the above example can specify a time of period of 9 AM
to 12 PM every Monday of the current month. In this example, the
set of time periods specified may thus potentially include time
periods in both the future and the past, depending on the current
time. If there are 4 Mondays in the current month, four
non-contiguous set of time periods will be created.
[0090] In general, time information can specify a set of time
periods that are in the past or in the future, or both. The set of
time periods may include static time windows (e.g., December 15 to
December 25), a moving time window (e.g., between the present and 5
days into the past), or even a group of non-contiguous time windows
(e.g., the first day of each week for the next four weeks). The
time information relating to a baseline template may be stored as
part of baselines definitions 1040 depicted in FIG. 1 or may be
stored in a separate part of a system devoted specifically to
baseline template, according to another embodiment.
[0091] After reception of information relating to a set of time
periods, a set of baselines can be created based upon the time
information (step 6025). Performance metrics data can then be
associated with each of the newly created baselines (step
6030).
[0092] FIG. 7 is a simplified block diagram of a computer system
100 that may be used to practice an embodiment of the various
inventions described in this application. As shown in FIG. 7,
computer system 100 includes a processor 102 that communicates with
a number of peripheral subsystems via a bus subsystem 104. These
peripheral subsystems may include a storage subsystem 106,
comprising a memory subsystem 108 and a file storage subsystem 110,
user interface input devices 112, user interface output devices
114, and a network interface subsystem 116.
[0093] Bus subsystem 104 provides a mechanism for letting the
various components and subsystems of computer system 100
communicate with each other as intended. Although bus subsystem 104
is shown schematically as a single bus, alternative embodiments of
the bus subsystem may utilize multiple busses.
[0094] Network interface subsystem 116 provides an interface to
other computer systems, networks, and portals. Network interface
subsystem 116 serves as an interface for receiving data from and
transmitting data to other systems from computer system 100.
[0095] User interface input devices 112 may include a keyboard,
pointing devices such as a mouse, trackball, touchpad, or graphics
tablet, a scanner, a barcode scanner, a touch screen incorporated
into the display, audio input devices such as voice recognition
systems, microphones, and other types of input devices. In general,
use of the term "input device" is intended to include all possible
types of devices and mechanisms for inputting information to
computer system 100.
[0096] User interface output devices 114 may include a display
subsystem, a printer, a fax machine, or non-visual displays such as
audio output devices, etc. The display subsystem may be a cathode
ray tube (CRT), a flat-panel device such as a liquid crystal
display (LCD), or a projection device. In general, use of the term
"output device" is intended to include all possible types of
devices and mechanisms for outputting information from computer
system 100.
[0097] Storage subsystem 106 may be configured to store the basic
programming and data constructs that provide the functionality of
the present invention. Software (code modules or instructions) that
provides the functionality of the present invention may be stored
in storage subsystem 106. These software modules or instructions
may be executed by processor(s) 102. Storage subsystem 106 may also
provide a repository for storing data used in accordance with the
present invention. Storage subsystem 106 may comprise memory
subsystem 108 and file/disk storage subsystem 110.
[0098] Memory subsystem 108 may include a number of memories
including a main random access memory (RAM) 118 for storage of
instructions and data during program execution and a read only
memory (ROM) 120 in which fixed instructions are stored. File
storage subsystem 110 provides persistent (non-volatile) storage
for program and data files, and may include a hard disk drive, a
floppy disk drive along with associated removable media, a Compact
Disk Read Only Memory (CD-ROM) drive, an optical drive, removable
media cartridges, and other like storage media.
[0099] Computer system 100 can be of various types including a
personal computer, a portable computer, a workstation, a network
computer, a mainframe, a kiosk, a server or any other data
processing system. Due to the ever-changing nature of computers and
networks, the description of computer system 100 depicted in FIG. 7
is intended only as an example for purposes of illustrating the
preferred embodiment of the computer system. Many other
configurations having more or fewer components than the system
depicted in FIG. 7 are possible.
[0100] Although specific embodiments of the invention have been
described, various modifications, alterations, alternative
constructions, and equivalents are also encompassed within the
scope of the invention. The described invention is not restricted
to operation within certain specific data processing environments,
but is free to operate within a plurality of data processing
environments. Additionally, although the present invention has been
described using a particular series of transactions and steps, it
should be apparent to those skilled in the art that the scope of
the present invention is not limited to the described series of
transactions and steps.
[0101] Further, while the present invention has been described
using a particular combination of hardware and software, it should
be recognized that other combinations of hardware and software are
also within the scope of the present invention. The present
invention may be implemented using hardware, software, or
combinations thereof.
[0102] The specification and drawings are, accordingly, to be
regarded in an illustrative rather than a restrictive sense. It
will, however, be evident that additions, subtractions, deletions,
and other modifications and changes may be made thereunto without
departing from the broader spirit and scope of the inventions.
* * * * *