U.S. patent application number 15/898053 was filed with the patent office on 2018-11-22 for performance evaluation of applications that access external resources.
The applicant listed for this patent is New Relic, Inc.. Invention is credited to Lewis Karl Cirne, Saxon Michael D'Aubin, James Robert Gochee.
Application Number | 20180337817 15/898053 |
Document ID | / |
Family ID | 61711546 |
Filed Date | 2018-11-22 |
United States Patent
Application |
20180337817 |
Kind Code |
A1 |
Cirne; Lewis Karl ; et
al. |
November 22, 2018 |
PERFORMANCE EVALUATION OF APPLICATIONS THAT ACCESS EXTERNAL
RESOURCES
Abstract
Evaluating application performance includes: obtaining a
request, processing the request, including invoking a function that
accesses an external resource, determining whether a response time
associated with invoking the function exceeds a threshold, and in
the event that the response time exceeds the threshold, capturing a
runtime attribute associated with the function.
Inventors: |
Cirne; Lewis Karl; (Palo
Alto, CA) ; Gochee; James Robert; (Portland, OR)
; D'Aubin; Saxon Michael; (Portland, OR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
New Relic, Inc. |
San Francisco |
CA |
US |
|
|
Family ID: |
61711546 |
Appl. No.: |
15/898053 |
Filed: |
February 15, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12584833 |
Sep 10, 2009 |
9935812 |
|
|
15898053 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 2201/81 20130101;
H04L 29/08072 20130101; H04L 43/16 20130101; G06F 11/3409 20130101;
H04L 29/06 20130101; G06F 11/3466 20130101; H04L 43/00 20130101;
G06F 11/36 20130101; G06F 2201/865 20130101; H04L 43/0841
20130101 |
International
Class: |
H04L 29/08 20060101
H04L029/08; H04L 29/06 20060101 H04L029/06; H04L 12/26 20060101
H04L012/26; G06F 11/34 20060101 G06F011/34 |
Claims
1. (canceled)
2. A system for evaluating application performance, comprising: a
processor configured to: process a request, including invoking a
function that accesses an external resource; determine whether a
response time associated with the invocation of the function
exceeds a threshold; in response to a determination that the
response time exceeds the threshold: mark the invocation of the
function for post processing to obtain a marked invocation of the
function; and capture a runtime attribute associated with the
marked invocation of the function; and post process the marked
invocation of the function to obtain additional performance
information; and a memory coupled to the processor, configured to
provide the processor with instructions.
3. The system of claim 2, wherein: the runtime attribute includes a
stack trace associated with the marked invocation of the function;
and the processor is further configured to post process the marked
invocation of the function comprises to: analyze, based on the
stack trace, the marked invocation of the function to identify code
invoking the accessing of the external resource.
4. The system of claim 3, wherein the post processing is performed
asynchronously.
5. The system of claim 2, wherein the function is a database access
function.
6. The system of claim 5, and the processor is further configured
to generate an explain plan of the database access function.
7. The system of claim 6, wherein the runtime attribute includes
database connection information associated with the invocation of
the database access function, and generating the explain plan
includes opening another database connection using the database
connection information.
8. The system of claim 7, wherein the database connection
information includes user credential information, database host
information and port information.
9. The system of claim 2, wherein the runtime attribute includes a
parameter that is used to invoke the function.
10. The system of claim 2, wherein the runtime attribute includes a
return value of the function.
11. The system of claim 2, wherein the runtime attribute includes
performance statistics. The system of claim 2, wherein the function
was previously modified to include instrumentation code configured
to measure response time.
12. The system of claim 2, wherein the external resource includes a
web service.
13. The system of claim 2, wherein the external resource includes a
networking resource.
14. A method for evaluating application performance, comprising:
processing a request, including invoking a function that accesses
an external resource; determining whether a response time
associated with invoking the function exceeds a threshold; in
response to a determination that the response time exceeds the
threshold: marking the function for post processing to obtain a
marked invocation of the function; and capturing a runtime
attribute associated with the marked invocation of the function;
and post processing the marked invocation of the function to obtain
additional performance information.
15. The method of claim 14, wherein the function is a database
access function.
16. A computer program product for evaluating application
performance, the computer program product being embodied in a
computer readable storage medium and comprising computer
instructions for: processing a request, including invoking a
function that accesses an external resource; determining whether a
response time associated with invoking the function exceeds a
threshold; in response to a determination that the response time
exceeds the threshold: marking the function for post processing to
obtain a marked invocation of the function; and capturing a runtime
attribute associated with the marked invocation of the function;
and post processing the marked invocation of the function to obtain
additional performance information.
17. The computer program product of claim 16, wherein the function
is a database access function.
Description
CROSS REFERENCE TO OTHER APPLICATIONS
[0001] This application is a continuation of co-pending U.S. patent
application Ser. No. 12/584,833, entitled PERFORMANCE EVALUATION OF
APPLICATIONS THAT ACCESS EXTERNAL RESOURCES filed Sep. 10, 2009
which is incorporated herein by reference for all purposes.
BACKGROUND OF THE INVENTION
[0002] As computer software becomes more sophisticated, performance
evaluation has also become a more complex task. In some
client-server applications, for example, many different calls are
made to the server application and it can be difficult and
time-consuming to pinpoint performance bottlenecks. One technique
often employed by software developers is to manually insert logging
code to facilitate performance evaluation. The addition of logging
code, however, can negatively impact the overall performance of the
system and may only provide limited information on how the
bottlenecks occurred.
BRIEF DESCRIPTION OF THE DRAWINGS
[0003] Various embodiments of the invention are disclosed in the
following detailed description and the accompanying drawings.
[0004] FIG. 1 is a block diagram illustrating an embodiment of a
performance evaluation system.
[0005] FIG. 2 is a flowchart illustrating an embodiment of an
application monitoring process.
[0006] FIG. 3 is a flowchart illustrating another embodiment of
another application monitoring process.
[0007] FIG. 4 is a flowchart illustrating an embodiment of a
process configured to perform post processing.
[0008] FIG. 5 is a diagram illustrating an example user interface
displaying performance monitoring results of an application.
[0009] FIG. 6 is a diagram illustrating an example user interface
displaying the diagnostic information pertaining to a selected
call.
DETAILED DESCRIPTION
[0010] The invention can be implemented in numerous ways, including
as a process; an apparatus; a system; a composition of matter; a
computer program product embodied on a computer readable storage
medium; and/or a processor, such as a processor configured to
execute instructions stored on and/or provided by a memory coupled
to the processor. In this specification, these implementations, or
any other form that the invention may take, may be referred to as
techniques. In general, the order of the steps of disclosed
processes may be altered within the scope of the invention. Unless
stated otherwise, a component such as a processor or a memory
described as being configured to perform a task may be implemented
as a general component that is temporarily configured to perform
the task at a given time or a specific component that is
manufactured to perform the task. As used herein, the term
`processor` refers to one or more devices, circuits, and/or
processing cores configured to process data, such as computer
program instructions.
[0011] A detailed description of one or more embodiments of the
invention is provided below along with accompanying figures that
illustrate the principles of the invention. The invention is
described in connection with such embodiments, but the invention is
not limited to any embodiment. The scope of the invention is
limited only by the claims and the invention encompasses numerous
alternatives, modifications and equivalents. Numerous specific
details are set forth in the following description in order to
provide a thorough understanding of the invention. These details
are provided for the purpose of example and the invention may be
practiced according to the claims without some or all of these
specific details. For the purpose of clarity, technical material
that is known in the technical fields related to the invention has
not been described in detail so that the invention is not
unnecessarily obscured.
[0012] FIG. 1 is a block diagram illustrating an embodiment of a
performance evaluation system. System 100 includes an application
server 102, on which one or more applications 104 are installed and
executed. The application server receives requests such as 120. The
requests may be sent by client devices, other applications (either
on the same server or on a different server), function calls within
application 104, or other appropriate request generating source.
For example, in some embodiments, the application server is a web
server, the application is a web application, and the request may
be a Hypertext Transfer Protocol (HTTP) request.
[0013] To process a request, the application will access an
external resource (i.e., a resource that is not an integral part of
the application), such as database 106, web service 108, network
service 110, or other appropriate resource. An external resource is
typically installed and executed on a device other than the
application server, but may sometimes be installed and executed on
the same application server in a different process. The application
is programmed to accesses the external resource using a suitable
protocol. For example, to access a database such as Oracle.TM. or
Microsoft SQL Server.TM., Structured Query Language (SQL) calls are
made; to access web services such as credit card processing service
or message queuing service, HTTP calls are made; and to access
network services such as socket services, networking calls such as
open_socket are made. These external resource access calls
sometimes cause performance bottlenecks, and can be specifically
identified and diagnosed using techniques described below.
[0014] A performance monitor 110 is configured to monitor the
performance of application 104. In some embodiments, the
performance monitor is implemented as special instrumentation code
(also referred to as probes) that cooperates with the application
to detect any slowdown in the application's access to external
resources. For example, in some embodiments where the application
is implemented using Ruby On Rails.TM. (Rails), the source code of
certain external resource access classes is replaced with
performance monitoring code that carries out the same function and
additionally performs monitoring functions. In a more specific
example, SQL database access calls all flow through a class named
ActiveRecord::Base, which is replaced with instrumented code that
invokes the original code, and additionally collects performance
information such as the amount of time spent on accessing the
external resource (referred to as the response time) and other
runtime attributes. In some embodiments where the application is
implemented using JAVA.TM., the byte code of certain external
resource access functions and libraries is replaced with byte code
with additional monitoring functions. Other implementations such as
binary code replacement may be used as appropriate.
[0015] FIG. 2 is a flowchart illustrating an embodiment of an
application monitoring process. Process 200 maybe performed by
application servers such as 102. At 202, a request is received. The
request may be received from a client, another server, a different
application running on the same server, or a function call in the
same application. At 204, the request is processed. A response, if
required, is generated and provided to the requester based on
results of the invocation. In some embodiments, the processing
includes invoking a function that accesses an external resource
such as a database, a web service, a network connection, etc. The
function has been previously instrumented to include monitoring
code to determine the function's response time. In some
embodiments, the total response time, i.e., the total amount of
time spent processing the request (e.g., a Process_HTTP_Request
function) is determined. In some embodiments, the remote function
call response time, i.e., the amount of time required to execute a
remote function call to the external resource (e.g., a SQL_Select
call that is invoked by the Process_HTTP_Request function) is
determined. In some embodiments, both types of response time are
recorded.
[0016] At 206, it is determined whether the response time exceeds a
predefined threshold. Depending on whether the response time is a
total response time or a remote function call response time,
different thresholds may be set. In some embodiments, both types of
response time are compared with their respective thresholds. If the
response time does not exceed the threshold, the performance of the
function's invocation is considered normal and no further process
is required. If, however, the response time exceeds the threshold,
at 208, further investigation is needed and one or more runtime
attributes associated with the invocation of the function are
captured. By examining these runtime attributes, the software
developer can identify certain causes for the slowdown. For
example, in some embodiments, the runtime attributes include a
stack trace of the function's stack frame(s), which can be used to
pinpoint certain code that invokes the external resource access and
causes the slowdown. In some embodiments, the runtime attributes
include parameter(s) associated with the invocation of the
function. For example, an incorrect parameter can slow down the
function call. In some embodiments, the runtime attributes include
the return value/payload information of the function, which may
indicate that too much data is being passed and therefore slowing
down the access. In some embodiments, the runtime attributes
include performance statistics, which may indicate that resource
constraints or a heavy load on the external resource is causing the
slowdown.
[0017] FIG. 3 is a flowchart illustrating another embodiment of
another application monitoring process. In this example, at 302, a
request is received at the application server. At 304, the request
is processed, and a database access function is invoked in response
to the request. For example, the request may ask for account
information of a user. To generate the response, the application
performs a database access function to get the account information
from a user database. At 306, the start and finish times of the
database access function are measured based on previously inserted
instrumentation code in the database access function. Thus, the
response time of the database access function call is determined
based on the measured start time and finish time. At 308, it is
determined whether the response time that is measured exceeds a
predefined threshold. No further action is required if the response
time does not exceed the predefined threshold. If, however, the
threshold is exceeded, at 310, one or more runtime attributes are
captured. In addition to the runtime attributes described above in
connection with process 200, in some embodiments additional runtime
attributes are saved for post processing. For example, database
connection information of the database access call, which includes
information pertaining to user credential, database host, port,
and/or any other appropriate information required for making the
database connection, is saved in data structures or other memory
that is accessible by the post processing code. The function is
marked for post processing.
[0018] At 312, post processing is performed to get more in-depth
diagnostic information. Since post processing operations can be
expensive to perform, they are sometimes performed asynchronously,
in a thread or a process that is separate from the function
invocation to more efficiently batch process multiple calls and
reduce performance impact on the main application.
[0019] The following is a pseudocode example of an instrumented
database access function:
TABLE-US-00001 Function Instrumented_SQLCall record start time
execute Original_SQLCall record finish time if (finish time - start
time) > threshold save configuration get runtime attributes such
as stack trace, etc.
[0020] FIG. 4 is a flowchart illustrating an embodiment of a
process configured to perform post processing. In this example,
process 400 starts by locating database access calls that are
deemed to be too slow (i.e., execution duration exceeding
threshold), at 402. In some embodiments, such function calls are
marked and/or stored in memory so that they can be located by the
post processing steps. In this example, for each slow function
call, at 404, database connection information is obtained from the
server application. In some embodiments, the connection information
is included in a configuration file of the application or
statically encoded in the application. In some embodiments, the
connection information is stored in memory location(s) and/or data
structure(s) accessible by the post processing steps. The database
connection information includes the user credential, database host,
port, etc. associated with the original database access call.
[0021] At 406, a new database connection is opened using the same
database connection information as the original call. At 408, an
explain plan is generated for the original database call, over the
new database connection. The explain plan is a database feature
that describes how the database would execute the database access
function. For example, whether indexes are used, which indexes are
used, what type of database operation is performed, if temporary
tables are used, and how many rows would be accessed. Analysis of
the explain plan can be helpful for determining the cause for the
slowdown. For example, queries on tables that don't use an index
run quickly when the number of rows is small but progressively get
slower as the number of rows grows. Knowing that a query is slow,
looking at an explain plan and seeing that a large number of rows
is accessed and that an index is not being used is enough
information for a software developer to quickly see that adding an
index will speed up queries.
[0022] In some embodiments, post processing further includes
invoking a debug version of the function. The debug function is
invoked under substantially the same condition, using the same
parameters as the invocation of the original function. The debug
version of the function will provide extra diagnostic information
such as log information, break points, or any other appropriate
debugging facilities.
[0023] The following is a pseudo code example for a post processing
function that operates in a separate thread and diagnoses slow SQL
calls:
TABLE-US-00002 Function Post_Process_SQL for each slow SQLCall get
database connection information open new database connection
generate explain plan (SQLCall(new database connection))
[0024] FIG. 5 is a diagram illustrating an example user interface
displaying performance monitoring results for a service call to an
application. The access duration of calls to external resources,
including various SQL calls and memory cache (MemCache) access
calls, are measured. In this example, call times in the "Exclusive"
column correspond to call durations minus any nested calls to child
functions, which more accurately represent how long the calls take
than the times recorded in the "Duration" column, which includes
time spent on nested calls. A threshold of 100 ms is used in this
example. Calls that took longer than 100 ms to complete, such as
the RealAgent#find_by_sql call that occurred at time 0.036,
find_and_aggregate call that occurred at 1.520, and
layouts/application.rhtml Template call at 276.643 are highlighted,
indicating that these calls warrant special attention since their
execution time exceeded the threshold.
[0025] The software developer can select a call from the table to
view further performance monitoring details. FIG. 6 is a diagram
illustrating an example user interface displaying the diagnostic
information pertaining to a selected call. In this example,
performance information of a SQL call that is made at time 0.036
and lasted for 640 ms is displayed (note that entries in the SQL
select statement are largely omitted for purposes of simplicity).
The explain plan for the SQL call indicates that a large number of
rows (172,753) are returned, and a filesort operation (invoked by
the "Order By" operation of the SQL call) is performed. These
factors contribute to the slow execution of the call. Additionally,
the stack trace shows the functions and line numbers of the chain
of calls that lead to this SQL call, starting at "load_time_zone"
of the "memory_watcher" file and ending at the "most_recent_run" of
the "virtual_agent" file. Knowing the causes and locations of the
slowdown can help the software developer make decisions about
modifying the database access call to reduce the amount of access
time.
[0026] Performance evaluation of applications that invoke function
calls that access external resource has been described. Although
database access and using explain plan to identify bottlenecks in
the access calls have been discussed extensively for purposes of
example, the technique can also be used to diagnose slowdowns in
other types of external resource access.
[0027] Although the foregoing embodiments have been described in
some detail for purposes of clarity of understanding, the invention
is not limited to the details provided. There are many alternative
ways of implementing the invention. The disclosed embodiments are
illustrative and not restrictive.
* * * * *