U.S. patent application number 13/659802 was filed with the patent office on 2013-05-09 for technique for inter-procedural memory address space optimization in gpu computing compiler.
This patent application is currently assigned to NVIDIA CORPORATION. The applicant listed for this patent is NVIDIA CORPORATION. Invention is credited to Vinod GROVER, Xiangyun KONG, Yuan LIN, Jian-Zhong WANG.
Application Number | 20130113809 13/659802 |
Document ID | / |
Family ID | 48223398 |
Filed Date | 2013-05-09 |
United States Patent
Application |
20130113809 |
Kind Code |
A1 |
KONG; Xiangyun ; et
al. |
May 9, 2013 |
TECHNIQUE FOR INTER-PROCEDURAL MEMORY ADDRESS SPACE OPTIMIZATION IN
GPU COMPUTING COMPILER
Abstract
A device compiler and linker is configured to optimize program
code of a co-processor enabled application by resolving generic
memory access operations within that program code to target
specific memory spaces. In situations where a generic memory access
operation cannot be resolved and may target constant memory,
constant variables associated with those generic memory access
operations are transferred to reside in global memory.
Inventors: |
KONG; Xiangyun; (Union City,
CA) ; WANG; Jian-Zhong; (Fremont, CA) ; LIN;
Yuan; (Cupertino, CA) ; GROVER; Vinod; (Mercer
Island, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
NVIDIA CORPORATION; |
Santa Clara |
CA |
US |
|
|
Assignee: |
NVIDIA CORPORATION
Santa Clara
CA
|
Family ID: |
48223398 |
Appl. No.: |
13/659802 |
Filed: |
October 24, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61556782 |
Nov 7, 2011 |
|
|
|
Current U.S.
Class: |
345/505 |
Current CPC
Class: |
G06F 8/445 20130101;
G06F 8/443 20130101; G06F 9/5066 20130101; G06F 8/41 20130101; G06F
8/45 20130101; G06F 8/456 20130101; G06F 8/4442 20130101; G06F
8/433 20130101 |
Class at
Publication: |
345/505 |
International
Class: |
G06F 15/80 20060101
G06F015/80 |
Claims
1. A computer-implemented method for optimizing program code
capable of being compiled for execution on a parallel processing
unit (PPU) having a non-uniform memory architecture, the method
comprising: identifying a first memory access operation that is
associated with a first pointer, wherein the first memory access
operation targets a generic memory space; ascending a
use-definition chain related to the first pointer; adding the first
pointer to a vector upon determining that the first pointer is
derived from a specific memory space in the non-uniform memory
architecture; and causing the first memory access operation to
target the specific memory space by modifying at least a portion of
the program code.
2. The computer-implemented method of claim 1, wherein the specific
memory space comprises a global memory space accessible by a set of
processing cores within the PPU, a local memory space associated
with a first processing core included in the set of processing
cores, a shared memory space accessible by two or more of the
processing cores included in the set of processing cores, or a
constant memory space residing in read-only memory.
3. The computer-implemented method of claim 1, wherein the
use-definition chain related to the first pointer is generated by
performing data flow analysis with the program code in order to
identify each definition and each use of the first pointer.
4. The computer-implemented method of claim 1, further comprising:
identifying a second memory access operation by descending a
definition-use chain related to a first memory address; determining
that the second memory access operation does not target a specific
memory space; identifying a first variable declaration associated
with the first memory address, wherein the first variable
declaration indicates that the first memory address relates to a
location in a constant memory space; and causing the first variable
declaration to indicate that the first memory address relates to a
location in a global memory space by modifying at least a portion
of the program code.
5. The computer-implemented method of claim 4, further comprising
promoting data associated with the first memory address from the
constant memory space to the global memory space.
6. The computer-implemented method of claim 4, wherein the
definition-use chain related to the first memory address is
generated by performing data flow analysis with the program code in
order to identify each definition and each use of the first memory
address.
7. The computer-implemented method of claim 4, further comprising:
identifying a third memory access operation that depends on the
first variable declaration; determining that the third memory
access operation also depends on a second variable declaration
associated with a second memory address, wherein the second
variable declaration indicates that the second memory address
relates to a location in the constant memory space; and causing the
second variable declaration to indicated that the second memory
address relates to a location in the global memory space by
modifying at least a portion of the program code.
8. The computer-implemented method of claim 7, further comprising:
causing the third memory access operation to target the global
memory space by modifying at least a portion of the program code;
and promoting data associated with the third memory address from
the constant memory space to the global memory space.
9. The computer-implemented method of claim 1, further comprising
performing at least one of a code re-ordering operation and an
alias analysis based on the at least a portion of the program code
that has been modified.
10. A non-transitory computer-readable medium storing program
instructions that, when executed by a processing unit, cause the
processing unit to optimize program code capable of being compiled
for execution on a parallel processing unit (PPU) having a
non-uniform memory architecture, by performing the steps of:
identifying a first memory access operation that is associated with
a first pointer, wherein the first memory access operation targets
a generic memory space; ascending a use-definition chain related to
the first pointer; adding the first pointer to a vector upon
determining that the first pointer is derived from a specific
memory space in the non-uniform memory architecture; and causing
the first memory access operation to target the specific memory
space by modifying at least a portion of the program code.
11. The non-transitory computer-readable medium of claim 10,
wherein the specific memory space comprises a global memory space
accessible by a set of processing cores within the PPU, a local
memory space associated with a first processing core included in
the set of processing cores, a shared memory space accessible by
two or more of the processing cores included in the set of
processing cores, or a constant memory space residing in read-only
memory.
12. The non-transitory computer-readable medium of claim 10,
wherein the use-definition chain related to the first pointer is
generated by performing data flow analysis with the program code in
order to identify each definition and each use of the first
pointer.
13. The non-transitory computer-readable medium of claim 10,
further comprising the steps of: identifying a second memory access
operation by descending a definition-use chain related to a first
memory address; determining that the second memory access operation
does not target a specific memory space; identifying a first
variable declaration associated with the first memory address,
wherein the first variable declaration indicates that the first
memory address relates to a location in a constant memory space;
and causing the first variable declaration to indicate that the
first memory address relates to a location in a global memory space
by modifying at least a portion of the program code.
14. The non-transitory computer-readable medium of claim 13,
further comprising the step of promoting data associated with the
first memory address from the constant memory space to the global
memory space.
15. The non-transitory computer-readable medium of claim 13,
wherein the definition-use chain related to the first memory
address is generated by performing data flow analysis with the
program code in order to identify each definition and each use of
the first memory address.
16. The non-transitory computer-readable medium of claim 13,
further comprising the steps of: identifying a third memory access
operation that depends on the first variable declaration;
determining that the third memory access operation also depends on
a second variable declaration associated with a second memory
address, wherein the second variable declaration indicates that the
second memory address relates to a location in the constant memory
space; and causing the second variable declaration to indicated
that the second memory address relates to a location in the global
memory space by modifying at least a portion of the program
code.
17. The non-transitory computer-readable medium of claim 16,
further comprising the steps of: causing the third memory access
operation to target the global memory space by modifying at least a
portion of the program code; and promoting data associated with the
third memory address from the constant memory space to the global
memory space.
18. The non-transitory computer-readable medium of claim 9, further
comprising the step of performing at least one of a code
re-ordering operation and an alias analysis based on the at least a
portion of the program code that has been modified.
19. A computing device configured to optimize program code capable
of being compiled for execution on a parallel processing unit (PPU)
having a non-uniform memory architecture, including: a processing
unit configured to: identify a first memory access operation that
is associated with a first pointer, wherein the first memory access
operation targets a generic memory space, ascend a use-definition
chain related to the first pointer, add the first pointer to a
vector upon determining that the first pointer is derived from a
specific memory space in the non-uniform memory architecture, and
cause the first memory access operation to target the specific
memory space by modifying at least a portion of the program
code.
20. The computing device of claim 19, further including: a memory
coupled to the processing unit and storing program instructions
that, when executed by the processing unit, cause the processing
unit to: identify the first memory access operation, ascend the
use-definition chain related to the first pointer, add the first
pointer to the vector, and cause the first memory access operation
to target the specific memory space.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. provisional
patent application titled "Method for Inter-Procedural Memory Space
Optimization in GPU Computing Compiler" filed on Nov. 7, 2011 and
having Ser. No. 61/556,782. The entire content of the foregoing
application is hereby incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] The present invention generally relates to graphics
processing unit (GPU) computing compilers, and, more specifically,
to a technique for inter-procedural memory address space
optimization in a GPU computing compiler.
[0004] 2. Description of the Related Art
[0005] Graphics processing units (GPUs) have evolved over time to
support a wide range of operations beyond graphics-oriented
operations. In fact, a modern GPU may be capable of executing
arbitrary program instructions. Such a GPU typically includes a
compiler that compiles program instructions for execution on one or
more processing cores included within the GPU. Each such core may
execute a particular execution thread in parallel with other
processing cores executing execution threads.
[0006] A given core within a GPU may be coupled to a local memory
space that is available to the GPU for memory access operations
when executing a thread. Each core may also be coupled to a shared
memory space to which one or more other cores may also be coupled.
With this configuration, multiple cores may share data via the
shared memory space. The cores within the GPU may also be coupled
to a global memory space that is accessible to all processing cores
and possibly to other processing units aside from the GPU
itself.
[0007] The configuration of multiple different memory spaces
described above is referred to in the art as a "non-uniform memory
architecture." In general, a non-uniform memory architecture
includes multiple different memory spaces where data may reside. A
program designed to execute on a GPU may access data that resides
in any or all of the different memory spaces in the non-uniform
memory architecture.
[0008] Within such a program, different memory access operations
may be specified, such as load/store operations or atomic
operations, each of which target a different address. However, a
given memory access operation targeting a given memory address may
not specify any particular memory space. In conventional
approaches, at run time, the GPU executing the program typically
reads a tag associated with the address that indicates the specific
memory space in which to perform the memory access operation. A tag
is required for each address because, for example, two different
variables may both reside at the same address within different
memory spaces. Without such a tag, the two variables would be
indistinguishable based on the addresses alone.
[0009] Relying on the tagging approach described above is
problematic for two reasons. First, reading a tag for each memory
access operation is a costly operation and wastes GPU resources.
Second, since variables having the same address are
indistinguishable until run-time, the GPU compiler is prevented
from performing program code optimizations prior to run time,
including memory access re-ordering or alias analysis.
[0010] Accordingly, what is needed in the art is a more effective
technique for compiling GPU program instructions.
SUMMARY OF THE INVENTION
[0011] One embodiment of the present inventions sets forth a
computer-implemented method for optimizing program code capable of
being compiled for execution on a parallel processing unit (PPU)
having a non-uniform memory architecture, including identifying a
first memory access operation that is associated with a first
pointer, where the first memory access operation targets a generic
memory space, ascending a use-definition chain related to the first
pointer, adding the first pointer to a vector upon determining that
the first pointer is derived from a specific memory space in the
non-uniform memory architecture, and causing the first memory
access operation to target the specific memory space by modifying
at least a portion of the program code.
[0012] One advantage of the disclosed technique is that a graphics
processing unit is not required to resolve all generic memory
access operations at run time, thereby conserving resources and
accelerating the execution of the application. Further, the
graphics processing unit is enabled to perform additional program
code optimizations with the application program code, including
memory access re-ordering and alias analysis, further accelerating
program code execution.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] So that the manner in which the above recited features of
the present invention can be understood in detail, a more
particular description of the invention, briefly summarized above,
may be had by reference to embodiments, some of which are
illustrated in the appended drawings. It is to be noted, however,
that the appended drawings illustrate only typical embodiments of
this invention and are therefore not to be considered limiting of
its scope, for the invention may admit to other equally effective
embodiments.
[0014] FIG. 1 is a block diagram illustrating a computer system
configured to implement one or more aspects of the present
invention;
[0015] FIG. 2 is a block diagram of a parallel processing subsystem
for the computer system of FIG. 1, according to one embodiment of
the present invention;
[0016] FIG. 3 illustrates a build process used to compile a
co-processor enabled application, according to one embodiment of
the present invention;
[0017] FIG. 4 is a flow diagram of method steps for optimizing
memory access operations, according to one embodiment of the
present invention;
[0018] FIG. 5 is a flow diagram of method steps for transferring
constant variables to a global memory space, according to one
embodiment of the present invention; and
[0019] FIG. 6 sets forth a pseudocode example to illustrate the
operation of a device compiler and linker, according to one
embodiment of the present invention.
DETAILED DESCRIPTION
[0020] In the following description, numerous specific details are
set forth to provide a more thorough understanding of the present
invention. However, it will be apparent to one of skill in the art
that the present invention may be practiced without one or more of
these specific details.
System Overview
[0021] FIG. 1 is a block diagram illustrating a computer system 100
configured to implement one or more aspects of the present
invention. Computer system 100 includes a central processing unit
(CPU) 102 and a system memory 104 communicating via an
interconnection path that may include a memory bridge 105. System
memory 104 includes an image of an operating system 130, a driver
103, and a co-processor enabled application 134. Operating system
130 provides detailed instructions for managing and coordinating
the operation of computer system 100. Driver 103 provides detailed
instructions for managing and coordinating operation of parallel
processing subsystem 112 and one or more parallel processing units
(PPUs) residing therein, as described in greater detail below in
conjunction with FIG. 2. Driver 103 also provides compilation
facilities for generating machine code specifically optimized for
such PPUs, as described in greater detail below in conjunction with
FIGS. 3-6. Co-processor enabled application 134 incorporates
instructions capable of being executed on the CPU 102 and PPUs,
those instructions being implemented in an abstract format, such as
virtual assembly, and mapping to machine code for the PPUs within
parallel processing subsystem 112. The machine code for those PPUs
may be stored in system memory 104 or in memory coupled to the
PPUs.
[0022] In one embodiment, co-processor enabled application 134
represents CUDA.TM. code that incorporates programming instructions
intended to execute on parallel processing subsystem 112. In the
context of the present description, the term "application" or
"program" refers to any computer code, instructions, and/or
functions that may be executed using a processor. For example, in
various embodiments, co-processor enabled application 134 may
include C code, C++ code, etc. In one embodiment, co-processor
enabled application 134 may include a language extension of a
computer language (e.g., C, C++, etc.).
[0023] Memory bridge 105, which may be, e.g., a Northbridge chip,
is connected via a bus or other communication path 106 (e.g., a
HyperTransport link) to an input/output (I/O) bridge 107. I/O
bridge 107, which may be, e.g., a Southbridge chip, receives user
input from one or more user input devices 108 (e.g., keyboard,
mouse) and forwards the input to CPU 102 via communication path 106
and memory bridge 105. Parallel processing subsystem 112 is coupled
to memory bridge 105 via a bus or second communication path 113
(e.g., a Peripheral Component Interconnect Express (PCIe),
Accelerated Graphics Port (AGP), or HyperTransport link); in one
embodiment parallel processing subsystem 112 is a graphics
subsystem that delivers pixels to a display device 110 that may be
any conventional cathode ray tube, liquid crystal display,
light-emitting diode display, or the like. A system disk 114 is
also connected to I/O bridge 107 and may be configured to store
content and applications and data for use by CPU 102 and parallel
processing subsystem 112. System disk 114 provides non-volatile
storage for applications and data and may include fixed or
removable hard disk drives, flash memory devices, and compact disc
(CD) read-only memory (ROM), digital video disc (DVD) ROM, Blu-ray,
high-definition (HD) DVD, or other magnetic, optical, or solid
state storage devices.
[0024] A switch 116 provides connections between I/O bridge 107 and
other components such as a network adapter 118 and various add-in
cards 120 and 121. Other components (not explicitly shown),
including universal serial bus (USB) or other port connections, CD
drives, DVD drives, film recording devices, and the like, may also
be connected to I/O bridge 107. The various communication paths
shown in FIG. 1, including the specifically named communication
paths 106 and 113 may be implemented using any suitable protocols,
such as PCIe, AGP, HyperTransport, or any other bus or
point-to-point communication protocol(s), and connections between
different devices may use different protocols as is known in the
art.
[0025] In one embodiment, the parallel processing subsystem 112
incorporates circuitry optimized for graphics and video processing,
including, for example, video output circuitry, and constitutes a
graphics processing unit (GPU). In another embodiment, the parallel
processing subsystem 112 incorporates circuitry optimized for
general purpose processing, while preserving the underlying
computational architecture, described in greater detail herein. In
yet another embodiment, the parallel processing subsystem 112 may
be integrated with one or more other system elements in a single
subsystem, such as joining the memory bridge 105, CPU 102, and I/O
bridge 107 to form a system on chip (SoC).
[0026] It will be appreciated that the system shown herein is
illustrative and that variations and modifications are possible.
The connection topology, including the number and arrangement of
bridges, the number of CPUs 102, and the number of parallel
processing subsystems 112, may be modified as desired. For
instance, in some embodiments, system memory 104 is connected to
CPU 102 directly rather than through a bridge, and other devices
communicate with system memory 104 via memory bridge 105 and CPU
102. In other alternative topologies, parallel processing subsystem
112 is connected to I/O bridge 107 or directly to CPU 102, rather
than to memory bridge 105. In still other embodiments, I/O bridge
107 and memory bridge 105 might be integrated into a single chip
instead of existing as one or more discrete devices. Large
embodiments may include two or more CPUs 102 and two or more
parallel processing subsystems 112. The particular components shown
herein are optional; for instance, any number of add-in cards or
peripheral devices might be supported. In some embodiments, switch
116 is eliminated, and network adapter 118 and add-in cards 120,
121 connect directly to I/O bridge 107.
[0027] FIG. 2 illustrates a parallel processing subsystem 112,
according to one embodiment of the present invention. As shown,
parallel processing subsystem 112 includes one or more parallel
processing units (PPUs) 202, each of which is coupled to a local
parallel processing (PP) memory 204. In general, a parallel
processing subsystem includes a number U of PPUs, where U is
greater than or equal to 1. (Herein, multiple instances of like
objects are denoted with reference numbers identifying the object
and parenthetical numbers identifying the instance where needed.)
PPUs 202 and parallel processing memories 204 may be implemented
using one or more integrated circuit devices, such as programmable
processors, application specific integrated circuits (ASICs), or
memory devices, or in any other technically feasible fashion.
[0028] Referring to FIGS. 1 as well as FIG. 2, in some embodiments,
some or all of PPUs 202 in parallel processing subsystem 112 are
graphics processors with rendering pipelines that can be configured
to perform various operations related to generating pixel data from
graphics data supplied by CPU 102 and/or system memory 104 via
memory bridge 105 and the second communication path 113,
interacting with local parallel processing memory 204 (which can be
used as graphics memory including, e.g., a conventional frame
buffer) to store and update pixel data, delivering pixel data to
display device 110, and the like. In some embodiments, parallel
processing subsystem 112 may include one or more PPUs 202 that
operate as graphics processors and one or more other PPUs 202 that
are used for general-purpose computations. The PPUs may be
identical or different, and each PPU may have a dedicated parallel
processing memory device(s) or no dedicated parallel processing
memory device(s). One or more PPUs 202 in parallel processing
subsystem 112 may output data to display device 110 or each PPU 202
in parallel processing subsystem 112 may output data to one or more
display devices 110.
[0029] In operation, CPU 102 is the master processor of computer
system 100, controlling and coordinating operations of other system
components. In particular, CPU 102 issues commands that control the
operation of PPUs 202. In some embodiments, CPU 102 writes a stream
of commands for each PPU 202 to a data structure (not explicitly
shown in either FIG. 1 or FIG. 2) that may be located in system
memory 104, parallel processing memory 204, or another storage
location accessible to both CPU 102 and PPU 202. A pointer to each
data structure is written to a pushbuffer to initiate processing of
the stream of commands in the data structure. PPU 202 reads command
streams from one or more pushbuffers and then executes commands
asynchronously relative to the operation of CPU 102. Execution
priorities may be specified for each pushbuffer by an application
program via device driver 103 to control scheduling of the
different pushbuffers.
[0030] Each PPU 202 includes an I/O (input/output) unit 205 that
communicates with the rest of computer system 100 via communication
path 113, which connects to memory bridge 105 (or, in one
alternative embodiment, directly to CPU 102). The connection of PPU
202 to the rest of computer system 100 may also be varied. In some
embodiments, parallel processing subsystem 112 is implemented as an
add-in card that can be inserted into an expansion slot of computer
system 100. In other embodiments, a PPU 202 can be integrated on a
single chip with a bus bridge, such as memory bridge 105 or I/O
bridge 107. In still other embodiments, some or all elements of PPU
202 may be integrated on a single chip with CPU 102.
[0031] In one embodiment, communication path 113 is a PCIe link, as
mentioned above, in which dedicated lanes are allocated to each PPU
202, as is known in the art. Other communication paths may also be
used. An I/O unit 205 generates packets (or other signals) for
transmission on communication path 113 and also receives all
incoming packets (or other signals) from communication path 113,
directing the incoming packets to appropriate components of PPU
202. For example, commands related to processing tasks may be
directed to a host interface 206, while commands related to memory
operations (e.g., reading from or writing to parallel processing
memory 204) may be directed to a memory crossbar unit 210. Host
interface 206 reads each pushbuffer and outputs the command stream
stored in the pushbuffer to a front end 212.
[0032] Each PPU 202 advantageously implements a highly parallel
processing architecture. As shown in detail, PPU 202(0) includes a
processing cluster array 230 that includes a number C of general
processing clusters (GPCs) 208, where C.gtoreq.1. Each GPC 208 is
capable of executing a large number (e.g., hundreds or thousands)
of threads concurrently, where each thread is an instance of a
program. In various applications, different GPCs 208 may be
allocated for processing different types of programs or for
performing different types of computations. The allocation of GPCs
208 may vary dependent on the workload arising for each type of
program or computation.
[0033] GPCs 208 receive processing tasks to be executed from a work
distribution unit within a task/work unit 207. The work
distribution unit receives pointers to processing tasks that are
encoded as task metadata (TMD) and stored in memory. The pointers
to TMDs are included in the command stream that is stored as a
pushbuffer and received by the front end unit 212 from the host
interface 206. Processing tasks that may be encoded as TMDs include
indices of data to be processed, as well as state parameters and
commands defining how the data is to be processed (e.g., what
program is to be executed). The task/work unit 207 receives tasks
from the front end 212 and ensures that GPCs 208 are configured to
a valid state before the processing specified by each one of the
TMDs is initiated. A priority may be specified for each TMD that is
used to schedule execution of the processing task. Processing tasks
can also be received from the processing cluster array 230.
Optionally, the TMD can include a parameter that controls whether
the TMD is added to the head or the tail for a list of processing
tasks (or list of pointers to the processing tasks), thereby
providing another level of control over priority.
[0034] Memory interface 214 includes a number D of partition units
215 that are each directly coupled to a portion of parallel
processing memory 204, where D.gtoreq.1. As shown, the number of
partition units 215 generally equals the number of dynamic random
access memory (DRAM) 220. In other embodiments, the number of
partition units 215 may not equal the number of memory devices.
Persons of ordinary skill in the art will appreciate that DRAM 220
may be replaced with other suitable storage devices and can be of
generally conventional design. A detailed description is therefore
omitted. Render targets, such as frame buffers or texture maps may
be stored across DRAMs 220, allowing partition units 215 to write
portions of each render target in parallel to efficiently use the
available bandwidth of parallel processing memory 204.
[0035] Any one of GPCs 208 may process data to be written to any of
the DRAMs 220 within parallel processing memory 204. Crossbar unit
210 is configured to route the output of each GPC 208 to the input
of any partition unit 215 or to another GPC 208 for further
processing. GPCs 208 communicate with memory interface 214 through
crossbar unit 210 to read from or write to various external memory
devices. In one embodiment, crossbar unit 210 has a connection to
memory interface 214 to communicate with I/O unit 205, as well as a
connection to local parallel processing memory 204, thereby
enabling the processing cores within the different GPCs 208 to
communicate with system memory 104 or other memory that is not
local to PPU 202. In the embodiment shown in FIG. 2, crossbar unit
210 is directly connected with I/O unit 205. Crossbar unit 210 may
use virtual channels to separate traffic streams between the GPCs
208 and partition units 215.
[0036] Again, GPCs 208 can be programmed to execute processing
tasks relating to a wide variety of applications, including but not
limited to, linear and nonlinear data transforms, filtering of
video and/or audio data, modeling operations (e.g., applying laws
of physics to determine position, velocity and other attributes of
objects), image rendering operations (e.g., tessellation shader,
vertex shader, geometry shader, and/or pixel shader programs), and
so on. PPUs 202 may transfer data from system memory 104 and/or
local parallel processing memories 204 into internal (on-chip)
memory, process the data, and write result data back to system
memory 104 and/or local parallel processing memories 204, where
such data can be accessed by other system components, including CPU
102 or another parallel processing subsystem 112.
[0037] A PPU 202 may be provided with any amount of local parallel
processing memory 204, including no local memory, and may use local
memory and system memory in any combination. For instance, a PPU
202 can be a graphics processor in a unified memory architecture
(UMA) embodiment. In such embodiments, little or no dedicated
graphics (parallel processing) memory would be provided, and PPU
202 would use system memory exclusively or almost exclusively. In
UMA embodiments, a PPU 202 may be integrated into a bridge chip or
processor chip or provided as a discrete chip with a high-speed
link (e.g., PCI Express) connecting the PPU 202 to system memory
via a bridge chip or other communication means. In the embodiment
of the invention described in conjunction with FIGS. 3-6, each PPU
202 is implemented with a non-uniform memory architecture, and,
accordingly, each such PPU 202 may have access to multiple
different memory spaces as directed by co-processor enabled
application 134.
[0038] As noted above, any number of PPUs 202 can be included in a
parallel processing subsystem 112. For instance, multiple PPUs 202
can be provided on a single add-in card, or multiple add-in cards
can be connected to communication path 113, or one or more of PPUs
202 can be integrated into a bridge chip. PPUs 202 in a multi-PPU
system may be identical to or different from one another. For
instance, different PPUs 202 might have different numbers of
processing cores, different amounts of local parallel processing
memory, and so on. Where multiple PPUs 202 are present, those PPUs
may be operated in parallel to process data at a higher throughput
than is possible with a single PPU 202. Systems incorporating one
or more PPUs 202 may be implemented in a variety of configurations
and form factors, including desktop, laptop, or handheld personal
computers, servers, workstations, game consoles, embedded systems,
and the like.
[0039] As mentioned, in the embodiment of the invention described
in conjunction with FIGS. 3-6, each PPU 202 is implemented with a
non-uniform memory architecture. Accordingly, each such PPU 202 may
have access to multiple different memory spaces, such as, e.g.,
system memory 104 or PP memory 204, among others, as directed by
co-processor enabled application 134. A compiler and linker
application derived from device driver 103 is configured to
optimize and compile program code in order to generate co-processor
enabled application 134. That program code may initially include
different memory access operations, such as load/store operations
or atomic operations, that may not specify a particular memory
space with which to perform the memory access operations. Such
memory access operations are referred to herein as "generic memory
access operations." In order to optimize the program code, the
compiler and linker application is configured to modify that
program code, as needed, to resolve generic memory access
operations into specific memory access operations that target a
particular memory space, as described in greater detail below in
conjunction with FIGS. 3-6.
[0040] FIG. 3 illustrates the build process used to compile the
co-processor enabled application 134 of FIG. 1, according to one
embodiment of the present invention. Program code 310 includes host
source code 312 and device source code 314. Host source code 312
incorporates programming instructions intended to execute on a
host, such as an x86-based personal computer (PC) or server. The
programming instructions in source code 312 may include calls to
functions defined in device source code 314. Any technically
feasible mechanism may be used to specify which functions are
designated as device source code 314.
[0041] Host source code 312 is pre-processed, compiled, and linked
by a host compiler and linker 322. The host compiler and linker 322
generates host machine code 342, which is stored within
co-processor enabled application 134.
[0042] Device source code 314 is pre-processed, compiled and linked
by a device compiler and linker 324. This compile operation
constitutes a first stage compile of device source code 314. Device
compiler and linker 324 generates device virtual assembly 346,
which is stored within a device code repository 350, residing with
or within co-processor enabled application 134. A virtual
instruction translator 334 may generate device machine code 324
from device virtual assembly 346. This compile operation
constitutes a second stage compile of device source code 314.
Virtual instruction translator 334 may generate more than one
version of device machine code 344, based on the availability of
known architecture definitions. For example, virtual instruction
translator 334 may generate a first version of device machine code
344, which invokes native 64-bit arithmetic instructions (available
in the first target architecture) and a second version of device
machine code 344, which emulates 64-bit arithmetic functions on
targets that do not include native 64-bit arithmetic
instructions.
[0043] Architectural information 348 indicates the real
architecture version used to generate device machine code 344. The
real architecture version defines the features that are implemented
in native instructions within a real execution target, such as the
PPU 202. Architectural information 348 also indicates the virtual
architecture version used to generate device virtual assembly 346.
The virtual architecture version defines the features that are
assumed to be either native or easily emulated and the features
that are not practical to emulate. For example, atomic addition
operations are not practical to emulate at the instruction level,
although they may be avoided altogether at the algorithmic level in
certain cases and, therefore, impact which functions may be
compiled in the first compile stage.
[0044] In addition to the device machine code 344 and device
virtual assembly 346, the device code repository also includes
architecture information 348, which indicates which architectural
features were assumed when device machine code 344 and device
virtual assembly 346 where generated. Persons skilled in the art
will recognize that the functions included within device machine
code 344 and virtual assembly 346 reflect functions associated with
the real architecture of PPU 202. The architecture information 348
provides compatibility information for device machine code 344 and
compiler hints for a second stage compile operation, which may be
performed by a device driver 103 at some time after the development
of co-processor enabled application 134 has already been
completed.
Inter-Procedural Memory Space Optimization
[0045] Device compiler and linker 324 is also configured to perform
various optimization routines with different procedures and/or
functions within program code 310. As mentioned, program code 310
may initially include generic memory access operations that do not
specify a particular memory space, and device compiler and linker
324 is configured to modify that program code to resolve the
generic memory access operations into memory access operations that
target a particular memory space. FIG. 4 describes an approach for
optimizing memory access operations, FIG. 5 describes an approach
to transferring constant variables to reside in a global memory
space, and FIG. 6 outlines an exemplary scenario in which the
approaches discussed in conjunction with FIGS. 4 and 5 may be
beneficial.
[0046] FIG. 4 is a flow diagram of method steps for optimizing
memory access operations, according to one embodiment of the
present invention. Although the method steps are described in
conjunction with the systems of FIGS. 1-2, persons skilled in the
art will understand that any system configured to perform the
method steps, in any order, is within the scope of the present
invention. Device compiler and linker 324 shown in FIG. 3 is
configured to implement the method steps.
[0047] As shown, a method 400 begins at step 402, where device
compiler and linker 324 collects memory access operations within
program code 310 that target a generic memory space. The memory
access operations may be load/store operations or atomic operations
such as, e.g., pointer de-referencing. At step 404, for each memory
access operation collected at step 402, device compiler and linker
324 ascend a use-definition chain generated for the pointer
associated with the memory access operation in order to determine
the specific memory space from which the pointer is derived. Device
compiler and linker 324 may generate the use-definition chain using
conventional techniques, such as data flow analysis, in order to
identify the use of the pointer and any previous definitions
involving the pointer. In one embodiment, device compiler and
linker 324 generates the use-definition chain using live
analysis-based techniques.
[0048] At step 406, device compiler and linker 324 adds each
pointer derived from a specific memory space (such as, e.g., global
memory, local memory, shared memory, etc.) to a vector. At step
408, for each pointer in the vector generated at step 406, device
compiler and linker 324 modifies the memory access operation
associated with that pointer to target the specific memory space
from which the pointer was derived. For example, a particular
pointer p derived from global memory may be de-referenced during a
load operation. By implementing the method 400, device compiler and
linker 324 could replace the pointer de-reference with a load
operation specifically targeting global memory.
[0049] In some situations, device compiler and linker 324 may not
be able to implement the method 400 to modify a given memory access
operation to target a specific memory space within program code
310. Such a situation may occur when program code 310 includes a
branch instruction. Since the outcome of a branch instruction is
unknown until run time, memory access operations that target
different memory spaces depending on the outcome of the branch
instruction may not be modifiable in the fashion described above.
In some cases those memory access operations may be left untouched
as generic memory access operations and resolved at run time.
[0050] However, memory access operations to a memory space reserved
for constant variables may not be effectively resolved at run time.
Since constant memory space typically resides within a read-only
memory, memory access operations associated with constant memory
are fundamentally different than other memory access operations,
and, as such, may not be resolvable at run time. Accordingly,
device compiler and linker 324 is configured to transfer certain
constant variables and the associated memory access operations
within program code 310 to reside in and target, respectively, a
global memory space, as discussed in greater detail below in
conjunction with FIG. 5.
[0051] FIG. 5 is a flow diagram of method steps for transferring
constant variables to reside in global memory space, according to
one embodiment of the present invention. Although the method steps
are described in conjunction with the systems of FIGS. 1-2, persons
skilled in the art will understand that any system configured to
perform the method steps, in any order, is within the scope of the
present invention. Device compiler and linker 324 shown in FIG. 3
is configured to implement the method steps.
[0052] As shown, a method 500 begins at step 502, where, for each
constant address in program code 310, device compiler and linker
324 descends the definition-use chain for the constant address
until a memory access operation is reached. Device compiler and
linker 324 may generate the definition-use chain using conventional
techniques, such as data flow analysis, in order to identify the
declaration of the constant address and any subsequent uses. In one
embodiment, device compiler and linker 324 generates the
definition-use chain using live analysis-based techniques.
[0053] At step 504, for each memory access operation reached in
step 502 and associated with a particular constant address, device
compiler and linker 324 marks a constant declaration associated
with the constant address as "must-transfer" if the memory access
operation is not resolved to a specific memory space.
[0054] At step 506, device compiler and linker 324 generates a
dependency list for each memory access operation. At step 508,
device compiler and linker 324 identifies any dependency lists that
include constant addresses with declarations marked as
"must-transfer." At step 510, device compiler and linker 324 marks
any memory access operations associated with the dependency lists
identified in step 508 as "must-transfer." At step 512, device
compiler and linker 324 marks any constant declarations associated
with constant addresses within the identified dependency lists as
"must-transfer." At step 514, device compiler and linker 324
modifies each transferable constant declaration to specify a
location in global memory space. At step 516, device compiler and
linker 324 modifies each transferable memory access operation to
target global memory. The method 500 then ends.
[0055] By implementing the method 500, device compiler and linker
324 is capable of transferring constant variables to reside in a
global memory space in situations where branch instructions would
otherwise leave memory access operations involving those constant
variables as generic memory access operations. Furthermore, device
compiler and linker 324 is also configured to transfer any constant
variables and associated memory access operations that depend on
previously-transferred variables, thereby ensuring that all
dependent constant variables are transferred together.
[0056] The methods 400 and 500 described above in conjunction with
FIGS. 4 and 5, respectively, are described in greater detail below
in conjunction with FIG. 6 by way of an example.
[0057] FIG. 6 sets forth a pseudocode example to illustrate the
operation of a device compiler and linker, according to one
embodiment of the present invention. As shown, pseudocode 600
includes pseudocode blocks 610, 620, 630, and 640. Pseudocode block
610 includes two constant int declarations for variables c1 and c2
and a shared int declaration for variable s. Pseudocode block 620
includes three pointer assignments p1, p2 and p4 to addresses of
the variables c1, s, and c2. Pseudocode block 630 includes branch
instructions 632 and 634 that assign pointers p3 and p5,
respectively, differently depending on which branch is followed at
run time. Pseudocode block 640 includes memory access operations
that set the data stored at pointers p3, p5, and p1 to variables x,
y, and z, respectively. Persons skilled in the art will understand
that pseudocode 600 described above could be easily implemented in
a variety of programming languages. In one embodiment, pseudocode
600 may be implemented in the CUDA.TM. programming language and may
represent some or all of program code 310.
[0058] The following description represents just one example of
device compiler and linker 324 performing the method 400 described
above in conjunction with FIG. 4. In this example, device compiler
and linker 324 first identifies the memory access operations within
pseudocode block 640, similar to step 402 of the method 400. Those
memory access operations are associated with pointers p1, p3, and
p5, as is shown.
[0059] Device compiler and linker 324 then ascends the
use-definition chain of each such memory access operation, similar
to step 404 of the method 400. In pseudocode 600, device compiler
and linker 324 ascends the use-definition chain of p3 by following
each branch of branch instruction 632 up to the pointer assignments
of p1 and p2 in pseudocode block 620, then tracing variables c1 and
s back to the declaration of those variables within pseudocode
block 610. Similarly, device compiler and linker 324 ascends the
use-definition chain of p5 by following each branch of branch
instruction 634 up to the pointer assignments of p1 and p4 in
pseudocode block 620, then tracing variables c1 and c2 back to the
declaration of those variables within pseudocode block 610. Device
compiler and linker 324 ascends the use-definition chain of p1 by
tracing that pointer back to the pointer assignment in pseudocode
block 620, then tracing variable c1 back to the declaration of that
variable within pseudocode block 610.
[0060] For each pointer associated with the memory access
operations collected in step 404, device compiler and linker 324
adds the pointer to a vector if that pointer is derived from a
specific memory space, similar to step 406 in the method 400. In
pseudocode 600, pointer p1 is derived from constant variable c1,
which resides in constant memory. Accordingly, device compiler and
linker 324 adds p1 to the vector. Pointer p3 is derived from either
p1 or p2, depending on branch instruction 632. Since p1 and p2 are
derived from constant memory and shared memory, respectively, the
memory access associated with p3 cannot be resolved to a specific
memory space and pointer p3 is not added to the vector. Pointer p5
is derived from either of constant variables c1 and c2, and so
regardless of which branch of branch instruction 634 is followed at
run time, p5 will still be derived from constant memory.
Accordingly, device compiler and linker 324 adds p5 to the
vector.
[0061] Device compiler and linker 324 traverses the vector and, for
each pointer in the vector, modifies the associated memory access
operation to target the specific memory space from which the
pointer was derived, similar to step 408 of the method 400. In
doing so, device compiler and linker 324 modifies the memory access
operations of p1 and p5 to specifically target constant memory. The
memory access operation associated with p3 is left as a generic
memory access operation.
[0062] Once the method 400 of FIG. 4 has been performed on the
pseudocode 600, the device compiler and linker 324 may then
re-process pseudocode 600 by performing the method 500 of FIG. 5 on
the pseudocode 600, as discussed by way of example below.
[0063] The following description represents just one example of
device compiler and linker 324 performing the method 500 described
above in conjunction with FIG. 5. In this example, device compiler
and linker 324 first descends the definition-use chain of each
constant address until a memory access is reached, similar to step
502 of the method 500. Device compiler and linker 324 descends the
definition-use chain of constant variables c1 and c2 declared in
pseudocode block 610, until reaching the memory access operations
associated with those constant variables. As shown, c1 can be
traced down to memory access operations involving pointers p1, p3,
and p5, while c2 can be traced down to memory access operations
involving just pointer p5.
[0064] For each of those memory access operations derived from a
particular constant declaration, device compiler and linker 324
marks that constant declaration as "must-transfer" if the memory
access is not resolved to a specific memory space, similar to step
504 of the method 500. As discussed above in the previous example,
the memory access operation associated with pointer p3 was left as
a generic memory access operation, and so device compiler and
linker 324 marks the constant declaration associated with that
memory access operation (the declaration for c1) as
"must-transfer."
[0065] Device compiler and linker 324 then generates a dependency
list for each memory access, similar to step 506 of the method 500.
Device compiler and linker 324 is configured to identify any
dependency lists that include constant addresses with constant
declarations marked as "must-transfer," similar to step 508 of the
method 500. In pseudocode 600, the memory access operation
associated with pointer p1 depends on c1, which was marked as
"must-transfer." Likewise, the memory access operation associated
with pointer p3 depends on c1 and the memory access operation
associated with pointer p5 also depends on c1. Accordingly, device
compiler and linker 324 would identify the dependency lists
associated with those memory access operations.
[0066] Device compiler and linker 324 would then mark the memory
access operations associated with the identified dependency lists
as "must-transfer," similar to step 510 of the method 500. In the
example described herein, device compiler and linker 324 would mark
all of the memory access operations shown in pseudocode block 640
as "must-transfer."
[0067] Device compiler and linker 324 would then mark any other
constant declarations associated with constant addresses in the
identified dependency lists as "must-transfer," similar to step 512
of the method 500. In pseudocode 600, device compiler and linker
324 would determine that the memory access operation for p5 depends
on constant variable c2, and since the dependency list for that
memory access operation was identified previously, then the
constant variable declaration for c2 would also be marked as
"must-transfer."
[0068] Device compiler and linker 324 would then modify each
"must-transfer" constant variable declaration to reside in global
memory, similar to step 514 of the method 500, and then modify each
"must-transfer" memory access operation to target global memory,
similar to step 516 of the method 500. In doing so, device compiler
and linker 324 may also promote data from the constant memory space
to the global memory space, as needed. By performing the technique
described in this example, device compiler and linker 324 transfers
all constant memory variables and memory access operations to
reside in and target, respectively, global memory, thus avoiding
situations where a generic memory access operation may or may not
target constant memory depending on the outcome of a branch
instruction.
[0069] In sum, a device compiler and linker is configured to
optimize program code of a co-processor enabled application by
resolving generic memory access operations within that program code
to target specific memory spaces. In situations where a generic
memory access operation cannot be resolved and may target constant
memory, constant variables associated with those generic memory
access operations are transferred to reside in global memory.
[0070] Advantageously, a graphics processing unit (GPU) is not
required to resolve all generic memory access operations at run
time, thereby conserving resources and accelerating the execution
of the application. Further, the GPU is enabled to perform
additional program code optimizations with the application program
code, including memory access re-ordering and alias analysis,
further accelerating program code execution.
[0071] One embodiment of the invention may be implemented as a
program product for use with a computer system. The program(s) of
the program product define functions of the embodiments (including
the methods described herein) and can be contained on a variety of
computer-readable storage media. Illustrative computer-readable
storage media include, but are not limited to: (i) non-writable
storage media (e.g., read-only memory devices within a computer
such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM
chips or any type of solid-state non-volatile semiconductor memory)
on which information is permanently stored; and (ii) writable
storage media (e.g., floppy disks within a diskette drive or
hard-disk drive or any type of solid-state random-access
semiconductor memory) on which alterable information is stored.
[0072] The invention has been described above with reference to
specific embodiments. Persons skilled in the art, however, will
understand that various modifications and changes may be made
thereto without departing from the broader spirit and scope of the
invention as set forth in the appended claims. The foregoing
description and drawings are, accordingly, to be regarded in an
illustrative rather than a restrictive sense.
* * * * *