U.S. patent application number 16/791361 was filed with the patent office on 2022-01-06 for system and method for generation of event driven, tuple-space based programs.
The applicant listed for this patent is Reservoir Labs, Inc.. Invention is credited to Muthu M. Baskaran, Thomas Henretty, M. H. Langston, Richard A. Lethin, Benoit J. Meister, Nicolas T. Vasilache, David E. Wohlford.
Application Number | 20220004425 16/791361 |
Document ID | / |
Family ID | 1000006034638 |
Filed Date | 2022-01-06 |
United States Patent
Application |
20220004425 |
Kind Code |
A9 |
Baskaran; Muthu M. ; et
al. |
January 6, 2022 |
SYSTEM AND METHOD FOR GENERATION OF EVENT DRIVEN, TUPLE-SPACE BASED
PROGRAMS
Abstract
In a system for automatic generation of event-driven,
tuple-space based programs from a sequential specification, a
hierarchical mapping solution can target different runtimes relying
on event-driven tasks (EDTs). The solution uses loop types to
encode short, transitive relations among EDTs that can be evaluated
efficiently at runtime. Specifically, permutable loops translate
immediately into conservative point-to-point synchronizations of
distance one. A runtime-agnostic which can be used to target the
transformed code to different runtimes.
Inventors: |
Baskaran; Muthu M.; (Old
Tappan, NJ) ; Henretty; Thomas; (Brooklyn, NY)
; Langston; M. H.; (Beacon, NY) ; Lethin; Richard
A.; (New York, NY) ; Meister; Benoit J.; (New
York, NY) ; Vasilache; Nicolas T.; (New York, NY)
; Wohlford; David E.; (Portland, OR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Reservoir Labs, Inc. |
New York |
NY |
US |
|
|
Prior
Publication: |
|
Document Identifier |
Publication Date |
|
US 20210255891 A1 |
August 19, 2021 |
|
|
Family ID: |
1000006034638 |
Appl. No.: |
16/791361 |
Filed: |
February 14, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14492899 |
Sep 22, 2014 |
10564949 |
|
|
16791361 |
|
|
|
|
61880592 |
Sep 20, 2013 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 9/4843
20130101 |
International
Class: |
G06F 9/48 20060101
G06F009/48 |
Goverment Interests
GOVERNMENT LICENSE RIGHTS
[0002] This invention was made with Government support under Award
No. DE-SC0008717, awarded by the Department of Energy. The
Government has certain rights in the invention.
Claims
1. A method of specifying event-driven tasks (EDTs) for an
EDT-based runtime, the method comprising: for an EDT structure
corresponding to a loop structure in code to be executed using an
EDT-based runtime, determining by a processor one or more
dependencies between a pair of instances, a first instance
corresponding to the EDT structure and a second instance
corresponding to the EDT structure or another different EDT
structure, and the determination being based on, at least: (i) a
type of the loop structure, and (ii) a union of respective
individual iteration domains of one or more statements associated
with the loop structure.
2. The method of claim 1, wherein the EDT-based runtime comprises
at least one of SWARM, OCR, and CnC.
3. The method of claim 1, wherein the EDT structure comprises a
tuple comprising: (a) a unique identifier, and (b) start and stop
levels associated with the corresponding loop structure.
4. The method of claim 3, wherein: the code comprises a loop nest,
and the loop nest comprises the loop structure corresponding to the
EDT structure and another loop structure corresponding to a
different EDT structure; and the start level corresponds to a depth
of the other loop structure, and the stop level corresponds to a
depth of the loop structure corresponding to the EDT structure.
5. The method of claim 3, wherein: the code comprises a loop nest,
and the loop nest comprises the loop structure corresponding to the
EDT structure; and the stop level corresponds to a depth of the
loop structure corresponding to the EDT structure.
6. The method of claim 3, wherein determination of a dependency
within the one or more dependencies is further based on the start
and stop levels in the tuple.
7. The method of claim 1, further comprising generating the union
of respective individual iteration domains of the one or more
statements associated with the loop structure.
8. The method of claim 1, further comprising: synthesizing by the
processor an EDT-instance generation statement causing the
EDT-based runtime to spawn a plurality of EDT instances, all
instances corresponding to the EDT structure.
9. The method of claim 1, further comprising synthesizing at least
one dependency statement specifying at least one of the one or more
dependencies, if the at least one dependency is determined to exist
between the pair of instances.
10. The method of claim 9, wherein: the type of the loop structure
corresponding to the EDT structure is sequential; and the at least
one dependency statement comprises a first dependency statement and
a second dependency statement, wherein: the first dependency
statement causes a dummy task to wait for completion of all
operations that correspond to the one or more statements associated
with the loop structure and that are designated to a first EDT
instance of the pair; and the second dependency statement causes
all operations that correspond to the one or more statements
associated with the loop structure and that are designated to a
second EDT instance of the pair to wait for completion of the dummy
task.
11. The method of claim 9, wherein: the type of the loop structure
corresponding to the EDT structure is a permutable, the loop
structure comprising an n.sub.d-dimensional loop nest comprising
n.sub.d permutable loops; at least one antecedent instance in each
of the n.sub.d dimensions, and at least one subsequence instance
are associated with the EDT structure; and the dependency statement
causes operations designated to the subsequent instance to wait for
completion of all operations that are designated to at most one
antecedent instance in each of one or more of the n.sub.d
dimensions.
12. The method of claim 9, wherein: the second instance corresponds
to the other different EDT structure, having associated therewith
another different loop structure; the union of respective iteration
domains further comprises respective iteration domains of one or
more statements associated with the other loop structure; and the
at least one dependency statement causes a task associated with the
first instance to wait for completion of at least one operation
that correspond to the one or more statements associated with the
other loop structure and that is designated to the second EDT
instance.
13. The method of claim 9, wherein synthesis of the at least one
dependency statement comprises deriving by the processor a
templated task tag comprising a tuple comprising: (a) a unique
identifier, and (b) start and stop levels associated with the
corresponding loop structure.
14. The method of claim 13, wherein the derivation of the templated
task tag comprises: computing a number of dimensions (n.sub.d) of
loops causing iteration of statements associated with the loop
structure corresponding to the EDT structure; and generating a
statement for computing a number of iterations based on respective
bounds of a loop in each dimension.
15. The method of claim 1, further comprising: marking by the
processor, one or more loop nodes in a tree of nested loops
representing loops in the code, based on at least one of: (i) a
type of the loop, (ii) a position of the loop within the tree of
nested loops, and (iii) user specification.
16. The method of claim 15, wherein the type of the loop is
sequential.
17. The method of claim 15, wherein the position of the loop within
the tree of nested loops comprises one of: (i) a loop at tile
granularity, and (ii) a loop having a sibling in the tree of nested
loops.
18. The method of claim 15, wherein: the type of the loop is
permutable; and a parent of the loop is within a different band;
and the parent is unmarked.
19. The method of claim 15, further comprising: constructing by the
processor a tree of EDT structures comprising the EDT structure,
each node in the tree of EDT structures representing a different
EDT structure corresponding to a respective marked loop node in the
tree of nested loops.
20. The method of claim 15, further comprising: constructing, by
the processor, a tree of nested loops representing loops in the
code, each loop node in the tree of nested loops corresponding to a
different loop in the code.
21. The method of claim 20, further comprising transforming loops
in the code.
22. The method of claim 20, further comprising tiling loops in the
code.
23. The method of claim 1, further comprising designating the
structure as a parent EDT structure and extracting by the processor
from the parent EDT structure a child-EDT structure, the child
structure being associated with a child loop structure that
excludes at least one loop from the loop structure associated with
the parent structure, wherein: the first instance of pair of
instances corresponds to the child-EDT structure; and the second
instance of the pair of instances corresponds to the child
EDT-structure or the parent EDT-structure.
24-47. (canceled)
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is related to an claims benefit of priority
to U.S. Provisional Patent Application Ser. No. 61/880,592, file on
Sep. 20, 2013 and entitled "System and Method for Generation of
Event Driven, Tuple-Space Based Programs," which is incorporated
herein by reference in its entirety.
FIELD OF THE INVENTION
[0003] This specification relates generally to optimizing compilers
and, in particular, to automatic techniques that facilitate
parallel spawning of tasks on runtimes that support event-driven
tasks (EDTs).
BACKGROUND
[0004] Despite predictions on the end of Moore years, for both
physical and economic reasons Intel has recently declared Moore's
law alive and well. However, as the number of transistors fitting a
given chip area continues to grow, so does the energy required to
enable them, resulting in the heat envelope supported by the
packaging being reached. The era of sequential computing relying on
ever increasing clock speeds and decomposition of the processing
pipeline into ever shorter stages indeed appears to have come to an
end. As Gflops per Watt replaced traditional GHz, clock speeds
stopped increasing and performance metrics started shifting.
Subsequently, due to the same power wall which halted frequency
scaling, the end of multi-core scaling was predicted. Some
commentators estimate that for any chip organization and topology,
multi-core scaling will also be power limited. To meet the power
budget, they project, ever more significant portions of the chip
will have to be turned off to accommodate the increase in static
power loss from increasing transistor count. We are thus entering
the "dark silicon" era.
[0005] From the point of view of programming models, in order to
meet the requirements on power consumption and the necessary levels
of parallelism for keeping the hardware busy, one answer pursued by
researchers is the exploration of large-scale dataflow-driven
execution model. In the dark silicon era as well as at Exascale
levels of parallelism, the envisioned architectures are likely
ill-balanced and will likely exhibit highly volatile performance
and failure characteristics. It is envisioned that applications
will, at least partially, steer away from the MPI bulk-synchronous
model and may rely on relocatable tasks, scheduled by a dynamic,
adaptive, work-stealing runtime.
[0006] These relocatable tasks are known as Event-Driven Tasks
(EDTs). At least one of the runtimes, the Open Community Runtime
(OCR), can support the execution model on the Intel Runnemede
research architecture. In this context, communication and locality
are performance and energy bottlenecks. Latencies to remote data
will generally grow to accommodate lower energy budgets devoted to
communication channels. As such, to hide these long latency
operations, it is beneficial to overprovision the software and
massive amounts of parallelism may need to be uncovered and
balanced efficiently and dynamically. In some systems, such as
GPGPU based systems, and in particular in CUDA, a user may specify
more parallelism than can be exploited for the purpose of hiding
latencies. The user specification of parallelism, however, is
generally not based on any systematic analysis of the loop-carried
dependencies and, as such, may not lead to parallelization
necessary to meet simultaneously the performance requirements and
power budgets.
[0007] Traditional approaches to parallelism typically require the
programmer to describe explicitly the sets of operations that can
be parallelized in the form of communicating sequential processes
(CSPs). The fork-join model and the bulk-synchronous model are
commonly used methodologies for expressing CSPs, for shared and
distributed memory systems, respectively. As multi-socket,
multi-core computers are becoming ubiquitous and are trending
towards ever more cores on chip, new parallel programming patterns
are emerging. Among these patterns, the task-graph pattern is being
actively pursued as an answer to the overprovisioning and
load-balancing problems. This model can support a combination of
different styles of parallelism (data, task, pipeline). At a very
high-level, the programmer writes computation tasks which can: (1)
produce and consume data, (2) produce and consume control events,
(3) wait for data and events, and (4) produce or cancel other
tasks. Dependences between tasks must be declared to the runtime
which keeps distributed queues of ready tasks (i.e., whose
dependences have all been met) and decides where and when to
schedule tasks for execution. Work-stealing can be used for
load-balancing purposes. Specifying tasks and dependences that are
satisfied at runtime is common to CnC, OCR, SWARM and to other
Event Driven runtimes.
[0008] The user specification tasks, however, is generally not
based on any systematic analysis of the program to be executed, so
as to enable a portioning of the operations of the program into
tasks that can fully exploit the parallel-processing power of a
target runtime. Because the tasks themselves are often defined
without the benefit of a systematic analysis, the dependencies
associated with the tasks are usually not expressed to the
parallelization necessary to achieve the required performance
and/or to meet a power budget.
[0009] One transformation system for expressing tasks and
dependencies therebetween is based on the polyhedral model. Some
transformation systems allows for intricate transformation
compositions, but the applicability of these system is generally
limited because they employ static dependence analysis. Such
transformation systems generally decide at compile time whether to
parallelize a loop structure or not and, as such, typically require
excessive compile times and/or may not achieve the parallelization
that can be obtained using EDT-based runtimes. Some techniques can
expand the scope of analyzable codes by (1) computing
inter-procedural over- and under-approximations that present a
conservative abstraction to the polyhedral toolchain, and (2) by
introducing more general predicates that can be evaluated at
runtime through fuzzy-array dataflow analysis. In practice,
conservative solutions mix well with the polyhedral toolchain
through a stubbing (a.k.a. blackboxing) mechanism and parallelism
can be expressed across irregular code regions. Unfortunately this
is not sufficient because the decision to parallelize or not
remains an all-or-nothing compile-time decision performed at the
granularity of the loop. In contrast EDT-based runtimes allow the
expression of fine-grain parallelism down to the level of the
individual instruction (overhead permitting), and the
transformation systems discussed above do not permit runtime
exploration of parallelism. Some techniques allow for performing
speculative and runtime parallelization using the expressiveness of
the polyhedral model. In these techniques, the speculation may be
erroneous and/or the compile time can be too long.
[0010] In some techniques, a dependence analysis based on a
directed acyclic graph (DAG) of linear-memory array descriptors can
generate lightweight and sufficient runtime predicates to enable
adaptive runtime parallelism. These methods may enable runtime
evaluation of predicates, and can result in significant speedups on
benchmarks with difficult dependence structures. In these
techniques, however, parallelism is still exploited in a fork-join
model via the generation of OpenMP annotations and, as such, these
techniques generally limit the parallelization and performance
benefits that can be achieved using EDT-based runtimes that use the
event-driven task model.
SUMMARY
[0011] In various embodiments, the systems and methods described
herein enable automatic generation of event-driven,
tuple-space-based programs from a sequential program specification.
A hierarchical mapping solution using auto-parallelizing compiler
technology can generate EDT instances for several different
EDT-based runtimes. This is achieved, at least in part, by
employing (1) a mapping strategy with selective trade-offs between
parallelism and locality to extract fine-grained EDTs, and (2) a
retargetable runtime application program interface (API) that can
capture common aspects of the EDT programming model and can allow
for uniform translation, porting, and comparisons between runtimes.
Specifically, complex loop nest restructuring transformations are
applied to construct a logical tree representation of a program to
be executed using an EDT-based runtime. This representation is
mapped to a tree of EDT types. Each EDT type is associated with a
unique (id, tag tuple) pair in the generated program. Dependency
statements based on tag tuples can be generated at compile time,
and these statements allow for determining, at runtime, whether a
required dependency is met. A runtime-agnostic layer (RAL) (e.g., a
C++ layer) may be used for retargeting the statements that spawn,
at runtime, EDT instances corresponding to each EDT type and/or for
retargeting one or more dependency statements to any selected
runtime e.g., Intel's CnC, ETI's SWARM, and the Open Community
Runtime (OCR).
[0012] In general, various embodiments of the systems and methods
described herein perform program analysis and transformation in a
systematic, automated manner. An analyzable sequential
specification may be converted into an intermediate representation.
Thereafter, instance-wise (corresponding to loop instances)
dependence analysis with extensions to support encapsulated
non-affine control-flow hidden within summary operations (a.k.a.
blackboxes), may be performed. Scheduling may be used to optimize a
trade-off between parallelism, locality, and/or other metrics
(e.g., estimated peak power, estimated total energy, etc.).
Non-orthogonal tiling of imperfectly nested loops with a heuristic
that balances a model of data reuse, cache sizes, and performance
of streaming prefetches may also be performed, and may be followed
by EDT formation from a tree representation of the tiled program.
Dependencies between EDT instances of various EDT types are then
generated. RAL code may be generated, which when targeted to a
selected runtime (i.e., compiled for the selected runtime), can
enable that runtime to determine dependencies between different
tasks to be executed by the runtime. The RAL can allow for
performing EDT-instance-dependency analysis independently of the
implementation details of any selected runtime, while
simultaneously allowing for expressing such dependencies to not
just one particular runtime but to any selected runtime.
[0013] In relation to related techniques, the various embodiments
described herein are significantly different at least as follows.
First, the analysis and synthesis process, that may be implemented
by a system configured to perform one or more process steps, is
generic and can target different runtimes. The process is
extensible to other runtimes that may become available in the
future. The experiments discussed below show the variability
between three different runtimes and the benefit of a nimble,
adaptive strategy, as facilitated by various embodiments. Second,
the process can be decentralized and can be fully asynchronous in
the creation of tasks and the dependences therebetween. Other
solutions generally must first construct a full graph and then only
begin useful work, which can be computationally expensive, if not
prohibitive. Considering Amdahl's law, the process according to
various embodiments can scale on a large numbers of processors and
distributed memory. Third, the baseline dependence specification
mechanism according to various embodiments is scalable at both
compile-time and runtime by virtue of exploiting loop types and
dependence information on restructured loops available from the
scheduler.
[0014] Accordingly, in one aspect a method is provided for
specifying event-driven tasks (EDTs) for execution by an EDT-based
runtime. The method includes analyzing by a processor an EDT
structure corresponding to a loop structure in code to be executed
using an EDT-based runtime. Specifically, the method includes
determining by the processor one or more dependencies between a
pair of instances. A first instance may correspond to the EDT
structure and a second instance may correspond to the EDT structure
or to another different EDT structure. The determination is based
on, at least: (i) a type of the loop structure, and (ii) a union of
respective individual iteration domains of one or more statements
associated with the loop structure. A loop structure, in general,
includes two or more nested loops but can include a single loop.
Examples of EDT-based runtimes include, but are not limited to,
SWARM, OCR, and CnC.
[0015] In some embodiments, the EDT structure comprises a tuple
that includes: (a) a unique identifier, and (b) start and stop
levels associated with the corresponding loop structure. A
tuple-based tag associated with an EDT structure/type is typically
different than a tuple-based tag associated with instances of the
EDT type/structure. Specifically, in tags of the EDT
type/structure, the start level typically corresponds to a parent
of the EDT type/structure and the stop level may correspond to the
outermost loop of the loop structure. In the tags associated with
the instances, the start and stop levels may correspond to the
levels of the outermost and innermost loops of a loop structure
associated with the EDT type/structure corresponding to the EDT
instances.
[0016] In some embodiments, the code includes a loop nest, and the
loop nest includes the loop structure corresponding to the EDT
structure. The loop nest may include another loop structure,
corresponding to a different EDT structure. The start level may
correspond to a depth of the other loop structure, and the stop
level may correspond to a depth of the loop structure that
corresponds to the EDT structure. In some embodiments, the code
includes a loop nest, and the loop nest includes the loop structure
corresponding to the EDT structure. The stop level may corresponds
to a depth of the loop structure corresponding to the EDT
structure, and the start level may correspond to a level/depth of a
root node, which can be designated to be zero, one, or any other
suitable number. The determination of a dependency that is included
within the one or more dependencies may be further based on the
start and stop levels in the tuple.
[0017] In some embodiments, the method further includes generating
the union of respective individual iteration domains of the one or
more statements associated with the loop structure. The method may
also include synthesizing by the processor an EDT-instance
generation statement causing the EDT-based runtime to spawn a
number of EDT instances. All of the spawned instances may
corresponding to the EDT structure that is analyzed. Alternatively
or in addition, the method may include synthesizing at least one
dependency statement specifying at least one of the one or more
dependencies, if at least one dependency is determined to exist
between the pair of instances.
[0018] In one embodiment, the type of the loop structure
corresponding to the EDT structure is sequential. The one or more
dependency statements that may be synthesized include a first
dependency statement and a second dependency statement. The first
dependency statement may cause a dummy task to wait for completion
of all operations that correspond to the one or more statements
associated with the loop structure and that are designated to a
first EDT instance of the pair. Thus, the dummy statement waits
till all operations that corresponds to a certain iteration of the
sequential loop have been completed. The second dependency
statement may cause all operations that correspond to the one or
more statements associated with the loop structure and that are
designated to a second EDT instance of the pair to wait for
completion of the dummy task. Thus, operations corresponding to a
next iteration of the sequential loop must wait for the completion
of the dummy task and, in effect, must wait for all
tasks/operations that are associated with a previous iteration of
the sequential loop.
[0019] In some embodiments, the type of the loop structure
corresponding to the EDT structure is a permutable, and the loop
structure includes an n.sub.d-dimensional loop nest that includes
n.sub.d permutable loops. At least one antecedent instance in each
of the n.sub.d dimensions, and at least one subsequence instance
are associated with the EDT type/structure. The dependency
statement may cause operations designated to the subsequent
instance to wait for completion of all operations that are
designated to at most one antecedent instance in each of one or
more of the n.sub.d dimensions. Thus, for a particular task
associated with a band of permutable loops, dependencies may be
evaluated at runtime with respect to at the most one task, and not
all tasks, associated with a loop in each dimension of the band of
permutable loops.
[0020] In some embodiments, the second instance corresponds to the
other different EDT structure/type. Another different loop
structure is associated with the other EDT structure/type. The
union of respective iteration domains may further include
respective iteration domains of one or more statements associated
with the other loop structure. As such, at least one dependency
statement may cause a task associated with the first instance to
wait for completion of at least one operation (e.g., if the other
loop structure is permutable, and all operations, if the other loop
structure is sequential) that correspond to the one or more
statements associated with the other loop structure and that is
designated to the second EDT instance.
[0021] Synthesis of one or more dependency statements may include
deriving by the processor a templated task tag that includes a
tuple that includes: (a) a unique identifier, and (b) start and
stop levels associated with the corresponding loop structure. The
derivation of the templated task tag may include computing a number
of dimensions (n.sub.d) of loops that may cause iterations of
statements associated with the loop structure corresponding to the
EDT structure. The derivation may also include generating a
statement for computing a number of iterations based on respective
bounds of a loop in each dimension.
[0022] In some embodiments, the method includes marking by the
processor, one or more loop nodes in a tree of nested loops
representing loops in the code. The marking may be performed, based
on at least one of: (i) a type of the loop, (ii) a position of the
loop within the tree of nested loops, and (iii) user specification.
The type of the loop can be sequential. The position of the loop
within the tree of nested loops may include one of: (i) a loop at
tile granularity, and (ii) a loop having a sibling in the tree of
nested loops. In some embodiments, the type of the loop is
permutable, a parent of the loop is within a different band, and
the parent is unmarked.
[0023] The method may further include constructing by the processor
a tree of EDT structures that includes the EDT structure that is
analyzed. Each node in the tree of EDT structures may represent a
different EDT structure corresponding to a respective marked loop
node in the tree of nested loops. In some embodiments, the method
includes constructing, by the processor, a tree of nested loops
representing loops in the code. Each loop node in the tree of
nested loops may correspond to a different loop in the code. The
method may further include transforming loops in the code.
Alternatively or in addition, the method may include tiling loops
in the code.
[0024] In some embodiments, the method is hierarchical and includes
designating the structure as a parent EDT structure. The method
also includes extracting by the processor from the parent EDT
structure a child-EDT structure. The child structure is associated
with a child loop structure that excludes at least one loop from
the loop structure associated with the parent structure. The first
instance of pair of instances may correspond to the child-EDT
structure, and the second instance of the pair of instances may
correspond to the child EDT-structure or to the parent
EDT-structure.
[0025] In another aspect, a computer system includes a first
processor and a first memory coupled to the first processor. The
first memory includes instructions which, when executed by a
processing unit that includes the first processor and/or a second
processor, program the processing unit to determine one or more
dependencies between a pair of instances. A first instance may
correspond to the EDT structure and a second instance may
correspond to the EDT structure or to another different EDT
structure. The determination is based on, at least: (i) a type of
the loop structure, and (ii) a union of respective individual
iteration domains of one or more statements associated with the
loop structure. A loop structure, in general, includes two or more
nested loops but can include a single loop. Examples of EDT-based
runtimes include, but are not limited to, SWARM, OCR, and CnC. In
some embodiments, the second memory coupled to the second processor
can receive through a network the instruction stored in the first
memory. In various embodiments, the instructions can program the
processing unit to perform one or more of the method steps
described above.
[0026] In another aspect, an article of manufacture that includes a
non-transitory storage medium has stored therein instructions
which, when executed by a processor program the processor to
determine one or more dependencies between a pair of instances. A
first instance may correspond to the EDT structure and a second
instance may correspond to the EDT structure or to another
different EDT structure. The determination is based on, at least:
(i) a type of the loop structure, and (ii) a union of respective
individual iteration domains of one or more statements associated
with the loop structure. A loop structure, in general, includes two
or more nested loops but can include a single loop. Examples of
EDT-based runtimes include, but are not limited to, SWARM, OCR, and
CnC. In various embodiments, the instructions stored in the article
of manufacture can program the processor to perform one or more of
the method steps described above.
BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The implementations depicted therein are provided by way of
example, not by way of limitation, wherein like reference numerals
generally refer to the same or similar elements. In different
drawings, the same or similar elements may be referenced using
different reference numerals. The drawings are not necessarily to
scale, emphasis instead being placed upon illustrating aspects of
the invention. In these drawings:
[0028] FIG. 1 schematically depicts an example of an EDT antecedent
graph;
[0029] FIG. 2 illustrates an exemplary process of generating a
structure of EDT types, according to one embodiment;
[0030] FIG. 3 illustrates an example of a process of determining
dependencies in permutable loops, according to one embodiment;
[0031] FIG. 4 depicts an example of a loop nest that includes
sequential loops;
[0032] FIG. 5 depicts an example of a templated expression that can
be used to express and/or evaluate dependency between tasks,
according to one embodiment;
[0033] FIG. 6 schematically depicts an organization of EDTs
spawning and synchronization;
[0034] FIG. 7 schematically depicts hierarchical EDTs;
[0035] FIGS. 8-12 respectively include Tables 1-5, showing various
experimental results for three different runtimes; and
[0036] FIG. 13 depicts an example of a system according to one
embodiment, and an example of the environment of such a system.
DETAILED DESCRIPTION
[0037] In various embodiments, an intermediate representation of a
code to be executed using an EDT-based runtime is based on a
hierarchical dependence graph. The nodes of the graph may be
statements that represent operations grouped together in the
internal representation. A typical unit of program analysis and
transformation is a statement. A statement S can be simple or
arbitrarily complex (i.e., an external precompiled object), as long
as it can be approximated conservatively. The edges of the graph
are dependences as defined below. An iteration domain for S,
D.sup.S is an ordered multi-dimensional set of iterations. An
instance of an iteration is written as i.sub.S. The (lexicographic)
order relation between two iterations i and j is defined by
i<<j if and if and only if i occurs before j in the program.
By introducing y, the symbolic, constant parameters of the program,
an iteration domain is the set {i.sub.s D.sup.S (y)}. Operations to
manipulate domains and their inverse include projections to extract
information along a sub-domain; image by a function to transform a
domain into another domain; intersection to construct the
iterations that are common to a list of domains; and index-set
splitting to break a domain into disjoint pieces. Exact projection
operations are computationally expensive, sometimes prohibitively
so, even at compile time. The implications are discussed below.
[0038] A scheduling function .THETA..sup.S is a linear affine
function that partially reorders the iterations of S in time. The
order << extends to time after scheduling is applied. In this
context, a dependence (T.fwdarw.S) is a relation between the set of
iterations of S and T and expresses that T depends on S. This
relation conveys the information that some iteration i.sup.T
D.sup.T (y) depends on i.sup.S D.sup.S (y) (i.e., they access the
same memory location by application of a memory reference) and that
i.sup.S<<i.sup.T in the original program. Set relation
{(i.sup.T, i.sup.S).di-elect cons..sub.T.fwdarw..sub.S(y))} or
.sub.T.fwdarw..sub.S(y) are typically generated to refer to the
specific iterations of T and S that take part in the dependence.
The multigraph of statement nodes and dependence edges is referred
to as the generalized dependence graph (GDG), and a GDG=(V,E),
i.e., a set of vertices and edges in the graph, respectively, is
generated in various embodiments.
[0039] The code to be executed using the EDT-based runtime may be
parallelized using a scheduler (e.g., Pluto) that can optimize
parallelism and locality in sequences of imperfectly nested loops.
Optimization may be obtained by unifying the tilability conditions
with scheduling techniques. The following is a brief review the
affine scheduling formulation. In one embodiment, the input of the
affine scheduling problem is a GDG. Following the standard
conventions, .PHI..sub.S is used to denote a 1-dimensional affine
schedule for statement S. For each edge in the GDG:
.delta.(y).gtoreq..PHI..sub.T(i.sub.T,y)-.PHI..sub.S(i.sub.S,y).gtoreq.0-
, (i.sub.T,i.sub.S).di-elect cons..sub.T.fwdarw.S(y)
By combining all of the dependences of the program, a feasible
linear space that can be subject to various optimization problems
can be formed. The parametric affine form .delta.(y) can be
interpreted as the maximal dependence distance between any two
schedules. In particular, if .delta.(y) can be minimized to 0, then
the solution .PHI. is communication-free and is thus parallel.
Similarly, if .delta.(y) can be minimized to a positive constant c,
then only local communication is needed and broadcast can be
eliminated. One iterative algorithm allows finding an independent
solution that is valid for the same set of dependence edges. This
implies the induced loops are permutable.
[0040] Scalable Dependence Computation Between EDTs: The EDT-based
runtimes typically require the programmer to specify dependences
between EDTs to constrain the order of execution for correctness
purposes. Dependence relations may be exploited by the runtime to
determine when a task is ready and may be scheduled for execution.
An EDT-specific challenge lies in the tractable computation of
dependences at compile-time and the overhead of their exploitation
at runtime.
[0041] The requirements for dependence relations between EDTs are
significantly different than for dependence analysis between
statements. The analysis of dependence between statements is only
concerned with original program order of statements and can be
captured by the set.
.sub.T.fwdarw.S(y)={(i.sup.S,i.sup.T).di-elect
cons.D.sup.S.times.D.sup.T|i.sup.S<<i.sup.T,M.sub.S[i.sup.S|=M.sub.-
T|i.sup.T]},
where M.sub.S and M.sub.T are memory access functions (typically
Read-After-Write affine indices in the same array). Array dataflow
analysis goes a step further and takes into account all possible
interleaved writes to keep only the true producer-consumer
dependences. A dataflow dependence can then be expressed as:
.times. .times. r -> s .function. ( y ) = .times. T -> s
.function. ( y ) - { W .times. T .times. S .times. ( .times. T
-> w .function. ( y ) .times. W -> s .function. ( y ) ) } .
##EQU00001##
where .PI. is the projection operator from T.times.W.times.S to
T.times.S. The formulation with set differences and the projector
operator merely simplifies the exposition of the problem. Even if
the projection operator is not explicitly stated, the dependency
analysis nevertheless requires the solution of a parametric Integer
Linear Programming (ILP) problem, which can be computationally
intensive.
[0042] In contrast, while determining dependences between EDTs one
or more of the following factors are also taken into account. In
some embodiments, ordering may need to be computed on a transformed
schedule, and the scheduling may include tiling transformations,
possibly at several levels. Different instances of a statement may
belong to the same tile. This is a projection operation that cannot
be avoided when computing dependences exactly. By virtue of
exploiting parallelism, the "last-write" information generally
becomes dynamic and may introduce the need for sets of dependence
relations.
[0043] One way to specify these relations automatically is to
compute exactly the dependences between tasks at compile-time based
on producer-consumer relationships. This, however, can lead to the
following problems: First, the dependences may be redundant. A
straightforward dependence algorithm only considers
producer-consumer relations on accesses to memory locations. In the
context of EDTs, without special treatment to prune redundancies,
all these dependences would be generated and can translate into a
high runtime overhead.
[0044] Second, perfectly pruning dependences statically requires
the static computation of the "last-write" the general solution of
which is a Quasi-Affine Selection Tree (QUAST). Computing this
information exactly is often very expensive computationally, and is
sometimes prohibitively expensive on original input programs or
code to be executed using a EDT-based runtime. After scheduling and
tiling, the complexity of the "last-write" computation is further
increased by the application of a projection operator (e.g.,
because several statement instances may belong to the same tile
instance).
[0045] Finally, the dependence relations between tasks are
generally non-convex, arising from the projection of the dependence
relation on a sub-space. The projection operator is non-convex.
With reference to FIG. 1, consider the possible dependence paths
between the origin=(0, 0) and (i, j)=(1,1). These correspond to the
number of paths of Manhattan distance two between these points on a
uniform 2-D grid. In particular, task (i, j)=(0, 0) has no
antecedents and can start immediately whereas (i, j)=(1, 1) has two
antecedents. Task (i, j) has ij redundant dependences (i.e. the
"volume" of the i.times.j region) which reduces to 0, 1, or 2
transitively unique dependences. In general, for one level of
hierarchy, the number of these dependences varies according to
whether the task coordinates lie on the edge vertex, edge line, or
on the interior of the 2D task space. This case can be handled by
creating a centralized, sequential loop that scans the Cartesian
product of iteration spaces of source and destination EDTs for each
single dependence. This mechanism, however, incurs high
computational overhead. To amortize/minimize such overhead, tile
sizes are generally kept large. However, if overprovisioning is to
be achieved, it is desirable to have significantly smaller EDTs,
and the overhead of having small EDTs is computationally expensive,
if not prohibitive, when a Cartesian-product based technique is
used for exploring the dependencies.
[0046] Various embodiments described below facilitate efficient
instantiation and execution of tasks by an EDT-based runtime,
without incurring significant runtime and compile-time computation
overheads. The operation of an exemplary system and an exemplary
method is based on loop properties rather than requiring explicit
computation all possible dependences between tasks.
[0047] Tree Representation and EDT Formation: Specifically, in one
embodiment, after scheduling and tiling, the transformed
program/code is represented as a tree of imperfectly nested loops,
similar to an abstract syntax tree (AST). While the discussion
below generally refers to a tree of loops, any suitable structure
such as annotated lists or arrays, a database, etc., can be used to
represent the program/code to be executed using a selected
EDT-based runtime and, in particular, the loop structures in such
program/code. Two main differences are noted below between a
traditional AST and a tree of loops. First, the tree of nested
loops representation is oblivious to the effects of loop
transformations (which may include peeling, loop shifting,
parameter versioning, and index-set splitting). Advantageously,
this representation thus allows for compositions of transformations
that can maximize parallelism that may be needed for
overprovisioning, as discussed above. The code generation can
address the intricacies of reforming the control-flow-efficient
transformed loop nests. Second, subsequent loop transformations may
be further composable and can preserve the independence of the
representation with respect to complex control flow.
[0048] In various embodiments, the tree of nested loops structure
is characterized by the integer "beta vector" that specifies
relative nesting of statements. Statements have identical first d
beta component if and only if they are nested under d common loops.
The bounds of the loops may be completely different for each
statement. As soon as the beta component differ, the loops are
distributed, the order of the loops being consistent with the order
of the beta component.
[0049] Each node in the tree therefore corresponds to a loop and
has a loop type associated therewith. To uniformize the EDT
extraction process, a root node may be introduced in the tree. The
root node does not correspond to any loop but is the antecedent of
all nodes of the tree. With reference to FIG. 2, in one embodiment,
a breadth-first traversal is performed on the tree structure that
is induced by the beta vectors, so as to mark certain nodes of the
tree. This process can form sequences of perfectly nested
consecutive loops with compatible types for dependence inference
purposes, as discussed below. In particular, permutable loops of
the same band can be mixed with parallel loops. Permutable loops
belonging to different bands, however, are not be mixed in one
implementation.
[0050] The BFS traversal may be stopped when the granularity of a
tile is reached. This can create EDTs at the granularity of tiles.
Alternatively the user may specify the nodes to be marked, ignoring
the tile granularities. The process introduces the remaining nodes
necessary to accommodate: (a) changes in permutable bands, (b)
sequential loops, and (c) imperfectly nested loops that may require
too many dependences (as one example in some situations, nodes that
have siblings). The processing of all of these nodes is described
below.
[0051] Once a tree of nested loops (a structure, in general) is
marked, one compile-time EDTs is formed for each marked non-root
node, as follows: A new unique identifier (ID) is selected for the
EDT type that is being constructed for a visited node. The start
and stop levels for this EDT type are also determined. The start
level is the level of the first marked ancestor of the node for
which the EDT type is being constructed, and the stop level is the
level of the node. The statements nested below the node are
filtered and attached to the current EDT type, i.e., the EDT type
being constructed. A union of iteration domains is designated as
the iteration domain of the EDT type. The union of iteration domain
includes a union of the individual domains of all of the statements
associated with the EDT type being constructed.
[0052] This process yields a tree of EDT types. It should be
understood that, the tree is only one structure used for
illustration and other structures such as annotated lists, arrays,
a database, etc., are within the scope of various embodiments. The
coordinates of each EDT type can be expressed in a multidimensional
tag space, and are uniquely determined by the loops [0, stop] in
the code. Coordinates [0, start) are received from the parent EDT
type, and the coordinates [start, stop] can be determined locally
from loop expressions and from loop levels. The combination of an
EDT ID and its coordinates can uniquely identify each EDT
instance.
[0053] Dependence Specification With Loop Types Information: In one
embodiment, while performing EDT instance-level dependence
analysis, a direct relation between iterations in the source and
target domains is obtained. These relations tend to be
computationally less expensive than Cartesian-product-based
expressions and, as such, can be used to determine efficiently at
compile time the expression of EDT-level dependencies. Such
determination of dependencies and synthesis of corresponding
dependency statements usually does not require computation of
projections. The synthesized dependency statements can be evaluated
at runtime in a distributed, asynchronous mode and generally not
require iterating over a high-dimensional loop nest at runtime. In
particular, to avoid iterations over a high-dimensional loop nest
at runtime, a decentralized view of the problem employs a
get-centric approach in which at runtime, an EDT instance can query
one or more predecessors thereof whether those predecessors have
finished respective executions thereof. This approach minimizes the
number of puts in a concurrent hash table, which are notoriously
more expensive than gets.
[0054] Individual dependence relations between statements are
generally non-invertible. Consider the relation [i, i].fwdarw.[i,
j], i.e., iteration [i, i] depends on iteration [i, j]. Forming the
inverse relation requires a projection, which typically gives rise
to many dependences at runtime. A particular advantage of various
embodiments is that although individual dependence relations
between statements may require projections, aggregate dependence
relations between EDT instances may not. Loop types exhibited by
the scheduler allows for a scalable computation of these aggregate
dependences, as described below. Parallel loops are the simplest
type of dependence relation in the program: they carry no
dependence. As a consequence, no special conditional needs to be
evaluated at runtime.
[0055] A permutable band of loops over induction variables
(i.sub.1, . . . , i.sub.n) has only forward dependences. These can
always be expressed conservatively by the set of n invertible
relations:
{[i.sub.1,i.sub.2, . . . ,i.sub.n]+e.sub.k.fwdarw.[i.sub.1,i.sub.2,
. . . ,i.sub.n], k.di-elect cons.[1,n]},
where e.sub.k is the canonical unit vector in dimension k. In order
to infer dependences for a nest of permutable loops each task
determines from its coordinates in the task space whether it is a
boundary task or an interior task and which other tasks it depends
on. For the purpose of illustration, consider a three-dimensional
(3D) task (i.e., n=3, and loops (i, j, k) are loops across tasks or
are inter-task loops). It should be understood that three as the
number of dimensions is illustrative only and that in general, a
permutable loop structure can include a single loop, two permutable
loops, or more than three (e.g., 4, 6, 10) permutable loops. In
various embodiments, the tags corresponding to EDT types or
structures are different than the tags associated with EDT
instances corresponding to a particular EDT type/structure. The
tags associated with the EDT instances can account for the loops of
the loop structure that is associated with the corresponding EDT
type/structure. In this example, the instance tags are based on the
bounds of loops i, j, and k, and can uniquely identify the
different instances of the permutable loop band (i, j, k).
[0056] An example of a code segment depicted in FIG. 3 can exploit
dependences of distance one along each dimension, i.e., Boolean
expressions are formed by plugging i-1, j-1, and k-1 into the
expression of the loop bounds. For each permutable loop dimension,
we introduce a condition to determine whether the antecedent of the
task along that dimension is part of the interior of the inter-task
iteration space. When the condition evaluates to true, the task
must wait (i.e. get) for its antecedent to complete. To illustrate,
FIG. 3 depicts an example of the interior_1 computation for the
dimension corresponding to loop i. Booleans interior_2 and
interior_3 for the dimensions j and k, respectively, can also be
determined.
[0057] Sequential loop is the most restrictive type for loops. It
imposes a fully specified execution order on the current loop with
respect to any loop nested below it. To visualize these effects,
consider the code depicted in FIG. 4 as an example. In this
example, the function f reads a portion of array A such that the
dependence structure is (seq, doall, seq, doall). Suppose that a
task has the granularity of a single (t, i, j, k) iteration of the
innermost statement. The dependence semantic from loop t is that
any task (t, i, j, k) depends on all of its antecedents (t-1, *, *,
*). Similarly from loop j, any task (t, i, j, k) depends on all of
its antecedents (t, i, j-1, *). If all dependence relations were to
be exposed to the runtime, the t loop would require evaluation of
N.sup.6 dependences which generally prohibitive for a typical loop
bound N.
[0058] In one embodiment, this problem can be addressed by
generating a dummy, one-dimensional (1D) fan-in/fan-out task,
similar to a tbb::empty_task. This has the effect of reducing the
"Cartesian product" effect of dependence relations. The dependence
semantic then becomes: any task (t, i, j, k) depends on sync(t) and
sync(t) depends on all tasks (t-1, i, j, k). The number of
dependences thus reduces to 2 N.sup.3 which is significantly
smaller than N.sup.6 dependencies. Depending on the value of N, the
reduced number of dependencies 2 N.sup.3 can still impose a
significant runtime overhead. As such, in one embodiment employs
hierarchical separation of concerns for sequential loops.
Specifically, an additional level of hierarchy can be generated in
the task graph associated with a sequential loop (e.g., the loop
j), effectively acting as a tbb::spawn_root_and_wait. To
accommodate this separation of concerns, different runtime EDTs,
each targeted for a particular EDT-based runtime can be generated
for each compile time EDT.
[0059] Runtime Agnostic Layer: In one embodiment, a runtime
agnostic layer (RAL) includes a set of C++ templated classes to
build expressions evaluated at runtime, along with an application
program interface (API) to be used by a compiler targeting a
selected runtime. Languages other than C++ allowing templated
structures or permitting emulation of templated structures (e.g.,
via macros) can also be used. The API aims at being a greatest
common denominator for features across runtimes. In one embodiment,
a central element is the templated TaskTag which encapsulates the
tuple holding the coordinates of the EDT type being analyzed in the
tag space. Specialized tuples for each runtime can be derived from
this TaskTag and may optionally extend the TaskTag with
synchronization constructs to implement async-finish. TaskTags may
be passed along with EDT instantiation statements as function
parameters in a selected runtime such as SWARM, OCR, and CnC.
[0060] Templated Expressions: In one embodiment, template
expressions were used to capture complex loop expressions and to
dynamically evaluate, at runtime, inverse dependence relations. An
exemplary assignment depicted in FIG. 5 can encode the
multi-dimensional ranges generated by some embodiments of the
EDT-dependency determination method described above. Operations on
these ranges take tuples as input and may return Booleans or other
tuples. The operations include evaluation of the expression at a
tuple, comparisons at a tuple, and computations of the minimum and
maximum given a tuple range (bounding box computation). These
expressions may be used as described below, referring back to FIG.
3. First a tuple of "terms" is created that encapsulates the
induction variables (t1, t2, t3), i.e., (i, j, k) in FIG. 3, and
parameters (T, N) of the code, any of which may appear in one or
more loop-bound expressions. Then, templated expressions p1, p2,
and p3 are declared which capture the lower and upper bound
expressions governing the iterations of the terms. Lastly, based on
the lower and upper bound expression, runtime dependences are
generated using a templated construct which can dynamically capture
the non-convex Boolean evaluations from FIG. 3. These expressions
are oblivious to the complexity of the loop expressions which can
become a severe bottleneck in a polyhedral IR based on dual
representation of constraints and vertices. The tradeoff is the
runtime overhead for constructing and evaluating the expression
templates. Experiments with vtune and particular efforts in one
embodiment in keeping this overhead low by using C++ 11's constexpr
types and declaring the expressions static, show an overhead below
3% in the worst cases encountered during testing.
[0061] EDT Code Generation: The code generation process can create
different runtime EDTs for each compile-time EDT. FIG. 6
illustrates the organization of spawning and synchronizations
across the three types of EDTs. Specifically, a code generator
(e.g., CLOOG) may be used to walk a tree of EDT types in a
recursive, top-down traversal. Each EDT type may be represented in
its own separate file. These EDT types can be compiled
independently (e.g., using gcc or any suitable compiler) and may be
linked with the runtime library to produce the final executable.
SHUTDOWN EDTs, as depicted in FIG. 6, do not require special
treatment, they include similar code, parameterized by the TASKTAG.
Each STARTUP and WORKER EDT is parameterized by a start and a stop
level. Until the start level, three behaviors can be observed: (1)
induction variables and parameters may be retrieved directly from
the EDT tag by using the overloading of TaskTag::operator=, (2)
loops are forcibly joined by forming the union of their domains,
and (3) loops are forcibly emitted as conditionals. Between the
start and stop levels, the code generation process follows the
normal behavior of CLOOG, separating statements and generating
loops and conditionals in each subbranch of the code tree. After
the stop level, behavior depends on the type of EDT. Specifically,
for a STARTUP, a counting variable increment in the first loop is
generated and the spawning of WORKER EDTs in the second loop is
also generated. For a non-leaf WORKER, code is generated to spawn
recursively STARTUP, and for a leaf WORKER, the computations and
communications corresponding to the actual work are generated, as
described above.
[0062] The hierarchical code for spawning tasks that is depicted in
FIG. 6 corresponds to a loop nest of loops: (i, j, k, l, m, n, o)
that have types: (seq, perm, perm, seq, perm, perm, doall). As
such, all tasks corresponding to a particular iteration of the loop
i and that associated with the non-leaf WORKER wait for the
completion of all of the tasks associated with the previous
iteration of the sequential loop i. Within a particular iteration
of the loop i, a task associated with the permutable loops j and k
need not depend on all of the previous iterations of the loops j
and k. Instead, a task associated with a particular iteration
depends only on a antecedent task in the j-1 dimension and/or an
antecedent task in the k-1 dimension. These tasks, however, include
the iterations of the sequential loop l. Here again, all of the
tasks associated with a particular iteration of loop l depend on
the completion of all of the tasks associated with the previous
iteration of the sequential loop l. A task associated with a
particular iteration of the loops m, n, and o need not wait for the
completion of all of the tasks corresponding to all the previous
iterations in the m and n dimensions. Instead, such as task may
depend on nothing more than the completion of antecedent tasks in
either one or both of the m-1 and n-1 dimensions.
[0063] Concurrent Collections (CnC) is a high-level coordination
language that lets a domain expert programmer specify semantic
dependences in a program without worrying about what runs where and
when. CnC has a task-graph model that is implicit in the program
representation by using hash tables. Intel-CnC is supplied with a
work-stealing runtime whose default scheduler is built on top of
the scheduler provided by Intel's Threading Building Blocks (TBB).
CnC uses tbb::concurrent_hashmap to implement step and item
collections. A step is a C++ object that implements an execute
method; it represents a scheduled unit of execution. The CnC
scheduler decides at runtime which step::execute methods are called
on which hardware thread and on which processor. This step::execute
method takes a step tag reference and a context reference. A step
becomes available when an associated step tag is put in the proper
step collection. A step may perform multiple gets and puts from/to
item collections. Item collections act as dataflow dependence
placeholders. By default, a CnC get is blocking. If it fails,
control is given back to the scheduler which re-enqueues the step
to await the corresponding tag put. Once that put occurs, the step
restarts. In a worst-case scenario, each step with N dependences
may invoke N-1 failing gets and be requeued as many times.
Additionally, on a step suspension, the gets are rolled back.
Performing all gets of a step before any put offers determinism
guarantees.
[0064] ETI's SWift Adaptive Runtime Machine (SWARM) is a low-level
parallel computing framework that shares similarities with CnC.
Additionally, SWARM handles resource objects and allows active
messages and continuation passing style. SWARM is a C API that
makes extensive use of pre-processor macros. In SWARM, an EDT is
declared as a C macro and scheduled into the runtime by calling the
swarm_schedule function. An EDT accepts a context parameter THIS
and an optional parameter INPUT that come in the form of pointers.
SWARM allows more complex behaviors where a parent EDT specifies a
NEXT and NEXT_THIS parameter to allow chaining of multiple EDTs.
SWARM also allows an EDT to bypass the scheduler and dispatch
another EDT immediately using swarm_dispatch. The tagTable put and
get mechanisms in SWARM are fully non-blocking. It is the
responsibility of the user to handle the synchronization properly,
to re-queue EDTs for which all gets did not see matching puts, and
to terminate the flow of execution for such EDTs. SWARM presents a
lower-level runtime and API and allows many low level
optimizations.
[0065] The Open Community Runtime (OCR) is a another runtime system
based on EDTs and work-stealing principles. OCR represents the task
graph explicitly and does not rely on tag hash tables. In OCR,
different objects can be specified as "events," whether they
represent EDTs, blocks of data ("datablocks"), or synchronization
objects. OCR does not natively rely on a tag space. Instead, when
an EDT is spawned, all the events it depends on must have already
been created by the runtime and must be passed as dependence
parameters to the EDT. By contrast, in CnC and SWARM, when a get is
performed, the corresponding hash table entry can be viewed as a
type of "synchronization future." There is effectively a race
condition between the first get, the subsequent gets and the first
put with a given tag. Additionally, mapping to a tag tuple to an
event is necessary to create the synchronizations. Without a hash
table, OCR requires the pre-allocation of a large number of
synchronization events (as is demonstrated in the Cholesky example
that is supplied with OCR). In one embodiment, a prescriber in the
OCR model was implemented to solve this race condition. Puts and
gets are performed in a tbb::concurrent_hash_map following the CnC
philosophy. In various embodiments, the PRESCRIBER step is
completely oblivious to the compiler and is fully handled by the
RAL. In the targeted OCR according to some embodiments, each WORKER
EDT is dependent on a PRESCRIBER EDT which increases the total
number of EDTs. Lastly, in some of the embodiments that use OCR as
the runtime hierarchical async-finish may be supported natively via
the use of a special "finish-EDT." CnC, SWARM can run on both
shared and distributed memory systems, and OCR may be extended for
distributed systems
[0066] Runtime Support for Hierarchical Async-Finish: Various
embodiments support hierarchical async-finish tasks in OCR, SWARM
and CnC. In various embodiments, the system and method describe
herein can generate EDTs that conform to a hierarchical execution
model from sequential input code. In particular, FIG. 7 illustrates
parallelism across hierarchical WORKER EDTs. WORKER instances in
the non-leaf worker (center circle) are connected by point-to-point
dependences. Within each top-level WORKER, bottom-level WORKER are
spawned, and may themselves connected by point-to-point
dependences. Instances that are not connected by dependences (i.e.
the unordered bottom-left and bottom-right instances in the example
depicted in FIG. 7) can be executed in parallel by the runtime.
This is a coarse level of parallelism. Additionally, within each
leaf worker, finer grained parallelism can also exploited by the
runtime.
[0067] OCR natively supports hierarchical async-finish by virtue of
the "finish EDT." OCR also provides "latch" objects that can be
used to emulate this feature like in SWARM, as discussed below. The
other two runtimes do not currently provide native support and,
hence, in various embodiments a layer of emulation that a
source-to-API compiler targets automatically is constructed.
[0068] SWARM natively supports "counting dependence" objects which
are similar to OCR latches. In some embodiments, this feature is
used as follows: Within each STARTUP code which determines how many
WORKER are spawned is generated. A swarm_Dep_t object is allocated
and default initialized to the number of WORKS that can be spawned.
When both the counter and the counting dependence are ready, a
SHUTDOWN is chained to await on the dependence object with the
associated count value. When the dependence count reaches zero, the
SHUTDOWN is awoken. A pointer to the swarm_Dep_t object is passed
as a parameter into the tag of each WORKER instance. At this point,
the current instance of STARTUP can spawn all its WORKERs. When
several levels of hierarchy are involved, each instance of a leaf
WORKER may satisfy the dependence to the SHUTDOWN spawned by their
common enclosing STARTUP. A non-leaf WORKER may relagate the
dependence satisfaction to the SHUTDOWN spawned by the same STARTUP
instance. SHUTDOWN may satisfy the counting dependence of their
respective callers, up until the main SHUTDOWN, which stops the
runtime.
[0069] CnC does not natively support async-finish or even counting
dependences. A reduction operator may be developed. In one
embodiment, using a C++ 11 atomic<int>, each WORKER upon
completion of the tasks designated thereto performs an atomic
decrement of the shared counter. As for SWARM, the counter is
constructed and passed by calling STARTUP. Unlike SWARM, the
ability to notify the SHUTDOWN on the event that the counter
reaches zero is lacking. Therefore, in various embodiments to
perform this synchronization in CnC, a SHUTDOWN performs a "get" of
an item that is only put in the corresponding item collection by
the unique WORKER EDT that decrements the counter to zero (i.e. the
dynamically "last" one). Unlike SWARM and OCR which provide their
own mechanisms, this emulation relies on the item collection (a
hashtable) to perform the signaling. However, accesses to this
hashtable are very rare: only the last WORKER and the associated
SHUTDOWN write and read the hashtable, respectively.
[0070] Experiments: The numbers presented herein may be viewed as a
baseline performance achievable from a sequential specification
automatically translated into EDTs before single thread tuning is
applied and in the absence of data and code placement hints to the
runtime. In particular, no single thread performance optimization
for SIMD, no data-layout transformation, and no tile size selection
heuristic or tuning were applied except where specified. The
mapping decisions were the same in all EDT cases except where
specified. Tile sizes for EDTs in these experiments were fixed to
64 for the innermost loops and 16 for non-innermost loops. This is
by no means optimal but just a heuristic for overdecomposition to
occur while keeping a reasonable streaming prefetch and single
thread performance. These numbers are illustrative only. The
results were compared to automatically generated OMP using a
framework that includes a static heuristic for tile size selection.
The static tile sizes selected for OMP are expected to load-balance
the execution over a statically fixed number of cores and may also
account for streaming memory engines.
[0071] Table 1 in FIG. 8 gives a characterization of the
experiments. For each benchmark, it was specified whether the
benchmark contains symbolic parameters (and if so, how many), the
data and iteration space size as well as the number of EDTs
generated and the maximum number of floating point operations per
full EDT (at the tile size granularities described above). In order
to characterize latencies and stress-test the different runtimes,
the experiments were diverse in their sizes, running from a mere 53
ms in single thread sequential mode (JAC-3D-1) up to 97 s
(JAC-3D-27P).
[0072] Experiments were performed on a two socket, eight core per
socket Intel Sandy Bridge E5-2690 @ 2.90 GHz running Fedora Core
19. Each core was additionally hyperthreaded for a maximum of 32
threads of execution. All experiments were run using "g++-4.8.0-O3"
and linked with a C++ RAL that was targeted to Intel's CnC v0.8,
ETI's SWARM v0.13, and to the Open Community Runtime (OCR)
v0.8.
[0073] CnC Dependence Specification Alternatives: CnC allows for
three different modes of specifying dependences. In one embodiment,
the RAL for CnC uses blocking "get" and is referred to as BLOCK.
This mechanism may introduce unnecessary overhead. In another
embodiment, the RAL was retargeted to target CnC's
unsafe_get/flush_gets mechanism to provide more asynchrony. This
mechanism is similar conceptually to the non-blocking gets in
SWARM. A third CnC mechanism, according to another embodiment, is
the so-called depends mechanism. For each task, all of its
dependences were pre-specified at the time of task creation. This
is similar to the PRE-SCRIBEREDT that may be generated
automatically for OCR, in one embodiment. Table 2 in FIG. 9 shows
the baseline performance achieved by the CnC generated codes when
varying the way dependences are specified. Unsurprisingly, blocking
"gets" result in significant overheads in cases where many smaller
EDTs are generated, which require more calls into the runtime. This
effect is not problematic in the larger 3D cases. More surprising
is the fact that DEP performs significantly worse in the cases
GS-3D-7P, GS-3D-27P, JAC-3D-7P and JAC-3D-27P. This was conjectured
not to be due to runtime overhead but due to scheduling decisions.
To confirm this, the following experiment was conducted: Two levels
of hierarchical EDTs were generated, which effectively increases
the potential runtime overhead for DEP. In these codes, the
non-leaf WORKER had the granularity of the two outermost loops,
whereas the leaf WORKER has the granularity of an original EDT
(16-16-16-64). Despite the increased runtime overhead to manage
these nested tasks, up to 50% speedup was achieved, as shown in
Table 3 in FIG. 10.
[0074] SWARM, OCR and OpenMP: The numerical results obtained with
SWARM, OCR and OpenMP, depicted in Table 4 in FIG. 11 are now
discussed, according to different categories of benchmarks. This
discussion applied to the results obtained for CnC as well.
Embarrassingly Parallel Examples are ones for which no runtime
dependences are required (DIV-3D-1, JAC-3D-1 RTM-3D and MATMULT).
The runtimes for the first the examples is very low (53-210 ins on
1 thread), and can be viewed as a test of runtime latency overhead
on very short runs, without dependences. MATMULT is a somewhat
larger example. These examples show that SWARM has a smaller
overhead than CnC and OCR for running parallel tasks, until
reaching the hyperthreading mode where SWARM performance generally
drops.
[0075] EDT granularity: LUD, POISSON and SOR illustrate relatively
small examples for which the statically selected tile sizes is not
adequate for EDT granularity purposes. In the case of POISSON
pipeline startup cost is prohibitively expensive; choosing tile
sizes of 2-32-128 yields around 7 Gflop/s with OCR on 32 threads, a
6.times. speedup. In the case of SOR, the tile sizes yield small
tasks of merely 1024 iterations corresponding to 5K instructions;
selecting larger tile sizes also improved performance. Overall,
these examples show that relatively small tile sizes that achieve
over-provisioning may not be beneficial, as discussed further below
for SOR and LUD.
[0076] OpenMP Efficient Examples: STRSM and TRISOLV illustrate two
cases which mix both parallel and permutable loops and for which
OpenMP performs significantly better than any of the known EDT
solutions. In this case, it was determined that the problem related
to tile size selection for reuse. In the case of STRSM, by
selecting a square tiles of size 64-64-64, a performance of up to
76 Gflop/s was obtained with OCR. The performance did not increase
further with hyperthreading, however. In addition, forcing the
OpenMP tile sizes to 16-16-64 capped the performance at 50 Gflop/s.
In the case of TRISOLV, by selecting a tile of size 64-64-256,
performance of up to 26 Gflop/s was obtained with OCR. This further
demonstrated the need for a proper tile size selection in EDT-based
runtimes. There is a difficult trade-off between
over-decomposition, reuse, single thread performance, streaming
prefetch utilization and problem size that should be solved in a
dynamic and adaptive fashion.
[0077] 2-D and 3-D Time Tiling. The remaining examples shows the
benefit of EDTs. In those cases, performance for EDT-based codes
scales significantly better than OpenMP performance, especially as
the Jacobi examples (explicit relaxation scheme) move twice as much
memory as GaussSeidel examples (implicit relaxation scheme) and do
not scale as well from 16 to 32 threads (hyperthreading).
[0078] Effects of EDT Granularity: For LUD and SOR on which initial
EDT performance was lower than expected, a few different tile sizes
were explored and, additionally, two levels of granularity was
explored for LUD, as shown in Table 5 in FIG. 12. The granularity
parameter represents the number of loops in an EDT type. FIG. 3,
for example, shows that there is a fine trade-off between EDT
granularity, number of EDTs, and the cost of managing these EDTs.
To confirm the runtime overhead as EDTs shrink in size, performance
hotspots were collected using Intel Vtune amplxe-cl for LUD16-16-16
with granularities 3 and 4 at 16 threads. First, templated
expressions calculations were performed but the load thereof was
not noticeable, confirming the low extra overhead of evaluating
such expressions. Second, in the case of granularity 4, more than
85% of the non-idle time was spent executing work, the rest being
spent mostly in the OCR dequeInit function. However, at a finer
granularity, the ratio of effective work dropped to merely 10%
stealing and queue management taking up to 80%. The drop in
performance between 16-16-16 and 10-10-100 suggests there is a
critical threshold, possibly linked to last-level cache sizes, at
which the overhead of OCR increases substantially.
[0079] There are a number of other runtimes that can be targeted
using the various embodiments described herein. For example, the
QUARK runtime can speed up the PLASMA linear algebra library with
dynamic task scheduling and a task-oriented execution model, via
parallelization explored automatically and systematically, based
on, in part, loop types, as described in various embodiments. The
task-oriented implementations of the linear algebra library can be
used to regenerate implementations of such a linear algebra library
taking advantage of the features of CnC, SWARM, and OCR.
Furthermore, processes according to various implementations are
oriented toward porting the library for impending architectural
changes from exascale, such as very deep memory hierarchies. Other
EDT oriented runtimes suitable for retargeting include the Qthreads
Library and HPX.
[0080] In summary, various embodiments described herein present the
first fully automatic solution that can generate event-driven,
tuple-space based programs from a sequential specification for
several EDT-based runtimes. This solution can performs hierarchical
mapping and can exploit hierarchical async-finishes. This solution
can also use auto-parallelizing compiler technology to target
different runtimes relying on event-driven tasks (EDTs) via a
runtime-agnostic layer. In different embodiments, the RAL has been
retargeted to Intel's Concurrent Collections (CnC), ETI's SWARM,
and the Open Community Runtime (OCR). The event-driven, tuple-space
based programs obtained according to the embodiments described
above generally resulted in performance improvements. The solution
takes advantage of parallel and permutable loops to abstract
aggregate dependences between EDT types.
[0081] With reference to FIG. 13, sequential code 1302 is analyzed
as described by a processor 1304. The processor 1304 can include a
single processing unit and memory or, as depicted, several
processing units and memory modules so that the analysis of the
code 1302 and synthesis of statements that can spawn EDT instances
and that can facilitate evaluation of dependencies between such
instances can be performed using more than one processing units.
The one or more statements can include the RAL. A code module 1306
that includes at least parts of the sequential code 1302 and the
one or more statements generated by the processor 1304 are compiled
by another processor 1308. Like the processor 1304, the processor
1306 can also include one or more processing units and/or one or
more memory units. In some embodiments, a single hardware system
can be configured as both the processor 1308 and the processor
1304.
[0082] The processor 1308 retargets the code module 1306 to one or
more EDT-based runtimes, such as runtimes 1310-1314 depicted in
FIG. 13. Three runtimes are shown for illustration only. In
general, retargeting can be performed for fewer (e.g., 1 or 2) or
more (e.g., 5, 10, etc.) different EDT-based runtimes. Each runtime
typically includes several workers that can execute various tasks
associated with the code 1302 in parallel, while observing the
inter-task dependencies. The spawning of various tasks is
facilitated by the runtime codes 1316-1320 for the corresponding
runtimes. To this end, the processor 1304 facilitates, as described
above, synthesis of statements that can spawn tasks by the target
runtime and that enable the runtime to test whether a dependency
exists between pairs of tasks and whether those dependencies have
been satisfied.
[0083] It is clear that there are many ways to configure the device
and/or system components, interfaces, communication links, and
methods described herein. The disclosed methods, devices, and
systems can be deployed on convenient processor platforms,
including network servers, personal and portable computers, and/or
other processing platforms. Other platforms can be contemplated as
processing capabilities improve, including personal digital
assistants, computerized watches, cellular phones and/or other
portable devices. The disclosed methods and systems can be
integrated with known network management systems and methods. The
disclosed methods and systems can operate as an SNMP agent, and can
be configured with the IP address of a remote machine running a
conformant management platform. Therefore, the scope of the
disclosed methods and systems are not limited by the examples given
herein, but can include the full scope of the claims and their
legal equivalents.
[0084] The methods, devices, and systems described herein are not
limited to a particular hardware or software configuration, and may
find applicability in many computing or processing environments.
The methods, devices, and systems can be implemented in hardware or
software, or a combination of hardware and software. The methods,
devices, and systems can be implemented in one or more computer
programs, where a computer program can be understood to include one
or more processor executable instructions. The computer program(s)
can execute on one or more programmable processing elements or
machines, and can be stored on one or more storage medium readable
by the processor (including volatile and non-volatile memory and/or
storage elements), one or more input devices, and/or one or more
output devices. The processing elements/machines thus can access
one or more input devices to obtain input data, and can access one
or more output devices to communicate output data. The input and/or
output devices can include one or more of the following: Random
Access Memory (RAM), Redundant Array of Independent Disks (RAID),
floppy drive, CD, DVD, magnetic disk, internal hard drive, external
hard drive, memory stick, or other storage device capable of being
accessed by a processing element as provided herein, where such
aforementioned examples are not exhaustive, and are for
illustration and not limitation.
[0085] The computer program(s) can be implemented using one or more
high level procedural or object-oriented programming languages to
communicate with a computer system; however, the program(s) can be
implemented in assembly or machine language, if desired. The
language can be compiled or interpreted.
[0086] As provided herein, the processor(s) and/or processing
elements can thus be embedded in one or more devices that can be
operated independently or together in a networked environment,
where the network can include, for example, a Local Area Network
(LAN), wide area network (WAN), and/or can include an intranet
and/or the Internet and/or another network. The network(s) can be
wired or wireless or a combination thereof and can use one or more
communications protocols to facilitate communications between the
different processors/processing elements. The processors can be
configured for distributed processing and can utilize, in some
embodiments, a client-server model as needed. Accordingly, the
methods, devices, and systems can utilize multiple processors
and/or processor devices, and the processor/processing element
instructions can be divided amongst such single or multiple
processor/devices/processing elements.
[0087] The device(s) or computer systems that integrate with the
processor(s)/processing element(s) can include, for example, a
personal computer(s), workstation (e.g., Dell, HP), personal
digital assistant (PDA), handheld device such as cellular
telephone, laptop, handheld, or another device capable of being
integrated with a processor(s) that can operate as provided herein.
Accordingly, the devices provided herein are not exhaustive and are
provided for illustration and not limitation.
[0088] References to "a processor", or "a processing element," "the
processor," and "the processing element" can be understood to
include one or more microprocessors that can communicate in a
stand-alone and/or a distributed environment(s), and can thus can
be configured to communicate via wired or wireless communications
with other processors, where such one or more processor can be
configured to operate on one or more processor/processing
elements-controlled devices that can be similar or different
devices. Use of such "microprocessor," "processor," or "processing
element" terminology can thus also be understood to include a
central processing unit, an arithmetic logic unit, an
application-specific integrated circuit (IC), and/or a task engine,
with such examples provided for illustration and not
limitation.
[0089] Furthermore, references to memory, unless otherwise
specified, can include one or more processor-readable and
accessible memory elements and/or components that can be internal
to the processor-controlled device, external to the
processor-controlled device, and/or can be accessed via a wired or
wireless network using a variety of communications protocols, and
unless otherwise specified, can be arranged to include a
combination of external and internal memory devices, where such
memory can be contiguous and/or partitioned based on the
application. For example, the memory can be a flash drive, a
computer disc, CD/DVD, distributed memory, etc. References to
structures include links, queues, graphs, trees, and such
structures are provided for illustration and not limitation.
References herein to instructions or executable instructions, in
accordance with the above, can be understood to include
programmable hardware.
[0090] Although the methods and systems have been described
relative to specific embodiments thereof, they are not so limited.
As such, many modifications and variations may become apparent in
light of the above teachings. Many additional changes in the
details, materials, and arrangement of parts, herein described and
illustrated, can be made by those skilled in the art. Accordingly,
it will be understood that the methods, devices, and systems
provided herein are not to be limited to the embodiments disclosed
herein, can include practices otherwise than specifically
described, and are to be interpreted as broadly as allowed under
the law.
* * * * *