U.S. patent application number 08/792077 was filed with the patent office on 2001-06-14 for write purge partial in sci based system.
Invention is credited to HORNUNG, BRYAN, MARIETTA, BRYAN.
Application Number | 20010003838 08/792077 |
Document ID | / |
Family ID | 25155724 |
Filed Date | 2001-06-14 |
United States Patent
Application |
20010003838 |
Kind Code |
A1 |
HORNUNG, BRYAN ; et
al. |
June 14, 2001 |
WRITE PURGE PARTIAL IN SCI BASED SYSTEM
Abstract
In a SCI based multi-node system, the write purge command joins
the new node that is requesting to write to the memory of the
sharing list, while maintaining the connection between the memory
and the sharing list. The new node then issues the purging command
to each node in the sharing list, while still maintaining the
connection of the sharing list to the memory. Next, the new node
issues the collapsing command to separate the sharing list from the
memory after the purging command has been issued to each node. A
send request data packet is used to distribute the write purge
command to the memory node.
Inventors: |
HORNUNG, BRYAN; (PLANO,
TX) ; MARIETTA, BRYAN; (AUSTIN, TX) |
Correspondence
Address: |
HEWLETT PACKARD COMPANY
P O BOX 272400
INTELLECTUAL PROPERTY ADMINISTRATION
FORT COLLINS
CO
80527-2400
US
|
Family ID: |
25155724 |
Appl. No.: |
08/792077 |
Filed: |
January 31, 1997 |
Current U.S.
Class: |
711/119 ;
711/133; 711/144; 711/E12.025 |
Current CPC
Class: |
G06F 12/0813
20130101 |
Class at
Publication: |
711/119 ;
711/133; 711/144 |
International
Class: |
G06F 012/08 |
Claims
What is claimed is:
1. A method for performing a write operation on a memory with a
sharing list in a multi-node system, the method comprising the
steps of: issuing a write linking command that attaches a new node
to the sharing list; and maintaining a connection between the
memory and the sharing list while the new node is attaching to the
sharing list.
2. The method according to claim 1, wherein: the linking command is
issued by the new node.
3. The method according to claim 1, wherein: the sharing list, the
new node, and the memory are connected by a plurality of
pointers.
4. The method according to claim 1, wherein: the new node is
connected between the memory and the shared list.
5. The method according to claim 1, wherein: the connection is
between the new node and the memory.
6. The method according to claim 1, further comprising the steps
of: issuing a purging command to each node in the sharing list; and
issuing a collapsing command to separate the sharing list from the
memory after the purging command has been issued to each node.
7. The method according to claim 6, wherein: the purge command
invalidates the data stored at each node.
8. The method according to claim 6, wherein: the purge command is
issued by the new node.
9. The method according to claim 6, wherein: the step of issuing
the collapsing command also separates each node in the sharing list
from each other.
10. The method according to claim 6, wherein: the step of issuing
the collapsing command also separates the new node from the
memory.
11. The method according to claim 6, wherein: the collapsing
command is issued by the new node.
12. The method according to claim 1, further comprising the step
of: using a data packet to distribute the command for the write
operation throughout the system.
13. The method according to claim 1, further comprising the step
of: using a write mask as the write command for the write
operation.
14. The method according to claim 1, wherein: the system is a SCI
system with cache coherent non-uniform memory access.
15. A method for performing a write purge partial operation on a
memory with a sequential sharing list in a SCI based multi-node
system cache coherent non-uniform access, the method comprising the
steps of: issuing a memory_write_purge_attach_to_list command that
attaches a new node to the sharing list; and maintaining a
connection between the memory and the sharing list while the new
node is attaching to the sharing list.
16. The method according to claim 15, wherein: the memory is marked
fresh prior to the step of issuing a
memory_write_purge_attach_to_list command.
17. The method according to claim 15, wherein: the sharing list
comprises a plurality of nodes, with a first node of the list
marked head_fresh, and with a last node of the list marked
tail_valid.
18. The method according to claim 17, wherein: the remaining nodes
of the list are marked mid_valid.
19. The method according to claim 15, further comprising the steps
of: issuing a cpurge_fresh_to_invalid command from the new node to
a first node of the list; issuing a cpurge_valid_to_invalid command
from the new node to the next node of the list; repeating the
cpurge_valid_to_invalid command until the node in the list has
received the command; and issuing a memory update_list-to-home
command after the step of repeating has been completed.
20. The method according to claim 19, wherein: the memory is marked
home subsequent to the step of issuing a memory update_list-to-home
command.
21. The method according to claim 15, further comprising the step
of: using a send request packet to distribute the command for the
write operation throughout the system.
22. The method according to claim 15, further comprising the step
of: using a write mask as the write command for the write
operation.
23. A multi-node system that uses a write operation on a memory
with a sharing list, the system comprising: means for attaching a
new node to the sharing list; and means for maintaining a
connection between the memory and the sharing list while the new
node is attaching to the sharing list.
24. The system according to claim 23, wherein: the sharing list,
the new node, and the memory are connected by a plurality of
pointers; and the connection is between the new node and the
memory.
25. The system according to claim 23, further comprising: means for
purging each node in the sharing list; and means for separating the
sharing list from the memory after each node has been purged.
26. The system according to claim 25, wherein: the means for
purging invalidates the data stored at each node.
27. The system according to claim 25, further comprising: means for
separating each node in the sharing list from each other.
28. The system according to claim 23, wherein: the system is a SCI
system with cache coherent non-uniform memory access.
Description
TECHNICAL FIELD OF THE INVENTION
[0001] This invention relates in general to memory accesses in
multi-node, multi-processor, cache coherent non-uniform access
system and relates in particular to a system and method for
performing a write purge operation in such a system.
BACKGROUND OF THE INVENTION
[0002] The Scalable Coherent Interface (SCI) Direct Memory Access
(DMA) write operations in the standard SCI specification completely
over-writes the cache lines. If the DMA devices are only updating
some of the bytes in the line, the information contained in the
memory for the other bytes are lost in the complete overwrite.
[0003] The DMA write operation also relies on software to maintain
cache coherency. If two devices are writing the same line, the
second device can believe that it has finished purging all caches
that contain the old data, even though the lines are being purged
because of the first device. This second device can then allow
other software to read stale data. The use of stale data by the
software will cause program errors.
[0004] Other prior art methods for writing partial lines rely on
reading the line into a local cache before the specified bytes are
updated. This results in other desirable data being swapped out of
the cache because of conflicts with the stored read data. In the
end, this results in poor performance of other processes that are
currently running because of the extra memory operations and the
latency associated with refetching affected data.
[0005] Therefore, there is a need in the art for a method and
system that has a write command that does not allow the use of
stale data by the software.
[0006] In addition, there is a need in the art for a method and
system that does not require the reading memory lines into a local
cache before the updating memory bytes.
SUMMARY OF THE INVENTION
[0007] These and other objects and features are achieved in a
system in which follows the same general flow as the DMA Write as
described in the SCI specification, however, the inventive system
and method does not detach the cache sharing list from memory.
Instead, it joins the sharing list. This prevents another write
purge from another node from believing it has finished its
operation while memory lines are still encached. If no sharing list
exists, a mask supplied by the command is used to merge the new
data into memory. The system and method tracks down stale data in
remote caches and merges it into the memory line using a mask
instead of discarding it.
[0008] One technical advantage of the present invention is to issue
a write purge command that joins the new node to the sharing list,
while maintaining the connection between the memory and the sharing
list.
[0009] Another technical advantage of the present invention is to
have the new node issue the purging command to each node in the
sharing list, while maintaining the connection between the memory
and the sharing list.
[0010] A further technical advantage of the present invention is to
have the new node issue the collapsing command to separate the
sharing list from the memory after the purging command has been
issued to each node. The collapsing command completes the
destruction of the sharing list.
[0011] A further technical advantage of the present invention is to
use a write mask with the write purge command.
[0012] The foregoing has outlined rather broadly the features and
technical advantages of the present invention in order that the
detailed description of the invention that follows may be better
understood. Additional features and advantages of the invention
will be described hereinafter which form the subject of the claims
of the invention. It should be appreciated by those skilled in the
art that the conception and the specific embodiment disclosed may
be readily utilized as a basis for modifying or designing other
structures for carrying out the same purposes of the present
invention. It should also be realized by those skilled in the art
that such equivalent constructions do not depart from the spirit
and scope of the invention as set forth in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] For a more complete understanding of the present invention,
and the advantages thereof, reference is now made to the following
descriptions taken in conjunction with the accompanying drawings,
in which:
[0014] FIGS. 1A to 1C show an example of the SCI specification
write purge operation;
[0015] FIGS. 2A to 2C show an example of the inventive write purge
partial operation;
[0016] FIG. 3 describes the operations performed by an
initialization state machine to execute the write purge
operation;
[0017] FIG. 4 describes the operations performed by a request state
machine to execute the write purge operation;
[0018] FIGS. 5A to 5F describe the operations performed by a
response state machine to execute the write purge operation;
[0019] FIGS. 6A to 6C describe the operations performed by a
conflicting state machine to execute the write purge operation;
[0020] FIG. 7 depicts the send request data packet for carrying the
write purge command in the bytemask; and
[0021] FIG. 8 shows the write mask.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] FIGS. 1A to 1C show how the prior art SCI specification
performs a DMA write to a line that is fresh with a sharing list.
The list starts out with memory 101 in the state of fresh with
pointer to node 0 102. Node 0 102 is head_fresh and has a forward
pointer to node 1 103, which is mid_valid. Node 1 103 has a back
pointer back to the head, node 0 102, and a forward pointer to node
2 104. Node 2 104 is tail_valid and has a back pointer to the mid
or node 1 103.
[0023] The first step that is performed by the standard SCI flow is
to determine if another node wants to write this memory line, if
so, then it will issue an MWRITE_FRESH_TO_HOME. As shown in FIG.
1A, node 3 105 wants to write to the memory line and issues the
MWRITE command to begin the write operation. FIG. 1B shows that the
list fresh_to_home is actually detached from the memory 101 so that
the sharing list of nodes 0, 1, and 2 102, 103, 104 is no longer
connected to the memory 101. So the memory is now marked home and
any new accesses can retrieve the memory line directly from memory
101.
[0024] Now node 3 105 goes in and cleans up or purges the sharing
list. Node 3 105 sends CREAD00_FRESH_TO_INVALID from node 3 to node
0. When node 3 105 receives a response, it uses the response to get
the forward pointer. Node 3 105 then uses the forward pointer to
send CREAD00_VALID_TO_INVALID from node 3 105 to node 1 103. When
node 3 105 receives a response, it then uses the forward pointer
from the response to send the CREAD00_VALID_TO_INVALID from node 3
105 to node 2 104. When this operation is complete or done, only
memory in the home state contains a copy of the data.
[0025] The SCI specification relies on software interlocking in
such a way that it does not access the fresh memory until the list
is totally purged. This can be very difficult if multiple write
purges occur to the same memory line almost simultaneously, as the
second one will immediately see that the line is marked HOME and
allow software to access this line, which could still have stale
copies.
[0026] FIGS. 2A to 2C shows that the operation of the invention
commences similarly to that shown in FIG. 1A, beginning with the
initial state with memory 101 being marked fresh and its pointer to
node 0 102. Node 0 102 is marked head_fresh with a forward pointer
to node 1 103 which is mid_valid. Mid_valid has a back pointer to
the head and a forward pointer to node 2 104, which is tail_valid.
Again, node 3 105 wants to do a write_purge or an update_memory
with the new data. So node 3 105 sends an MWRITE_purge command,
MWRITE_PU_ATTACH_TO_LIST, from node 3 105 to memory 101. As shown
in FIG. 2B, memory 101 and node 3 105 uses the mask 500 from the
write_purge_partial command to merge in the new data and does not
detach the list as in FIG. 1B.
[0027] So now the list is still connected with the memory 101
marked fresh. The forward pointer of the memory 101 points to node
3 105. Node 3 105 points to node 0 102. Node 0 102 remains in the
head_fresh state since it has not received any communication from
node 3 105. Node 0 102 has a forward pointer pointing to node 1 103
mid_valid, and the mid_valid has a forward pointer to tail_valid
node 2 104.
[0028] After node 3 105 receives its response from the mwrite_purge
command, it then does a CPURGE not a CREAD, because in this case
the line is FRESH and does not need new data. So after the
mwrite_purge response, node 3 105 starts purging the list with a
CPURGE_FRESH_TO_INVALID from node 3 105 to node 0 102. Node 3 105
then receives a response from node 0 containing the next forward
pointer, and issues a CPURGE_VALID_TO_INVALID from node 3 105 to
node 1 103.
[0029] When node 3 105 receives a response from node 1 103 with the
next forward pointer, it then performs a CPURGE_VALID_TO_INVALID
from node 3 105 to node 2 104. When node 3 105 sees that the
tail_valid node, here node 2 104, has been reached, it then does an
MUPDATE_LIST_TO_HOME from node 3 105 to the memory 101 and the
memory is left in the home state as shown in FIG. 2C. Thus, this
method achieves the same results as shown in FIG. 1C.
[0030] FIGS. 3 to 6 describe the operations handled by the various
state machines inside the SCI controller to do the
write_purge_partial flow.
[0031] FIG. 3 describes the operations for the memory access
controller Request State Machine Logic. This state machine will
take requests from the memory access controller, and given the
state passed by the memory access controller and the flow, will
decide on the first state that is inside the SCI controller. For
example, in the write_purge case, where the memory access
controller requests with head_fresh, the SCI controller would
initialize this request to a CS_HF_MODS_HD state, or for in a
different case, head_fresh_MODS_only_head_dirty state.
[0032] FIG. 4 describes how that state is turned into a request on
the ring. In the previous case where the state is CS_HF_MODS_HD,
this state machine would then make a ring request of
mupdate_list_to_gone.
[0033] FIGS. 5A-F describes the operations for the Response State
Machine. This state machine describes what is done when a response
is received from the remote memory or cache. The machine decides
whether to make another request to the ring, make a response to the
memory access controller, or in some cases, perform both. Again
following the previous case where CS_HF_MODS_HD, if the response is
not nullified and is FRESH, in that case, then the machine will
transition to the CS_HD_INVAL_OD state and then the FIG. 3B state
machine will take this state and generate a new request.
[0034] The state machine of FIGS. 6A-C describes what happens if a
request for the same line is received from another node while
actively in the write purge flow. Again following the previous case
where CS_HF_MODS_HD state, the state machine will allow a
cupdate_prevmid or cupdate_prevtail to complete. All other requests
will be nullified. The prev-TAIL will update the forward pointer
and change the state.
[0035] FIG. 7 depicts the symbols in a send request packet 400 on
the SCI rings. The significant field in this particular packet is
the second symbol in which we have a field labeled bytemask 401.
The bytemask field 401 is used to carry the byte mask information
for the write_purge_partial.
[0036] FIG. 8 describes the mask 500 in more detail. The mask
consists of a start 501 and an end 502, each being 5 bits. If the
start is all 0's and the end is all 1's that means we had started
at 0 and ended at 31, meaning we write the whole line per
write_purge. Other values may be used, for example, start at 5 and
end at 20, however the sub-field must always be of continuous
bytes.
[0037] Although the present invention and its advantages have been
described in detail, it should be understood that various changes,
substitutions and alterations can be made herein without departing
from the spirit and scope of the invention as defined by the
appended claims.
* * * * *