U.S. patent application number 10/151582 was filed with the patent office on 2003-11-20 for system for reordering sequenced based packets in a switching network.
Invention is credited to Shaikli, Nadim.
Application Number | 20030214949 10/151582 |
Document ID | / |
Family ID | 29419460 |
Filed Date | 2003-11-20 |
United States Patent
Application |
20030214949 |
Kind Code |
A1 |
Shaikli, Nadim |
November 20, 2003 |
System for reordering sequenced based packets in a switching
network
Abstract
System for reordering sequenced based packets in a switching
network. The system includes time stamp logic that operates to
associate a receive time indicator with each received data packet.
The system also includes Enqueue logic that operates to compute an
expiration time for each received packet based on the receive time
indicator, and stores the expiration time and the sequence
identifier for each received packet into a table. Dequeue logic
operates to read the table to determine the received data packets
to output in an output stream so that the received data packets are
output in a selected order. The Dequeue logic also operates to
determine a true expiration time for one or more unreceived data
packets, and if the true expiration time for a selected unreceived
data packet is reached, the Dequeue logic operates to omit the
selected unreceived data packet from the output stream.
Inventors: |
Shaikli, Nadim; (San Diego,
CA) |
Correspondence
Address: |
SOCAL IP LAW GROUP
310 N. WESTLAKE BLVD. STE 120
WESTLAKE VILLAGE
CA
91362
US
|
Family ID: |
29419460 |
Appl. No.: |
10/151582 |
Filed: |
May 16, 2002 |
Current U.S.
Class: |
370/394 ;
370/412; 370/428 |
Current CPC
Class: |
H04L 47/10 20130101;
H04L 47/34 20130101; H04L 47/50 20130101; H04L 47/286 20130101;
H04L 47/564 20130101 |
Class at
Publication: |
370/394 ;
370/412; 370/428 |
International
Class: |
H04L 012/28; H04L
012/66 |
Claims
What is claimed is:
1. A system for reordering data packets at a destination processor
in a packet switching network, wherein a plurality of source
processors transmit the data packets to the destination processor
via one or more communication fabrics, and wherein a sequence
identifier is associated with each data packet, the system
comprising: time stamp logic that operates to associate a receive
time indicator with each received data packet; Enqueue logic that
operates to compute an expiration time for each received packet
based on the receive time indicator, and to store the expiration
time into a table based on the sequence identifiers associated with
the received packets; and Dequeue logic that operates to read the
table to determine the received data packets to output in an output
stream, so that the received data packets are output in a selected
order, wherein the Dequeue logic also operates to determine a true
expiration time for one or more unreceived data packets from the
expiration times associated with received packets, and wherein if
the true expiration time for a selected unreceived data packet is
reached, the Dequeue logic operates to omit the selected unreceived
data packet from the output stream.
2. The system of claim 1, further comprising: a memory coupled to
the Enqueue logic; and a pointer FIFO coupled to the Enqueue logic,
wherein the Enqueue logic operates to obtain a pointer to the
memory from the pointer FIFO for each received data packet and
stores the received data packets in the memory based on the
pointers.
3. The system of claim 2, wherein the memory and the pointer FIFO
are coupled to the Dequeue logic.
4. The system of claim 3, further comprising a read pointer
register that is used to read selected entries from the table.
5. The system of claim 4, further comprising a delay store register
that is used to store the pointers into the table.
6. The system of claim 5, further comprising a current value
register that is used to determine the true expiration time.
7. The system of claim 6, further comprising a largest value
register that is used to maintain a largest sequence identifier
value.
8. A method for reordering data packets at a destination processor
in a packet switching network, wherein a plurality of source
processors transmit the data packets to the destination processor
via one or more communication fabrics, and wherein a sequence
identifier is associated with each data packet, the method
comprising steps of: associating a receive time indicator with each
received data packet; determining an expiration time for each of
the received data packets based on their associated receive time
indicators; determining a true expiration time for one or more
unreceived data packets based on the expiration times associated
with the received data packets; forming an output stream of
received data packets, wherein the output stream contains the
received data packets in the same order that they were transmitted
from their respective source processors; and omitting unreceived
data packets from the output stream if the true expiration time for
the unreceived data packets is reached.
9. The method of claim 8, further comprising steps of: storing the
expiration time and the sequence identifier for each received
packet into a table; and reading the table to determine the
received data packets to output in the output stream.
10. The method of claim 9, further comprising steps of: obtaining a
memory pointer from an address FIFO for each received packet; and
storing each received packet in a memory based on the associated
memory pointer.
11. The method of claim 10, further comprising a step of returning
the memory pointer associated with each packet in the output stream
to the address FIFO.
Description
FIELD OF THE INVENTION
[0001] The present invention relates generally to the operation of
switching networks, and more particularly, to a system for
reordering sequence based packets in a switching network.
BACKGROUND OF THE INVENTION
[0002] Communications networks are now required to handle data at
very high data rates. For example, a data rate of 10 gigabits per
second (Gbps) is common. When it is required to process data at
these speeds, multiple high-speed parallel connections may be used
to increase the effective bandwidth. However, this may result in
one or more transmission problems, since the data streams must be
divided to be distributed over the multiple parallel connections,
and then at some point after parallel transmission, recombined to
form the original streams.
[0003] FIG. 1 shows a block diagram 100 of a typical network
structure for transmitting data frames (or data packets) from
source processors 102 to a destination processor 104 via multiple
communication fabrics 106. The data streams include packets or
frames that may comprise a fixed amount of data. For example,
stream A may include frames A0, A1, and A2 that are received by the
source processor A and transmitted to each of the fabrics as shown.
The stream B may include frames B0, B1 and B2 that are transmitted
to the fabrics by source processor B as shown, and the stream C may
include frames C0, C1 and C2 that are transmitted to the fabrics by
source processor C as shown.
[0004] Once the frames are received by the fabrics, they are
transmitted to the destination processor 104 as shown. The
destination processor receives the frames and combines them in the
order they arrive to form output stream D for transmission to
another destination. In some systems, the destination processor
breaks up stream D into multiple streams and transmits the multiple
streams to another destination via multiple communication
fabrics.
[0005] A significant problem that exists with current transmission
systems, such as the system shown in FIG. 1, is that the frames may
end up in the wrong order when stream D is formed and transmitted
from the destination processor D. For example, if frames are
originally transmitted via multiple transmission paths, they might
arrive at the destination in an order that is different from how
they were transmitted. For example, the frames may be output in
stream D in the order shown at 108. In this case, frame B2 is
output before frame B1, and frame C2 is output before frame C1.
Thus, the frames for source processors B and C are transmitted out
of order. In such a case, it may be necessary to discard out of
order frames of data and request a new transmission of those
frames. As a result, additional overhead will be used and a
corresponding loss of transmission bandwidth will be realized.
[0006] Another problem that may occur in current transmission
systems involves frames that may be lost in transmission, so that
they never arrive at the destination processor. In this situation,
it becomes difficult to determine when a frame is lost or merely
delayed in transmission. Furthermore, if a frame is lost, the
transmission of the received frames may be excessively delayed or
blocked because of delays associated with the lost frame. Thus, the
received frames pay a penalty for the lost or delayed frames in
that the transmission of the received frames may be delayed or
blocked completely.
[0007] Therefore, it would be desirable to have a system to reorder
frames of data in a transmission system so that the frames are
output in the correct order, thereby improving transmission
efficiency. The system should also provide a way to process lost or
delayed frames so that the transmission of received frames at a
destination processor is not excessively delayed or blocked.
SUMMARY OF INVENTION
[0008] The present invention includes a system for reordering
sequenced based packets in a switching network. For example, in one
embodiment of the invention, a reordering system is provided that
receives packets from multiple sources and reorders the received
packets based on a unique sequence number and arrival time
associated with each packet. In another embodiment of the
invention, the packets are further provided with a priority, and
the priority is used to determine the order that the packets are
output.
[0009] In one embodiment included in the present invention, a
system is provided for reordering data packets at a destination
processor in a packet switching network, wherein a plurality of
source processors transmit the data packets to the destination
processor via one or more communication fabrics, and wherein a
sequence identifier is associated with each data packet. The system
comprises time stamp logic that operates to associate a receive
time indicator with each received data packet. The system also
comprises Enqueue logic that operates to compute an expiration time
for each received packet based on the receive time indicator, and
to store the expiration time into a table based on the sequence
identifiers associated with the received packets. The system also
comprises Dequeue logic that operates to read the table to
determine the received data packets to output in an output stream,
so that the received data packets are output in a selected order,
wherein the Dequeue logic also operates to determine a true
expiration time for one or more unreceived data packets, and
wherein if the true expiration time for a selected unreceived data
packet is reached, the Dequeue logic operates to omit the selected
unreceived data packet from the output stream.
[0010] In another embodiment included in the present invention, a
method is provided for reordering data packets at a destination
processor in a packet switching network, wherein a plurality of
source processors transmit the data packets to the destination
processor via one or more communication fabrics, and wherein a
sequence identifier is associated with each data packet. The method
comprises the steps of associating a receive time indicator with
each received data packet, computing an expiration time for each
received packet based on the receive time indicator, storing the
expiration time and the sequence identifier for each received
packet into a table, reading the table to determine the received
data packets to output in an output stream so that the received
data packets are output in a selected order, determining a true
expiration time for one or more unreceived data packets, and
omitting a selected unreceived data packet from the output stream
if the true expiration time for the selected unreceived data packet
is reached
BRIEF DESCRIPTION OF DRAWINGS
[0011] The foregoing aspects and the attendant advantages of this
invention will become more readily apparent by reference to the
following detailed description when taken in conjunction with the
accompanying drawings wherein:
[0012] FIG. 1 shows a block diagram of a typical network structure
for transmitting data packets from source processors to destination
processors;
[0013] FIG. 2 shows a diagram of one embodiment of a system for
reordering packets constructed in accordance with the present
invention;
[0014] FIG. 3 shows a diagram of one embodiment of a reordering
processor constructed in accordance with the present invention;
[0015] FIG. 4 shows a diagram that illustrates the computation of a
true expiration time for a data packet associated with a sequence
number in accordance with the present invention;
[0016] FIG. 5 shows a flow diagram of one embodiment of a method
for reordering packets in accordance with the present invention;
and
[0017] FIG. 6 illustrates an example of reordering packets in
accordance with the present invention.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENT
[0018] The present invention includes a system for reordering
sequenced based packets in a switching network. For example, in one
embodiment of the invention, a reordering system is provided that
receives packets from multiple sources and reorders the received
packets based on a time stamp associated with each received packet.
Thus, various embodiments of the system included in the present
invention are discussed in detail in the following text.
[0019] Exemplary Embodiment
[0020] FIG. 2 shows a diagram of one embodiment of a system 200 for
reordering frames constructed in accordance with the present
invention. The system 200 includes sequence encoders 202 at each
source processor that encode packets in each stream with a unique
sequence identifier before transmission via fabrics 210. The system
also includes a reordering system 204 constructed in accordance
with the present invention that includes time stamp logic 206 that
operates to associate a received time stamp with each received
packet. The reordering system 204 operates to output packets at
output stream D that are in the correct order with respect to how
they were transmitted from their respective source processors.
Furthermore, the reordering system uses the time stamp logic 206 to
make a determination about lost or delayed packets so that these
packets do not excessively delay or block transmission of received
packets from the destination processor D'.
[0021] During operation of the system 200, the source processors
receive data streams containing data packets or frames. The source
processors operate to give each of the data frames a unique
sequence identifier prior to transmitting them to one or more
fabrics 210. For example, the sequence identifier may be a sequence
number and source identifier that uniquely identifies the packet
and its transmitting source. In one embodiment of the invention,
the source processors also assign a priority to each frame in
addition to the sequence identifier. Thus, each frame that is
transmitted to the fabrics 210 includes a sequence identifier, and
optionally, priority information. Any technique can be used to
encode a sequence identifier and/or priority with the data frames.
For example, the source processors can include encoder logic that
can encode each data frame with a unique sequence number that acts
as the sequence identifier. Furthermore, the encoder logic can
include a priority indicator with each frame. The priority
indicator can be selected from one of several priority levels.
Thus, any suitable technique can be used within the scope of the
invention to associate a sequence identifier and priority
information with each data frame.
[0022] The system 200 also includes the reordering system 204 at
the destination processor D'. The reordering system 204 operates to
receive the frames from the fabrics 210 and process the frames
based on their respective sequence identifiers to reorder the
frames for output from the destination processor D'. The frames are
reordered so that they are placed in an identical order as when
they were transmitted from their respective source processors.
Thus, stream D' will include all received frames in correct
order.
[0023] The reordering system 204 also operates to assign an
expiration time to unreceived frames based on receive time
parameters associated with received frames. The expiration time is
used to determine when unreceived frames are lost, so that the
transmission of received frames as part of reordered stream D' will
not pay a penalty by being excessively delayed or blocked.
[0024] FIG. 3 shows a diagram of one embodiment of a reordering
system 300 constructed in accordance with the present invention.
The reordering system forms part of the destination processor D'
and operates to provide reordering of received frames in accordance
with the invention. In addition to reordering in accordance with
the present invention, the destination processor D' may operate on
the data frames in other ways to facilitate their transmission,
however, these other processes are not essential to the operation
of one or more embodiments of the invention, and so will not be
described in detailed. For example, the destination processor D'
may serialize the frames for transmission, or provide known error
detection and correction processes that are independent from the
reordering system.
[0025] A receiver 302 receives one or more data streams 304 that
have arrived at the destination processor D'. For example, the
receiver 302 receives the frames transmitted via the fabrics 210
from the source processors. The receiver 302 is coupled to a memory
306 so that each of the received frames may be stored in the memory
306.
[0026] Time stamp logic 308 is coupled to the receiver 302 and
operates to provide a time stamp for each received data frame.
Information about each received frame and its associated time stamp
are then stored in a table 310 for later processing.
[0027] Enqueue logic 312 is coupled to the memory 306, so that the
Enqueue logic 312 can control the process of loading the memory 306
with the frames as they are received and also process information
in the table 310. The Enqueue logic may comprise any suitable
hardware such as a CPU, gate array or other hardware logic, and may
also include any suitable software to operate in conjunction with
the hardware. The Enqueue logic 312 operates to load expiration
times for received packets into the table 310 for later use during
operation of the reordering system.
[0028] The memory 306 is coupled to a transmitter 314 that receives
the frames as they are transferred out of the memory 306 in the
correct order in accordance with the present invention. Thus, as
the data frames are received, they are time stamped and their data
is entered into memory, and as a result of the operation of one or
more embodiment of the present invention, the data will flow from
the memory to the transmitter 314 in the correct order for
transmission to another destination. In one embodiment, the
transmitter 314 transmits the frames in a single stream 316 toward
their final destination. For example, the transmitter may transmit
the single stream 316 into another transmission fabric. In another
embodiment, the transmitter 314 operates to break-up the reordered
stream 316 for transmission to another destination via several
transmission fabrics.
[0029] Dequeue logic 316 is coupled to the memory 306 and operates
to control the reordering of frames and the transfer of frames from
the memory to the transmitter 314. For example, in one embodiment,
the Dequeue logic operates to control the transfer of frames from
the memory based on the sequence number and time stamp associated
with each frame that is stored in the table 310. The Dequeue logic
may comprise any suitable hardware such as a CPU, gate array or
other hardware logic, and may also include any suitable software to
operate in conjunction with the hardware.
[0030] The Dequeue logic also provides a timing function that can
determine when unreceived frames have expired. For example, the
Dequeue logic may wait for a selected time period before
determining that an unreceived frame has been lost in transmission.
In one embodiment, this time period provides a way to compensate
for a latency time associated with the transmission fabrics, thus
allowing frames that have been only slightly delayed during
transmission through the transmission fabrics to be included in the
reordered output stream. In another embodiment, the selected time
period allows data frames that are either lost or excessively
delayed to be declared expired, so that the Dequeue logic operates
to output other frames received within acceptable time parameters.
Thus, the reordering system 300 operates to perform reordering in
accordance with the present invention and to prevent lost frames
from excessively delaying the continued transmission of received
frames.
[0031] FIG. 4 shows a diagram 400 that illustrates the computation
of a true expiration time for unreceived data packets in accordance
with the present invention. The algorithm for determining the true
expiration time is based on the following two conditions.
[0032] 1. As packets are sent to the destination, they are
sequenced in order.
[0033] 2. If a packet with sequence number S+1 arrives at time T,
then the packet with sequence number S must have either arrived or
will arrive no later than time T+d, where d is the delay through
the communication fabric.
[0034] As packets arrive at the destination, their data is stored
away into a memory and a table 401 is used to track their reception
based on their sequence number. The packets are time stamped upon
reception and the arrival time of received packets is noted and a
maximum timeout period is added to that arrival time to determine
an expiration time for unreceived packets having earlier sequence
numbers. For example, the resultant expiration time associated with
a selected received packet is extended down to every packet having
a earlier sequence number from that received packet. The process of
extending time down to the locations that precede the current
packet indicate that for every non-valid table entry (i.e., packet
not received), if the time at the destination were to equal the
timeout value, it could be safely assumed that the packet
associated with this entry will never arrive. Since packets may
arrive completely out of sequence, and since this extension is done
on a per packet basis, a true expiration time for the non-valid
entries is determined by taking a minimum (MIN) function of all the
times that extend beyond those non-valid entries.
[0035] The following text describes the process of determining a
true expiration time for non-valid entries in the table 401, (i.e.,
for packets that have not been received at the destination). The
following computation assumes a timeout period that is equal to
ten, however, any desirable timeout period may be selected. To
determine expiration times associated with received packets, the
time stamp associated with the received packets are used. For
example, referring to FIG. 4, a valid packet is received at time
(4) having sequence number twenty-one, as shown at 402. An
expiration time for that packet is determined by adding the timeout
period to the received time to determine an expiration time equal
to fourteen. This expiration time is extended down to packets
having sequence numbers less than 21 (i.e., 20, 19, 18, etc.) as
shown at 404.
[0036] The next packet to arrive has sequence number (28), which
arrives at time five, as shown at 406. As a result, an expiration
time of fifteen (5+10) is determined and extended down to packets
having sequence numbers less than 28 as shown at 408.
[0037] Another packet arrives at time (6) having sequence number
thirty-two, as shown at 410. This received packet results in an
expiration time of sixteen (6+10) being extended down to packets
having sequence numbers less than 32, as shown at 412. Similarly, a
packet received at time (8), shown at 414, and having a sequence
number thirty-five establishes an expiration time of eighteen
(8+10) for packets having sequence numbers less than 35, as shown
at 416. Another packet arrives at time (12), as shown at 418, and
has a sequence number of twenty-five. This received packet results
in an expiration time of twenty-two (12+10) being assigned to
packets having sequence numbers less than 25, as shown at 420.
[0038] To determine a true expiration time for non-valid table
entries (i.e., unreceived packets), a MIN function is used so that
for each non-valid entry, the minimum downward extended expiration
time is selected as the true expiration time. For example, the
minimum downward extended expiration time for sequence numbers
34-33 is 18, so the true expiration time for those unreceived
packets is 18, as shown at 422. The minimum expiration time for
sequence numbers 31-29 is 16, so the true expiration time for those
unreceived packets is 16, as shown at 424. In a similar fashion,
the minimum expiration time for the remaining unreceived packets
can be used to determine their respective true expiration times.
For packets 27, 26, 24, 23, and 22, the true expiration is 15, as
shown at 426. For packets 20-18 the true expiration time is 14, as
shown at 428. Therefore, as expiration times are set and extended
downward for each received packet, and true expiration times for
non-valid table entries can be determined using the MIN
function.
[0039] FIG. 5 shows a flow diagram 500 for one embodiment of a
method for reordering frames in accordance with the present
invention. At block 502, streams of packets are received at one or
more source processors for transmission over one or more fabrics to
at least one destination processor. At block 504, the packets
received at each source processor are given a unique sequence
number, and optionally prioritized. For example, the sequence
number may include a source identifier so that each packet may be
uniquely identified.
[0040] At block 506, the sources transmit their respective streams
of sequenced packets, via one or more fabrics, to at least one
destination processor. At block 508, the streams of sequenced
packets are received at the destination processor. Each received
packet is provided with a time stamp that indicates a receive time
for the packet.
[0041] At block 510, the sequence numbers, source identifier and
any priority information is extracted from the received packets. At
block 512, the packet data is stored into memory at the destination
processor. A memory locator is associated with each packet, so that
the packet data may be retrieved from the memory.
[0042] At block 514, a memory table is updated to map the arrival
of the packets, their respective sequence numbers, expiration
times, and memory pointers that identify the locations in the
memory where the data associated with each packet is stored.
[0043] The operations discussed with respect to blocks 502 through
514 are repeated so that as additional packets are received from
the source processors, their data is stored in memory and the
memory table is updated in accordance with the above steps. The
remaining steps in the method 500, shown below, are used to reorder
the received packets in accordance with the present invention and
transmit the received packets in the correct order to another
destination.
[0044] At block 516, a determination is made to determine a packet
to be unloaded from the memory for output. The packet is determined
from the memory table based on sequence number, and optionally, a
priority level. For example, in one embodiment, the packets are
determined from their associated sequence numbers without regard to
priority levels. In another embodiment, the packets with higher
priority are output in sequence before packets with lower
priority.
[0045] At block 518, a test is performed to determine if the
determined packet is available for output. If the packet has not
arrived, the method proceeds to block 520 where a test is performed
to determine if the true expiration time for the packet has been
reached by the current time. For example, the destination processor
has a time source that is used to keep track of current time and to
time stamp received packets. The true expiration time for any
unreceived packet includes a selectable transmission factor that
represents a transmission time through the communication fabric
that the packet flows through to get to the destination. Thus, if
the current time exceeds the true expiration time for the
determined packet, and that packet has not been received, the
packet is considered to be lost.
[0046] If the expiration time has not been reached by the current
time, the method proceeds back to block 518 to continue checking to
see if the packet has arrived. If the expiration time has been
reached by the current time, then it is assumed that the packet has
been lost, and the method proceeds to block 522 where the packet is
dropped and a next packet is determined for output at block
516.
[0047] At block 524, assuming the packet is available, the selected
packet is unloaded from the memory, and the freed memory is made
available to store newly received data packets. For example, the
memory pointer associated with the packet is reused by the Enqueue
logic to store a newly received packet. At block 526, the selected
packet retrieved from the memory is output from the reordering
system to another destination. The method then proceeds to block
516 where another packet is determined for output in accordance
with the present invention.
[0048] In accordance with the present invention, frames transmitted
from a particular source are reordered to have the same order as
when initially transmitted. Therefore, one or more embodiments
included in the present invention provide a system for reordering
frames in a switching network, so that lost frames can be
determined, and as a result, the transmission of received frames is
not excessively delayed.
[0049] Exemplary Operation
[0050] FIG. 6 illustrates an example of one embodiment for
reordering packets in accordance with the present invention. To
demonstrate the operation of this embodiment of the invention, it
will be assumed that a stream of packets transmitted from a
particular source processor is received at a destination processor
and reordered in accordance with the present invention. Although
one or more embodiments included in the present invention operate
to reorder packets received from multiple source processors that
may have multiple priority levels, for the purposes of clarity, the
present example demonstrates reordering of packets received from a
single source processor. However, it will be apparent to one with
skill in the art that the embodiments of the invention described
herein can be expanded, modified, or adjusted to reorder packets
received from multiple source processors.
[0051] In one embodiment, the reordering system may be implemented
in a single integrated circuit (IC chip), where tables and/or
memory that are used to reorder the packets are implemented in an
off-chip memory (SDRAM or SRAM). However, it is also possible to
implement the tables and/or memory on-chip.
[0052] Referring now to FIG. 6 there is shown Enqueue logic 312,
Dequeue logic 316, time logic 308, and memory 602 with associated
address FIFO 604. Also shown in FIG. 6 is memory table 310. In the
present embodiment, the memory 602, FIFO 604 and table 310 are part
of the memory 306 of FIG. 3. However, in accordance with the
invention, these elements may be formed from the same memory or
separate memories, or may be formed from any other combination of
logic, memory or registers.
[0053] There are four registers that are also part of the
reordering system shown in FIG. 6. The four registers are
associated with each transmitting source processor for reordering
packets associated with each respective source processor. For
clarity, the registers are shown as individual registers, however,
the registers may be part of the Enqueue logic 312, the Dequeue
logic 316 or implemented in any other hardware or software
associated with the reordering system. The four registers are
defined as follows.
[0054] 1. A read pointer (read_ptr) register 606 points at a
current location in the table 310 and is used when forming the
reordered output stream.
[0055] 2. A delayed store (delayed_store) register 608 stores
parameters associated with incoming packets until the parameters
are ready to be written into the table 310.
[0056] 3. A current value (current_value) register 610 holds an
ongoing current sequence number and expiration time used when
forming the reordered output stream.
[0057] 4. A largest value (largest_value) register 612 holds the
largest sequence number and expiration time seen while reading
parameters from the table 310 when forming the reordered output
stream.
[0058] As packets arrive at the destination processor, the Enqueue
logic 312 obtains a memory pointer from the address FIFO 604 and
uses this pointer to store the packet's data into the memory 602.
As packets enter the reorder system, a time-stamp will be
associated with each packet as will a timeout-period (this
timeout-period is under software control and can be modified for
any reason). For example, the time out period represents the time
it is expected to take for a packet to flow from the source to the
destination via the communication fabric.
[0059] An expiration time is calculated by adding the time-stamp
and timeout-period together. The sequence number of the packet and
expiration time are packet parameters that are referred to as the
data-pair of the packet. The data-pair of the packet are stored in
the table 310 at a table location indicated by a current sequence
number that is contained in the delayed_store register 608. The
table location pointed to by the delayed_store register represents
a pointer from the previous packet.
[0060] To illustrate the above process, it will be assumed that a
stream of packets is received at the reordering system, and
sequence numbers, time stamps and expiration times for a portion of
those packets are shown at 614. It will be assumed that the packet
with sequence number (5) has been processed by the Enqueue logic of
the reordering system. When this packet is processed, its sequence
number and memory pointer are stored into the delay_store register
608. The next packet to be processed has sequence number (18) as
shown at 616.
[0061] The Enqueue logic obtains a memory pointer from the address
FIFO 604 and uses this pointer to store the data associated with
packet (18) into the memory 602. The Enqueue logic then stores the
data-pair (seq/exp) for packet 18 in the table 310 at the address
pointed to by the delay_store register. Thus, the data-pair for
packet 18 is stored at location (5) in the table 310 as shown at
618. Also stored at location (5) in the table 310 is the memory
pointer for packet (5) data. The Enqueue logic then stores the
sequence number and memory pointer for packet (18) into the
delay_store register 608 as shown at 620. Thus, the next packet
processed by the Enqueue logic will have its parameters stored at
location (18) in the table 310.
[0062] To summarize the above process, the data pointer for the
packet having sequence number (5) that is stored in the table 310
is correct for that sequence number, but the accompanying data-pair
points to the next packet that has arrived. At the same time, the
new packet (18) has its sequence number and memory pointer stored
into the delayed_store register 608. Thus, the delayed_store
register lives-up to its name by delaying stores into the table 310
until a subsequent packet is received. The delayed store register
also skews where the stores happen (i.e. addressing) so as to
adhere to the underlying algorithm included in the present
invention.
[0063] The above process is used by the reordering system included
in the present invention to receive packets, time stamp the
packets, compute an expiration time for the packets, store the
packet data into memory, and store packet parameters into the table
310. The following text will describe how the reordering system
operates to retrieve the packet data from memory and reorder the
packets to form a reordered output stream in accordance with the
present invention.
[0064] The Dequeue logic operates to retrieve the packet data from
the memory 602 using the read-ptr register 606 and the table 310 to
form a reordered output stream 632. The read_ptr register points to
the next sequence number in the table 310 to determine the packet
data to be retrieved from the memory. If the value read from the
table at the selected sequence number is valid, its accompanying
memory data pointer is used to retrieve the data from memory for
output. If the value from the table is not valid, the Dequeue logic
operates to determine if the packet associated with that sequence
number has expired.
[0065] To determine whether the packet associated a sequence number
has expired, the Dequeue logic compares the current time (system
time) from the time logic 308 to the expiration time stored in the
current_value register 610. If the expiration time stored in the
current_value register is greater than the system time, the Dequeue
logic operates to determine if the packet has been received and is
being processed by the Enqueue logic. Thus, the packet has not
expired and the system operates to wait to receive that packet.
However, once the system timer equals the expiration time and the
packet has still not arrived, the Dequeue logic deems that packet
to be lost, and the read_ptr is incremented to point to the next
entry in the table to find the next sequenced packet to output. If
the next entry is not valid, a comparison of expiration time (which
hasn't changed because no valid packet has been read) and the
system time will indicate that that entry is lost as well. This
process continues until a valid entry is read.
[0066] The current_value register holds the next data-pair (i.e.,
the next sequence number and expiration time) that can be used to
define the true expiration time for one or more packets. For
example, entries for the current_value register for the above
example are shown at 622. The current_value register is updated
when the read_ptr equals the current_value sequence number. For
example, when the read_ptr points to sequence number (18) in the
table 310, the current value register is updated to have a sequence
number of (20) and an expiration time of (19), as shown at 624.
[0067] If the next sequence value read from the table is less than
the current_value's sequence number, the largest_value register 612
is used to determine the entries in the current_value register 610.
This prevents a situation where a previously accessed sequence
number's expiration time is used. For example, when the read_ptr is
at sequence number (20) the sequence number read from the table is
(10). This number is less than the value in the current_value
register. In this case, the next entries for the current_value
register are determined from the largest_value register 612. Thus,
the sequence number (40) and expiration time (25) are entered into
the current_value register, as show at 626.
[0068] The largest_value register holds the largest sequence number
read thus far from the table. As each entry in the table is read,
this register is updated with every sequence number that is higher
than the current contents of the register. The associated
expiration time is also entered. Therefore, the use of the
current_value register and the largest_value register operate to
provide true expiration times for the packets, as shown at 628.
Upon reset, the current value register will equal the first
read_out value pair.
[0069] There are two valid bits (v1 and v2), shown at 630, that are
part of the table 310 and are used to reduce the number of writes
that are necessary to invalidate entries as they are read. A
selected column, for instance v1, is used to indicate whether or
not the current table entry is valid, and the other column, v2 in
this case, is reset when v1 is written. For example, referring to
table 310, v2 is used to indicate that current table entries are
valid when a "1" is present. At the same time, v1 is set to "0."
When enough frames are received so that the table fills and
eventually wraps-around, v1 will then be set to "1" when a new
entry is valid, and v2 will be reset to "0." Thus, by alternating
between v1 and v2 to indicate when a table entry is valid (i.e., a
packet has arrived), it is only necessary to perform one write
operation to validate an entry, which eliminates the need to
perform a second write operation to invalidate the entry when the
corresponding packet is output.
[0070] There is a global setting that indicates which valid bit (v1
or v2) to use to indicate that a table entry is valid. For example,
the global setting is controlled by the Enqueue logic 312. In this
way, when all the table entries are exhausted, the system changes
the global setting to reverse the role of the valid bits (v1, v2)
so that the Dequeue logic knows to look at the correct valid bit to
determine when a table entry is valid. Therefore, an extra write
cycle is not needed to invalid an entry.
[0071] As the Dequeue logic 316 reads the table 310 to obtain
address pointers for the packet data to be output, those address
pointers are returned to the FIFO 604 so that they may be reused by
the Enqueue logic 312 to store newly received packet data. By using
the read_ptr register to sequentially read the table 310 to obtain
address pointers to received packet data, the Dequeue logic
operates to output the packet data in the correct order with
respect to how it was transmitted from the originating source
processor. The Dequeue logic also operates to determine true
expiration times for unreceived packets and skips those packets
when it is determined that they have expired, thereby preventing
lost packets from unreasonably delaying or blocking the formation
of the output stream 632.
[0072] Pseudo Code
[0073] To further illustrate the inter-workings of the reordering
algorithm included in the present invention, the following
pseudo-code is presented that describes how the current_value and
largest_value registers are updated in accordance with the present
invention.
[0074] Current_Value Register
[0075] To initialize this register, after a reset occurs the
sequence number included with the first incoming packet is entered
as the current_value. The following code is used to update the
current_value register after initialization.
[0076] if (read_ptr==current_value.sequence_number)
[0077] current_value.sequence_number=read_out.sequence_number
[0078] current_value.expiration_time=read_out.expiration_time
[0079] if (read_ptr=current_value.sequence_number)
[0080] if
(current_value.sequence_number>read_out.sequence_number)
[0081]
current_value.sequence_number=largest_value.sequence_number
[0082]
current_value.expiration_time=largest_value.expiration_time
[0083] if ((read_ptr==current_value.sequence_number) &
!valid)
[0084] send delay_store register
[0085] Largest_Value Register
[0086] The largest_value register operates as a place holder for
the highest sequence numbers read from the table during operation
of the reordering system. The following code is used to update the
largest_value register after initialization.
[0087] if
(read_out.sequence_number>largest_value.sequence_number)
[0088] largest_value.sequence_number=read_out.sequence_number
[0089] largest_value.expiration_time=read_out.expiration_time
[0090] Although described with reference to the utilization of the
table 310, it will be apparent to one with skill in the art that a
separate table, memory, or memory map may be used to process data
packets from each transmitting source processor, and if required,
for each respective priority used. Thus, in other embodiments of
the present invention, the Enqueue logic and the Dequeue logic are
coupled to multiple tables, and processing steps similar to those
described above are performed for each table to reorder the
received data packets for output in the reordered stream 632.
[0091] The present invention includes a system for reordering
sequenced based packets in a switching network. The embodiments
described above are illustrative of the present invention and are
not intended to limit the scope of the invention to the particular
embodiments described. Accordingly, while several embodiments of
the invention have been illustrated and described, it will be
appreciated that various changes can be made therein without
departing from the spirit or essential characteristics thereof.
Accordingly, the disclosures and descriptions herein are intended
to be illustrative, but not limiting, of the scope of the
invention, which is set forth in the following claims.
* * * * *