U.S. patent application number 11/838152 was filed with the patent office on 2008-01-31 for method and apparatus for improving the fairness of new attaches to a weighted fair queue in a quality of service (qos) scheduler.
This patent application is currently assigned to IBM. Invention is credited to William John Goetzinger, Glen Howard Handlogten, James Francis Mikos, David Alan Norgaard.
Application Number | 20080025215 11/838152 |
Document ID | / |
Family ID | 28040142 |
Filed Date | 2008-01-31 |
United States Patent
Application |
20080025215 |
Kind Code |
A1 |
Goetzinger; William John ;
et al. |
January 31, 2008 |
METHOD AND APPARATUS FOR IMPROVING THE FAIRNESS OF NEW ATTACHES TO
A WEIGHTED FAIR QUEUE IN A QUALITY OF SERVICE (QoS) SCHEDULER
Abstract
In a first aspect, a network processor includes a scheduler in
which a scheduling queue is maintained. A last frame is dispatched
from a flow queue maintained in the network processor, thereby
emptying the flow queue. Data indicative of the size of the
dispatched last frame is stored in association with the scheduler.
A new frame corresponding to the emptied flow queue is received,
and the flow corresponding to the emptied flow queue is attached to
the scheduling queue. The flow is attached to the scheduling queue
at a distance D from a current pointer for the scheduling queue.
The distance D is determined based at least in part on the stored
data indicative of the size of the dispatched last frame.
Inventors: |
Goetzinger; William John;
(Rochester, MN) ; Handlogten; Glen Howard;
(Rochester, MN) ; Mikos; James Francis;
(Rochester, MN) ; Norgaard; David Alan;
(Rochester, MN) |
Correspondence
Address: |
IBM CORPORATION, INTELLECTUAL PROPERTY LAW
DEPT 917
3605 HIGHWAY 52 NORTH
ROCHESTER
MN
55901-7829
US
|
Assignee: |
IBM
|
Family ID: |
28040142 |
Appl. No.: |
11/838152 |
Filed: |
August 13, 2007 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10102166 |
Mar 20, 2002 |
7257124 |
|
|
11838152 |
Aug 13, 2007 |
|
|
|
Current U.S.
Class: |
370/230 |
Current CPC
Class: |
H04L 47/10 20130101;
H04L 47/2441 20130101 |
Class at
Publication: |
370/230 |
International
Class: |
H04L 12/56 20060101
H04L012/56 |
Claims
1. A method of operating a network processor, comprising:
dispatching a last frame from a flow queue maintained in the
network processor, thereby emptying the flow queue; and storing
data indicative of a size of the dispatched last frame.
2. The method of claim 1, further comprising: receiving a new frame
corresponding to the emptied flow queue; and attaching to a
scheduling queue a flow corresponding to the emptied flow queue;
wherein the flow is attached to the scheduling queue a distance D
from a current pointer for the scheduling queue, the distance D
being determined based at least in part on the stored data
indicative of the size of the dispatched last frame.
3. The method of claim 2, wherein the distance D is determined
based in part on a Quality of Service to which the flow is
entitled.
4. The method of claim 1, wherein the data indicative of the size
of the dispatched last frame is stored in a flow queue control
block which corresponds to the emptied flow queue.
5. The method of claim 4, wherein the flow queue control block is
stored in a random access memory associated with a scheduler.
6. A network processor, comprising: a scheduler which includes a
scheduling queue having flows attached thereto and defining a
sequence in which the attached flows are to be serviced; and
storage means, associated with the scheduler, for maintaining a
flow queue corresponding to each flow attached to the scheduling
queue; wherein the storage means stores, for each flow queue that
has been emptied, data indicative of a last frame dispatched from
the respective flow queue.
7. The network processor of claim 6, wherein: when a new frame is
received that corresponds to a flow queue that has been emptied, a
flow corresponding to the new frame is attached to the scheduling
queue at a distance D from a current pointer for the scheduling
queue, the distance D being determined based at least in part on
the stored data indicative of the size of the last frame dispatched
from the flow queue that has been emptied.
8. The network processor of claim 7, wherein the distance D is
determined based in part on a Quality of Service to which the flow
is entitled.
9. The network processor of claim 6, wherein the data indicative of
the size of the last frame dispatched from the respective flow
queue is stored in a flow queue control block which corresponds to
the respective flow queue.
10. The network processor of claim 6, wherein the storage means
includes a random access memory.
11. A computer program product for use with a network processor,
the computer program product comprising: a medium readable by a
computer, the computer readable medium having program code adapted
to: dispatch a last frame from a flow queue maintained in the
network processor, thereby emptying the flow queue; and store data
indicative of a size of the dispatched last frame.
Description
[0001] The present application is a continuation of and claims
priority to U.S. patent application Ser. No. 10/102,166, filed Mar.
20, 2002, which is hereby incorporated by reference herein in its
entirety.
CROSS REFERENCE TO RELATED PATENT APPLICATIONS
[0002] The present application is related to the following U.S.
Patent Applications, each of which is hereby incorporated by
reference herein in its entirety: [0003] U.S. patent application
Ser. No. 10/016,518, filed Nov. 1, 2001, titled "WEIGHTED FAIR
QUEUE HAVING EXTENDED EFFECTIVE RANGE" (IBM Docket No.
ROC920010199US1); [0004] U.S. patent application Ser. No.
10/015,994, filed Nov. 1, 2001, titled "WEIGHTED FAIR QUEUE SERVING
PLURAL OUTPUT PORTS" (IBM Docket No. ROC920010200US1); [0005] U.S.
patent application Ser. No. 10/015,760, filed Nov. 1, 2001, titled
"WEIGHTED FAIR QUEUE HAVING ADJUSTABLE SCALING FACTOR" (IBM Docket
No. ROC920010201US1); [0006] U.S. patent application Ser. No.
10/002,085, filed Nov. 1, 2001, titled "EMPTY INDICATORS FOR
WEIGHTED FAIR QUEUES" (IBM Docket No. ROC920010202US1); [0007] U.S.
patent application Ser. No. 10/004,373, filed Nov. 1, 2001, titled
"QoS SCHEDULER AND METHOD FOR IMPLEMENTING PEAK SERVICE DISTANCE
USING NEXT PEAK SERVICE TIME VIOLATED INDICATION" (IBM Docket No.
ROC920010203US1); [0008] U.S. patent application Ser. No.
10/002,416, filed Nov. 1, 2001, titled "QoS SCHEDULER AND METHOD
FOR IMPLEMENTING QUALITY OF SERVICE WITH AGING STAMPS" (IBM Docket
No. ROC920010204US1); [0009] U.S. patent application Ser. No.
10/004,440, filed Nov. 1, 2001, titled "QoS SCHEDULER AND METHOD
FOR IMPLEMENTING QUALITY OF SERVICE WITH CACHED STATUS ARRAY" (IBM
Docket No. ROC920010205US1); and [0010] U.S. patent application
Ser. No. 10/004,217, filed Nov. 1, 2001, titled "QoS SCHEDULER AND
METHOD FOR IMPLEMENTING QUALITY OF SERVICE ANTICIPATING THE END OF
A CHAIN OF FLOWS" (IBM Docket No. ROC920010206US1).
FIELD OF THE INVENTION
[0011] The present invention is concerned with data and storage
communication systems and is more particularly concerned with a
network processor that includes a scheduler component.
BACKGROUND OF THE INVENTION
[0012] Data and storage communication networks are in widespread
use. In many data and storage communication networks, data packet
switching is employed to route data packets or frames from point to
point between source and destination, and network processors are
employed to handle transmission of data into and out of data
switches.
[0013] FIG. 1 is a block diagram illustration of a conventional
network processor in which the present invention may be applied.
The network processor, which is generally indicated by reference
numeral 10, may be constituted by a number of components mounted on
a card or "blade". Within a data communication network, a
considerable number of blades containing network processors may be
interposed between a data switch and a data network.
[0014] The network processor 10 includes data flow chips 12 and 14.
The first data flow chip 12 is connected to a data switch 15 (shown
in phantom) via first switch ports 16, and is connected to a data
network 17 (shown in phantom) via first network ports 18. The first
data flow chip 12 is positioned on the ingress side of the switch
15 and handles data frames that are inbound to the switch 15.
[0015] The second data flow chip 14 is connected to the switch 15
via second switch ports 20 and is connected to the data network 17
via second network ports 22. The second data flow chip 14 is
positioned on the egress side of the switch 15 and handles data
frames that are outbound from the switch 15.
[0016] As shown in FIG. 1, a first data buffer 24 is coupled to the
first data flow chip 12. The first data buffer 24 stores inbound
data frames pending transmission of the inbound data frames to the
switch 15. A second data buffer 26 is coupled to the second data
flow chip 14, and stores outbound data frames pending transmission
of the outbound data frames to the data network 17.
[0017] The network processor 10 also includes a first processor
chip 28 coupled to the first data flow chip 12. The first processor
chip 28 supervises operation of the first data flow chip 12 and may
include multiple processors. A second processor chip 30 is coupled
to the second data flow chip 14, supervises operation of the second
data flow chip 14 and may include multiple processors.
[0018] A control signal path 32 couples an output terminal of
second data flow chip 14 to an input terminal of first data flow
chip 12 (e.g., to allow transmission of data frames
therebetween).
[0019] The network processor 10 further includes a first scheduler
chip 34 coupled to the first data flow chip 12. The first scheduler
chip 34 manages the sequence in which inbound data frames are
transmitted to the switch 15 via first switch ports 16. A first
memory 36 such as a fast SRAM is coupled to the first scheduler
chip 34 and stores data frame pointers (in the form of flow queues)
and flow control information (in the form of flow queue control
blocks ("FQCBs") 37). Flow queues are discussed further below. The
first memory 36 may be, for example, a QDR (quad data rate)
SRAM.
[0020] A second scheduler chip 38 is coupled to the second data
flow chip 14. The second scheduler chip 38 manages the sequence in
which data frames are output from the second network ports 22 of
the second data flow chip 14. Coupled to the second scheduler chip
38 are at least one and possibly two memories (e.g., fast SRAMs 40)
for storing data frame pointers and flow control information. The
memories 40 may, like the first memory 36, be QDRs. The additional
memory 40 on the egress side of the network processor 10 may be
needed because of a larger number of flows output through the
second network ports 22 than through the first switch ports 16.
[0021] FIG. 2 schematically illustrates conventional queuing
arrangements that may be provided for a data flow chip/scheduler
pair (either the first data flow chip 12 and the first scheduler
chip 34 or the second data flow chip 14 and the second scheduler
chip 38) of the network processor 10 of FIG. 1. In the particular
example illustrated in FIG. 2, the first data flow chip 12 and the
first scheduler chip 34 are illustrated, but a very similar queuing
arrangement may be provided in connection with the second data flow
chip 14 and the second scheduler chip 38. In the queuing
arrangement for the first data flow chip 12 and the first scheduler
chip 34, incoming data frames (from data network 17) are buffered
in the input data buffer 24 associated with the first data flow
chip 12 (FIG. 1). Each data frame is associated with a data flow or
"flow". As is familiar to those who are skilled in the art, a
"flow" represents a one-way connection between a source and a
destination.
[0022] Flows with which the incoming data frames are associated are
enqueued in ("attached to") a scheduling queue 42 maintained in the
first scheduler chip 34. The scheduling queue 42 defines a sequence
in which the flows attached thereto are to be serviced. The
particular scheduling queue 42 of interest in connection with the
present invention is a weighted fair queue which arbitrates among
flows entitled to a "best effort" or "available bandwidth" Quality
of Service (QoS).
[0023] As shown in FIG. 2, the scheduling queue 42 is associated
with a respective output port 44 of the first data flow chip 12. It
is to be understood that the output port 44 is one of the first
switch ports 16 illustrated in FIG. 1. (However, if the data flow
chip/scheduler pair under discussion were the egress side data flow
chip 14 and scheduler chip 38, then the output port 44 would be one
of the network ports 22.) Although only one scheduling queue 42 and
one corresponding output port 44 are shown, it should be understood
that in fact there may be plural output ports and corresponding
scheduling queues each assigned to a respective port. (However,
according to an alternative embodiment, disclosed in
above-referenced co-pending patent application Ser. No. 10/015,994,
filed Nov. 1, 2001 (Attorney Docket No. ROC920010200US1), a group
of output ports may be associated with each scheduling queue
42).
[0024] Although not indicated in FIG. 2, the first scheduler chip
34 also includes flow scheduling calendars which define output
schedules for flows which are entitled to a scheduled QoS with
guaranteed bandwidth, thus enjoying higher priority than the flows
governed by the scheduling queue 42.
[0025] The memory 36 associated with the first scheduler chip 34
holds pointers ("frame pointers") to locations in the first data
buffer 24 corresponding to data frames associated with the flows
enqueued in the scheduling queue 42. The frame pointers are listed
in flow queues (not separately shown), each of which corresponds to
a respective flow that is or may be attached to the scheduling
queue 42. The flow queue indicates an order in which frames
associated with the flow were received and are to be
dispatched.
[0026] The memory 36 also stores flow control information, such as
information indicative of the QoS to which flows are entitled. The
flow control information is stored in flow queue control blocks
("FQCBs"), each of which corresponds to a respective one of the
flow queues.
[0027] When the scheduling queue 42 indicates that a particular
flow attached thereto is the next to be serviced, reference is made
to the first frame pointer in the corresponding flow queue in the
memory 36 and the corresponding frame data is transferred from the
first data buffer 24 to an output queue 46 associated with the
output port 44. At the same time, the flow is detached from the
scheduling queue 42, and, assuming that at least one more frame
pointer remains in the corresponding flow queue, is reattached to
the scheduling queue in accordance with a procedure that is
described below.
[0028] A more detailed representation of the scheduling queue 42 is
shown in FIG. 3. As noted above, the scheduling queue 42 is used
for weighted fair queuing of flows serviced on a "best effort"
basis. In a particular example of a scheduling queue as illustrated
in FIG. 3, the scheduling queue 42 has 512 slots (each slot
represented by reference numeral 48). Other numbers of slots may be
employed. In accordance with conventional practice, flows are
enqueued to the scheduling queue 42 in the case of a "reattachment"
based on a formula that takes into account both a length of a data
frame associated with a flow to be reattached and a weight which
corresponds to a QoS to which the flow is entitled.
[0029] More specifically, the queue slot in which a flow is placed
upon reattachment is calculated according to the formula
CP+((WF.times.FS)/SF), where CP is a pointer ("current pointer")
that indicates a current position (the slot currently being
serviced) in the scheduling queue 42; WF is a weighting factor
associated with the flow to be enqueued, the weighting factor
having been determined on the basis of the QoS to which the flow is
entitled; FS is the size of the frame currently being dispatched
for the flow to be reattached; and SF is a scaling factor chosen to
scale the product (WF.times.FS) so that the resulting quotient
falls within the range defined by the scheduling queue 42. (In
accordance with conventional practice, the scaling factor SF is
conveniently defined as an integral power of 2--i.e., SF=2.sup.n,
with n being a positive integer--so that scaling the product
(WF.times.FS) is performed by right shifting.) With this known
weighted fair queuing technique, the weighting factors assigned to
the various flows in accordance with the QoS assigned to each flow
govern how close to the current pointer of the queue each flow is
enqueued. In addition, flows which exhibit larger frame sizes are
reattached farther from the current pointer of the queue, to
prevent such flows from appropriating an undue proportion of the
available bandwidth of the queue. Upon reattachment, data that
identifies a flow (the "Flow ID") is stored in the appropriate
queue slot 48.
[0030] In addition to the "reattachment" situation described above,
there are two other cases in which flows are attached to the
scheduling queue 42. The first of these two cases is concerned with
attachment to the scheduling queue 42 upon arrival of the first
frame for a new flow. The second of the two cases is concerned with
attachment of a flow to the scheduling queue 42 upon arrival of the
first frame after the flow queue for the flow in question has been
emptied (i.e., after the last frame pointed to by the flow queue is
dispatched). In both of these cases, there is no frame currently
being dispatched, and accordingly, there is no size information
available for such a currently dispatched frame. It has therefore
been proposed in both cases to attach the flow to the scheduling
queue 42 at a predetermined fixed distance from the current pointer
CP for the scheduling queue 42. However, the present inventors have
recognized that this proposed practice may undermine the desired
weighted fair queuing in certain situations that may be encountered
in the second case, namely attachment of the flow to the scheduling
queue 42 after the corresponding flow queue has been emptied. In
particular, if a given flow is made up of large but relatively
infrequent frames, the predetermined fixed enqueuement distance may
be too short to limit the flow in question to the Quality of
Service to which it is entitled. Furthermore, where a flow is made
up of relatively infrequent short frames, the predetermined fixed
enqueuement distance may work to "short change" the flow, i.e., to
prevent it from receiving the Quality of Service to which it is
entitled.
[0031] It is an object of the present invention to assure that a
contracted-for QoS is maintained for a flow upon attachment of the
flow to a weighted fair queue in a case where a new frame is
received for the flow after the corresponding flow queue has
emptied.
SUMMARY OF THE INVENTION
[0032] According to a first aspect of the invention, a method of
operating a network processor is provided. The method includes
dispatching a last frame from a flow queue maintained in the
network processor, thereby emptying the flow queue, and storing
data indicative of a size of the dispatched last frame.
[0033] In at least one embodiment, the inventive method may further
include receiving a new frame corresponding to the emptied flow
queue, and attaching to a scheduling queue a flow corresponding to
the emptied flow queue. The flow may be attached to the scheduling
queue a distance D from a current pointer for the scheduling queue,
where the distance D is determined based at least in part on the
stored data indicative of the size of the dispatched last
frame.
[0034] According to a second aspect of the invention, a network
processor is provided, including a scheduler which includes a
scheduling queue. The scheduling queue has flows attached thereto
and defines a sequence in which the attached flows are to be
serviced. The network processor according to this aspect of the
invention further includes a storage device that is associated with
the scheduler, and maintains a flow queue corresponding to each
flow attached to the scheduling queue. Further in accordance with
the first aspect of the invention, the storage device stores, for
each flow queue that has been emptied, data indicative of a size of
a last frame dispatched from the respective flow queue.
[0035] In at least one embodiment, when a new frame is received
that corresponds to a flow queue that has been emptied, the flow
corresponding to the new frame may be attached to the scheduling
queue at a distance D from a current pointer for the scheduling
queue. The distance D is determined based at least in part on the
stored data indicative of the size of the last frame dispatched
from the flow queue that has been emptied.
[0036] Numerous other aspects are provided, as are computer program
products. Each inventive computer program product may be carried by
a medium readable by a computer (e.g., a carrier wave signal, a
floppy disk, a hard drive, a random access memory, etc.).
[0037] With the apparatus and method of the present invention, a
flow may be attached to the scheduling queue, after emptying of the
corresponding flow queue and receipt of a new frame for the flow,
on the basis of the size of the last frame dispatched upon emptying
of the flow queue. Consequently, flows that attempt to "misbehave"
by sending very large but infrequent frames, are nevertheless
accorded their appropriate Quality of Service. Furthermore, flows
made up of relatively infrequent short frames will not be penalized
due to the small size of the frames in the flow.
[0038] Other objects, features and advantages of the present
invention will become more fully apparent from the followed
detailed description of exemplary embodiments, the appended claims
and the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0039] FIG. 1 is a block diagram of a conventional network
processor in which the present invention may be applied;
[0040] FIG. 2 is a block diagram representation of conventional
queuing arrangements provided in a data flow chip/scheduler pair
included in the network processor of FIG. 1;
[0041] FIG. 3 is a pictorial representation of a weighted fair
queuing scheduling queue provided in accordance with conventional
practices; and
[0042] FIG. 4 is a flow chart that illustrates a process provided
in accordance with the invention for attaching a flow to a
scheduling queue.
DETAILED DESCRIPTION
[0043] Attachment of a flow to the scheduling queue 42 in
accordance with the invention will now be described, with reference
to FIG. 4 and the conventional network processor 10 of FIGS. 1-3.
It will be understood that the present invention may be employed
with any suitable conventional network processor.
[0044] FIG. 4 is a flow chart that illustrates a process provided
in accordance with the invention for attaching a flow to the
scheduling queue 42.
[0045] The process of FIG. 4 starts at block 50 and proceeds to
block 52, at which the last frame for a flow attached to the
scheduling queue 42 is dispatched by the network processor 10.
Accordingly, the flow queue corresponding to the flow in question
is emptied, and the flow is detached from the scheduling queue 42,
without being reattached.
[0046] Following, or in conjunction with, block 52 is block 54. At
block 54 data indicative of the size of the frame dispatched at
block 52 is stored. For example, this data may be stored in the
flow queue control block (FQCB) corresponding to the flow queue
which was emptied at block 52.
[0047] Following block 54 is a decision block 56, at which it is
determined whether the next frame has arrived for the flow
corresponding to the emptied flow queue. Until the next frame
arrives, the process of FIG. 4 idles. Once the next frame has
arrived, block 58 follows decision block 56. At block 58, the
location (e.g., the particular slot 48) at which the flow
corresponding to the arriving frame is to be attached to the
scheduling queue 42 is determined based at least in part on the
data stored at block 54 that is indicative of the size of the frame
dispatched at block 52. In particular, the slot at which the flow
is to be attached may be determined in accordance with the same
formula CP+((WF.times.FS)/SF) referred to above, except that FS in
this case is taken to be the size of the frame dispatched at block
52, as indicated by the data stored at block 54. The other symbols,
namely CP, WF and SF have the same meaning referred to above and
therefore need not be explained again.
[0048] Following block 58 is block 60. At block 60, the flow in
question is attached to the scheduling queue 42 at the slot
determined at block 58. The process then ends, at 62.
[0049] With the method and apparatus of the present invention,
flows that "misbehave" by sending very large frames infrequently
can be prevented from misappropriating a quantity of bandwidth to
which such flows are not entitled. At the same time, the inventive
method and apparatus prevent flows exhibiting infrequent, small
frames from being "short changed".
[0050] The process of FIG. 4 may be implemented in hardware,
software or a combination thereof. In at least one embodiment of
the invention, the process of FIG. 4 is implemented in hardware
employing a suitable combination of conventional logic circuitry
such as adders, comparators, selectors, etc. Such hardware may be
located, for example, within the scheduler 34 and/or the scheduler
38 (FIG. 1), and/or within the data flow chip 12 and/or the data
flow chip 14. A person of ordinary skill in the art may develop
logic circuitry capable of performing the inventive process
described with reference to FIG. 4. In a software embodiment of the
invention, the process of FIG. 4 may comprise one or more computer
program products. Each inventive computer program product may be
carried by a medium readable by a computer (e.g., a carrier wave
signal, a floppy disk, a hard drive, a random access memory,
etc.).
[0051] The foregoing description discloses only exemplary
embodiments of the invention; modifications of the above disclosed
apparatus and method which fall within the scope of the invention
will be readily apparent to those of ordinary skill in the art.
According to one alternative embodiment, a scheduling queue may
have plural subqueues of different ranges and resolutions,
according to an invention disclosed in above-referenced co-pending
patent application Ser. No. 10/016,518, filed Nov. 1, 2001
(Attorney Docket No. ROC920010199US1).
[0052] Moreover, in the above description, the invention has been
implemented in connection with a separate scheduler chip associated
with a network processor. However, it is also contemplated to
implement the invention in a scheduler circuit that is implemented
as part of a data flow chip or as part of a processor chip.
[0053] Accordingly, while the present invention has been disclosed
in connection with exemplary embodiments thereof, it should be
understood that other embodiments may fall within the spirit and
scope of the invention, as defined by the following claims.
* * * * *