U.S. patent application number 11/376666 was filed with the patent office on 2006-07-27 for method and apparatus for managing the flow of data within a switching device.
Invention is credited to H. Earl Ferguson, Mike K. Noll, Derek H. Pitcher, Jeffrey Prince, Randy Ryals.
Application Number | 20060165055 11/376666 |
Document ID | / |
Family ID | 36147469 |
Filed Date | 2006-07-27 |
United States Patent
Application |
20060165055 |
Kind Code |
A1 |
Ryals; Randy ; et
al. |
July 27, 2006 |
Method and apparatus for managing the flow of data within a
switching device
Abstract
A method and apparatus for managing the flow of data within a
switching device is provided. The switching device includes network
interface cards connected to a common backplane. Each interface
card is configured to support the maximum transfer rate of the
backplane by maintaining a "pending" queue to track data that has
been received but for which the appropriate routing destination has
not yet been determined. The switching device includes a switch
controller that maintains a central card/port-to-address table.
When an interface card receives data with a destination address
that is not known to the interface card, the interface card
performs a direct memory access over a bus that is separate from
the backplane to read routing data directly from the central table
in the switch controller. Each interface card builds and maintains
a routing information table in its own local memory that only
includes routing information for the destination addresses that the
interface card is most likely to receive.
Inventors: |
Ryals; Randy; (Menlo Park,
CA) ; Prince; Jeffrey; (Sunnyvale, CA) ;
Ferguson; H. Earl; (Los Altos, CA) ; Noll; Mike
K.; (San Jose, CA) ; Pitcher; Derek H.;
(Newark, CA) |
Correspondence
Address: |
BLAKELY SOKOLOFF TAYLOR & ZAFMAN
12400 WILSHIRE BOULEVARD
SEVENTH FLOOR
LOS ANGELES
CA
90025-1030
US
|
Family ID: |
36147469 |
Appl. No.: |
11/376666 |
Filed: |
March 15, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10330640 |
Dec 27, 2002 |
7031296 |
|
|
11376666 |
Mar 15, 2006 |
|
|
|
10093290 |
Mar 5, 2002 |
6788671 |
|
|
10330640 |
Dec 27, 2002 |
|
|
|
08896485 |
Jul 17, 1997 |
6427185 |
|
|
10093290 |
Mar 5, 2002 |
|
|
|
08536099 |
Sep 29, 1995 |
|
|
|
08896485 |
Jul 17, 1997 |
|
|
|
Current U.S.
Class: |
370/351 ;
370/362 |
Current CPC
Class: |
H04L 49/3009 20130101;
H04L 49/3045 20130101; H04L 49/254 20130101; H04L 49/30 20130101;
H04L 49/3036 20130101; H04L 49/252 20130101; H04L 49/901 20130101;
H04L 49/40 20130101; H04L 49/25 20130101; H04L 49/90 20130101 |
Class at
Publication: |
370/351 ;
370/362 |
International
Class: |
H04L 12/50 20060101
H04L012/50; H04Q 11/00 20060101 H04Q011/00; H04L 12/28 20060101
H04L012/28 |
Claims
1-14. (canceled)
15. A switching device comprising: a master control process
including a memory to store routing information; and a plurality of
devices coupled together over a first connection and in
communication with the master control process, each of the
plurality of devices comprising (i) a local memory including a
plurality of entries that collectively contain a portion of the
routing information stored in the memory of the master control
process and (ii) a controller to update contents of at least one
entry of the local memory from the memory of the master control
process over a second connection separate from the first
connection.
16. The switching device of claim 15, wherein the plurality of
devices comprises a first device including a plurality of
ports.
17. The switching device of claim 16, wherein controller of the
first device updates the contents in response to an event being a
condition in which contents of the entry have not been accessed for
longer than a specified amount of time.
18. The switching device of claim 16, wherein the controller of the
first device updates the contents in response to an event being a
condition in which a cell having a destination address is received
and no routing information associated with the cell is currently
stored in the local memory.
19. The switching device of claim 18, wherein the device further
comprising a control unit that controls the controller in
performing a transfer of the portion of the routing information
from the memory of the master control process into the local
memory.
20. The switching device of claim 18, wherein the cell is a packet
including the destination address.
21. The switching device of claim 19, wherein the control unit of
the first device further comprising: a buffer; a plurality of
queues including a pending queue; and an arrival handling unit
coupled to the pending queue, the arrival handling unit to place a
pointer into the pending queue, the pointer being associated with a
cell slot of the buffer into which the cell having the destination
address is placed.
22. The switching device of claim 21, wherein the control unit of
the first device further comprising a route determination unit that
sequentially removes pointers from the pending queue and determines
where the cell having the destination address should be routed.
23. The switching device of claim 16, wherein the first device is
an interface card.
24. The switching device of claim 19, wherein the controller and
the control unit are implemented as separate circuitry.
25. A switching device comprising: a master control process
including a memory to store routing information; and a plurality of
devices coupled together to exchange data over a first connection,
each of the plurality of devices, being in communication with the
master control process, comprises (i) a local memory adapted to
contain a portion of the routing information stored in the memory
of the master control process and (ii) a controller to receive the
portion of the routing information from the memory of the master
control process over a second connection separate of the first
connection upon receiving data that includes a destination address
about which no routing information is currently stored in the local
memory.
26. The switching device of claim 25, wherein the plurality of
devices comprises a first device that comprises (i) a first local
memory including a first portion of the routing information stored
in the memory of the master control process and (ii) a first
controller to receive the first portion of the routing information
from the memory of the master control process upon receiving data
that includes a destination address about which no routing
information is currently stored in the first local memory; and a
second device that comprises (i) a second local memory including a
second portion of the routing information stored in the memory of
the master control process and (ii) a second controller to receive
the second portion of the routing information from the memory of
the master control process upon receiving data that includes a
destination address about which no routing information is currently
stored in the second local memory.
27. The switching device of claim 26, wherein the second portion of
the routing information stored in the second local memory differs
from the first portion of the routing information stored in the
first local memory.
28. The switching device of claim 25, wherein the portion of the
routing information is a subset of the routing information stored
in the memory of the master control process.
29. The switching device of claim 26, wherein the master control
process is coupled to each of the plurality of devices via the
second connection being a multiple line bus that interconnects the
first controller and the second controller.
30. The switching device of claim 25, wherein at least a first
device of the plurality of devices further comprises a control unit
that controls the controller to perform a transfer of the portion
of the routing information from the memory of the master control
process into the local memory.
31. The switching device of claim 30, wherein the control unit of
the first device further comprising: a buffer; a plurality of
queues; and an arrival handling unit coupled to at least one of the
plurality of queues, the arrival handling unit to place a pointer
into an entry of the one of the plurality of queues, the pointer
being associated with a location of the buffer into which the data
having the destination address is placed.
32. The switching device of claim 31, wherein the data is a
cell.
33. The switching device of claim 32, wherein the control unit of
the first device further comprising a route determination unit that
sequentially removes pointers from the one of the plurality of
queues and determines where the cell having the destination address
should be routed.
34. The switching device of claim 25, wherein the master control
process resides separately from the plurality of devices.
35. The switching device of claim 25, wherein the memory of the
master control process stores a card/port-to-address table to
contain the routing information.
36. The switching device of claim 35, wherein the routing
information of the card/port-to-address table is indexed by
performing a hash operation on at least a portion of the
destination address.
37. The switching device of claim 35, wherein an address space of
the memory of the master control process is non-overlapping with
address spaces of the local memories associated with the plurality
of devices.
38. The switching device of claim 26, wherein an address space of
the memory of the master control process is non-overlapping with an
address space of the first local memory associated with the first
device and an address space of the second local memory associated
with the second device.
39. The switching device of claim 38, wherein the address spaces of
the first local memory and the address space of the second local
memory are non-overlapping.
40. The switching device of claim 39, wherein the first controller
is adapted to receive routing information contained in the second
local memory of the second device.
41. The switching device of claim 25, wherein the local memory
contains the portion of the routing information by either (i)
reading the portion of the routing information from the memory of
the master control process or (ii) writing the portion of the
routing information to the local memory.
42. The switching device of claim 35 wherein at least one of the
plurality of devices is an interface card, the first connection
being a bus and the second connection being a backplane.
43. A switching device comprising: a memory to store the routing
information; and a plurality of components in communication with
the memory, each of the plurality of components including a local
memory to store a table featuring a portion of the routing
information and a memory controller to receive particular routing
information from the memory upon receiving a cell that includes a
destination address about which the particular routing information
is currently not stored.
44. The switching device of claim 43, wherein the plurality of
components are a plurality of interface cards.
45. The switching device of claim 43, wherein the portion of the
routing information stored in a local memory of a first component
of the plurality of components differs from the portion of the
routing information stored in a local memory of a second component
of the plurality of components.
46. The switching device of claim 43, wherein the portion of the
routing information is a subset of the routing information stored
in the memory of a master control process.
47. The switching device of claim 43, wherein the cell is a packet
including the destination address.
48. A switching device comprising: a first connection; a second
connection; a master control process coupled to the second
connection, the master control process including a memory to store
routing information; and a first device coupled to the first
connection and the second connection, the first device comprising a
local memory and a memory controller to read the routing
information from the memory of the master control process via the
second connection upon receiving data that includes a destination
address about which no routing information is currently stored in
the local memory; and a second device coupled to the first device
via the first connection to transfer data to the first device and
further coupled to the master control process via the second
connection, the second device comprises a local memory and a memory
controller to read the routing information from the memory of the
master control process via the second connection upon receiving
data that includes a destination address about which no routing
information is currently stored in the local memory.
49. The switching device of claim 48, wherein the second connection
is a multiple line bus that interconnects the memory controller of
the first device with the memory controller of the second
device.
50. The switching device of claim 48, wherein the memory controller
of the first device with the memory controller of the second device
exclusively using the second connection to read the routing
information from the memory of the master control process.
51. A method comprising: exchanging data between a plurality of
components coupled together by a first connection and collectively
forming a switching device; reading routing information; inspecting
a local table to determine if an entry associated with the routing
information is stored within the local table; using information
from the entry for routing a cell if an entry of the local table
associated with the routing information is stored within the local
table; accessing information from a central table over a second
connection separate and independent from the first connection if no
entry associated with the routing information is stored within the
local table; and receiving the information from the central table
for use in routing the cell having the routing information.
52. The method of claim 51 further comprising: storing the
information from the central table into the local table for
subsequent use upon receipt of the cell including the routing
information.
53. The method of claim 51, wherein the cell is a packet including
a destination address.
54. The method of claim 53, wherein the reading of the routing
information from the cell includes reading the destination address
of the packet.
55. The method of claim 53, wherein the inspecting of the local
table includes performing a hashing operation on the destination
address of the cell to produce a result, the result being used as
an index for the local table.
56. The method of claim 51, wherein the accessing of the
information from the central table is conducted by a direct memory
access from one of the plurality of components to a master process
control located remotely from the one of the plurality of
components.
57. A method comprising: receiving a cell over a port coupled to a
first memory of a first device of a switching device, the first
device being coupled to a second device over a first connection to
exchange data; reading routing information from the cell;
inspecting a local table of the first memory to determine if an
entry associated with the routing information is stored within the
local table; using information from the entry for routing the cell
if an entry of the local table associated with the routing
information is stored within the local table; and populating the
local table with information accessed from a secondary table stored
in a second memory and retrieved over a second connection different
than the first connection if no entry associated with the routing
information is stored within the local table, the information being
used for continued routing of the cell.
58. The method of claim 57, wherein the cell is a packet including
a destination address.
59. The method of claim 58, wherein the inspecting of the local
table includes performing a hashing operation on a destination
address of the packet to produce a result, the result being used as
an index for the local table.
60. The method of claim 59, wherein the populating of the local
table comprises exclusively using the second connection that is
physically separate and independent from the first connection to
route the information from the central table to the local table
with the first connection being used to transfer information from
the cell other than the routing information between the first and
second devices of the switching device.
61. The method of claim 57, wherein the populating of the local
table includes accessing an entry of the secondary table of the
second memory having an address space non-overlapping with an
address space assigned to the first memory using a direct memory
access operation.
62. A method comprising: storing data from a first connection into
a location within a buffer; snooping address information from the
data detected on the first connection; placing the address
information and a pointer to the location into an entry of a queue;
determining a routing destination based on the address information;
and transmitting the data from the location within the buffer of
the device based on the routing information.
Description
FIELD OF THE INVENTION
[0001] The present invention relates to controlling data flow
within a switch, and more specifically, to information distribution
and buffering mechanisms for controlling data flow between multiple
network interfaces over a common backplane.
BACKGROUND OF THE INVENTION
[0002] Computer networks consist of a plurality of network devices
connected in a way that allows the network devices to communicate
with each other. Special purpose computer components, such as hubs,
routers, bridges and switches, have been developed to facilitate
the process of transporting information between network
devices.
[0003] FIG. 1 is a block diagram of a typical switching device 100.
Switching device 100 contains slots for holding network interface
cards 102, 104 and 106 and a backplane 108 for transporting
information between the various interface cards 102, 104 and 106.
The interface cards 102, 104 and 106 have ports that are connected
to network devices that are located external to switching device
100. For example, port 110 of interface card 104 is connected to a
network device 112. Port 114 of network interface card 106 is
connected to network devices 116, 118 and 120.
[0004] Before switching device 100 can accurately route data
between devices connected to different ports of different cards,
switching device 100 must know which devices are connected to which
ports. Switching device 100 obtains this knowledge by inspecting
the source addresses associated with incoming data.
[0005] Typically, incoming data is encapsulated in a structure
(e.g. a packet or a cell) that contains both a destination address
and a source address. The destination address identifies the device
or group of devices to which the data is to be sent. The source
address identifies the device from which the data is sent. For the
purposes of explanation, the term "cell" shall be used herein to
refer to any data structure that contains data and
source/destination information, including but not limited to
Ethernet packets, ATM cells, and token ring packets.
[0006] When an interface card receives a cell that specifies a
previously unknown source address, the interface card stores a
port-to-address entry in a locally-maintained table. The interface
card transmits a message to a central switch controller 122 to
notify the switch controller 122 that a device with the specified
address is connected to the particular port of the network
interface card. Based on the information contained in this message,
the switch controller 122 adds a card/port-to-address entry in a
centrally-maintained card/port-to-address table. The
card/port-to-address entry indicates that the specified port of the
interface card that received the cell is connected to a device that
has the source address that was specified in the cell. When all
interface cards on the switching device 100 report to the switch
controller 122 the addresses of the devices to which they are
connected in this fashion, the switch controller 122 is able to
create and maintain a relatively complete table that maps addresses
to ports on controller cards.
[0007] As mentioned above, switching device 100 allows network
devices connected to the ports of one interface card to communicate
with network devices connected to the ports of other interface
cards by allowing the network interface cards to transmit
information to each other over backplane 108. For example, network
device 112 can communicate with network device 116 by transmitting
data to interface card 104 in a cell that identifies network device
116 as the desired destination. For the purposes of explanation, it
shall be assumed that the address of network device 116 is
ADDRX.
[0008] Circuitry on the interface card 104 transmits a message over
backplane 108 to the switch controller 122 to indicate that the
interface card 104 has received a cell with the destination address
ADDRX. Switch controller 122 searches for ADDRX in its
card/slot-to-address table to determine where interface card 104
should send the cell. In the present example, the
card/slot-to-address table in switch controller 122 would contain
an entry to indicate that ADDRX is the address of a device
connected to port 114 of interface card 106. The switch controller
122 transmits a message containing this information over backplane
108 to interface card 104.
[0009] Based on the information contained in the message, interface
card 104 routes the cell over the backplane 108 to interface card
106. Interface card 106 receives the cell from backplane 108. Upon
receiving the cell, interface card 106 transmits a message over
backplane 108 to switch controller 122 to inform switch controller
122 that interface card 106 has received a message with the
destination address ADDRX. Switch controller 122 transmits a
message over backplane 108 to interface card 106 to inform
interface card 106 that address ADDRX is associated with a device
connected to port 114 of interface card 106. Based on this
information, interface card 106 transmits the cell through port 114
to device 116.
[0010] One significant disadvantage of routing cells through
switching device 100 in the manner described above is that the
messages that are sent between the switch controller 122 and the
various interface cards to determine the appropriate routing paths
are sent over backplane 108. Consequently, less bandwidth is
available for transmitting the actual cells between the interface
cards.
[0011] A second disadvantage to the switching scheme described
above is the amount of time that elapses between when an interface
card receives a cell and the time at which the interface card knows
where to send the cell. During this time interval, the interface
card may receive a whole series of cells for the same destination.
To prevent data loss, the interface card would have to include a
much larger buffer than would otherwise be required if the
interface card could immediately retransmit the cell.
[0012] In an alternate approach, the interface card does not wait
for the reply from the switch controller 122. Rather, the interface
card simply transmits the cells that it does not know how to route
to the switch controller 122. The switch controller 122 retransmits
the cells to the appropriate destination based on information
contained in its table. The switch controller also transmits the
appropriate routing information to the interface card so that the
interface card will be able to route cells with that destination
address in the future.
[0013] One disadvantage of this approach is that the same cells are
routed over the backplane twice, increasing the amount of traffic
on the backplane. A second disadvantage of this approach is that it
places a heavy burden on the switch controller, because at any
given time every interface card in the switch could be sending
cells for the switch controller to reroute.
[0014] A third disadvantage is that cells may arrive at their
intended destination out of sequence. For example, an interface
card may receive a series of cells 1..N for an unknown destination.
Initially, the interface card sends the cells to the switch
controller. After the interface card has sent M (where 1<M<N)
cells to the switch controller, the interface card receives the
reply message from the switch controller that indicates how to
route the cells. The interface card will then transmit the
remaining cells (M+1)..N directly to the appropriate card/port. In
this scenario, cell M+1 is likely to arrive at the destination
card/port prior to cell M, since cell M is sent to the switch
controller first.
[0015] In another alternative approach, each interface card within
switching device 100 can locally store a complete
card/port-to-address table. According to this approach, each
interface card that detects a new source address will place a
message on backplane 108 that indicates the card and slot on which
the message with the new source address was received. Each
interface card would read the message off the backplane 108 and add
the appropriate card/port-to-address entry in its own table. Thus,
when an interface card receives a cell, the interface card can
quickly determine the appropriate destination for the cell from its
own table without having to send or receive messages over the
backplane 108. By reducing the messaging traffic over backplane
108, the throughput of switching device 100 is increased. By
reducing the delay between receipt of a cell and transmission of
the cell, the buffer size within each interface card can be
reduced.
[0016] An approach in which each interface card maintains a
complete card/port-to-address table, such as the approach described
above, has the disadvantage that the table may become very large.
Each card would have to reserve large amounts of memory to store
the table, and contain the control circuitry for managing a large
table. Further, each interface card will typically only use a
relatively small amount of the data stored in the table. For
example, interface card 104 will never need to know the addresses
of devices connected to interface card 106 that only communicate
with other devices connected to interface card 106.
[0017] Even when a local card/port-to-address table is used by an
interface card to determine how to route a cell, there is some
interval between the time at which the cell is received by the
interface card and the time at which the cell is transmitted by the
interface card. During this interval, the cell must be stored
somewhere within the interface card. FIG. 2 illustrates a typical
buffer mechanism 200 that may be employed by interface card 104 to
store cells during this interval.
[0018] Referring to FIG. 2, it illustrates the buffering circuitry
of interface card 104 of FIG. 1 in greater detail. Interface card
104 includes a first-in-first-out (FIFO) buffer 202, a buffer
control unit 206 and a main buffer 204. FIFO buffer 202 is coupled
between backplane 108 and main buffer 204. Buffer control unit 206
is coupled to and controls both FIFO buffer 202 and main buffer
204. In addition to port 110, interface card 104 includes ports 208
and 210. Ports 110, 208 and 210 are coupled to main buffer 204.
[0019] The main buffer 204 is used to temporarily store both data
cells that are to be transmitted out ports 110, 208 and 210, and
data cells that are to be transmitted to backplane 108. The main
buffer 204 includes a plurality of cell slots, each of which is
capable of storing one cell of data.
[0020] Buffer control unit 206 maintains a plurality of queues,
including one queue (212, 214 and 216) for each of ports 110, 208,
and 210, a queue 220 for backplane 108, and a "free slot" queue
218. Buffer control unit 206 stores pointers to the cell slots of
main buffer 204 in the various queues. Specifically, each cell slot
within main buffer 204 has a unique pointer. The pointer to each
cell slot of main buffer 204 is stored in the queue that
corresponds to the destination of the data that is currently stored
within the cell slot. For example, if a cell slot currently stores
data to be sent on the backplane 108, then the pointer to the cell
slot is stored in the queue 220 that corresponds to the
backplane.
[0021] When interface card 104 receives a cell, the interface card
104 must determine the destination of the cell, as described above.
Once the destination of the cell has been determined, buffer
control unit 206 causes the cell to be stored in main buffer 204
and updates the queues to reflect that the cell is to be sent to
the appropriate destination.
[0022] Specifically, when a cell is to be sent out a port, the
pointer to the cell slot in which the cell resides is placed in the
queue associated with the port. When a cell is to be transmitted
over the backplane 108, the buffer control unit 206 places the
pointer to the cell slot in which the cell resides in the queue 220
associated with the backplane 108.
[0023] When a cell of data is transmitted by interface card 104,
the cell slot that contained the cell no longer contains data to be
transmitted. To indicate that the cell slot is now available to
store incoraing data, the buffer control unit 206 places the
pointer to the cell slot in the free slot queue 218. When storing a
cell of data in main buffer 204, the buffer control unit 206 pulls
a pointer from the free slot queue 218, and uses the cell slot
indicated by the pointer to store the cell of data.
[0024] At any given moment all of the other interface cards on
backplane 108 may be placing cells on backplane 108 that are
destined for devices connected to interface card 104. To process
the cells without losing data, interface card 104 must be able to
sustain, for at least brief periods of time, a backplane-to-card
transfer rate equal to the maximum transfer rate supported by
backplane 108 (e.g. 3.2 gigabytes per second).
[0025] Typically, buffer control unit 206 determines the
appropriate destination for cells at a rate (e.g. 1.2 gigabytes per
second) that is slower than the maximum transfer rate of the
backplane 108. Therefore, to sustain brief periods in which cells
arrive at the high transfer rate of the backplane 108, the
information contained on backplane 108 is transferred from the
backplane 108 into the high speed FIFO buffer 202 of interface card
104. During the interval in which a cell is stored in FIFO buffer
202, interface card 104 determines the destination of the cell.
Once the destination of the cell has been determined, buffer
control unit 206 removes the cell from FIFO buffer 202 and stores
the cell in the main buffer 204.
[0026] The buffering system illustrated in FIG. 2 has the
significant disadvantage that high speed FIFO buffers are
expensive. Further, every interface card in the switch must have
its own of FIFO buffer to support the maximum transfer rate of the
backplane. Consequently, the increased cost of high speed FIFO
buffers is incurred for every interface card in the switch.
[0027] Based on the foregoing, it is clearly desirable to provide a
switching device in which the delay between when an interface card
receives a cell and the time at which the interface card transmits
the cell is reduced. It is further desirable to provide a switching
device in which the amount of traffic on the backplane that is used
to determine how to route cells is reduced. Further, it is
desirable to reduce the size and complexity of tables that are
maintained locally in interface cards. In addition, it is desirable
to provide a buffering system that allows interface cards to
receive data off the backplane at the backplane's maximum transfer
rate without requiring the use of high speed FIFO buffers.
SUMMARY OF THE INVENTION
[0028] A method and apparatus for managing the flow of data within
a switching device is provided. The switching device includes
network interface cards connected to a common backplane. Each
interface card is configured to support the maximum transfer rate
of the backplane by maintaining a "pending" queue to track data
that has been received but for which the appropriate destination
has not yet been determined. The switching device includes a switch
controller that maintains a central card/port-to-address table.
When an interface card receives data with a destination address
that is not known to the interface card, the interface card
performs a direct memory access over a bus that is separate from
the backplane to read routing data directly from the central table
in the switch controller. Each interface card builds and maintains
a routing information table in its own local memory that only
includes routing information for the destination addresses that the
interface card is most likely to receive.
BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The present invention is illustrated by way of example, and
not by way of limitation, in the figures of the accompanying
drawings and in which like reference numerals refer to similar
elements and in which:
[0030] FIG. 1 is a block diagram of a generic switching device;
[0031] FIG. 2 is a block diagram that illustrates the buffering
system used in a prior art network interface card;
[0032] FIG. 3 is a block diagram that illustrates a buffering
system according to an embodiment of the present invention; and
[0033] FIG. 4 is a block diagram that illustrates a switching
device in which destination address is distributed to local tables
using direct memory access techniques according to an embodiment of
the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
Improved Buffer Mechanism
[0034] Referring to FIG. 3, it illustrates the interface card 104
of FIG. 2 with a buffer control unit 306 according to an embodiment
of the present invention. Similar to the buffering mechanism
illustrated in FIG. 2, buffer control unit 306 includes a plurality
of queues, including queues 212, 214, and 216 that correspond to
ports 110, 208 and 210, a queue 220 that corresponds to the
backplane 108, and a free slot queue 218. In addition, buffer
control unit 306 includes a pending queue 308. As shall be
described hereafter, buffer control unit 306 uses pending queue 308
in a way that allows buffer control unit 306. to transfer cells off
the backplane 108 directly into main buffer 204 at the maximum
transfer rate of the backplane 108. Consequently, the need for a
high speed FIFO buffer is eliminated.
[0035] When interface card 104 detects a cell on backplane 108 that
is destined for interface card 104, an arrival handling unit (AHU)
312 within buffer control unit 306 pulls the pointer from the head
of free slot queue 218 and places the cell directly in the cell
slot that corresponds to the pointer. Ideally, the arrival handling
unit 312 would then attach the pointer to the tail of the queue
that corresponds to the destination of the cell. However, as
mentioned above, there is not enough time to determine the
destination of a cell before arrival handling unit 312 must be
available to handle the arrival of a subsequent cell.
[0036] Rather than immediately determine the destination of the
cell as it is stored in main buffer 204, arrival handling unit 312
simply snoops the destination address from the cell and places into
the pending queue 308 both the destination address and the pointer
to the cell slot into which the cell was placed. Because no
destination determination is performed during this process, arrival
handling unit 312 is able to place cells into main buffer 204 at
the maximum transfer rate of backplane 108.
[0037] Buffer control unit 306 includes a route determination unit
(RDU) 310 that sequentially removes entries from the head of the
pending queue 308 and determines where cells that have the
destination address contained in the entries should be routed. When
route determination unit 310 determines the proper route for the
destination address contained in an entry, the pointer in the entry
is placed in the queue that corresponds to the determined
route.
[0038] Typically, the route determination unit 310 will not be able
to determine the route that corresponds to a destination address in
the amount of time that it takes for a subsequent cell to arrive on
backplane 108. Consequently, when cells for interface card 104 are
arriving on backplane 108 at the maximum transfer rate of backplane
108, the size of the pending queue will increase. However, in
typical applications, there will also be periods of relatively low
cell arrival rates. During these periods, route determination unit
310 will be removing entries from pending queue 308 faster than the
arrival handling unit 312 is adding entries to the pending queue
308.
[0039] According to one embodiment, buffer control unit 306 is
implemented in an application specific integrated circuit (ASIC).
However, many alternative implementations are possible. For
example, the destination determination unit 310 and the arrival
handling unit 312 may be implemented by separate hardwired
circuitry, or by programming one or more processors to perform the
described functions. The present invention is not limited to any
particular implementation of buffer control unit 306.
Route Determination Mechanism
[0040] The mechanism used by route determination unit 310 to
determine the proper route for cells based on the destination
address contained within the cells significantly affects the cost
and efficiency of the switching device in which buffer control unit
306 is used. As mentioned above, interface cards can determine this
information by communicating over the backplane 108 with a switch
controller 122 that contains a central card/port-to-address table,
or by each maintaining their own card/port-to-address table.
However, each of these options has significant disadvantages. The
approach in which cards send messages to request the required
information from the switch controller 122 is relatively slow and
uses valuable bandwidth on the backplane. The approach in which
each interface card maintains its own complete table requires each
card to dedicate a relatively large amount of resources for
creating and maintaining the table.
[0041] Referring to FIG. 4, it illustrates a switching device 400
configured according to one embodiment of the invention. Switching
device 400 includes a backplane 428, two interface cards 448 and
402, and a master control process (MCP) 432. Master control process
432 includes memory 450, a memory controller 454, and a processor
452. For the purposes of explanation, the master control process
432 is illustrated as an independent component of switching device
400. However, the master control process 432 may actually reside on
one of the network interface cards in switching device 400.
[0042] Each of the interface cards 402 and 448 includes a plurality
of ports, a main buffer, a control unit, a memory controller and a
local memory. Specifically, interface card 448 includes three ports
442, 444 and 446, main buffer 418, control unit 420, memory
controller 456 and memory 426. Interface card 402 includes three
ports 436, 438 and 440, a main buffer 404, control unit 406, memory
controller 458 and memory 414. The control units 420 and 406
generally represent control circuitry which performs various
control functions, including the functions described above with
respect to buffer control units. Control units 420 and 406 may be
implemented using hard-wired circuitry, a programmed processor, or
a combination of hard-wired and programmed circuitry.
[0043] In addition to backplane 428, switching device 400 includes
a bus 430. Bus 430 is a multiple-line bus (e.g. 16 lines) that
interconnects the memory controllers on all of the interface cards
and on the memory controller in the master control process 432. Bus
430 is separate from the path of the normal data flow within
switching device 400, which occurs over backplane 428.
Consequently, the presence of signals on bus 430 has no effect on
the overall data flow over backplane 428.
Combined Memory Space
[0044] Each of memories 426, 450 and 414 is assigned an address
space that does not overlap with the address space assigned to any
other of the memory devices within switching device 400. For
example, memories 426, 450 and 414 may be respectively assigned the
address spaces 0 to N, (N+1)+M, and (M+1) to L. The address space
covered by all of the memories 426, 450 and 414 (e.g. 0 to L) is
referred to herein as the combined address space.
[0045] Each of memory controllers 454, 456, and 458 is able to
directly perform memory transactions on data stored anywhere within
the combined address space. An operation that is to be performed on
data that is stored in memory that is not local (with respect to
the memory controller that will perform the operation) is performed
over bus 430. Bus 430 may be, for example, a multiplexed channel in
which a memory controller places an address on the bus 430 during
the first half of a clock cycle, and the memory that includes the
memory location specified in the address places the data that
resides in the memory location on the bus 430 during the second
half of a clock cycle.
[0046] Because each memory controller is able to perform transfer
operations on any data stored within the combined address space,
the control units 406 and 420 and processor 452 are able to request
memory operations to be performed on any data in the combined
address space, regardless of the location of the memory that
contains the data involved in the operation. Further, according to
one embodiment of the invention, each control unit is completely
unaware of the address boundaries between the various memory
devices. The fact that some of the transfer operations are taking
place over bus 430 is completely transparent to the control units
420, 406 and processor 452.
[0047] For example, control unit 406 may request data to be copied
from a first memory location to a second memory location. The
control unit 406 is completely unaware of where the memory
device(s) that correspond to the first and second memory locations
actually reside. For the purposes of explanation, it shall be
assumed that the first memory location resides in the address space
of memory 426, while the second memory location resides in the
address space of memory 414.
[0048] To perform the requested operation, control unit 406 simply
requests memory controller 458 to perform the memory transfer.
Memory controller 458 retrieves data from the first location by
reading the data from the appropriate memory location in memory 426
over bus 430. Memory controller 458 then stores the data at the
appropriate location in memory 414.
Direct Access of the Central Table
[0049] According to one embodiment of the invention, the central
card/port-to-address table 434 is stored in the memory 450 of the
master control process 432. The location at which each entry is
stored in the table 434 is determined by performing a hash function
on the address associated with the entry. The hash function may
consist of extracting the lower N bits from the address, for
example. Collisions may be resolved by any one of numerous
collision resolution techniques. The present invention is not
limited to any particular hash function or collision resolution
technique.
[0050] When an interface card receives a cell, the control unit on
the interface card reads the destination address from the cell and
performs the hash function on the destination address. The result
of the hash function is used as an index into central table 434.
The control unit transmits control signals to the memory control
unit associated with the control unit, causing the memory control
unit to retrieve data from the memory location that corresponds to
the index. The memory control unit performs the transfer operation
over bus 430. The control unit uses the retrieved information to
determine where to transmit the received cell.
Local Tables
[0051] The embodiment described above overcomes many of the
disadvantages of the prior art. However, the numerous memory
accesses that would be performed over bus 430 in a switching device
that has a large number of interface cards may exceed the
throughput of bus 430. Further, if the central table 434 became
corrupted or MCP 432 ceased to function properly, the entire
switching device 400 would cease to function.
[0052] To resolve these problems, the interface cards are
configured to store in their local memories the portions of the
central table 434 in which they are most interested. According to
one embodiment, each interface card maintains its own local table.
In the illustrated embodiment, interface card 402 includes a local
table 410 stored in memory 414. Interface card 448 includes a local
table 422 stored in memory 426. Initially, these tables are
empty.
[0053] Interface cards add entries to their local tables only as
needed. For example, assume that interface card 402 receives a
cell. The cell is stored in a cell slot of main buffer 404. An
entry containing (1) the pointer to the cell slot and (2) the
destination address contained in the cell is placed on the pending
queue. When the entry is at the head of the pending queue, the
control unit 406 does not immediately invoke a memory operation to
retrieve information from the central table 434 in MCP 432. Rather,
control unit 406 inspects the local table 410 to determine whether
an entry associated with the destination address resides in the
table 410. If the local table contains an entry associated with the
destination address, then the appropriate routing for the packet is
determined based on the information in the entry ,and the pointer
to the cell slot is placed in the appropriate queue(s).
[0054] If the local table 410 does not contain an entry associated
with the destination address contained within the cell, then the
control unit 406 causes information from the appropriate entry of
the central table 434 to be retrieved over bus 430 through a direct
memory access operation, as described above. The entry from the
central table 434 identifies the card and/or port to which the
received cell is to be transmitted. Based on this information, the
control unit 406 places the pointer to the cell slot in which the
cell resides in the appropriate queue.
[0055] In addition to placing the pointer in the appropriate queue,
control unit 406 generates an entry for table 410 based on the
information retrieved from central table 434. The information
stored in the local table 410 includes information that indicates
to where cells that contain the destination address in question
should be routed. Therefore, when interface card 402 receives cells
with the same destination address in the future, control unit 406
will be able to determine the appropriate routing of the cells
without having to access the central table 434 stored in the MCP
432.
[0056] According to one embodiment of the invention, the entry that
corresponds to a given address is stored in the local table at a
location based on the index created by performing a hash function
on the address. The entry for a particular address may contain, for
example, the following information: a routing tag, address
information, a collision pointer, an "identical address" flag, and
aging information.
[0057] The routing tag is a tag that indicates the card(s) and
port(s) to which cells with the particular address are to be
routed. According to one embodiment of the invention, the routing
tag is a "destination tag", and each interface card is configured
with a mechanism to route cells to the appropriate cards and ports
based on the destination tag. A system that employs destination
tags to route packets between interface cards connected to a common
backplane is described in U.S. patent application Ser. No.
08/501,483 entitled METHOD AND APPARATUS FOR CONTROLLING DATA FLOW
WITHIN A SWITCHING DEVICE that was filed on Jul. 12, 1995 by
Ferguson, the contents of which are incorporated herein by
reference.
[0058] The address information contained in an entry is information
that allows an interface card to determine the full address that
corresponds to the entry. For example, in a system where a portion
of the final address is used as an index to the entry, the entry
may contain the portion of the address that was not used as the
index. The full address associated with an entry may therefore be
determined based on the location of the entry within the local
table and the portion of the address that is contained within the
entry. It may be important to be able to determine the full address
associated with an entry in order to resolve collisions when two
addresses hash to the same index.
[0059] The collision pointer is index that points to the memory
location of an entry that collided with the current entry. If no
collision has occurred, then the collision pointer is NULL. The
"identical address" flag indicates whether the destination address
that corresponds to this entry is identical to the destination
address that corresponds to another entry.
[0060] The aging information indicates how much time has elapsed
from the transmission of the last packet destined to the address.
According to one embodiment, the control unit on a card uses the
aging information to delete entries that correspond to addresses
that have not been used for longer than some specified amount of
time. By deleting entries that correspond to addresses that have
not been used for some period of time, the size of the local table
is kept to a minimum. If a packet with an address that corresponds
to an entry arrives after the entry has been deleted, the interface
card must again read information from the central table 434 to
recreate the entry in the local table.
[0061] In the worst case scenario, the operations involved in
determining the destination for a cell include accessing local
tables, calculating a memory location, performing a direct memory
access to retrieve data from the memory location, and updating
local tables. None of these operations require the involvement of
off-card processors or control units, nor do they generate any
traffic over the backplane. Consequently, even in the worst case,
an interface card is able to determine the appropriate routing of a
cell prior to the time at which the interface card must begin to
process a subsequent cell.
[0062] The local table within any given interface card will
typically only contain a relatively small portion of the
information stored in the central table 434. Consequently, the
amount of resources required in each interface card to create and
maintain the tables is relatively small. Further, the information
contained in the local memory of an interface card will be the most
relevant information for that particular interface card.
[0063] Using the techniques described above, each interface card
gradually builds its own local table. If an interface card has
already received a cell for a particular destination, then the
interface card will not have to perform memory accesses over bus
430 to process subsequent cells for the same destination. Rather,
the interface card will already have stored enough information
locally to determine where to send the cell without need for
further interaction with the master control process 432. As a
result, the amount of traffic over bus 430 is significantly
reduced.
[0064] A second advantage of storing local tables within each
interface card is that the failure of any one component in
switching device 400 will not cause the entire switching device 400
to fail. For example, if MCP 432 ceases to function properly, then
the interface cards may not be able to retrieve destination address
from the central table 434. However, the interface cards will be
able to continue to process cells for destinations for which the
interface cards have previously received cells.
[0065] According to one embodiment, an interface card may access
the local tables in other interface cards when the MCP 432 ceases
to function. These accesses are performed through the same direct
memory access process that interface cards use to read the entries
from the central table 434. When an interface card finds an entry
that corresponds to a particular address in the local table of
another card, the interface card copies information from the entry
into its own table, just as it does when retrieving information
from the central table 434.
[0066] In the foregoing specification, the invention has been
described with reference to specific embodiments thereof. It will,
however, be evident that various modifications and changes may be
made thereto without departing from the broader spirit and scope of
the invention. The specification and drawings are, accordingly, to
be regarded in an illustrative rather than a restrictive sense.
* * * * *