U.S. patent application number 09/849144 was filed with the patent office on 2001-11-22 for fault management in a multichannel transmission system.
Invention is credited to Gerstel, Ornan Alexander, Hall, William Eric, Ramaswami, Rajiv, Sasaki, Galen Hajime.
Application Number | 20010043563 09/849144 |
Document ID | / |
Family ID | 26679305 |
Filed Date | 2001-11-22 |
United States Patent
Application |
20010043563 |
Kind Code |
A1 |
Gerstel, Ornan Alexander ;
et al. |
November 22, 2001 |
Fault management in a multichannel transmission system
Abstract
Efficient methods for rerouting failed channels on to spare
channels in a multichannel transmission system are described. One
method uses a minimal amount of rerouting to reroute all the
channels if one or two line cards fail. Another set of methods
reroute up to two failed channels on to spare channels in the event
of channel failures. With this method only the failed channels are
rerouted. However the failed channels may have to be routed through
a slightly larger, but minimal, number of additional hops than with
the first method.
Inventors: |
Gerstel, Ornan Alexander;
(Riverdale, NY) ; Hall, William Eric; (Clinton,
CT) ; Ramaswami, Rajiv; (Ossining, NY) ;
Sasaki, Galen Hajime; (Honolulu, HI) |
Correspondence
Address: |
MCANDREWS HELD & MALLOY, LTD
500 WEST MADISON STREET
SUITE 3400
CHICAGO
IL
60661
|
Family ID: |
26679305 |
Appl. No.: |
09/849144 |
Filed: |
May 4, 2001 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09849144 |
May 4, 2001 |
|
|
|
09039665 |
Mar 16, 1998 |
|
|
|
6256293 |
|
|
|
|
09039665 |
Mar 16, 1998 |
|
|
|
08618652 |
Mar 19, 1996 |
|
|
|
5729527 |
|
|
|
|
60009300 |
Dec 29, 1995 |
|
|
|
Current U.S.
Class: |
370/228 ;
370/216; 714/1 |
Current CPC
Class: |
H04J 14/0278 20130101;
H04J 14/0297 20130101; H04J 14/0282 20130101; H04Q 11/0001
20130101 |
Class at
Publication: |
370/228 ;
370/216; 714/1 |
International
Class: |
G01R 031/08 |
Claims
Having thus described our invention, what we claim as new and
desire to secure by letters patents is:
1. A fault tolerant multichannel communications system having a
plurality of communication channels transmitted over a trasmission
medium, said communications system comprising: (a) a plurality of
line cards 1, 2, . . . , N at each end of a transmission medium,
with one of said line cards at each end of said transmission medium
for each of said channels and with one of said line cards s at each
end of said medium being designated as a spare line card, each said
line card having a plurality of local ports, a link port, and a
input/output port; (b) a plurality of switches with one switch at
each end of said medium for each of said line cards, each switch
capable for interconnecting a link port of a corresponding one of
said line cards to an I/O port or a local port of latter said
corresponding one line card, and each switch also capable of
interconnecting said I/O port to either said link port or a local
port of said corresponding one line card, with a local port of each
line card i being hard wired to a local port of line card i +1 mod
N and a local port of line card i-1 mod N; and (c) means for
rerouting the I/O of a failed line card f as follows: for each line
card i belonging to a set [s, f) of line cards, switching the I/O
port of line card (i+1) mod N to a link port of line card i, where
s f, wherein said failed channel is restored, where said set Is,f)
is the set of all line cards such that 3 [ s , f ) = { { s , s + 1
, , f - 1 } , if s < f ; { s , s + 1 , , N - 1 , 0 , 1 , , f - 1
} , if s > f ; 0 , if s = f .
2. A fault tolerant multichannel communications system having a
plurality of communication channels, said communications system
comprising: (a) a plurality of line cards 1, 2, . . . , N at each
end of said communications medium, with one of said line cards at
each end of said medium for each of said channels and with one of
said line cards s at each end of said medium being designated as a
spare line card, each said line card having a plurality of local
ports for a link port, and a input/output port for; (b) a plurality
of switches with one switch at each end of said medium for each of
said line cards, each switch capable for interconnecting a link
port of a corresponding one of said line cards to an I/O port or a
local port of latter said corresponding one line card, and each
switch also capable of interconnecting said I/O port to either said
link port or a local port of said corresponding one line card, with
a local port of each line card i being hard wired to a local port
of line card i+1 mod N and a local port of line card i-1 mod N; and
(c) means for rerouting a failed line card f as follows: for each
line card belonging to a set of line cards (f, s), switching the
I/O port of line card i to a link port of line card (i+1) mod N,
where s f wherein said failed channel is restored, where (f, s) is
the set of all line cards such that 4 [ f , s ) = { { f , f + 1 , ,
f - 1 } , if f < s ; { f , f + 1 , , N - 1 , 0 , 1 , , s - 1 } ,
if f > s ; 0 , if f = s .
3. In a multichannel communications system having N channels with
line cards 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
link port and a plurality of local ports, with one line card
connected to each end of said transmission medium for each of said
channels, a method of rerouting channels upon detection of a
failure of two of said line cards f.sub.0 and f.sub.1, said method
comprising: (a) designating two of said line cards as spare line
cards s.sub.0 and s.sub.1; (b) hardwiring one of said local ports
of each line card i mod N to one of said local ports of each line
card (i+1) mod N and (i-1) mod N; (c) rerouting said failed line
cards f.sub.0 and f.sub.1 as follows: i. if
s.sub.0.ltoreq.f.sub.0.ltoreq- .f.sub.1.ltoreq.s.sub.1 then doing
spare-to-failed rerouting from s.sub.0 to f.sub.0 and
failed-to-spared rerouting from s.sub.1 to f.sub.1, and ii. if
s.sub.0.ltoreq.f.sub.0.ltoreq.s.sub.1.ltoreq.f.sub.1 then doing
spare-to-failed rerouting from s.sub.0 to f.sub.0 and
spare-to-failed rerouting from s.sub.1 to f.sub.1, said
spare-to-failed rerouting comprising switching each said I/0 port
of line card (i+1) mod N to said link port of line card i mod N for
all i .di-elect cons.( s, f), where: 5 [ s , f ) = { { s , s + 1 ,
, f - 1 } , if s < f ; { s , s + 1 , , N - 1 , 0 , 1 , , f - 1 }
, if s > f ; 0 , if s = f , and said failed-to-spare rerouting
from f to s comprising switching said I/O port of line card i mod N
to said link port of line card (i+1) mod N for all i.di-elect
cons.(f, s), where: 6 [ f , s ) = { { f , f + 1 , , f - 1 } , if f
< s ; { f , f + 1 , , N - 1 , 0 , 1 , , s - 1 } , if f > s ;
0 , if f = s .
4. A fault tolerant multichannel communication system, comprising:
(a) a transmission medium; (b) N line cards 0 through N-1, each
line card connecting a separate channel to an end of said
transmission medium, each line card having an I/O port, a link
port, and a plurality of local ports, with two of said line cards
s.sub.0 and s.sub.1 being designated as spare line cards, with one
of said local ports of each line card i mod N being connected to
one of said local ports of line card (i+1) mod N and line card
(i-1) mod N; (c) A plurality of switches, each of said switches for
interconnecting said I/O port, said link port, and said local ports
of a corresponding one of said line cards; and (d) a means for
rerouting I/Os from failed line cards f.sub.0 and f.sub.1 using
said switches as follows: i. if
s.sub.0.ltoreq.f.sub.0.ltoreq.f.sub.1.ltoreq.s.sub.1 then doing
spare-to-failed rerouting from s.sub.0 to f.sub.0 and
failed-to-spared rerouting from s.sub.1 to f.sub.1, and ii. if
s.sub.0.ltoreq.f.sub.0.ltoreq.s.sub.1.ltoreq.f.sub.1 then doing
spare-to-failed rerouting from s.sub.0 to f.sub.0 and
spare-to-failed rerouting from s.sub.1 to f.sub.1, said
spare-to-failed rerouting comprising switching each said I/O port
of line card (i+1) mod N to said link port of line card i mod N for
all i.di-elect cons.(s, f), where: 7 [ s , f ) = { { s , s + 1 , ,
f - 1 } , if s < f ; { s , s + 1 , , N - 1 , 0 , 1 , , f - 1 } ,
if s > f ; 0 , if s = f , and said failed-to-spare rerouting
from f to s comprising switching said I/O port of line card i mod N
to said link port of line card (i+1) mod N for all i.di-elect
cons.(f, s), where: 8 [ f , s ) = { { f , f + 1 , , f - 1 } , if f
< s ; { f , f + 1 , , N - 1 , 0 , 1 , , s - 1 } , if f > s ;
0 , if f = s .
5. In a multichannel communications system having N channels with
line cards 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
link port and a plurality of local ports, with one line card
connected to each end of said transmission medium for each of said
channels, a method of rerouting channels upon detection of a
failure of one of said line cards, said method comprising: (a)
partitioning said line cards into groups of size at most k+1, where
each group has at least one spare line card, and where k equals the
number of local ports on each line card; (b) for each line card in
each of said groups that is not a spare line card, hardwiring one
of latter said each line card's local ports to a local port of said
one spare line card in said each group; and (c) upon detection of a
failure of anv of said line cards, rerouting said I/O port of
latter said failed line card to the link port of said designated
spare line card for the group of line cards containing said failed
line card.
6. In a multichannel communications system having N channels with
line cards 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
bypass port, a link port and a plurality of k local ports where
k<N, with one line card connected to each end of said
transmission medium for each of said channels, a method of
rerouting channels upon detection of a failure of one of said line
cards, said method comprising: (a) designating one of said line
cards as a spare line card s; (b) hard wiring the local ports of
each of said line cards such that there is a path from each of said
line cards to said spare line card; and (c) rerouting the I/O port
of a failed line card by switching said I/O port of said failed
line card to its said bypass port which is hard wired to said local
port of another one of said line cards and switching said latter
local port to said link port of spare line card s.
7. A fault tolerant multichannel communication system, comprising:
(a) a transmission medium; (b) N line cards 0 through N-1, each
line card connecting a separate channel to an end of said
transmission medium, each line card having an I/O port, a bypass
port, a link port, and a plurality of k local ports where k<N,
with one of said line cards being designated as spare line card s,
with said local ports of said line cards being hardwired so that
there is a path from each of said line cards to line card s; (c) A
plurality of switches, each of said switches for interconnecting
said I/O port, said bypass port, said link port, and said local
ports of a corresponding one of said line cards; and (d) a means
for rerouting I/Os from a failed one of said line cards by using
said one of said switches corresponding to said line card to switch
said I/O port of said failed line card to its said bypass port
whichis hardwired to said local port of another one of said line
cards and switching said latter local port to said link port of
spare line card s.
8. In a multichannel communications system having N channels with
line cards 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
bypass port, a link port and a plurality of local ports, with one
line card connected to each end of said transmission medium for
each of said channels, a method of rerouting channels upon
detection of a failure of one of said line cards, said method
comprising: (a) designating one of said line cards as a spare line
card; (b) hard wiring one of said local ports of each of up to k of
said line cards to each of said of said spare line cards, where k
is the number of local ports each of said line cards; (c) hard
wiring said local ports of each of said line cards hard wired to
said local ports of said spare line card to an additional k-1 of
said line cards; (d) repeating step 8c for each line card of said
k-1 line cards by hard wiring each of latter said line cards in
turn to an additional k-1 of said line cards and so on until all of
said line cards form a tree configuration with a plurality of
subtrees; (e) hard wiring the bypass port of each line card that is
a leaf of said tree configuration to the local port of another line
card that is a leaf of said tree configuration; (f) hard wiring the
bypass port of each of said line cards that is not a leaf to the
local port of a line card that is a leaf; and (g) rerouting the I/O
port of a failed one of said line cards by switching its I/O port
to its bypass port and switching appropriate local ports of said
line cards so that said I/O port of said failed line card is
connected to said link port of said spare line card.
9. In a multichannel communications system having N channels with
line cards 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
bypass port, a link port and a plurality of k local ports where
k<N, with one line card connected to each end of said
transmission medium for each of said channels, a method of
rerouting channels upon detection of a failure of two of said line
cards, said method comprising: (a) designating two of said line
cards as spare line cards s.sub.1 and s.sub.2; (b) hardwiring said
local ports of each of said line cards such that for any pair of
line cards f.sub.1 and f.sub.2 there are either two paths P.sub.1
from f.sub.1 to s.sub.1 and P.sub.2 from f.sub.2 to s.sub.2 where
P.sub.1 and P.sub.2 have no common wires, or two paths P.sub.1 from
f.sub.1 to s.sub.2 and P.sub.2 from f.sub.2 to s.sub.1 where
P.sub.1 and P.sub.2 have no common wires; (c) rerouting the I/O of
up to two failed line cards by switching the I/O of said failed
line cards to said spare cards by setting switches along paths from
said failed line cards to said spares, where said paths have no
common wires.
10. A fault tolerant multichannel communication system, comprising:
(a) a transmission medium; (b) N line cards 0 through N-1, each
line card connecting a separate channel to an end of said
transmission medium, each line card having an I/O port, a bypass
port, a link port, and a plurality of k local ports k<N, with
two of said line cards being designated as spare line cards s.sub.1
and s.sub.2, with said local ports of said line cards being
hardwired such that for any pair of line cards f.sub.1 and f.sub.2
there are either two paths P.sub.1 from f.sub.1 to s.sub.1 and
P.sub.2 from f.sub.2 to S.sub.2, where P.sub.1 and P.sub.2 have no
common wires, or two paths P.sub.1 from f.sub.1 to S.sub.2 and
P.sub.2 from f.sub.2, to s.sub.1 where P.sub.1 and P.sub.2 have no
common wires; (c) A plurality of switches, each of said switches
for interconnecting said I/O port, said bypass port, said link
port, and said local ports of a corresponding one of said line
cards; and (d) a means for rerouting I/Os from up to two failed
line cards by setting said switches along of those of said paths
from said failed line cards to said spares. where said paths have
no common wires.
11. In a multichannel communications system having N channels with
line crds 0 through N-1 connected to each end of a transmission
medium of said system with each line card having an I/O port, a
bypass port, a link port and a plurality of local ports, with one
line card connected to each end of said transmission medium for
each of said channels, a method of rerouting channels upon
detection of failures of up to two of said line cards, said method
comprising: (a) designating two of said line cards s.sub.1 and
s.sub.2 as spare cards; (b) hardwiring another one of said line
cards i to said local ports of s.sub.1 and s.sub.2 to form a dual
tree, where i is a leaf card of said dual tree and s.sub.1 and
s.sub.2 are internal cards of said dual tree, and said dual tree
comprises two trees T.sub.1 and T.sub.2, where s.sub.1 and line
card i belong to T.sub.1, and s.sub.2 and line card i belong to
T.sub.2; (c) hardwiring another line card x of said line cards to
said dual tree according to the following rules: i. if there are
two internal cards j and k belonging to T.sub.1 and T.sub.2
respectively, with each of said cards j and k hardwired to each
other by less than k-1 leaf cards then hardwire local ports of line
card .dagger-dbl. to said local ports of j and k, where
.dagger-dbl. is a leaf card of said trees T.sub.1 and T.sub.2, ii.
if rule 11(c)i cannot be applied, and if a single leaf card l is
hardwired to two internal cards i.sub.1 and i.sub.2 belonging to
T.sub.1 and T.sub.2 respectively, and any other leaf card m
hardwired to two internal cards is at least as far as I is from the
furthest spare card, then disconnect l from i.sub.1, hardwire a
local port of .dagger-dbl. to a local port of i.sub.1, hardwire a
local port of .dagger-dbl. to a local port of 1, where .dagger-dbl.
is now a leaf card belonging to T.sub.1 and l now belongs
exclusively to T.sub.2, and iii. if rules 11(c)i and 11(c)ii cannot
be applied, and two leaf cards p and q are hardwired to each other,
then disconnect p from q, hardwire local ports of .dagger-dbl. to
local ports of p and q, where .dagger-dbl. is a leaf card belonging
to both T.sub.1 and T.sub.2 and p and q become internal cards; and
(d) rerouting the I/Os of up to two failed line cards f.sub.1 and
f.sub.2 by finding the shortest paths P.sub.1,1 from f.sub.1 to
s.sub.1, P.sub.1,2 from f.sub.1 to s.sub.2, p.sub.2,1 from f.sub.2
to s.sub.1, and p.sub.2,2 from f.sub.2 to s.sub.2, choosing q.sub.1
as p.sub.1,2 and q.sub.2 as p.sub.2,2 if p.sub.1,1 and p.sub.2,2 do
not have a common wire, and choosing q.sub.1 as p.sub.1,2 and
q.sub.2 as p.sub.2,1 otherwise, setting switches along said paths
q.sub.1 and q.sub.2 such that said I/Os of said failed line cards
are routed to the link ports of said spare line cards.
Description
TECHNICAL FIELD
[0001] The present invention relates generally to point-to-point
transmnission links employing multiple channels. More specifically,
it relates to providing fault tolerance on optical
wavelength-division-multi- plexed multichannel links.
DESCRIPTION OF THE PRIOR ART
[0002] A multichannel link comprises a number of channels, say N,
between two sites. These channels may be transmitted separately
(for example over parallel wires or fiber cables) or multiplexed on
to one or a small niumber of wires or fibers using time or
wavelength division multiplexing. Usually these links are realized
in the form of line cards, one for each channel. A line card can be
any device that provides an interface between the I/O for the
channel and the transmission medium. Line cards may fail. In many
applications it may be desirable to reroute inputs/outputs (I/Os)
of failed line cards on to other line cards (or channels).
[0003] One example is the IBM MuxMaster system [1]. This system
multiplexes up to ten full-duplex channels on to a single
transmission link. Typically, fault management is done by having a
fully redundant configuration [2]. For each operating link with 10
channels, an additional link with 10 channels is used as a backup.
Switching is provided outside the system so that if a card
(channel) on the operating link fails, the I/O for that card is
switched to the card on the backup link. However this is an
expensive solution since it doubles the hardware needed. In cases
where the failure of cards is rare and it is unlikely that more
than a few cards fail at any given time, it is desirable to
minimize the number of additional spare cards. U.S. Pat. No.
5,313,456 "Data link protecting system" This patent considers a
transmission system comprising of two ends A and B with N
transmission lines between them. Each transmission line can carry
data signals and an additional low-speed control signal. At any
given time there is one control signal that is carried on one of
the N transmission lines. At each end A and B this control signal
is connected via a 1.times.N switch to any of the N transmission
lines. If the current transmission line fails, the patent teaches
how to then switch the control signal onto another unused
transmission line using the 1.times.N switches. Thus this patent
provides for rerouting 1 input signal to a spare transmission line
in the event of a failure using two 1.times.N switches. This patent
requires complex and expensive 1.times.N switching components when
N is large (e.g., N=100). 5,434,691 "Communications system having
optical transmission line . . . . . ." This patent provides for
rerouting inputs on a failed transmission channel to a spare
channel, like the present invention but uses more inefficient
methods. The first method taught assumes a 1.times.N switch on the
spare line card which is connected to each of N other line cards.
If one of the latter line cards fails the input of that card is
then routed on to the spare transmission channel. This patent
requires complex and expensive 1 .times.N switching components when
N is large (e.g., N=100). The second and third methods taught in
this patent connect the link cards in a line or ring network,
wherein card i is connected to i-1 and i +1, as in our invention.
If a card fails, its input is rerouted through the network to the
spare card without affecting the other channels. In the worst case
this can take up to N additional "hops", since the input may have
to be routed through N-1 other cards before it reaches the spare
card. Such a large number of hops can adversely impact signal
quality. U.S. Pat. No. 4,365,247 "End station digital transmission
link relieving system" This patent essentially teaches a specific
implementation of the first method of U.S. Pat. No. 5,434,691
wherein a 1.times.N switch is used on the spare line card to
connect it directly to the N other link cards. Again, this patent
requires complex and expensive 1.times.N switching components when
N is large (e.g., N=100). U.S. Pat. No. 5,113,391 "Intelligent
Channel Unit" This patent teaches cascading "intelligent channel
units" (ICUs) so that ICU i is connected to ICU i-1 and ICU i+1. If
ICU i fails, the patent teaches a method to bypass ICU i so that
ICU i-1 can be directly connected to ICU i+1, while ICU i is
repaired. Here, the I/O of ICU (line card) i is lost. This patent
does not teach how to connect link cards so that if two cards i and
j fail simultaneously, they can both be routed to different
designated spare cards. U.S. Pat. No. 5,214,692 "Bypass for
telephone switching system" This patent considers a system wherein
N digital lines arrive and pass through a "main distribution frame"
(MDF) to a "automatic call distributor" (ACD) which contains a
N.times.N switch that connects each digital line to a digital
telephone at the output of the switch. In the event of the switch
in the ACD failing, the patent teaches two techniques (a) to have
each digital telephone connected directly to a digital line (no
switching capability provided) bypassing the failed switch, or (b)
the digital telephones are connected to another switch in the MDF
which is used to connect them to the digital lines if the switch in
the ACD fails. This patent does not thus teach the idea of having
line cards (or other entities) connected together so that if one or
two of these entities fails their inputs can be routed to
designated spare cards. U.S. Pat. No. 5,331,631 "N+K sparing in a
telecommunications switching environment" This patent teaches how
to provide fault-tolerance within a multistage packet-switched
interconnection network. It does not teach how to reroute inputs on
failed line cards to other spare cards using limited switching and
small number of additional hops.
SUMMARY OF THE INVENTION
[0004] The present invention describes methods for rerouting
channels on failed line cards to spare line cards. It uses very
limited switching in combination with carefully designed
interconnection patterns to achieve rerouting of multiple failed
channels to spare line cards by going through a minimal number of
additional switching components. The invention provides some
switching in the line cards to allow rerouting of the I/O channels
in the event of line cards failing. While this invention is
applicable to any multichannel system, it has particular advantages
when using wavelength division multiplexing, where the number of
channels per fiber is not very large and the probability for a line
card to fail is relatively high.
[0005] Two types of rerouting are considered: (1) where we are
allowed to reroute all I/Os in the event of a line card failure
(full rerouting), and (2) where we are allowed to reroute only the
I/O corresponding to the failed line card (limited rerouting).
[0006] Two different types of switching arrangements are
considered. In the first arrangement, referred to as limited
switching, each line card is provided with a switch that
interconnects the I/O either to the link port (for transmission on
the link) or to one of k other local ports. It is also possible to
interconnect a local port with the link port. Each port can be
interconnected with only one other port at any given time. Local
ports of different line cards may be "hardwired" together. The term
"wire" shall refer to any physical connection such as a copper wire
of optical fiber between two elements being "hardwired" to each
other. This allows an I/O of a line card to be switched to the I/Os
and link ports of other line cards.
[0007] With the first arrangement one line card is designated as a
spare card. If another line card fails then the invention proposes
a method to reroute all the I/Os on to other channels (full
rerouting). Each I/O is routed through only one additional line
card. The number of additional cards (or "hops") a signal is routed
through is important since the degradation in signal quality is
approximately proportional to this number. Thus this rerouting
method minimizes the degradation in signal quality. A similar
method is proposed for the case of two line card failures.
[0008] For the limited rerouting case the invention proposes to use
one spare card for each group of k line cards and allows the I/O of
a failed line card to be routed directly to a spare line card, or
again one additional hop.
[0009] In the serond arrangement, referred to as crossbar switching
the invention provides a higher-degree of switching in the line
card. Specifically, each line card is provided with a crossbar
switch that can realize any permutation of its k local ports, link
port and an inside port. It is provided with an additional switch
that under normal operation connects the I/O signal to the inside
port but under a failure, connects the I/O to a separate bypass
port. Local ports and bypass ports of different cards may be
"hardwired" together. This allows an I/O to be switched to other
line cards if its associated line card fails.
[0010] With the second arrangement the invention provides methods
for limited rerouting in the case of one or two line-card failures.
In each case it provides a method for rerouting only the failed
I/Os on to spare cards going through approximately log N line
cards, N being the total number of channels. While this results in
a poorer signal quality compared to the limited-switching
arrangement, it allows the number of spares to be kept small,
allows the failed line card to be removed for servicing without
bringing down that I/O channel, and does not affect all the other
channels if a channel fails.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG. 1 shows the block diagram of a multichannel
transmission system.
[0012] FIG. 2 shows a more detailed diagram of a mux/demux
unit.
[0013] FIG. 3 shows the switching arrangement present in a line
card for the case of limited switching.
[0014] FIG. 4 shows the wiring of the link cards for full rerouting
in the case N =6 and various changes in the configuration in case
of a single failure.
[0015] FIG. 5 shows the wiring of the link cards for full rerouting
in the case N =6 and the configuration in case of two failures.
[0016] FIG. 6 shows the wiring of the link cards for limited
rerouting using limited switching, in a group for the case N=5, and
the configuration after a fault.
[0017] FIG. 7 shows the switching arrangement present in a line
card and its associated I/O card for the case of crossbar
switching,
[0018] FIG. 8 shows a tree configuration for wiring the line cards
using crossbar switching, in an N=17 channel system with k=4 local
ports on each link card. The undirected lines indicate wires
between two local ports and a directed line indicates a wire from a
bypass port to a local port,
[0019] FIG. 9 shows the same tree configuration, in case of a
failure of a line card, the dashed lines represent the route of the
I/O of a failing line card #12 to the liok of the spare card
#0,
[0020] FIG. 10 shows an initial configuration and transformations
for larger configurations for the wiring of link cards for the case
of limited rerouting to two spare cards and crossbar switching,
[0021] FIG. 11 shows an example for the growth of an initial
configuration into a configuration of 12 line cards and two spare
cards. The figure shows only the wiring between local ports,
[0022] FIG. 12 shows the last configuration in FIG. 11 in detail,
including the bypass wiring.
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0023] FIG. 1 shows the block diagram of a multichannel
transmission system. It consists of a multiplexer/demultiplexer
(mux/demux) unit 100 at each end of the transmission link 107 with
6 channels 101-106 (N channels in general), a line card 200 for
each channel, and a mux/demux device 108 that physically
merges/separates the channels onto the link in this embodiment.
Other embodiments may not require a mux/demux device if no
multiplexing is done on the transmission medium.
[0024] FIG. 2 shows a more detailed diagram of a mux/demux unit
100. It consists of N line cards 200 (N=6 in this example). In the
embodiment, each line card is actually realized by using a pair of
cards: a link card 201 and an I/O card 202. The
multiplexing/demultiplexing and transmission hardware is found on
the link cards, and the I/O cards simply receive the input/output
signals and route them to the link cards. Thus, link cards are more
prone to failures than I/O cards. Other realizations may combine
the functionality of both cards into one card. Each I/O card 202
has an I/O port 203 to which the external input/output is
connected. Each link card has a number of local ports 204 and a
link port 205 for connecting it to the transmission link via wire
30 and a mux/demux device 108. A controller 209 is connected to all
of said line cards and is based on any processor, e.g.,
Pentium.
[0025] This invention describes a wiring scheme that supports
switching of I/O from failed to spare line cards. The control
mechanism required to detect a failure and to implement the desired
configuration changes is based on standard methods. See reference
[3] which is hereby incorporated herein by reference. Thus,
detection of a failing line card can be done by detecting the loss
of light from the link port 205, this failure and others are
signaled to the controller 209, which, in turn, computes the
required change in the configuration, based on its knowledge of the
current state of each of the components in the unit. The controller
then signals the switches in each line card a request to change
their state and updates its view of the state of the unit.
[0026] Limited Switching
[0027] FIG. 3 shows the switching arrangement present in a line
card 200. The switch component 301 interconnects the I/O port 203
to either the link port 205 for transmission on the link, or on to
one of four local ports (204) (k local ports in general). The
switch can also interconnect the link port either to the I/O port
or on to one of the local ports. Here, N>k+1, where switch
component 301 has two 1.times.(k+1) switches.
[0028] Full Rerouting
[0029] Single Failure: Referring to FIG. 4A, under normal operation
all N line cards 200 (and channels) can be used. For the sake of
discussion, number the line cards (and corresponding I/Os) from 0
to N-1. More specifically, for N=6, the line cards 200 are numbered
from 0 to 5 as shown. At least one line card s (line card #1) is
considered a spare, i.e., its I/O can be discarded. In case of a
single line card failure, the goal is to reroute N-1 I/Os (other
than ,he I/O s port of) to the link ports of the remaining line
cards (#'s 0, and 2-5).
[0030] This can be done using the following configuration with k=2.
Each line card z has its local ports hardwired to a local port of
line card (i-1) mod N and a local port of line card (i+1) mod N
(See 45 of FIG. 4A). FIG. 4A shows the wiring of the link cards for
the case N=6. Link card 0 is hardwired to link card 1 and 5, link
card 1 to link card 0 and 2 etc. The switches 301 are set as shown
in 40 the figure, with each I/O being interconnected to its
corresponding link port. Also shown are links 30 out of the link
ports 205.
[0031] Referring to FIG. 4B, in case of a single line card failure,
say at line card f (#3 in FIG. 4B), successful rerouting can be
accomplished using one of the following two algorithms. The
following notation will be used: 1 [ x , y ) = { { x , x + 1 , , y
- 1 } , if x < y ; { x , x + 1 , , N - 1 , 0 , 1 , , y - 1 } ,
if x > y ; 0 , if x = y .
[0032] Spare-to-failed rerouting from s to f: For i .di-elect
cons.[s, f), have I/O (i+1) mod N be switched to the link port of
line card i.
[0033] Failed-to-spare rerouting from f to s: For i .di-elect
cons.[f, s), have I/O i be switched to the link port of line card
(i+1) mod N.
[0034] Note that for both rerouting algorithms, if s=f then no
rerouting takes place. Example 1: Suppose that link card 1 in FIG.
4A is designated as the spare card and link card 3 fails. Then we
do either spare-to-failed rerouting from 1 to 3, or failed-to-spare
rerouting from 3 to 1.
[0035] FIG. 4B illustrates spare-to-failed rerouting from 1 to 3.
I/O port of line card #3 is switched to link port 2 (See 44 and 46
of line cards #2 and #3, respectively), and I/O port of line card
#2 is interconnected to the link port of spare line card #1. See
42-43.
[0036] FIG. 4C illustrates failed-to-spare rerouting from line card
#5 to #1. The I/O port of line card #5 is switched to the link port
of line card #0 (See 49, 45', 47) and I/O #0 to link port #1 (See
41).
[0037] Suppose link card 1 fails. Then no rerouting is done, as
shown in FIG. 4D. Two Failures: Referring to FIG. 5 this is similar
to the case above except that there are at least two spare line
cards, s.sub.0 and s.sub.1. In case of two line card failures, the
goal is to reroute N-2 I/Os (other than spare I/O ports s.sub.0 and
s.sub.1) to the link ports of the remaining line cards. If there is
no failure, the wiring is shown in FIG. 5A, and is identical to the
wiring of FIG. 4A.
[0038] Let the two failures be at line cards f.sub.0 and f.sub.1.
Scan the line cards starting at s.sub.0 towards line card #N, and
continue from line card #0 to #s.sub.0-1. Note that without loss of
generality, only two cases need to be considered: Case 1 (see FIG.
5B) is when after s.sub.0, f.sub.0 is met by the scanning process,
then f.sub.1 and then s.sub.1. Case 2 (see FIG. 5C) is when f.sub.0
is met first, then s.sub.1 and only then f.sub.1.
[0039] The following algorithm successfully reroutes the I/O ports.
For Case 1, spare-to-failed rerouting from s.sub.0 to f.sub.0 is
done, and failed-to-spare rerouting from f.sub.1 to s.sub.1 is
done. For Case 2, spare-to-failed rerouting from s.sub.0 to f.sub.0
and then from s.sub.1 to f.sub.1 is done. (Note that the rerouting
for Case 2 can be simplified if f.sub.0=s.sub.1. Then
failed-to-spare rerouting from f.sub.1 to s.sub.0 is sufficient.)
Example 2: Referring to FIG. 5B, suppose line cards #1 and #5 are
designated spare cards. Suppose also cards #2 and #4 fail. This
corresponds to Case 1 above, with s.sub.0=1, f.sub.0=2, f.sub.1=4,
and s.sub.1=5. The I/O port of line #2 is switched to link port of
line card #1 (See 53, 45' and 52). The I/O port of line card #4 to
link 5 (see 54, 45', and 56).
[0040] Referring to FIG. 5C, suppose instead that cards #0 and #2
fail. This corresponds to Case 2 above, with s.sub.0=1, f.sub.0=2,
s.sub.0=5 and f.sub.1=0. The I/O port of line #2 is switched to
link port of line card #1 (See 53, 45', and 52), and the I/O port
of line card #0 to link port of line card #5 (See 51,45', and
57).
[0041] Limited Rerouting
[0042] With limited rerouting if line card i fails, only I/O i
needs to be rerouted. Let there be at least [N/(k+1)] spare line
cards.
[0043] We propose the following configuration, illustrated in FIG.
6. Partition the line cards into groups of size at most k+1 such
that each group has at least one spare. This is possible since
there are at least [N/(k+1)] spares. For each line card that is not
a spare, wire one of its local ports to a local port of a spare in
its group (See 45). This wiring is possible since the group sizes
are at most k+1 and there are k local ports on a line card. FIG. 6A
illustrates this configuration for the case N=15 and k=4.
[0044] If a line card fails, its I/O is rerouted to the link port
of its spare. For example, in FIG. 6B line card #0 fails and its
I/O is routed to link 30' of card #2 on the spare via 61, 45' and
62.
[0045] Crossbar Switching
[0046] FIG. 7 shows the second switching arrangement, present in a
line card 200' (i.e., a link card 201' and its associated I/O card
202'). The bypass switch 501 on the I/O card interconnects the I/O
port 203 to either the inside port 502 or the bypass port 503. The
crossbar switch 504 on the link card provides full permutation
connectivity of the inside port 502, the link port 205 and the 4
local ports 204 (k local ports in general).
[0047] We use the following conventions in the figures: A wire
between two local ports will be designated the number 45 (drawn as
a solid line) during the normal operation of the system, and 45'
(drawn as a dashed line) if it is used to reroute the I/O of a
failing line card to a spare. A wire between a bypass port to a
local port will be designated 55 in the figures (drawn as a solid
arrow from the bypass to the local port), in the normal operation
mode and 55' (drawn as a dashed arrow) if it is used by the failing
card for rerouting. The enabled data paths in a switch will be
designated the number 40 (drawn an a solid two-way arrow) in the
normal mode and 40' (dashed two-way arrow) if used as part of a
route from a failed line card to a spare. These designations will
change whenever a specific reference is required.
[0048] We propose two configurations to interconnect the cards to
deal with failures. For each configuration we give algorithms to
reroute the I/O of the failed card(s) onto the spare card.
[0049] The first is a tree configuration. This handles failure of
one line card and uses a minimum hop route to reroute a failed
channel. The second is a dual tree configuration. This handles two
line card failures but uses slightly longer routes to reroute
failed channels.
[0050] Single Failure-Tree Configuration
[0051] Referring to FIG. 8, our approach is to interconnect the
line cards 200' to realize a network wherein there is a path from
each line card to the spare line card using wires 45. This path may
go through other line cards. In addition the bypass port 503 of any
line card #i must be wired to a local port 204 of a card #j such
that card #i is not in the path between card #j and the spare. Such
a configuration is shown in FIG. 8 for the case N=17 and k=4. Here,
for example, card #4 is bypassed to card #13, and the path from
card #13 to card #0 (the spare) goes through card #3 but not card
#4. In FIG. 9 when card #12 fails, its input is bypassed to card #3
via 80, 55' (to card #11), 81, 45' (to card #3), 82, 45' (to card
#0) and eventually 83 the link port 205 of spare card #0.
[0052] In order to rminimize the number of hops between a card and
the spare, our preferred approach is to interconnect the line cards
to realize a tree network of maximum degree k (using local port to
local port wires 45). The designated spare line card is the root of
the tree. It is hardwired to up to k other line cards. Each of
these line cards is in turn hardwired to k-1 other line cards,
until all the cards are interconnected in the configuration. Cards
that are hardwired to a single other card via wires 45 (local port
to local port) are termed "leaves". In FIG. 8, cards #5-#16 are
leaves. If there are N line cards and 1 additional spare line card,
then the height h (the maximum length path from a leaf to the root)
of the tree connecting all the line cards is 2 h = 1 + log k - 1 (
N + 1 - 2 k ) .
[0053] In addition, we hardwire the bypass port of each line card
#i with a free local port of a line card #1 (using a conection 55)
so that:
[0054] j is a leaf in the tree (this is possible since the number
of leaves in such a tree is always larger than the number of
non-leaves),
[0055] i does not lie on the path from j to the root. If i is a
leaf card, it is possible to meet this requirement by defining a
cycle of all leaves and hardwiring the bypass of each leaf to the
next leaf on the cycle. If i is not a leaf, the requirement is met
by hardwiring i to a leaf in another subtree of the root.
[0056] Generalizing the example in the beginning of this section,
if a failure of a line card i occurs, the I/O is switched by its
501 switch via the bypass port 503 and wire 55 to the local port
204 of line card j. At j, it is switched by the 504 switch to the
local port 204 hardwiring j it to its parent k in the tree. The 504
switch on k's line card is set to enable data transfer from i to
k's parent and so forth, in order to realize a path from the I/O on
line card i to the spare card at the root of the tree. This
procedure is repeated at the other end of the faulty link, so that
the data between the same I/O cards is now transffered via a
different link port. The maximum number of cards required to
participate in the rerouting of the failed channel is h+1.
[0057] Because of the line card modular strcuture, and because of
the way that the bypass ports are wired in this embodiment, the
failed link card 201' can be replaced without affecting any of the
channels in the mux/demux unit. Observe that the maximum number of
hops required to reroute a failed channel is h+1, where h is the
height of the tree, computed above.
[0058] Two Failures-Dual Tree Configuration
[0059] If we designate two line cards as spare cards, we now create
two k-degree trees T.sub.1 and T.sub.2, each with N/2 cards
(assuming we have N line cards) and 1 spare, by hardwiring the
local ports of the cards (wire 45) as described above. The height
of each tree does not exceed log.sub.k-1 N. The trees are
interconnected at the leaf level in two ways: either a leaf is
common to both trees or two leaves of the trees are hardwired to
each other. This structure realizes a dual tree. Example 3: A dual
tree configuration for interconnecting 12 line cards and 2 spare
cards is shown in FIG. 12. In the figure we have T, rooted at card
#1, which includes cards #2 to #9, and tree T.sub.2 rooted at card
#14, which includes cards #6 to #14. The trees are interconnected
in both abovementioned ways: leaves #6 to #9 are common to both
T.sub.1 and T.sub.2, while leaves #4 and #12 belong exclusively to
T.sub.1 and T.sub.2 respectively, and are connected to each other.
Leaves #5 and #13 are also connected to each other.
[0060] The transformations descibed below define the rules for
hardwiring the local ports of the line cards to each other via wire
45, while requirements B1 and B2 described below and comments C2
and C3 define the rules for hardwiring the bypass ports to local
ports via wire 55.
[0061] The following requirements are necessary in this embodiment
to make sure that once an I/O of some card is bypassed to another
card, it can reach a spare card, even if the link card 201 fails
entirely (including its switching function): (B1) The bypass of a
leaf card i is not hardwired to a leaf j whose bypass is hardwired
to i, and (B2) The bypass port of a card i is not hardwired to a
leaf j whose path to a spare goes through i. More formally, if Li
is the set of leaves of a tree whose path to the root goes through
i, then j f Li. In particular, a bypass port of card i is not
hardwired to the same card i.
[0062] As explained below, this configuration is feasible with a
switch degree of k.gtoreq.4. Furthermore, the above conditions
ensures that, for any card i, if both i and the card to which i's
bypass is hardwired, j, fail, at least one of them (namely j) will
be successfully rerouted. At any other two failure scenario, both
failures will be rerouted to the spares.
[0063] The dual tree configuration is defined by a basic
configuration of a single line card (plus two spares), and three
transformations that gradually increase the number of cards in the
configuration, one card at a time, as demonstrated in FIG. 11, up
to any desired number of line cards N.
[0064] Basic configuration. The basic configuration is composed of
a single leaf card (depicted as a full box), hardwired to the two
spares (depicted as circles), see FIG. 1OA.
[0065] Transformations. The basic configuration is extended one
card at a time, by applying one of the transformations in FIG. 10B.
In the figure, a transformation is depicted by an arrow, the spare
cards are depicted as circles, line cards which are leaves are
depicted as full boxes, while line cards that are non-leaves
(termed internal line cards) are depicted as empty boxes. The
transformations are applied in the following order: (1) T.sub.c,
(2) T.sub.a, (3) T.sub.b. Hence, transformations of type T.sub.c
are applied whenever possible, only if no such transformations are
possible, then T.sub.b is performed, and only if no transformations
T.sub.c, T.sub.b are applicable, is a T.sub.a transformation
applied.
[0066] Henceforth, the distance relating terms ("distance","far"
etc.) will refer to the number of hops (or line cards) between a
pair of line cards. The transformations are fully described by the
following table, in which "Ord" stands for the order in which
transformations are applied, "Card" stands for the increase in the
total number of line cards in the configuration resulting from the
transformation, and "Dist" for the maximal increase in the maximum
distance from a leaf to a spare card. The numbers in the table
refer to FIG. 10, in which the new card which is the outcome of the
transformation is marked by "new 200'".
1 Trans. Refer to Ord Precondition Description Card Leaf Dist
.tau..sub.a 2 (1) A single leaf #1 is Disconnect +1 +1 +1 hardwired
to two inter- card #1 from 45b, add nal or spare cards (wires a new
card 200', hard- 45a and 45b). (2) Any wire it to card #1 by other
leaf x satisfying a wire 45, and hardwire precondition 1 must be
the new card to the wire at least as far from the 45b. furthest
spare. .tau..sub.b 3 Two leaf cards #1 and Insert a new leaf card
+1 -1 +0 #2 are hardwired con- #3 between cards #1 nected to each
other and #2 (mutual to both (wire 45c). trees), by disconnect- ing
wire 45c, and hard- wiring card #3 to both #1 and #2 by wire 45.
Cards #1 and #2 are now non-leaves. .tau..sub.c 1 There exist two
inter- Hardwire a new leaf +1 +1 +0 nal cards, #1 .epsilon. T.sub.1
and card to both #1 and #2 #2 .epsilon. T.sub.2, that are inter-
using wire 45. connected by less than k - 1 leaves (the num- ber of
cards as #3 is less than k - 1).
[0067] The feasiblity of the construction is made possible by the
following comments:
[0068] (C1) Since bypass ports are hardwired to ports of leaves
only (by means of wire 55), there have to be enough leaves after
each transformation, to support connections from all cards to their
free ports. Note that each leaf has k-1 free ports (since it always
has two connections to other cards). Thus, if k.gtoreq.4,
transformations T.sub.a and T.sub.c only enlarge the number of free
ports (since only one card is added, and k-2.gtoreq.1 free ports
are added). The only case which may deem the solution infeasible is
transformation T.sub.1, which decreases the number of available
bypass ports at the leaves, while increasing the number of total
cards. This is taken care of by the precedence order of the
transformations, which ensures that T.sub.b only takes place when
there are extra ports at the leaves, to support the bypass
connections.
[0069] (C2) Complying with requirement B1 (See above) is easy if
there are more than 2 leaves (and impossible otherwise): Connect
the leaves in one big cycle. After this, each leaf still has at
least one free port.
[0070] (C3) To comply with requirement B2 (See above), arrange the
subtrees headed by the children of the root in a cyclic order
(S.sub.0, S.sub.1, . . . , S.sub.k, where each S.sub.i is a
subtree). Then, hardwire the internal nodes of S.sub.i to the free
ports in the leaves of S.sub.i+1 (all calculations are modulo k+1).
Since the transformation precedence order above creates a dual tree
with subtrees of equal size, except for at most one single subtree,
this is feasible for most S.sub.i. Assume S.sub.j is the only
smaller sub-tree. Then some set X of cards in S.sub.j-1 cannot be
hardwired to leaves of S.sub.j. However, some set Y of the leaves
in S.sub.j+1 is not mapped to by the (fewer) cards in S.sub.j.
Hence, it is possible to hardwire the bypass ports of cards in X to
the free ports of the leaves in Y.
[0071] The above description translates into the following
algorithm, which, in conjunction with the description in the above
table, is sufficient for a programmer to code on any processor,
such as a Pentium:
2 Construct the dual-tree: WHILE NumOfCards < N IF .tau..sub.c
can be applied THEN apply it ELSE IF .tau..sub.a can be applied
THEN apply it ELSE apply .tau..sub.b
[0072] Connect the bypass ports:
[0073] (1) Of leaf cards to other leaf cards in one cycle
[0074] (2) Of inner cards to the remaining ports of leaf cards (see
above). Example 4: A detailed dual tree configuration for
interconnecting 12 link cards to two spares is shown in FIG. 12.
The figure depicts in detail the last configuration from FIG. 11.
FIG. 11 depicts all the intermediate configurations, starting from
a basic 3 card configuration. In each stage, the arrows in FIG. 11
are tagged by the transformation from FIG. 10 that was used. While
the wires 55 are not depicted in FIG. 11, they appear in FIG. 12 as
unidirectional arrows from the bypass port to the local port. As
may be verified by matching the card numbers of FIG. 12 and the
full boxes in FIG. 11, the leaf cards are #4 to #9 and #12 to #13.
To comply with requirement Bl, their wires 55 connect them in a
cycle:
#5.fwdarw.#4.fwdarw.#6.fwdarw.#7.fwdarw.#8.fwdarw.#9.fwdarw.#12.-
fwdarw.#13.fwdarw.#5. Also, to comply with requirement B2, the
bypass of an internal card is hardwired to a leaf node in a
different subtree. For example, the bypass port of card #2 cannot
be hardwired to leaf cards #6 to #8, but can be hardwired to any
other leaf card because the other leaf cards are in different
subtrees from that of card #2. Specifically, card #2 is hardwired
to leaf card #5.
[0075] When up to two line cards f.sub.1 and f.sub.2 fail, the
controller unit 209 computes the shortest paths P.sub.i,j between
said failed line card f.sub.i and spare card sj (where
i=1..2,j=1..2), and finds if P.sub.1,1 and P.sub.2,2 have a common
wire or pass through one of the failed cards. In such a case, the
controller modifies the switch interconnections along P.sub.1,2 and
P.sub.2,1 so as to enable the I/O of f.sub.1 to reach the link port
of s.sub.2 and the I/O of f.sub.2 to reach the link port of
s.sub.1. Otherwise, the controller modifies the switches along
P.sub.1,1 and P.sub.2,2 so as to enable the I/O of f.sub.1 to reach
the link port of s.sub.1 and the I/O of f.sub.2 to reach the link
port of s.sub.2
[0076] References
[0077] [1] F. J. Janniello, R. A. Neuner, R. Ramaswami, and P. E.
Green, "Multi-protocol optical fiber multiplexer for remote
computer interconnection," in OFC'95 Tech. Digest, 1995.
[0078] [2] F. J. Janniello, R. A. Neuner, R. Ramaswami, and P. E.
Green, "A multi-protocol optical fiber multiplexer for remote
computer interconnection," tech. rep., IBM Research Report,
1995.
[0079] [3] J. Y. Hui, Switching and Traffic Theory For Integrated
Broadband Networks. Kluwer Academic Publishers, 1990.
* * * * *