U.S. patent application number 11/012108 was filed with the patent office on 2005-06-23 for multiplexing of control and data over an http connection.
This patent application is currently assigned to Solace Systems, Inc.. Invention is credited to Ashton, Peter, Barnes, Martin, Bertin, Greg, Betts, Craig, Burwell, Wayne, Pochopsky, David.
Application Number | 20050135418 11/012108 |
Document ID | / |
Family ID | 34680861 |
Filed Date | 2005-06-23 |
United States Patent
Application |
20050135418 |
Kind Code |
A1 |
Betts, Craig ; et
al. |
June 23, 2005 |
Multiplexing of control and data over an HTTP connection
Abstract
A method for exchanging control and customer data between
network element in a communications network involves establishing a
virtual connection between the routers, and exchanging the control
and customer data over an http layer.
Inventors: |
Betts, Craig; (Kanata,
CA) ; Pochopsky, David; (Ottawa, CA) ; Barnes,
Martin; (Kanata, CA) ; Bertin, Greg; (Ottawa,
CA) ; Ashton, Peter; (Nepean, CA) ; Burwell,
Wayne; (Ottawa, CA) |
Correspondence
Address: |
MARKS & CLERK
P.O. BOX 957
STATION B
OTTAWA
ON
K1P 5S7
CA
|
Assignee: |
Solace Systems, Inc.
Ottawa
CA
|
Family ID: |
34680861 |
Appl. No.: |
11/012108 |
Filed: |
December 16, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60530678 |
Dec 19, 2003 |
|
|
|
Current U.S.
Class: |
370/469 ;
370/465 |
Current CPC
Class: |
H04L 67/02 20130101;
H04L 45/00 20130101 |
Class at
Publication: |
370/469 ;
370/465 |
International
Class: |
H04L 001/00 |
Claims
We claim:
1. A method of passing application protocol streams between network
elements in a data communications network, comprising establishing
a virtual connection between said network elements, establishing an
HTTP layer within said virtual connection; and multiplexing said
application protocol streams over said HTTP layer.
2. The method of claim 1, wherein said network elements are XML
routers, and said network is a content based network.
3. The method of claim 1, wherein said application protocol streams
control application sub-systems within a network element, and said
application sub-systems are identified using HTTP URI
arguments.
4. The method of claim 1, wherein said application protocol streams
control application sub-systems within a network element, and said
application sub-systems are identified using "absolute path"
portions of HTTP URIs.
5. The method of claim 1, wherein said application protocol streams
control application sub-systems within a network element, and said
application sub-systems are identified using HTTP header
fields.
6. The method of claim 1, wherein said virtual connection is a TCP
connection.
7. The method of claim 2, wherein said protocol streams comprise a
first data flow connecting an XLSP protocol in each XML router, a
second flow connecting an XSMP protocol in each router, and a third
flow connecting a Data Plane Forwarding Engine in each router.
8. The method of claim 1, wherein said application protocol streams
are multiplexed over a single TCP connection, and said application
protocol streams are prioritized via queue servicing.
9. The method of claim 8, wherein each of said application protocol
streams is mapped onto an output queue, and wherein more than one
application protocol stream can be mapped onto the same output
queue.
10. The method of claim 9, wherein said output queues are serviced
using a weighted round robin scheme, with each queue being
allocated a different weight.
11. The method of claim 10, wherein said queues are assigned
weights M:M such that the bandwidth received by the control queue
is M/M+N.
12. The method of claim 9, wherein said application protocol
streams comprise control plane and data plane traffic, and said
control and data plane traffic is separated into separate output
queues.
13. A router for use in a data communications network, wherein said
router is configured to establish a virtual connection with other
routers in the network, establish an HTTP layer within said virtual
connection; and multiplex application protocol streams over said
HTTP layer.
14. The router of claim 13, which is an XML router for use in an
XML content based network.
15. The router of claim 13, further comprising a scheduler for
imposing a queueing discipline on outbound traffic over said HTTP
layer.
16. The router of claim 15, wherein said scheduler is a weighted
round robin scheduler.
17. The router of claim 13, wherein said application protocol
streams control application sub-systems within a network element,
and said router is configured to identify application sub-systems
using HTTP URI arguments.
18. The router of claim 13, wherein said application protocol
streams control application sub-systems within a network element,
and said router is configured to identify application sub-systems
using "absolute path" portions of HTTP URIs.
19. The router of claim 13, wherein said application protocol
streams control application sub-systems within a network element,
and said router is configured to identify application sub-systems
using HTTP header fields.
Description
CROSS REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit under 35 USC 119(e) or
prior U.S. provisional application Ser. No. 60/530,678 filed Dec.
19, 2003, the contents of which are incorporated herein by
reference.
FIELD OF THE INVENTION
[0002] This invention relates to the field of data communication
networks, and in particular to a method of passing application
protocol streams between any device in a data communications
network, such as a content based network having XML routers.
BACKGROUND OF THE INVENTION
[0003] Content-based networks are described in A. Carzaniga, M. J.
Rutherford, A. L. Wolf. A routing scheme for content-based
networking. Department of Computer Science, University of Colorado,
June 2003, the contents of which are herein incorporated by
reference.
[0004] In content routed networks, one of the factors that
determine network scalability is the limited number of TCP
connections supported by each router. Between any two adjacent
content routers, both control and customer data needs to be
exchanged. Within the control data flows, two distinct protocols
are defined; the XML Link State Protocol (XLSP) and XML
Subscription Management Protocol (XSMP), both of which are
components of the Implicit Routing Protocol (IRP). Refer to
co-filed application Ser. No. 60/530,615, which is herein
incorporated by reference. In this example, there are three
application protocol streams being exchanged between each pair of
content routers: an XSMP control stream, an XLSP control stream,
and a data stream.
[0005] A traditional method for passing these three data flows
between devices would be to establish separate TCP connections for
each data flow. Multiplexing and de-multiplexing of the data at
each end would be accomplished via distinct TCP port numbers.
However, this technique has two significant drawbacks:
[0006] 1. The number of TCP connections supported by any one
networking device may be (relatively) small, and hence the wasteful
usage of TCP connections would quickly limit the size and
scalability of the network.
[0007] 2. In a firewalled environment, the administrative overhead
of configuring the firewalls to allow each type of connection
significantly increases the administrative effort required to
deploy the content routed network.
[0008] As an example, in FIG. 1, consider the relatively small
network 1. Here, the topology includes seven content routers 2,
interconnected with thirteen XML links 3. Using TCP multiplexing, a
total of thirty nine connections would be required. Each XML link 2
between a pair of content routers requires three TCP connections
under this scheme since one TCP connection is needed to exchange
customer data, which in a content-routed network is the events or
documents that are being routed, a second TCP connection is needed
for the XLSP protocol, and a third TCP connection is needed for the
XSMP protocol. It is apparent that TCP connections would quickly
limit network scalability, as the number of routers increases into
the hundreds and number of XML links increases into the thousands
or tens of thousands. Moreover, as new inter-router protocols are
developed to enable new services or capabilities within the content
routed network 1, yet more TCP connections would be required
between a pair of routers connected with an XML link, further
compounding the problem.
SUMMARY OF THE INVENTION
[0009] The invention discloses a novel technique for multiplexing
flows for customer data and one or more control protocols over a
single HTTP/TCP connection, which allows the reduction by one half
or more in the number of TCP connections required for a given
network topology. The invention is applicable to any device where
multiple protocols can share a single HTTP connection.
[0010] According to the present invention there is provided a
method of passing application protocol streams between network
elements in a data communications network, comprising establishing
a virtual connection between said network elements, establishing an
HTTP layer within said virtual connection; and multiplexing said
application protocol streams over said HTTP layer.
[0011] The invention may be applied to XML routers in a content
based network, although it is not limited to use with XML
routers.
[0012] Embodiments of the invention can, for example, be used to
identify control and data plane sub-systems within an XML router
using HTTP Universal Resource Identifier (URI) arguments. HTTP and
the HTTP URI is defined in RFC2616, "HyperText Transfer
Protocol--HTTP/1.1", June 1999; RFC1945, "Hypertext Transfer
Protocol--HTTP/1.0", May 1996, and also in RFC2396, "Uniform
Resource Identifiers (URI): Generic Syntax.", August 1998, all from
The Internet Society.
[0013] Control and data traffic can be multiplexed over a single
TCP connection, in which case HTTP is used as the de-multiplexing
mechanism.
[0014] The invention also provides a method for ensuring that when
control and data plane traffic is multiplexed over a single TCP
connection, the prioritization of control plane messages is
achieved via queue servicing.
[0015] The invention still further provides a router for use in a
data communications network, wherein said router is configured to
establish a virtual connection with other routers in the network,
establish an HTTP layer within said virtual connection; and
multiplex application protocol streams over said HTTP layer.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The invention will now be described in more detail, by way
of example only, with reference to the accompanying drawings, in
which:
[0017] FIG. 1 shows a Content Routing Network;
[0018] FIG. 2: shows Multiplexing over HTTP/TCP; and
[0019] FIG. 3 shows Outbound Queue Scheduling.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] In accordance with the principles of the invention all data
exchanged between adjacent routers in the content based network of
FIG. 1 is carried via HTTP over TCP. A single TCP connection
between routers can be achieved, if the HTTP layer is used to
de-multiplex the control and data flows, as shown in FIG. 2. Two
routers 10 and 11 are connected by an HTTP over TCP connection 12,
which is terminated by the HTTP function 16 in each router. Within
the single HTTP over TCP connection 12, there are three flows 13,
14 and 15. Flow 13 connects the XLSP protocol 17 in each router 10
and 11. Flow 14 connects the XSMP protocol 18 in each router 10 and
11. Flow 15 connects the Data Plane Forwarding Engine (DP-FE) 19 in
each router 10 and 11.
[0021] HTTP identifies "paths" or locations for the exchanged data
via field called the Uniform Resource Identifier (URI). The
definition of the HTTP URI, specified in the reference above,
allows the passing of arguments between the communicating devices.
Arguments are identified by the leading character "?". The general
form of the HTTP Universal Resource Locator (URL), which is a form
of a Universal Resource Identifier (URI) is:
[0022] "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
[0023] Within the "query" portion, the "?name" argument is used to
identify and de-multiplex data flows to the appropriate sub-systems
within the content router, for example, the three subsystems 17, 18
and 19 shown in FIG. 2. For example,
[0024] http://<router_ip>/?name=<subsystem_name>
[0025] There are three currently recognized values for
<subsystem_name>, which are identified in FIG. 2. As an
example, when sending an XLSP message to the adjacent content
router with IP address 192.168.10.1, the URI would be:
[0026] http://192.168.10.1/?name=XLSP
[0027] The default subsystem is the DP-FE, and if an HTTP message
is received without the name=<subsystem_name> parameter
specified, then the DP-FE subsystem 19 is selected by default.
[0028] Another method to use a single HTTP session to
multiplex/demultiplex multiple protocol flows is to use the
"absolute path" portion of the URI to specify the subsystem.
[0029] For example:
[0030] http://192.168.10.1/<subsystem_name>
[0031] where subsystem name is one of those specified in FIG. 2.
For example, for the XLSP subsystem 17, the URI would be:
[0032] http://192.168.10.1/XLSP
[0033] Additionally, other HTTP header fields could be utilized
instead of the HTTP URI in order to specify the subsystem name for
the purposes of multiplexing/demultiplexing multiple protocol flows
over a single HTTP connection. HTTP consists of numerous header
fields to carry various control information, such as the HTTP
content length. One of these existing fields could be used to
define the subsystem, or a new header field could be defined. For
example, the defined HTTP "pragma" general-header field could be
used to specify the subsystem. For example, in order to specify the
XLSP subsystem:
[0034] Pragma: XLSP
[0035] Alternatively, if a new field "subsystem" was defined and
used as part of the HTTP header, the subsystem could be specified
as:
[0036] subsystem=XLSP
[0037] The technique of multiplexing control and data flows over a
single TCP connection presents a potential problem in the
prioritization of control traffic over data traffic (or more
generally higher priority application traffic over lower priority
application traffic). In the presence of heavy volumes of customer
dataplane traffic (handled by the DP-FE subsystem 19 of FIG. 2),
links can become congested, which can lead to significant delays or
lost messages. It is critical that the Implicit Routing Protocol
(IRP) protocol messages are isolated from these effects, as they
can lead to routing instabilities, routing cycles and lost
subscriptions; all of which lead to a disruption in customer data.
Note that the IRP function is composed of the XLSP subsystem and
the XSMP subsystem. The IRP is described in co-filed application
Ser. No. 60/530,615.
[0038] Design techniques are applied to the outbound traffic
direction to mitigate the effect of congestion on IRP protocol
traffic.
[0039] Congestion on a TCP connection is reflected by a backup of
outgoing messages ready to be sent on that connection. These
messages are stored internally in the content router in a software
queuing data structure (which can also be implemented as a hardware
queue if the HTTP over TCP function is implemented using hardware
acceleration). By separating the control and dataplane traffic into
separate queues, and imposing a queue servicing and scheduling
discipline across the queues, it is possible to minimize the delays
experienced by the control traffic.
[0040] The scheduling discipline chosen is a Work
Conserving-Weighted Round Robin, as depicted in FIG. 3. Work
conserving involves scheduling disciplines which always make use of
available link bandwidth. Weighted Round Robin is a scheduling
discipline which cycles through a set of queues, and grants access
to the physical medium based on weights assigned to the queues. In
this scheme, the two queues can be assigned weights M:N, such that
under congestion scenarios, the bandwidth received by the control
queue is M/(N+M) of the total available bandwidth. Typical values
for M:N are 5:1.
[0041] Since the ratio of data plane traffic to control messaging
is typically largely biased in favor of the data plane (i.e. in
typical network operations there are many more data plane messages
than control plane messages), it is rare that control messaging
will ever consume the full bandwidth available to it. In these
cases the work conserving aspect of the scheduler kicks in: if one
of the queues has no data to send in its timeslot, the other queue
is serviced.
[0042] The queuing is shown in FIG. 3. A single TCP socket (TCP
connection) 20 carries the three application flows of XLSP 23, XSMP
24 and DP-FE 25 as described above, using HTTP 22. The XLSP
subsystem 23 produces a message flow 26. The XSMP subsystem 24
produces a message flow 27. The DP-FE subsystem 25 produces a
message flow 28. These messages are formatted as HTTP by the HTTP
block 22. The XLSP 23 and XSMP 24 messages are placed into queue
30, and the DP-FE 25 messages are placed into queue 29. The Work
Conserving-Weighted Round Robin (WC-WRR) block 31 is responsible
for managing the removal of messages from queues 30 and 29 for
sending to the TCP connection manager block 21, which manages the
TCP connection 20. Queue 30 carries the control plane traffic, and
queue 29 carries the dataplane traffic.
[0043] In FIG. 3, a single queue 30 is used for both XLSP 23 and
XSMP 24, which together comprise the IRP. However, each of these
two subsystems could be given their own queue if the traffic
between XLSP and XSMP needed different priority treatment. In
addition, additional subsystems could be added to the system in
addition to the three shown, and this could result in the addition
of additional queues, or the new subsystems could share existing
queues 29 and 30.
[0044] It will be appreciated by persons skilled in the art that
many variants of the invention are possible.
[0045] All references mentioned above are herein incorporated by
reference.
* * * * *
References