U.S. patent application number 14/613894 was filed with the patent office on 2015-09-10 for hybrid autonomous network and router for communication between heterogeneous subnets.
The applicant listed for this patent is ARCHITECTURE TECHNOLOGY, INC.. Invention is credited to Jafar Al-Gharaibeh, Ben Burnett, Ranga Sri Ramanujan, Barry A. Trent.
Application Number | 20150257081 14/613894 |
Document ID | / |
Family ID | 54018805 |
Filed Date | 2015-09-10 |
United States Patent
Application |
20150257081 |
Kind Code |
A1 |
Ramanujan; Ranga Sri ; et
al. |
September 10, 2015 |
HYBRID AUTONOMOUS NETWORK AND ROUTER FOR COMMUNICATION BETWEEN
HETEROGENEOUS SUBNETS
Abstract
A single routable network and an integrated router having a
plurality of network interfaces for such network having a plurality
of heterogeneous subnetworks having different network parameters,
each of the plurality of network interfaces configured to be
connected to a different one of the plurality of heterogeneous
subnetworks. The integration router is configured to automatically
connect with each of the plurality of heterogeneous subnetworks.
The integration router providing persistent network connectivity
between user nodes across the plurality of heterogeneous
subnetworks
Inventors: |
Ramanujan; Ranga Sri;
(Medina, MN) ; Burnett; Ben; (Prior Lake, MN)
; Trent; Barry A.; (Chanhassen, MN) ;
Al-Gharaibeh; Jafar; (Bloomington, MN) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
ARCHITECTURE TECHNOLOGY, INC. |
Eden Prairie |
MN |
US |
|
|
Family ID: |
54018805 |
Appl. No.: |
14/613894 |
Filed: |
February 4, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61935661 |
Feb 4, 2014 |
|
|
|
Current U.S.
Class: |
370/329 |
Current CPC
Class: |
H04L 45/125 20130101;
H04L 45/02 20130101; H04L 45/24 20130101; H04W 40/02 20130101; H04W
88/10 20130101 |
International
Class: |
H04W 40/26 20060101
H04W040/26; H04W 40/02 20060101 H04W040/02 |
Goverment Interests
GOVERNMENT CONTRACTS
[0002] This invention was made with government support under
contract FA8750-12-C-0213 awarded by the Air Force Research
Laboratory (AFRL) Small Business Innovation Research (SBIR). The
government has certain rights in the invention.
Claims
1. A single routable network comprising: a plurality of
heterogeneous subnetworks having different network parameters; and
an integration router containing a plurality of network interfaces,
each of said plurality of network interfaces configured to be
connected to a different one of said plurality of heterogeneous
subnetworks; said integration router being configured to
automatically connect with each of said plurality of heterogeneous
subnetworks; said integration router providing persistent network
connectivity between user nodes across said plurality of
heterogeneous subnetworks.
2. The network of claim 1 wherein said integration router is
configured to automatically connect with each of said plurality of
heterogeneous subnetworks without individual manual configuration
of parameters associated with each of said plurality of
heterogeneous subnetworks.
3. The network of claim 2 wherein said integration router is
configured to provide dynamic route selection between a first node
on one of said plurality of heterogeneous subnetworks and a second
node on another of said plurality of heterogeneous subnetworks.
4. The network of claim 3 wherein said integration router provides
dynamic route selection between said first node on said one of said
plurality of heterogeneous subnetworks and said second node on said
another of said plurality of heterogeneous subnetworks based on a
quality of service.
5. The network of claim 4 wherein said integration router
dynamically configures a route selection between said first node on
said one of said plurality of heterogeneous subnetworks and said
second node on said another of said plurality of heterogeneous
subnetworks based on a proactive failover based on degradation of
service quality before route communication failure.
6. The network of claim 3 wherein said integration router provides
dynamic route selection between said first node on said one of said
plurality of heterogeneous subnetworks and said second node on said
another of said plurality of heterogeneous subnetworks based on a
balancing loads over different routes between said first node on
said one of said plurality of heterogeneous subnetworks and said
second node on said another of said plurality of heterogeneous
subnetworks.
7. The network of claim 3 wherein said integration router further
comprises a BGP-MX module configured to dynamically discover BGP
peers across said plurality of heterogeneous subnetworks.
8. The network of claim 7 wherein said BGP-MX module is further
configured to automatically and dynamically establish and adjust a
route between said first node on said one of said plurality of
heterogeneous subnetworks and said second node on said another of
said plurality of heterogeneous subnetworks.
9. The network of claim 1 wherein said integration router is
configured to be integrated with a conventional software
router.
10. The network of claim 9 wherein said integration router is
configured to be integrated within said conventional software
router.
11. The network of claim 9 wherein said integration router is
configured as an add-on module to a conventional software
router.
12. The network of claim 9 wherein said integration router is
configured to be programmed within a programmable processing
appliance.
13. The network of claim 1 wherein said integration router
comprises: an autonomous router configuration module configured to
be operatively coupled to a conventional router; a dynamic address
map providing topology and address tracking; and a device
configuration interface configured to be coupled to each of said
plurality of heterogeneous subnetworks.
14. The network as in claim 13 wherein each of said plurality of
heterogeneous subnetworks are separate autonomous systems.
15. A single routable network comprising: a plurality of
heterogeneous subnetworks having different network parameters; and
a plurality of integration routers containing a plurality of
network interfaces, each of said plurality of network interfaces
configured to be connected to a different one of said plurality of
heterogeneous subnetworks; said integration router being configured
to automatically connect with each of said plurality of
heterogeneous subnetworks; said integration router providing
persistent network connectivity between user nodes across said
plurality of heterogeneous subnetworks; wherein a first type of
said plurality of integration routers is configured as an interior
router to be installed a backbone nodes providing routing to and
from a first node on one of said plurality of network interfaces
and a second node on another of said plurality of network
interfaces; wherein a second type of integration router is
configured as an edge router providing routing to and from a node
located on one of said plurality of network interfaces; and wherein
a third type of integration router is configured as a gateway
router to be installed a backbone nodes providing routing to and
from a first node on one of said plurality of network interfaces
and a second node on another of said plurality of network
interfaces and to provide links to a global information grid.
16. An integration router configured for use on a single routable
network having a plurality of heterogeneous subnetworks having
different network parameters; said integration router containing a
plurality of network interfaces, each of said plurality of network
interfaces configured to be connected to a different one of said
plurality of heterogeneous subnetworks; said integration router
being configured to automatically connect with each of said
plurality of heterogeneous subnetworks; and said integration router
providing persistent network connectivity between user nodes across
said plurality of heterogeneous subnetworks.
17. The integration router of claim 16 wherein said integration
router is configured to automatically connect with each of said
plurality of heterogeneous subnetworks without individual manual
configuration of parameters associated with each of said plurality
of heterogeneous subnetworks.
18. The integration router of claim 17 wherein said integration
router is configured to provide dynamic route selection between a
first node on one of said plurality of heterogeneous subnetworks
and a second node on another of said plurality of heterogeneous
subnetworks.
19. The integration router of claim 16 wherein said integration
router is further configured to automatically and dynamically
establish and adjust a route between said first node on said one of
said plurality of heterogeneous subnetworks and said second node on
said another--of said plurality of heterogeneous subnetworks.
Description
RELATED APPLICATION
[0001] This application claims priority from U.S. Provisional
Application No. 61/935,661, filed on Feb. 4, 2014 entitled "HYBRID
AUTONOMOUS ROUTER FOR TACTICAL COMMUNICATION".
TECHNICAL FIELD
[0003] The present invention relates to networking systems and
components in general and, more particularly to networking systems
and components having an ability to connect heterogeneous wireless
networks.
BACKGROUND
[0004] Various networking systems, protocols and networking
equipment and software exist to connect various discrete components
together into a communication network.
[0005] However, existing networking approaches are incapable of
integrating heterogeneous wireless networks. Such existing networks
have: [0006] (1) inability to adapt dynamically to topology changes
in the network; [0007] (2) need for excessive manual configuration;
[0008] (3) lack of scalability with network size; [0009] (4)
inability of routing protocols to factor wireless link quality into
routing decisions, resulting in sub-optimal network
performance.
SUMMARY
[0010] A hybrid router, an autonomous IP based network integration
solution that provides end-to-end sensor-to-shooter connectivity
across a heterogeneous tactical network is disclosed. This network
consists of IP sub-networks of various types such as TTNT (Tactical
Targeting Network Technology), mini-CDL (Common Data Link), free
space optics communications (FSOC), QNT (Quint Networking
Technology), and JCAN (Joint Capability for Airborne Networking).
These integrated networks provide improved tactical communications
and situational awareness. Network integration is the convergence
of many IP devices (wired, wireless, radio, optical) each forming
IP subnets into a single IP network. Each of the device subnets may
be IP capable on their own, but cannot integrate seamlessly and
automatically with others.
[0011] In an embodiment, a single routable network has a plurality
of heterogeneous subnetworks having different network parameters
and an integration router containing a plurality of network
interfaces, each of the plurality of network interfaces configured
to be connected to a different one of the plurality of
heterogeneous subnetworks. The integration router is configured to
automatically connect with each of the plurality of heterogeneous
subnetworks. The integration router providing persistent network
connectivity between user nodes across the plurality of
heterogeneous subnetworks.
[0012] In an embodiment, the integration router is configured to
automatically connect with each of the plurality of heterogeneous
subnetworks without individual manual configuration of parameters
associated with each of the plurality of heterogeneous
subnetworks.
[0013] In an embodiment, the integration router is configured to
provide dynamic route selection between a first node on one of the
plurality of heterogeneous subnetworks and a second node on another
of the plurality of heterogeneous subnetworks.
[0014] In an embodiment, the integration router provides dynamic
route selection between the first node on the one of the plurality
of heterogeneous subnetworks and the second node on the another of
the plurality of heterogeneous subnetworks based on a quality of
service.
[0015] In an embodiment, the integration router dynamically
configures a route selection between the first node on the one of
the plurality of heterogeneous subnetworks and the second node on
the another of the plurality of heterogeneous subnetworks based on
a proactive failover based on degradation of service quality before
route communication failure.
[0016] In an embodiment, the integration router provides dynamic
route selection between the first node on the one of the plurality
of heterogeneous subnetworks and the second node on the another of
the plurality of heterogeneous subnetworks based on a balancing
loads over different routes between the first node on the one of
the plurality of heterogeneous subnetworks and the second node on
the another of the plurality of heterogeneous subnetworks.
[0017] In an embodiment, the integration router further comprises a
BGP-MX module configured to dynamically discover BGP peers across
the plurality of heterogeneous subnetworks.
[0018] In an embodiment, the BGP-MX module is further configured to
automatically and dynamically establish and adjust a route between
the first node on the one of the plurality of heterogeneous
subnetworks and the second node on the another of the plurality of
heterogeneous subnetworks.
[0019] In an embodiment, the integration router is configured to be
integrated with a conventional software router.
[0020] In an embodiment, the integration router is configured to be
integrated within the conventional software router.
[0021] In an embodiment, the integration router is configured as an
add-on module to a conventional software router.
[0022] In an embodiment, the integration router is configured to be
programmed within a programmable processing appliance.
[0023] In an embodiment, the integration router has an autonomous
router configuration module configured to be operatively coupled to
a conventional router, a dynamic address map providing topology and
address tracking, and a device configuration interface configured
to be coupled to each of the plurality of heterogeneous
subnetworks.
[0024] In an embodiment, each of the plurality of heterogeneous
subnetworks are separate autonomous systems.
[0025] In an embodiment, a single network has a plurality of
heterogeneous subnetworks having different network parameters and a
plurality of integration routers containing a plurality of network
interfaces, each of the plurality of network interfaces configured
to be connected to a different one of the plurality of
heterogeneous subnetworks. The integration router is configured to
automatically connect with each of the plurality of heterogeneous
subnetworks. The integration router providing persistent network
connectivity between user nodes across the plurality of
heterogeneous subnetworks. A first type of the plurality of
integration routers is configured as an interior router to be
installed a backbone nodes providing routing to and from a first
node on one of the plurality of network interfaces and a second
node on another of the plurality of network interfaces. A a second
type of integration router is configured as an edge router
providing routing to and from a node located on one of the
plurality of network interfaces. A third type of integration router
is configured as a gateway router to be installed a backbone nodes
providing routing to and from a first node on one of the plurality
of network interfaces and a second node on another of the plurality
of network interfaces and to provide links to a global information
grid.
[0026] In an embodiment, an integration router configured for use
on a single routable network having a plurality of heterogeneous
subnetworks having different network parameters. The integration
router containing a plurality of network interfaces, each of the
plurality of network interfaces configured to be connected to a
different one of the plurality of heterogeneous subnetworks. The
integration router is configured to automatically connect with each
of the plurality of heterogeneous subnetworks. The integration
router provides persistent network connectivity between user nodes
across the plurality of heterogeneous subnetworks.
[0027] In an embodiment, the integration router is configured to
automatically connect with each of the plurality of heterogeneous
subnetworks without individual manual configuration of parameters
associated with each of the plurality of heterogeneous
subnetworks.
[0028] In an embodiment, the integration router is configured to
provide dynamic route selection between a first node on one of the
plurality of heterogeneous subnetworks and a second node on another
of the plurality of heterogeneous subnetworks.
[0029] In an embodiment, the integration router is further
configured to automatically and dynamically establish and adjust a
route between the first node on the one of the plurality of
heterogeneous subnetworks and the second node on another of the
plurality of heterogeneous subnetworks.
DRAWINGS
[0030] FIG. 1 illustrates an embodiment of an airborne network
built from mixed IP devices;
[0031] FIG. 2a is a block diagram of hardware of a router
constructed in accordance with an embodiment of the present
invention;
[0032] FIG. 2b is an illustration of the software architecture of a
router constructed in accordance with an embodiment of the present
invention;
[0033] FIG. 3 is a diagram of an airborne network with routers
constructed in accordance with an embodiment of the present
invention;
[0034] FIG. 4 is a block diagram of software architecture of the
present invention;
[0035] FIG. 5 is a conceptual view of a router constructed in
accordance with an embodiment of the present invention containing
multiple interfaces;
[0036] FIG. 6 illustrates the black side of a notional multi-subnet
airborne network;
[0037] FIG. 7 is an illustration of routers constructed in
accordance with an embodiment of the present invention placed in a
multi-subnet airborne network;
[0038] FIG. 8 is a more detailed block diagram of the software
architecture of a router constructed in accordance with an
embodiment of the present invention;
[0039] FIG. 9 is an illustration of an initial state of an
embodiment of multi-subnet network;
[0040] FIG. 10 is an illustration of the multi-subnet network of
FIG. 9 as a UAV joins the network;
[0041] FIG. 11 is an illustration of the multi-subnet network of
FIG. 10 after step 4 of the UAV joining the network;
[0042] FIG. 12 is an illustration of the multi-subnet network of
FIG. 11 with completed alias-tag tables;
[0043] FIG. 13 is an illustration of the multi-subnet network
showing an example of a data path route;
[0044] FIG. 14 is an illustration of the multi-subnet network
showing dynamic failover and rerouting along a different path;
[0045] FIG. 15 is a detailed architecture overview of a router
constructed in accordance with an embodiment of the present
invention;
[0046] FIG. 16 illustrates an alias tag table entry;
[0047] FIG. 17 is an illustration of a completed alias tag
table;
[0048] FIG. 18 is an example of the format of a policy table
entry;
[0049] FIG. 19 is an illustration of the multi-subnet network with
a UAV scenario;
[0050] FIG. 20 is a block diagram of an AV network configuration
illustrating ports available on a router constructed in accordance
with an embodiment of the present invention;
[0051] FIG. 21 is a block diagram of a router constructed in
accordance with an embodiment of the present invention utilized as
a ground control station coupled to the internet;
[0052] FIG. 22 is a block diagram of an embodiment of the software
architecture of a router constructed in accordance with an
embodiment of the present invention;
[0053] FIG. 23 is an illustration of test bed utilizing routers
constructed in accordance with an embodiment of the present
invention;
[0054] FIG. 24 is a diagram illustrating steps utilizing link
quality monitoring and proactive link failover;
[0055] FIG. 25a and FIG. 25b illustrate routers constructed in
accordance with an embodiment of the present invention installed in
a test bed with mini-CDL radios;
[0056] FIG. 26a is a diagram illustrating how data is routed in the
absence of load balancing;
[0057] FIG. 26b is a diagram illustrating data flow balanced across
available links;
[0058] FIG. 27 is an illustration of one scenario of maximized
network performance;
[0059] FIG. 28 is an illustration of a second scenario of maximized
network performance;
[0060] FIG. 29, FIG. 30 and FIG. 31 are diagrams illustrating
subnet convergence function with link quality monitoring;
[0061] FIG. 32 is a diagram illustrating flow control;
[0062] FIG. 33 is a diagram illustrating bandwidth and latency
monitoring;
[0063] FIG. 34 is a diagram illustrating packet loss detection;
[0064] FIG. 35 and FIG. 36 are diagrams illustrating subnet
convergence function utilizing flow control;
[0065] FIG. 37 and FIG. 38 are diagrams illustrating subnet
convergence utilizing traffic redirection;
[0066] FIG. 39 and FIG. 40 are diagrams illustrating quality of
service aware unicast routing service;
[0067] FIG. 41 and FIG. 42 are diagrams illustrating mobility
management function;
[0068] FIG. 43 and FIG. 44 are diagrams illustrating load balancing
function;
[0069] FIG. 45 is a diagram illustrating a test in which redundant
paths between subnetworks are created and on the paths is
impaired;
[0070] FIG. 46 is a diagram illustrating a test which recreates
redundant links and verifies load balancing enhancing multicast
efficiency;
[0071] FIG. 47 is a diagram illustrating a test platform for
network aware, traffic aware; proactive failover and load
balancing;
[0072] FIG. 48 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing an initial state and setup before join;
[0073] FIG. 49 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 1);
[0074] FIG. 50 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 2);
[0075] FIG. 51 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 3);
[0076] FIG. 52 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 4);
[0077] FIG. 53 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 5);
[0078] FIG. 54 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 6);
[0079] FIG. 55 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 7);
[0080] FIG. 56 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 8);
[0081] FIG. 57 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 9);
[0082] FIG. 58 is a diagram illustrating the use case of FIG. 48
when a node joins the network (step 10);
[0083] FIG. 59 is a diagram illustrating the use case of FIG. 48
when a node joins the network showing completed table entries;
[0084] FIG. 60 is a diagram illustrating a use case of routing a
packet (step 1);
[0085] FIG. 61 is a diagram illustrating the use case of FIG. 60
routing a packet (step 2);
[0086] FIG. 62 is a diagram illustrating the use case of FIG. 60
routing a packet (step 3);
[0087] FIG. 63 is a diagram illustrating a use case of routing a
packet;
[0088] FIG. 64 is a diagram illustrating a use case of re-routing a
packet (step 1);
[0089] FIG. 65 is a diagram illustrating the use case of FIG. 64
re-routing a packet (step 2);
[0090] FIG. 66 is a diagram illustrating the use case of FIG. 64
re-routing a packet (step 3);
[0091] FIG. 67 is a diagram illustrating a use case of routing a
packet illustrating a return path;
[0092] FIG. 68 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing a node moving from one TTNT subnet to another
TTNT subnet;
[0093] FIG. 69 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing routing using replication and balancing;
[0094] FIG. 70 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing a link fault while routing data;
[0095] FIG. 71 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing routing of control messages; and
[0096] FIG. 72 is a diagram illustrating a use case of routers
constructed in accordance with an embodiment of the present
invention showing routing of control messages with link
failure.
DESCRIPTION
[0097] The entire content of U.S. Provisional Application Ser. No.
61/935,661, filed Feb. 4, 2014 is hereby incorporated by
reference.
[0098] A hybrid router, an autonomous IP based network integration
solution that provides end-to-end sensor-to-shooter connectivity
across a heterogeneous tactical network is disclosed. This network
consists of IP sub-networks of various types such as TTNT (Tactical
Targeting Network Technology), mini-CDL (Common Data Link), free
space optics communications (FSOC), QNT (Quint Networking
Technology), and JCAN (Joint Capability for Airborne Networking).
These integrated networks provide improved tactical communications
and situational awareness. Network integration is the convergence
of many IP devices (wired, wireless, radio, optical) each forming
IP subnets into a single IP network. Each of the device subnets may
be IP capable on their own, but cannot integrate seamlessly and
automatically with others.
[0099] The Hybrid Autonomous Router for Tactical Networks (HART) is
a self-configuring integration router software module that
interconnects heterogeneous subnets of differing IP devices into a
single routable network implemented for integration with a mini-CDL
radio.
[0100] The HART integration router is a software tool that can be
added into existing commercial routers on an add-on card,
integrated into software routers such as quagga or vyatta, or can
be installed in a network appliance where data is sent through the
appliance. These HART Integration Routers may be installed on
appliances and nodes throughout the black-side of a network.
[0101] Many different wireless networking technologies are
currently used to build airborne networks today. These wireless
networks operate on different frequencies, use different waveforms,
and provide limited autonomous switching and routing capability.
This is shown in FIG. 1.
[0102] Existing approaches for integrating these heterogeneous
wireless networks suffer from four major limitations overcome by
the HART approach: [0103] (5) inability to adapt dynamically to
topology changes in the network; [0104] (6) need for excessive
manual configuration; [0105] (7) lack of scalability with network
size; [0106] (8) inability of routing protocols to factor wireless
link quality into routing decisions, resulting in sub-optimal
network performance.
[0107] HART addresses two major technical challenges that remain
unaddressed by the state of the art to achieve the desired
capability of an autonomous integration router for tactical edge
networks: [0108] 1. Autonomous Mission Persistent Network
Connectivity, i.e., the development of a scalable and automatic
approach for establishing and maintaining end-to-end connectivity
between user nodes in a dynamically changing tactical network
topology with intermittent connectivity to the global information
grid ("GIG"). [0109] 2. Autonomous Mission Optimized Network
Connectivity, i.e. the development of an enhanced tactical routing
approach that is aware of the dynamic changes in the quality of
wireless links within the tactical network and that factors link
quality into its computation of end-to-end routes to optimize
network and mission performance.
[0110] FIG. 2a shows a block diagram of the hardware where the
integration router (HART) contains multiple interfaces, i.e., a LAN
interface connected to a HAIPE-fronted platform LAN and WAN
interfaces connected to different wireless terminals, e.g., TTNT,
QNT, mini-CDL, FSO). Some of these wireless terminals implement
bridged IP subnets (e.g., mini-CDL, FSO) while others implement
routed IP subnets using proprietary routing protocols (e.g., TTNT,
QNT). The software architecture of the HART integration router,
shown in FIG. 2b, consists of a core OSPF routing function with
plug-in router extensions for each type of wireless terminal. The
terminal-specific router extensions implement the HART innovations
listed above to enable performance-optimized routing across the
heterogeneous collections of bridged and routed IP subnets.
[0111] The HART approach has two major beneficial features: [0112]
1. It requires no modifications to the software of existing
tactical IP radios and optical modems to deliver its network
integration and enhanced tactical routing capabilities. This
facilitates rapid deployment of the technology in the field by
eliminating the need to coordinate software changes with the
various vendors of wireless IP terminals. [0113] 2. It lends itself
to a modular implementation architecture where the HART mechanism
and techniques are implemented as plug-in software modules that can
either be (1) integrated within existing software routers (e.g.,
quagga, XORP); or (2) executed on add-on processor modules for
commercial hardware routers (e.g., within an AXP card for a Cisco
router); or (3) integrated within commercially-available
programmable high-speed packet processing appliances (e.g.,
EZappliance).
[0114] Underlying the HART approach for network integration and
enhanced tactical routing is a set of innovative techniques and
mechanisms that will be implemented within software module that can
be installed either as an add-on to existing routers, or as a
network appliance within existing networks. To address the
challenges and capability gaps described above, the HART software
may provide these capabilities: [0115] Hybrid routing; [0116]
Quality-aware routing; [0117] Integration with existing
Sub-networks.
[0118] Hybrid Routing
[0119] HART employs a hybrid routing approach to establish and
maintain end-to-end connectivity between nodes in a mobile tactical
network environment. It employs a direct routing approach for
connecting nodes within the tactical AS (Autonomous Systems) and
uses a mobile IP based overlay routing approach with tactical
enhancements for connecting tactical nodes to nodes in the GIG. The
enhanced Mobile IP based technique eliminates the packet
encapsulation overhead associated with overlay routing when packets
traverse the bandwidth-constrained tactical user sub-nets. The HART
Hybrid routing addresses both: [0120] Intra-Tactical-AS Routing;
[0121] Routing between the Tactical-AS and the GIG.
[0122] Quality-Aware Routing
[0123] The proposed approach for enhanced tactical routing is based
on dynamic route selection using link quality and using optical and
other link types as failover links when primary links failed or
degraded. This is now expanded to become more generic to allow more
flexibility for Quality aware routing. HART tracks link quality
using network monitors, link probes and other techniques. This
quality-aware link monitoring will be used to calculate a Link
Quality Metric (LQM). The LQM will be stored in HART and updated in
the router on the HART machine adding quality-aware routing to
existing routers (which do not have a way to monitor and probe the
link quality).
[0124] Integration with existing Sub-networks
[0125] The HART module integrates with existing subnets and
gateways (e.g.
[0126] JCAN: Joint Capability for Airborne Networking and BACN:
Battlefield Airborne Communications Node)as a separate AS
(Autonomous Systems). It uses the innovative BGP-MX (BGP with
Mobility Extensions) service to exchange network reachability
information with these other ASes. BGP-MX overcomes static manual
configuration of BGP peering arrangements by implementing
mechanisms that enable transit routers in a mobile AS environment
to (1) dynamically discover BGP peers from a different AS and
automatically establish a peering relationship; and (2) converge
rapidly to a new route in response to changes in the topology of
the backbone networks.
[0127] Discussion
[0128] The HART approach can be used for network integration and
enhanced tactical routing in a heterogeneous tactical networking
environment. The HART approach may be used to enhance the
following: [0129] Analyze Network Integration Requirements; [0130]
Develop Design of Protocols and Services; [0131] Evaluate
Performance using Simulations; [0132] Develop Product
Implementation Architecture.
[0133] Analysis
[0134] Use cases of diverse multi-subnet environments and
situations have been created such as: nodes joining or leaving a
network; or nodes moving from one network (using TTNT) to another
network (using Mini-CDL). Other use cases involved requested data
flows from the ground to an aerial node, the reverse, and data
flows from aerial node to aerial node. Some of these Use Cases are
included in Appendix B below.
[0135] In various embodiments, HART solves: [0136] No changes to
existing radios and COTS routers; [0137] Provide GIG to Airborne
Network (AN) endpoint connectivity (and reverse); [0138] Provide
GIG to Airborne Network (AN) endpoint connectivity (and reverse);
[0139] Provide quality aware link and route selection.
[0140] These use cases were analyzed to determine what an
integration router monitors, and what network dynamics are tracked,
and what options are configured in existing hardware (routers and
radios).
[0141] The HART design is refined by analyzing several real-world
use cases of multi-subnet environments with different radio and
routing capabilities.
[0142] After the design was refined, a subset of functionality was
selected to create a prototype to demonstrate the HART approach.
The features selected for prototyping were proactive failover
across many links and integration with a Mini-CDL radio and
interface. With the proactive failover mechanism, the HART router
monitors the quality of each radio link to detect when it falls
below a specified threshold. Upon detection of unacceptable link
degradation, the HART router proactively routes all IP packet flows
traversing the degraded link to a backup link of higher quality. If
or when the original link recovers its quality, the IP packet flows
are switched back to this link. Current systems provide no
capability for proactive link failover; failover occurs only after
a complete breakdown of a link. Application performance suffers as
a consequence. In addition to proactive link selection, the HART
prototype implements multi-level policy-based failover (1, 2, 3, 4,
5 or more backup links in a prioritized order), a capability that
is not supported within COTS routers. For instance, with
conventional routers, if the policy route for an IP flow were to
fail it will be routed over the default interface. Unlike HART,
secondary and tertiary policy routes cannot be specified with
conventional routers.
[0143] HART Overview
[0144] Network integration is the convergence of many IP devices
(wired, wireless, radio, optical) each forming IP subnets into a
single IP network. Each of these device subnets may be IP capable
on their own, but cannot integrate seamlessly and automatically
with others. Appendix A outlines the list of IP device targets, the
interface method, and device subnet integration.
[0145] HART software provides heterogeneous network integration.
HART addresses these problems by integrating the following
techniques into Autonomous Integration Routers: [0146] Hybrid
routing; [0147] Dynamic subnet domain name service; [0148]
Quality-aware link monitoring and routing; and [0149]
Mobility-enhanced BGP (or BGP-MX).
[0150] These HART Integration Routers may be installed throughout
the black-side network, as shown in FIG. 3.
[0151] The HART integration router is a software tool that can be
added into existing commercial routers on an add-on card,
integrated into software routers such as quagga or vyatta, or can
be installed in a network appliance and where data is sent through
the appliance.
[0152] The HART software module has six components shown in 4. FIG.
4 also shows the data flow between each of these components. The
components are: [0153] Topology and Address Tracking (TAT)--Dynamic
Address Map; [0154] Autonomous Router Configuration (ARC) Module;
[0155] Device Configuration Interfaces (DCI); [0156] HART Policy
Editor; [0157] BGP-MX module; [0158] Router.
[0159] Core Network Integration Requirements
[0160] This section describes what configuration options to route
IP packets on tactical networks. Each end node has an IP address
and default route/router and the routers track routes to other
subnets.
[0161] The configurations for each node are: [0162] IP address;
[0163] Default route/default router/gateway; [0164] Subnet mask and
settings; [0165] DNS settings (server/proxy addresses).
[0166] The configurations for each subnet are: [0167] Default
router; [0168] List of IPs; [0169] DNS settings; [0170] Address
allocation.
[0171] HART Components
[0172] The overall system for HART components: [0173] Both IPv4 and
IPv6; [0174] Compatibility, to work with: [0175] Existing
software/servers/systems: firewalls, proxy servers, caches,
switches, routers and appliances, HAIPEs, gateways, filtering
tools/appliances/apps, etc.; [0176] Hardware vendors: security and
network vendors, including: Cisco, Sun, Microsoft, Check Point, and
3com; [0177] Applications: ISR, C2 and targeting (sensors to
shooters); [0178] Various data types: voice, data, video, sound and
security; [0179] Scalability: 1000's of nodes, across 10's or 100's
of subnets and AS's, all mobile and moving; [0180] Work in
red/black networks.
[0181] The topology tracker and routing component integrates with
the various monitors to track topology and make routing decisions.
[0182] Integrate with monitors; [0183] IP address assignment, if
appropriate; [0184] Coordinate across the entire hybrid
tactical-AS; [0185] IPv4 & IPv6 address spaces; [0186] Push
address changes to node configuration writers; [0187] Routing: make
dynamic route decisions; [0188] Route from 1 subnet type to another
(TTNT to Mini-CDL); [0189] Leverage commercial router practices
that provide a common switching fabric while interchanging line
cards operating at different data rates (i.e. T1, DS-3, OC-192);
[0190] Subnet handoff (node moves from TTNT to QNT, or moves from
TTNT and adds mini-CDL, and other use cases); [0191] Perform
multi-link, multi-channel, multi-antenna multi-path routing and
communications topology configurations; [0192] Perform static
routing (mesh, ring, hub and spoke) between multiple subnets;
[0193] Perform mobile ad hoc routing between multiple subnets node
join/leave, net join/leave; [0194] Provide a way to provision
tunnels/links/flows; [0195] Track a link/flow detail record
(FDR).
[0196] Monitors integrate with the other components to provide
network monitoring data: [0197] Node trackers--SNMP readers
(routes, address, location, etc.); [0198] DNS monitor--(new IP's
linked to old names, IP changes for names, red/black concerns, DNS
probes, etc.); [0199] Link quality probes.
[0200] Node configuration writers provide the machine-to-machine
interface for automatic configuration. It integrates with the other
components to provide a single interface to write configuration
options and settings to local and remote routers and wireless IP
devices. These writer components may use SNMP and/or device
specific API's; [0201] Provide a single option writing API to other
components; [0202] Automatic configuration--HART without human
involvement; [0203] Local and remote programmatic (API)
configuration options to set various IP settings [programmatic
control of each radio/device/technology]; [0204] Use SNMP and/or
other protocols/services; [0205] May include radio/device specific
API requirement.
[0206] Dynamic DNS proxy integrates with the other components to
track DNS names, changes to DNS entries, and act as a DNS proxy to
respond to DNS requests with dynamic responses; [0207] Integrate
with DNS monitors; [0208] DNS name tracking; [0209] HART should act
as a proxy for a DNS server; [0210] Track DNS name and link to 1 or
all IPs for node (each subnet may have a different IP or range);
[0211] Link Mobile IP "permanent IP" to tactical-AS IP's and DNS
name; [0212] Works in Red/black networks (Black side only, but may
be aware of red side effects).
[0213] Quality aware proxy monitors the various links and paths
connected to HART and across the network, then update the link
metrics so that the routers can make smarter decisions: [0214]
Integrate with link quality monitors; [0215] Monitors link quality,
bandwidth, latency, packet loss, # of retransmits, usage level,
available bandwidth (total-used), etc. run these through a formula
to produce a "quality metric"; [0216] Characterize static link and
network performance, bit error rate, packet loss; [0217] Implement
an RFC 5578 proxy; [0218] Initiate link-quality measuring probes;
[0219] Make/help with routing decisions to optimize based on link
quality; [0220] Develop, demonstrate and analyze link fade, optical
polarization rotation, pointing and tracking, antenna gain, link
margins, bit error rates; [0221] Update link metrics in local and
remote routers and devices.
[0222] BGP-MX
[0223] HART treats other network integration system as separate
ASs. This provides automatic configuration and integration with
other networking systems in use [0224] Integrate seamlessly with
BGP routers [0225] Dynamically discover BGP peers from other
as--and setup peering relationships [0226] Converge new routes
rapidly in response to changes in topology [0227] Integrate with
external as--such as: [0228] JCAN: Joint Capability for Airborne
Networking subnets [0229] GIG: Global Information Grid
(backhaul/reachback) [0230] BACN: Battlefield Airborne
Communications Node [0231] DISN: Defense Information System
Network
[0232] The HART addresses at least two major technical challenges
to achieve desired capability of an autonomous integration router
for tactical edge networks: [0233] Autonomous Mission Persistent
Network Connectivity, i.e. the development of a scalable and
automatic approach for establishing and maintaining end-to-end
connectivity between user nodes in a dynamically changing tactical
network topology with intermittent connectivity to the GIG. [0234]
Autonomous Mission Optimized Network Connectivity, i.e. the
development of an enhanced tactical routing approach that is aware
of the dynamic changes in the quality of wireless links within the
tactical network and that factors link quality into its computation
of end-to-end routes to optimize network and mission
performance.
[0235] Underlying the HART approach for addressing these challenges
is a set of techniques that are listed in Table 1 below and
described later.
TABLE-US-00001 TABLE 1 Technical Challenges HART Innovations
Autonomous Mission Hybrid routing with address switching Persistent
Connectivity Autonomous Mission Autonomous quality aware routing
Optimized Connectivity
[0236] FIG. 5 shows a conceptual view of the HART solution where
the integration router contains multiple interfaces, i.e., a LAN
interface connected to a HAIPE-fronted platform LAN and WAN
interfaces connected to different wireless terminals, e.g., TTNT,
QNT, mini-CDL, FSO). Some of these wireless terminals implement
bridged IP subnets (e.g., mini-CDL, FSO) while others implement
routed IP subnets using proprietary routing protocols (e.g., TTNT,
QNT). The software architecture of the HART integration router,
shown in Error! Reference source not found., consists of a core
OSPF routing function with plug-in router extensions for each type
of wireless terminal. The terminal-specific router extensions
implement the HART innovations listed above to enable
performance-optimized routing across the heterogeneous collections
of bridged and routed IP subnets.
[0237] The HART approach has two major beneficial features: [0238]
It has no modifications to the software of existing tactical IP
radios and optical modems to deliver its network integration and
enhanced tactical routing capabilities. This facilitates rapid
deployment of the technology in the field by eliminating
coordination of software changes with the various vendors of
wireless IP terminals. [0239] It lends itself to a modular
implementation architecture where the HART mechanism and techniques
are implemented as plug-in software modules that can either be (1)
integrated within existing software routers (e.g., quagga, XORP);
or (2) executed on add-on processor modules for commercial hardware
routers (e.g., within an AXP card for a Cisco router); or (3)
integrated within commercially-available programmable high-speed
packet processing appliances (e.g., EZ appliance).
[0240] In an overview, HART software will provide heterogeneous
network integration. The HART approach for addressing these
problems will integrate the following innovative techniques into
Autonomous Integration Routers: [0241] Hybrid routing Dynamic
Topology and Address Tracking [0242] Quality-aware link monitoring
and routing [0243] Mobility-enhanced BGP (or BGP-MX)
[0244] HART Hybrid routing enables OSPF-based dynamic route
discovery and maintenance in a mobile ad hoc airborne network with
a diverse set of bridge IP subnets (mini-CDL, FSOC) and routed IP
subnets (TTNT, QNT). HART uses OSPF [Boe06, Ci05] between HART
nodes and then export routes to (and import routes from) existing
routers and radio hardware that may implement proprietary routing
protocols. This is in contrast to current solutions that use
tunneling of data packets across subnets such as TTNT and thereby
incurring excessive network overhead. To eliminate the overhead
associated with tunneling data packets across routed IP subnets,
such as TTNT, HART will use a novel packet forwarding technique
called "address switching" for TTNT, QNT and other tactical routed
IP subnets.
[0245] To support address switching, HART tracks the names and
addresses of node as they join and leave the various subnets. This
data is used to track aliases (names or labels) of the nodes in the
network as they move. This data forms a topology of the network
that HART uses to augment the routing within the network. The
topology information is similar to a routing table and is stored as
"alias--tag" entries which form the Topology and Address Tracking
(TAT) Database. HART uses this alias-tag table along with HART
policies to determine routes to use and configuration changes to
make on the routers. The policies define link augmentations like
replicate data across two or more links to provide for redundancy,
or failover priority if links fail or degrade.
[0246] In addition to topology data, HART tracks link quality using
network monitors, link probes and other techniques. This
quality-aware link monitoring is used to calculate a Link Quality
Metric (LQM). The LQM is stored in HART and updated in the router
on the HART machine adding quality-aware routing to existing
routers (which do not have a way to monitor and probe the link
quality).
[0247] In tactical and airborne networks other domains (groups of
subnets) form
[0248] ASes (Autonomous Systems). ASes use a protocol called BGP to
route between ASes. BGP does not support mobility and dynamic
configuration. BGP-MX is a mobility extension for BGP to provide
support for mobility and automatic discovery.
[0249] The HART Integration Routers are installed on appliances and
nodes throughout the black-side (CT side) of a network. FIG. 6
shows the black-side of a notional, multi-subnet airborne network.
FIG. 7 shows where HART routers may be installed in the network of
FIG. 6.
[0250] HART Routers come in three flavors (shown in Error!
Reference source not found.): [0251] HART-Edge (HART-E): These HART
Appliances are installed on each edge node in the network and will
route data to and from an endpoint, but not as an intermediate hop
in a multi-hop path. These appliances track the local nodes on the
platform, and routes and default routers to other networks. [0252]
HART-Interior (HART-I): These HART Appliances are installed on
backbone nodes and provide routing to and from nodes. These nodes
will use more storage for extensive tables to track nodes. HART-I
nodes will forward data about reachability of edge nodes, addresses
and status to the HART-Gateway nodes. [0253] HART-Gateway (HART-G):
Same functions as HART-Interior, and also provides links and routes
back to the GIG, Internet or other networks. The gateway appliances
track the nodes around the network and keep the dynamic address
links updated and current.
[0254] The HART suite of components is shown in the list below and
in Error! Reference source not found. The figure shows the data
flow between each of these components. The components are: [0255]
Topology and Address Tracking (TAT)--Dynamic Address Map [0256]
Autonomous Router Configuration (ARC) Module [0257] Device
Configuration Interfaces (DCI) [0258] HART Policy Editor [0259]
BGP-MX module [0260] Router
[0261] Use case: How HART routes packets
[0262] To route data through a network of mixed subnets, HART
monitors may track the nodes and build tables of the addresses and
how to get from subnet to subnet (Topology and Address Tracking).
These tables are called alias-tag tables. The alias-tag table is
similar to a routing table, and will be used by HART for routing
and address switching. The entries in the alias-tag table contain a
destination address (or subnet address), a next hop address (the
next hop from the local network to get packets to the final
destination address), number of hops, the Link Quality Metric
(LQM), and the capacity.
[0263] When a network is first set up, HART initializes itself.
Then as nodes join or leave, HART updates the alias-tag tables
throughout the network. Below is a use case of how this is
done.
[0264] Initial setup process (FIG. 9): [0265] HG1 detects GIG and
declares "I'm a gateway router" [0266] Broadcasts itself as
"default route" to GIG. [0267] HG2 detects GIG and declares "I'm a
gateway router" [0268] Broadcasts itself as "default route" to GIG.
[0269] HI1 detects no GIG and declares "I'm an Interior router"
(default) [0270] Receives "default route" from JALN backbone (HG1
and HG2, for specified subnets) [0271] HI2 detects no GIG and
declares "I'm an Interior router" (default) [0272] Receives
"default route" from JALN backbone (HG1 and HG2, for specified
subnets) [0273] Becomes "default router" for TTNT radio subnet to
link them to GIG (thru JALN backbone)
[0274] When a Node Joins the network (Error! Reference source not
found.), HART routers monitor and probe the network to test links
and configure the alias-tag tables (Error! Reference source not
found.) that will be used for address switching later. The process
is described below.
[0275] Step 1: [0276] Establish each link from that node to the
existing network [0277] Add alias-tags at various endpoints in the
network to enable address switching (faster than routing) (enabled
by HART dynamic address and label tracking)
[0278] Step 2: [0279] SatCom connects: Uses fixed SatCom IP to
connect to the other SatCom endpoint (12.1.4.8) [0280] HE1 adds an
alias-tag for 17.0.0.0->12.1.4.8 [0281] HG1 adds an alias-tag
for 17.1.2.3->12.1.3.1
[0282] Step 3: [0283] First Mini-CDL Radio connects [0284]
Establishes "link local" addresses on both endpoints [0285] HE1
adds an alias-tag for 17.0.0.0->224.1.1.1 [0286] HG1 adds an
alias-tag for 17.1.2.3->224.1.1.2
[0287] Step 4 (FIG. 11): [0288] Second Mini-CDL Radio connects
[0289] Establishes "link local" addresses on both endpoints [0290]
HI1 requests 224.1.1.2 first, but HE1 already is using, so responds
with 224.1.1.3 [0291] HE1 adds an alias-tag for
17.0.0.0->224.1.1.1 [0292] HI1 adds an alias-tag for
17.1.2.3->224.1.1.3 [0293] HI1 forwards alias-tag to HG1, HG1
adds it as:17.1.2.3->10.1.2.1 (green arrow in FIG. 11) [0294]
HI1 adds reverse alias-tag
[0295] Step 5: [0296] TTNT Radio connects [0297] Uses fixed TTNT IP
to connect to the TTNT cloud [0298] Finds default router in TTNT
cloud (HI2)->sets default router to HI2 [0299] HE1 adds an
alias-tag for 17.0.0.0->10.71.103.2 [0300] HI2 adds an alias-tag
for 17.1.2.3->10.77.91.1 [0301] HI2 forwards alias-tag to
HG1:17.1.2.3->10.2.2.1 [0302] HI2 addsreversealias-tag
[0303] The completed alias-tag tables are shown in Error! Reference
source not found.
[0304] Packet Routing and Rerouting
[0305] To route data from H2 (Gnd) to H1 (Air):
[0306] Packet leaves H2 [0307] Packet: Src=19.1.2.3; Dst=17.1.2.3;
Data [0308] H2.fwdarw.GIG.fwdarw.HG1
[0309] HG1 looks up H1 [0310] Next hop=224.1.1.2(M-CDL): <=1
hop.fwdarw.no address switching [0311] Packet: Src=19.1.2.3;
Dst=17.1.2.3; Data [0312] HG1.fwdarw.M-CDL.fwdarw.HE1
[0313] HE1 receives [0314] Packet: Src=19.1.2.3; Dst=17.1.2.3; Data
[0315] HE1.fwdarw.H1 [0316] See FIG. 13
[0317] But then an error causes the M-CDL1 link to fail. The flow
(H2 (Gnd) to H1 (Air)) is rerouted:
[0318] HG1 looks up next path to H1 [0319] Next hop=10.1.2.1(JALN):
2 hop.fwdarw.address switching [0320] Packet: Src=19.1.2.3;
Dst=10.1.2.1; (JALN); Dst`=17.1.2.3; Data [0321]
HG1.fwdarw.JALN.fwdarw.HI1
[0322] HI1 looks up H1 [0323] Next hop=224.1.1.3(M-CDL): <=1
hop.fwdarw.no address switching [0324] Restore packet:
Src=19.1.2.3; Dst=17.1.2.3; Data [0325]
HG1.fwdarw.M-CDL.fwdarw.HE1
[0326] HE1 receives [0327] Packet: Src=19.1.2.3; Dst=17.1.2.3; Data
[0328] HE1.fwdarw.H1 [0329] See FIG. 14
[0330] HART Router Features
[0331] The above use case illustrates the HART functionality to
provide dynamic network convergence. To do this HART uses three
flavors of the Integration router: [0332] HART-Edge Routers [0333]
HART-Interior Routers [0334] HART-Gateway Routers
[0335] The HART-Edge Routers have these features: [0336] Reads and
acts on HART Policies. [0337] Local topology and link quality
monitors (SNMP, and other APIs). [0338] Autonomous device, router
and radio configuration (SNMP, and other APIs). [0339] Forwards
data to HART-Interior nodes using (address switching, IP routing,
bridging, and repeating).
[0340] The HART-Interior Routers, same as HART-E, plus these
features: [0341] Same as HART-Edge Routers. [0342] Autonomous data
switching and routing (address switching, IP routing, bridging, and
repeating). [0343] Advanced routing/switching to other subnets and
hosts. [0344] Ability to setup/configure tunnels. [0345]
Regional/Domain monitoring of node availability, topology (next
hop, path), names, connection status, and quality (SNMP, and
other). [0346] Stores this data in an internal table to use for
routing/switching data. [0347] Shares this data with neighbor HART
routers. [0348] Capable of using OSPF and other generic or
"default" routing algorithms (enhanced with link quality
metrics).
[0349] The HART-Gateway Routers, same as HART-I, plus these
features: [0350] Same as HART-Interior Routers. [0351] Collects and
Tracks IP addresses and next-hop information for nodes on the
hybrid network. This data is used to update the dynamic address
links so nodes outside the dynamic hybrid network can reach the
nodes as they move and shift addresses within the hybrid network.
[0352] BGP and BGP-MX mobility extensions to interconnect (link,
route, switch) to external networks (GIG, other Autonomous Systems
[ASes]).
[0353] HART Architecture Overview
[0354] Each of the three flavors of routers is built from the same
core components. These components are shown in Error! Reference
source not found. and described in further detail below.
[0355] HART Monitors
[0356] This is the HART Topology and Address Tracking system. The
HART monitors are a collection of monitoring and capture tools that
gather data about the network, nodes, links, topology, quality,
etc. These tools use various methods to collect and gather this
data from many sources: SNMP, Radio APIs, network probes, etc. As
this data is collected it is stored in two databases. The first
stores the majority of monitored data (topology, link quality
metrics, etc.) the second is the address map database that stores
the alias-tag tables that are used for routing. The address map is
constantly maintained and kept small to enable fast routing
lookups. [0357] Node trackers--SNMP readers (routes, address,
location, etc.) [0358] Address and label monitor--(new IP's linked
to old/existing names, IP changes for names, red/black concerns,
DNS probes, etc.) [0359] Link quality probes [0360] Monitor link
quality, bandwidth, latency, packet loss, # of retransmits, usage
level, available bandwidth (total-used), etc. These are combined
through a formula to determine a Link Quality Metric (LQM) [0361]
Characterize static link and network performance, bit error rate,
packet loss.
[0362] As nodes are discovered they will be added to the tracking
data by using "Node Join" commands.
[0363] Dynamic address mapping and topology tracking module
integrates with the monitor and database components to follow
dynamic mobile nodes and update the mappings of links to the fixed
addresses of edge platforms. This allows GIG connected nodes to
find and route traffic to dynamic mobile end points with minimal
overhead. This service is also used to track and link DNS names of
mobile nodes with alternative IP addresses (maintained by HART) to
reach those nodes. [0364] Integrate with HART monitors [0365] Store
most current data about network status, IP addresses of nodes, link
status and link quality [0366] Track link/flow detail records
[0367] Topology and Address Tracking--Dynamic Address Mapping and
name tracking [0368] Track a platforms "permanent IP" and DNS names
(through different radio subnets) [0369] Link "permanent IP" to 1
or all dynamic IPs within the tactical-AS for the node (each radio
subnet may have a different IP or range) [0370] HART should act as
a proxy for a DNS server requests [0371] Level and amount of
Topology and Address Tracking data stored by HART on a node depends
on if the node is a HART-Edge (next hops, default routes),
HART-Interior (region or AS based data), or HART-Gateway (GIG
scale, multi-AS, very large scale) [0372] Works in Red/black
networks (Black side only, but be aware of red side effects)
[0373] Alias Tag Table
[0374] Each HART node maintains a table of "alias tags" (address
labels or "next hops") to reach specific end points or subnets.
Alias tags are the name of the labels used for the address
switching done by HART. These tags are also used as routes to nodes
and subnets.
[0375] Each minimal entry in the table has a: [0376] Destination
[0377] Local radio or subnet to use [0378] Link Quality Metric
(LQM) [0379] Link/route Capacity (or bandwidth)
[0380] An alias-tag entry is shown in FIG. 16.
[0381] The LQM is calculated based on some combination of number of
hops to get to the destination through that radio subnet (H); and
expected latency to get to the destination through that radio
subnet (L) calculated over some time period. The H, L and capacity
values are the minimal values for link quality selection. Other
values that may be used include: [0382] Name (Node name, DNS, etc.)
[0383] Location (lat., long.) [0384] Other link quality metrics,
such as: (ave. packet loss, ave. # of retransmits, bit error
rate)
[0385] HART-E routers maintain only a limited table of how to
connect to the larger network and default routers (or the closest
HART-I or HART-G router), the format of this data is shown below in
FIG. 17. HART-I and HART-G routers maintain entries that allow
cross routing and switching between radio subnets without requiring
to route through a GIG node.
[0386] (Note: H, L, B will be replaced with LQM and Capacity)
[0387] The HART autonomous router configuration (ARC) module uses
OSPF and the information from the TAT Database to make routing
decisions and to auto configure various aspects of the network and
resources. This module is able to send data by routing, address
switching, bridging or repeating. It is able to replicate and load
balance data across multiple links as well. This module also
integrates with existing routers through OSPF, RIP or other
standards. [0388] Integrate with (use) Topology and Address
Tracking tables (alias-tags). [0389] Act autonomously using
policies as framework. [0390] IP address assignment if appropriate
[0391] Coordinate across the entire hybrid tactical-AS [0392] IPv4
& IPv6 address spaces [0393] Push address changes to node
configuration writers [0394] Able to send data by address
switching, routing, tunneling, bridging or repeating [0395] Make
dynamic data forwarding decisions [0396] Implement OSPF between
HART nodes [0397] Route from 1 subnet type to another (TTNT to
Mini-CDL) [0398] Leverage commercial router practices (OSPF, RIP or
other standards) that provide a common switching fabric while
interchanging line cards operating at different data rates (i.e.
T1, DS-3, OC-192) [0399] Develop and demonstrate static routing
(mesh, ring, etc.) between subnets [0400] Develop and demonstrate
mobile ad hoc routing between multiple subnets node join/leave, net
join/leave [0401] Subnet handoff (node moves from TTNT to QNT, or
moves from TTNT and adds mini-CDL, and other use cases) [0402]
Replicate (increase reliability) and load balance data (increase
throughput) across multiple links. [0403] Develop, demonstrate and
analyze multi-link, multi-channel, multi-antenna multi-path routing
and communications topology configurations [0404] Autonomously
provision tunnels/links/flows [0405] Use quality aware extensions
[0406] Use monitored link quality data to update the link metrics
of routing protocols (OSPF, etc.) so routers (non-HART also) can
optimize based on link quality [0407] Develop, demonstrate and
analyze link fade, optical polarization rotation, pointing and
tracking, antenna gain, link margins, and bit error rates. [0408]
Update link metrics in local and remote routers and devices. [0409]
Integrate with existing routers through OSPF, RIP or other
standards.
[0410] HART routing priority and format [0411] Is next hop <2
hops? [0412] Y: Send direct to next hop. [0413] N: Does "next hop"
support HART address switching? [0414] Y: address switch to next
hop. [0415] N: Tunnel to next hop.
[0416] HART address switched packet format
[0417] Original Packet:
##STR00001##
[0418] Address switched packet (option 1):
##STR00002##
[0419] NDst--Next hop destination
[0420] Dest`--Original destination
[0421] Overhead: Adds>=5 bytes--Turn on options, then add the
Dest` address (4 bytes) (More in IPv6)
[0422] Address switched packet (option 2):
##STR00003##
[0423] NDst--Next hop destination
[0424] HAS-flag--Some special HART set of IP options to indicate a
HART address switched packet OR a specific DSCP value OR an IP
Protocol code (or combination of these indicators)
[0425] Dest`--Original destination
[0426] Overhead: Adds 4 bytes--Length of the Dest` address (More in
IPv6)
[0427] NOTE: Where possible HART will learn and save next hops
(paths and routes) to be used for a conversation. This will remove
the need to include the Dest` field in every packet, removing the
overhead from later packets in the conversation.
[0428] HART Tunneling packet format
[0429] Original Packet:
##STR00004##
[0430] Tunneled Packet:
##STR00005##
[0431] NDst--Address of "Next hop" HART router
[0432] HAS-flag--Some special HART set of IP options to indicate a
HART tunneled packet OR a specific DSCP value OR an IP Protocol
code (or combination of these indicators)
[0433] Overhead: Adds at least 40 bytes--full original IP packet is
wrapped inside a new IP packet (More in IPv6)
[0434] Inter-HART communication commands
[0435] Inter-HART communication commands define any HART to HART
messages to share data.
[0436] Leave/join update message
[0437] When a node leaves one subnet or joins a new subnet, the
HART routers send an update. This update will be sent on the old
subnet after a timeout period. Another update will be sent on the
new subnet after a join is completed. These updates will serve the
purpose of informing the HART-I and HART-G routers of where edge
and interior nodes can be found (after moving), and any new or
updated paths to get to those nodes or quality metrics along the
paths.
Node Join
[0438] Destination node [0439] HART-E router for node [0440] Route
(Next hop) (or NULL for endpoint) (this gets filled after the first
hop) [0441] #Hops (increment for each hop) [0442] Latency [0443]
Bandwidth [0444] TBD--Other Quality Metrics
Node Leave
[0444] [0445] Route/path Drop flag [0446] Destination node [0447]
HART-E router for node
[0448] HART Device Configuration Interfaces
[0449] HART device configuration interfaces provide the
machine-to-machine interface for automatic configuration. It
integrates with the other components to provide a single interface
to write configuration options and settings to local and remote
routers and wireless IP devices. These writer components may use
SNMP and/or device specific APIs. [0450] Provide a single
configuration API to other components [0451] Automatic
configuration--HART without human involvement [0452] Local and
remote programmatic (API) configuration options to set various IP
settings [programmatic control of each radio/device/technology]
[0453] Use SNMP and/or other protocols/services [0454] May include
radio/device specific API requirement
[0455] Device Configuration commands
[0456] Basic IP configuration of devices use (in an embodiment,
minimally) the commands described below. HART defines a single,
unified API to connect and set these commands for each radio device
supported by HART (Mini-CDL radios, SNMP devices, TTNT, etc.).
Set/Get IP Address
[0457] Get/set flag [0458] Interface to get/set address [0459]
Address [0460] Subnet Mask [0461] Default Router
Set/Get Route
[0461] [0462] Get/set flag [0463] # of Route to set (Entry # in a
table, 0=default route) [0464] Destination (address or subnet)
[0465] Route (Next hop) Set/Get DNS settings (server/proxy
addresses) [0466] Get/set flag [0467] Primary DNS Server (or HART
DNS Proxy) [0468] Secondary DNS Server (or HART DNS Proxy) Set/Get
IP Address Allocation settings (DHCP) [0469] Get/set flag [0470] IP
range Start address [0471] IP range Stop address [0472] Subnet Mask
of Range [0473] Default Router of Range [0474] Primary DNS Server
(or HART DNS Proxy) of Range [0475] Secondary DNS Server (or HART
DNS Proxy) of Range
[0476] HART Policy Editor
[0477] The HART Policy Configuration GUI allows the user to setup
and maintain the policies used by the HART routers. This tool
allows the user to define the default link for traffic and the
order of backup or failover links to use. Once the default is
setup, the user can specify different traffic types based on
matching DSCP (differentiated services code point) and/or protocol
ID fields. For each traffic type a primary link can be selected and
then the other links can be ordered as to which link(s) will be
used to failover that traffic type.
[0478] A policy option in the HART prototype is multi-link
forwarding. For a specified traffic type (specific DSCP and/or
protocol ID) multiple links can be specified to replicate packets
on. This option sends the same packet or data frame across multiple
links to provide improved reliability. If the primary link fails
the data will not be lost or interrupted, the flows will continue
across the other specified links without affecting the data flow at
all.
[0479] Policy types: [0480] Default Policy: primary and backup
links for all non-specified traffic. [0481] Automatic Failover
Policy: primary and backup links for all specific traffic. Specific
traffic defined by DSCP and protocol ID fields. [0482] Multi-Link
Forwarding Policy: specify primary link and replication links to
replicate specific traffic on. Specific traffic defined by DSCP and
protocol ID fields. [0483] Load Balancing Policy: specify group of
links to spread specific data across (not replicate). Each link
will be used in a rotating fashion. Different data packets will be
sent simultaneously across several links arriving at the same time.
This has the effect of increasing throughput. Specific traffic
defined by DSCP and protocol ID fields. Example: [0484] If DSCP==18
(AF21) then PrimaryLink=Mini-CDL1 and
FailoverLink0rder=AN/ARC-210(V); Mini-CDL2; WGS; Inmarsat [0485]
Provide a GUI tool to edit policies [0486] Write policy files.
[0487] Read existing policy files
[0488] Policy Table entry [0489] Each Policy will have these
values: [0490] DSCP value (or NULL) [0491] And/or flag (0-AND,
1-OR) [0492] IP Protocol Code (or NULL, especially if DSCP is NULL)
[0493] Policy Type (0-Default Policy; 1-Automatic Failover Policy;
2-Multi-Link Forwarding Policy; 3-Load Balancing Policy) [0494]
Primary Link Identifier [0495] Secondary Link Identifier Priority
List
[0496] BGP-MX Module
[0497] HART treats other network integration systems as separate
ASes. This provides automatic configuration and integration with
other networking systems in use. An extension to BGP is used to add
mobility awareness and dynamics.
[0498] BGP-MX: [0499] Integrates seamlessly with BGP routers [0500]
Dynamically discover BGP peers from other as--and setup peering
relationships [0501] Converge new routes rapidly in response to
changes in topology [0502] Integrate with external AS's such as:
[0503] JCAN: Joint Capability for Airborne Networking subnets
[0504] GIG: Global Information Grid (backhaul/reachback) [0505]
BACN: Battlefield Airborne Communications Node [0506] DISN: Defense
Information System Network
[0507] HART Design Details
[0508] HART is designed for several real-world use cases of
multi-subnet environments with different radio and routing
capabilities.
[0509] A subset of functionality was selected to create a prototype
to demonstrate the HART approach. The features selected for
prototyping were proactive failover across many links and
integration with a Mini-CDL radio and interface. With the proactive
failover mechanism, the HART router monitors the quality of each
radio link to detect when it falls below a specified threshold.
Upon detection of unacceptable link degradation, the HART router
proactively routes all IP packet flows traversing the degraded link
to a backup link of higher quality. If or when the original link
recovers its quality, the IP packet flows are switched back to this
link. Current systems provide no capability for proactive link
failover; failover occurs only after a complete breakdown of a
link. Application performance suffers as a consequence.
[0510] As shown in FIG. 19, the UAV platform has two Mini-CDL
radios, an AN/ARC-210(V) radio, an Inmarsat (BGAN) SatCom and a WGS
SatCom. The GCS (Ground Control Station) in this case will have the
same five endpoint radios. HART in this case exploits the redundant
data paths to provide increased reliability of the IP-based
connection between the GCS and the UAV. This system does not use
HAIPEs to create a red/black network.
[0511] In an embodiment, HART consists of two kinds of appliances:
the HART-Edge appliance resident on the UAV and the HART-Gateway
appliance resident at the GCS.
[0512] The HART-Edge Appliance has eight Ethernet ports. Five of
these ports are used to connect to the five RF links as shown in
FIG. 20. The other three ports are available for use by on-board
hosts and sensors on the UAV to connect to the tactical network via
the HART-Edge appliance. In this case, each on-board IP device
(sensor or CPU) will use the HART router as its "default
router".
[0513] Ground Control Station (GCS) Network
[0514] There is an Ethernet network (LAN) at the GCS to connect
various hosts and servers. This ground network uses a hub or switch
to connect all the devices (FIG. 21). The GCS has ground-based
transceivers for the various RF/wireless communications devices.
Each of these devices has an IP/Ethernet interface.
[0515] The HART-Gateway Appliance has eight Ethernet interfaces,
one connected to the ground network router and another connected
directly to each of the RF/wireless devices (FIG. 21). Optionally,
HART could connect to another LAN on the RF side that has the
RF/wireless devices connected to a hub or switch.
[0516] The GCS may have external networks (i.e. internets)
connected to the ground LAN. With proper routing configuration this
does not affect HART, and in fact HART will route data to and from
the UAV to the external networks as well.
[0517] Each RF/wireless radio device used in this scenario is
listed above with specifications that are used by HART to make
informed policy based routing decisions.
[0518] HART Features [0519] FreeBSD PC (version 7.3) [0520] Quad
NIC (network interface cards)--at least 5 Ethernet connections
[0521] FreeBSD router software [0522] Maintain IP data flows
between the GCS and one or more UAVs connected to the GCS [0523]
Policy based dynamic link selection [0524] Provides automatic link
multi-level failover and recovery [0525] Policy based multi-link
forwarding (Stretch Goal) [0526] Replication provides high
reliability communications [0527] Policy configuration GUI (Stretch
Goal) [0528] Integration with Cubic Mini-CDL radio hardware
[0529] An embodiment of the software architecture of HART is shown
in FIG. 22.
[0530] HART Policy Configuration
[0531] HART uses a Policy Configuration file to define the settings
used by HART for link failover. This allows the user to define the
default link for traffic and the order of backup or failover links
to use. Once the default is setup, the user can specify different
traffic types based on matching DSCP (differentiated services code
point) and/or protocol ID fields. For each traffic type a primary
link can be selected and then the other links can be ordered as to
which order each will be used to failover that traffic type.
[0532] The last policy option in the HART prototype is the
multi-link forwarding. For a specified traffic type (specific DSCP
and/or protocol ID) multiple links can be specified to replicate
packets on. This option will send the same packet or data frame
across multiple links to provide improved reliability. If the
primary link fails the data will not be lost or interrupted, the
flows will continue across the other specified links without
affecting the data flow at all.
[0533] Three policy types: [0534] Default Policy: primary and
backup links for all non-specified traffic. [0535] Automatic
Failover Policy: primary and backup links for all specific traffic.
Specific traffic defined by DSCP and protocol ID fields. [0536]
Multi-Link Forwarding Policy: specify primary link and replication
links to replicate specific traffic on. Specific traffic defined by
DSCP and protocol ID fields.
[0537] Example: [0538] If DS CP==18 (AF2 1) then
PrimaryLink=Mini-CDL1 and FailoverLink0rder=AN/ARC-210(V);
Mini-CDL2; WGS; Inmarsat
[0539] HART Emulation Testbed
[0540] Emulab was used to create a HART testbed (FIG. 23),
consisting of: [0541] Two HART routers running on either end of 5
links (hart0&hart1) [0542] Two user nodes running various
applications on the endpoints (ep1 & ep1) [0543] Four links
emulated by Emulab Delay nodes (D) and Radio Emulators (emul0 &
emull) [0544] Radio Emulator are adjusted during the experiment to
report lesser quality to HART, causing HART to make route
adjustments (e.g. failover)
[0545] Radio Link Specifications Details
[0546] In a live scenario, the wireless radio devices and
specifications used are: [0547] Mini-CDL Radio [0548] Bandwidth: 8
Mbps (5.4 Mbps-10.7 Mbps) [0549] Latency: 400ms RTT [0550]
Interface: Ethernet [0551] Data routing: Ethernet Bridge: data
comes in over Ethernet, then is transferred directly to the paired
endpoint [0552] Range: LOS: surface-to-surface (sts): ave:4.4 nm II
surface-to-air (sta): ave:14.8 nm) [0553] WGS (Wideband Global
SATCOM) SatCom [0554] Bandwidth: >2 Mbps [0555] Latency: 710 ms
RTT [0556] Interface: Ethernet [0557] Data routing: IP routed
through SatHub [0558] Range: BLOS [0559] Inmarsat (BGAN) [0560]
Bandwidth: 329 kbps (10 kbps-2 Mbps) [0561] Latency: 710 ms RTT
[0562] Interface: Ethernet [0563] Data routing: IP routed through
SatHub [0564] Range: BLOS [0565] AN/ARC-210(V) (ARC-210 gen5)
[0566] Bandwidth: 80 kbps (48.8 Kbps-97.7 Kbps) [0567] Latency: 400
ms RTT [0568] Interface: Ethernet [0569] Data routing: Ethernet
Bridge: data comes in over Ethernet, then is transferred directly
to the paired endpoint [0570] Range: LOS: surface-to-surface:
.about.55 nm (31-92 nm); surface-to-air: .about.176 nm (119-264
nm)
[0571] In the emulation environment to make the configuration and
setup simpler, ATC used these link specifications:
TABLE-US-00002 Link Throughput Latency Radio0 5 Mb 20 ms Radio1 1
Mb 200 ms Radio2 1 Mb 300 ms Radio3 1 Mb 400 ms
[0572] In addition to proactive link selection, HART implements
multi-level policy-based failover (1, 2, 3, 4, 5 or more backup
links in a prioritized order), a capability that is not supported
within COTS routers such as Cisco. For instance, with Cisco
routers, if the policy route for an IP flow were to fail it will be
routed over the default interface. Unlike HART, secondary and
tertiary policy routes cannot be specified with Cisco routers.
[0573] FIG. 24 (steps 1-4) shows HART link quality monitoring and
proactive link failover. FIG. 24 shows the Emulab experiment: 2
LANs connected by 4 links through HART routers. Each LAN has 3
nodes: an endpoint (ep), a radio emulator node (emul) and a HART
router (hart). The radio emulator node runs an application that ATC
built to emulate the Mini-CDL radios' link quality API, and allow
for manual quality adjustment. Step 1 shows a ping from ep0 to ep1.
Step 2 shows the Mini-CDL radio emulator (for RadioLink0) being set
from 100% to 60%.
[0574] In step 3, HART detects the new quality metric for
RadioLink0 is below threshold of 75%, and then triggers the
failover to RadioLink1. Step 4 shows the ping times have increased
which shows that RadioLink1 may be used instead of RadioLink0. HART
may successfully detect link quality degradation by interfacing
with the radio and then change the router on "hart0" to use the
failover path of RadioLink1.
[0575] HART Radio Integration testbed
[0576] HART may be used with two real Mini-CDL radios (FIG. 25a and
FIG. 25b). HART is integrated with the radios and then successfully
shows to monitor the status of the live radios (through the
Mini-CDL API), making correct autonomous failover configuration
updates to the routers (HARTO and HART1 in FIG. 25b). The
demonstration shows a video stream being sent over the radio link.
As the link degrades below a preset threshold, routing fails over
to a series of backup links.
[0577] HART neighbor discovery service automatically discovers IP
one-hop neighbor HART edge and transit routers on a radio subnet.
HART transmits periodic subnet multicast of Hello messages by each
HART router to enable dynamic neighbor discovery. A neighbor table
is maintained by each HART router with subnet-specific address and
alias addresses of each neighbor.
[0578] Subnet Convergence Function
[0579] The subnet convergence function provides a common IP subnet
layer interface for the routing function. It enables automatic
formation of virtual point-to-point link with each neighboring HART
router. It performs monitoring of quality metrics for each virtual
link. It implements network level flow control.
[0580] Virtual Link Formation & Maintenance
[0581] Virtual link formation and maintenance provides cut-through
routing for implementing a virtual link. It maintains per-flow
state for each IP data flow using a virtual link. It performs IP
packet header restoration for data packets received over a virtual
link.
[0582] Link-Quality Monitoring
[0583] Link-Quality monitoring functions implement a passive
technique for sensing packet loss rate on virtual link. It
implements an active, passive, or hybrid technique for virtual link
capacity sensing. It implements an active, passive, or hybrid
technique for sensing virtual link latency. It provides a link
quality query interface and asynchronous notification
mechanism.
[0584] Network-level Flow Control
[0585] Network-level flow control provides network-level control of
the rate at which packets are sent to an attached radio. It
implements adaptive per-virtual-link rate control based on dynamic
sensing of virtual link. It may augment radio-supported flow
control (e.g., RFC 5578).
[0586] Traffic Redirection
[0587] Traffic redirection implements a mechanism to provide
redirection of an IP packet flow to a different next-hop than the
current one for load balancing or for traffic-aware routing. Flow
redirection is based on source and destination addresses of IP
packets.
[0588] QoS-Aware Unicast Routing Service
[0589] The QoS-aware unicast routing service provides an OSPF-based
core routing protocol for unicast routing over inter-router virtual
links. It interconnects radio IP subnets into one HART network. It
maintains multiple routing metrics per virtual link. It implements
multiple routing tables, one per routing metric. It performs
link-aware route selection. It performs traffic-aware route
selection.
[0590] Load Balancing Function
[0591] Load balancing function performs distribution of traffic
exiting an IP subnet across multiple egress links, if applicable.
It performs distribution of traffic entering an IP subnet across
multiple ingress links, if applicable.
[0592] Dynamic Link Metrics
[0593] When a HART router has multiple connections to the HART
internetwork, the HART design allows that HART router to
dynamically configure the link metrics based on radio link
monitoring.
[0594] Dynamic Route Selection
[0595] When multiple paths are available to route IP traffic
through the HART inter-network, the HART design allows the dynamic
selection of the path that an IP packet flow will take based on
routing metrics.
[0596] Radio-aware flow control
[0597] The HART design allows for flow control between the HART
router and its connected radios.
[0598] Mission-Aware Traffic Management
[0599] The HART design allows the HART internetwork to be
configured with a set of mission-specific parameters that influence
dynamic link selection and dynamic route selection for specified
traffic classes.
[0600] Load Balancing
[0601] The HART design allows utilization of multiple communication
links when such links exist between elements of the HART
internetwork. For example, when multiple links exist between two
nodes, as shown in FIG. 26a and FIG. 26b, the HART design allows
both links (e.g., link a and link b) to be used in parallel to
balance traffic flows transiting between the nodes (e.g., R.sub.1
and R.sub.2). Error! Reference source not found.a shows how the
data is routed in the absence of load balancing, while Error!
Reference source not found.b demonstrates how the data flow is
balanced across the available links. By load balancing, more data
can ultimately be transferred between the two nodes.
[0602] An embodiment in which HART Maximizes Network
Performance
[0603] (Scenario 1) is illustrated in FIG. 27 in which C sends a
data file to X, a CSTAR router senses two links from UAV1 to TOC.
HART automatically load balances across both links to gain more
bandwidth.
[0604] Another embodiment in which HART Maximizes Network
Performance (Scenario 2) is illustrated in FIG. 28 in which: [0605]
L sends a data file to X [0606] H sends a data file to X [0607]
HART senses both traffic flows and chooses 2 independent
non-interfering routes from L X and H X [0608] This improves
network throughput and performance
[0609] A subnet convergence function is illustrated in FIG. 29 in
which HART: [0610] Provides a common IP subnet layer interface for
routing function [0611] Automatically forms virtual point-to-point
links with each neighboring HART router [0612] Monitors quality
metrics for each virtual link [0613] Provides network-level flow
control
[0614] A subnet convergence function: virtual link formation &
maintenance is illustrated in FIG. 30 and FIG. 31 in which HART:
[0615] Provides a virtual IP 1-hop channel between neighboring HART
routers [0616] Cuts-through routing for implementing virtual link
[0617] Provides maintenance of per-flow state for each IP data flow
using virtual link [0618] Facilitates IP packet header restoration
for data packets received over a virtual link
[0619] A subnet convergence function: link-quality monitoring is
illustrated in FIG. 32 in which HART: [0620] Measures link quality
to support link-aware routing [0621] Uses a passive technique for
sensing packet loss rate on virtual link [0622] Uses an
Active/Passive technique for virtual link capacity sensing [0623]
Uses an Active/Passive technique for sensing virtual link latency
[0624] Provides link quality query interface and asynchronous
notification mechanism
[0625] Bandwidth/latency monitoring is illustrated in FIG. 33 in
which HART: [0626] Uses existing network data as a probe packet
[0627] Sends probe packets of varying size [0628] Response time
(RTT) is a function of probe size [0629] Uses existing network data
as probe packet
[0630] Packet-Loss Sensing [0631] Packet loss is passively sensed
using IP header identification and fragment offset fields that are
currently unused [0632] Each packet is tagged [0633] IP
identification field contains two octets of the HART routers IP
address [0634] IP fragment-offset field contains a sequence number
[0635] The receiving HART node decodes the tag [0636] Missing
sequence numbers are indicative of packet loss [0637] A sliding
window is utilized to account for out-of-order packets
[0638] Packet loss detection is illustrated in FIG. 34 in which:
[0639] Packet loss is computed passively [0640] Each packet is
tagged by HART [0641] Sequence number (S) is injected into packet
header [0642] H+S is the same size as H [0643] The receiving HART
node decodes the tag [0644] Packet loss occurs when sequence
numbers are missing
[0645] Subnet convergence function: flow control is illustrated in
FIG. 35 in which: [0646] To prevent head-of-line blocking of
packets within the radio [0647] Network-level control of the rate
at which packets are sent to attached radios [0648] Adaptive
per-virtual-link rate control based on dynamic sensing of virtual
link [0649] May augment radio-supported flow control (e.g., RFC
5578)
[0650] Subnet convergence function: flow control is further
illustrated in FIG. 36 in which HART: [0651] Prevents radio buffer
overrun and packet loss due to head-of-line blocking [0652] Sends
at rate matched to receiver capability [0653] Token-bucket scheme
[0654] Window-based control
[0655] Subnet convergence function: traffic redirection is
illustrated in FIG. 37 in which: [0656] HART provides redirection
of an IP packet flow to a different next-hop than the current one
for load balancing or for traffic-aware routing [0657] Flow
redirection based source and destination addresses of IP
packets
[0658] Subnet convergence function: traffic redirection is further
illustrated in FIG. 38 in which: [0659] Transit Router Decision
[0660] Route Through Default [0661] Route Through Alternate [0662]
Redirect To Alternate
[0663] Quality of service (QoS) aware unicast routing service is
illustrated in FIG. 39 in which: [0664] Interconnection of radio IP
subnets into one HART network [0665] OSPF-based core routing
protocol for unicast routing over inter-router virtual links [0666]
Maintenance of multiple metrics per virtual link [0667] Multiple
routing tables, one per routing metric [0668] Link-aware route
selection [0669] Traffic-aware route selection
[0670] And in FIG. 40 in which: [0671] Linux kernel router [0672]
OSPF overlay network of HART routers [0673] Quagga router control
daemons [0674] Kernel routing tables for each traffic class [0675]
Latency-sensitive [0676] Latency-tolerant [0677] Use OSPF
TOS-specific Link State Advertisements (LSAs) [0678] Tag traffic
with DSCP/TOS fields of IP frame
[0679] Mobility-management function is illustrated in FIG. 41 in
which: [0680] a. Enable subnet-hopping of HART endpoints [0681] b.
Automatic detection of current radio subnet [0682] c. Automatic
selection of subnet-specific address for the endpoint [0683] d.
Mobility registry providing mapping between portable address of
endpoint and its current subnet-specific address [0684] e.
AODV-based mechanism to enable routing to portable addresses in
case the mobility registry is unreachable
[0685] And FIG. 42 in which: [0686] HART "portable" IP address
[0687] Reachable on any subnet for which it has been configured
[0688] HART routers maintain reachability information about
"portable" address to subnet-specific address mapping [0689]
Mobility registry maintains mapping between portable and
subnet-specific addresses [0690] Mobility registry "beacons" its
presence using network-wide multicast [0691] Transit router
periodically registers the portable and subnet-specific addresses
of nodes resident on any of its subnets [0692] Transit router
consults mobility registry if it does not have a mapping between
the portable address and subnet-specific address for a packet it is
forwarding [0693] If the mobility registry is not reachable, the
transit router invokes AODV to resolve the portable address of a
packet
[0694] Load-balancing function is illustrated in FIG. 43 in which:
[0695] Network-optimized data transport [0696] Distribution of
traffic exiting an IP subnet across multiple egress links, if
applicable
[0697] And in FIG. 44 in which: [0698] Linux `tc` traffic shaper
[0699] Quagga `equal-cost multi-path routing` feature
[0700] Quality of Service (QoS) Overview [0701] Goal to deliver
predictable data services [0702] Important for providing reliable
services that are sensitive to bandwidth, latency and error rate
[0703] Voice [0704] Video [0705] Defined by a set of parameters
that describe service level [0706] Bandwidth [0707] Buffer usage
[0708] Delay [0709] etc
[0710] Quality of Service (QoS) Overview [0711] HART Black side QoS
[0712] DiffSery with DSCP translation between domains [0713]
KG-250X can provide Red side QoS [0714] Red side service level maps
to Black side DSCP [0715] End-to-end QoS with HART [0716] SRS
complete, design in progress
[0717] Red side QoS Design [0718] Integrated Services: Flow-based
service level guarantees via RSVP [0719] Red flow source and
destination hosts initiate RSVP exchange [0720] All Red side
routers, including KG-250X, participates in the RSVP exchange
[0721] Establishes guaranteed level of service for the flow, or
fails [0722] Source host can try again for a lower level of
service
[0723] Test AE (illustrated in FIG. 45) creates redundant paths
between subnetworks within the network. One of the paths is
impaired to exercise the router's ability to switch to the "better"
redundant link. [0724] Creates redundant paths between subnetworks
within the HART network [0725] One of the paths is impaired to
exercise routers' ability to switch to the "better" redundant
link
[0726] Test AF, illustrated in FIG. 46, simplifies test description
by eliminating IP addressing specifics, recreates redundant links
and verifies load balancing enabling multicast efficiency. [0727]
Simplifies test description by eliminating IP addressing specifics
[0728] Recreates redundant links [0729] Verifies load balancing,
multicast efficiency
[0730] Test AG, illustrated in FIG. 47, is network aware and
traffic aware.
[0731] Network Aware: [0732] a. Automatically configured network
communication, without manual configuration, between/among networks
having differing IP protocols [0733] i. Hybrid routing with address
switching [0734] ii. Multi-cast video feed
[0735] Traffic Aware [0736] a. Choose network route, over a network
having a paths with a plurality of intermediate nodes, based, at
least in part, on the type of traffic/message [0737] i. E.g., file
transfer: choose route having a relatively high bandwidth [0738]
ii. E.g., talk: choose route having relatively low latency [0739]
b. Multiple routing tables based on different metrics [0740] i.
E.g., Table I: based on bandwidth [0741] ii. E.g., Table II: based
on latency [0742] c. IP header marked with type of traffic [0743]
i. Uses the routing table based on the type of traffic
[0744] Pro-active Failover [0745] a. Autonomous quality aware
routing [0746] b. Switch from existing route to a new route prior
to network communication failure [0747] i. i.e., before
communication is lost [0748] ii. built-in fault tolerance [0749] c.
If link (route) quality factor degrades below a [predetermined]
threshold, then switch route before communication is lost [0750] i.
E.g., miss two packets but one packet gets through [0751] d.
Monitor quality of each individual link [0752] e. Detect and
remember packet loss
[0753] Load Balancing [0754] a. Apportions traffic over different
routes based on [overall] network load, not just on an individual
message [0755] i. Helps reduce network clogging [0756] ii. May
result in a longer route for an individual message or some messages
but [overall] network performance improves for all or more
users
[0757] Appendix A: Target IP Radios, Devices and technologies to
integrate Primary integration technologies:
[0758] TTNT: Tactical Targeting Network Technology [0759] IP
Support: yes, radios provide IP support externally, and internally
route at the lower network and mac layers [0760] Config API (read):
SNMP [0761] Config API (write):SNMP [0762] Used for: Waveform
developed for JTRS (Joint Tactical Radio System) for airborne
networking. Similar to ANW (Airborne Networking Waveform). Used for
MIDS (Multi-functional Information Distribution System) data
traffic as well as other TCP/IP network traffic
[0763] Mini-CDL: Miniature CDL radio [0764] IP Support: yes,
provides point-to-point links, used to create IP bridges [0765]
Config API (read):custom API [0766] Config API (write):custom
API
[0767] Used for: Video and data links from mini and micro UAVs and
UASes. [0768] NOTE: CDL waveform is not currently compatible with
JTRS radios and cannot be used on them [0769] Background: Subset of
CDL (Common Data Link) radio family, smaller form factor. CDL
Family: [0770] TP-CDL: Team Portable CDL [0771] TCDL--Tactical
Common Data Link. Used for: N-CDL, USQ-123, ATARS, BGPHES, CHBDL,
SHARP, TIGDL I/II [0772] N-CDL: Networked CDL [0773] TIGDL-II:
Tactical Interoperable Ground Data Link II(2) [0774] MR-TCDL:
Multi-Role Tactical CDL [ABE: Auxiliary Bandwidth Efficient; and
Discovery]
[0775] WiMax: Worldwide Interoperability for Microwave Access
[0776] Config API (read): SNMP [0777] Config API (write): SNMP
[0778] Used for:as a generic test radio (Similar properties to
TTNT)
[0779] FSO: Free Space Optical [0780] Config API (read): assumed
SNMP [0781] Config API (write):assumed SNMP [0782] Used for:high
bandwidth point-to-point links
[0783] JCAN: Joint Capability for Airborne Networking subnets
[0784] Config API (read): BGP (BGP-MX) & SNMP [0785] Config API
(write):BGP (BGP-MX) & SNMP [0786] Used for:Integrating legacy
data links via an airborne gateway node
[0787] ORCA: Optical RF Communications Adjunct [0788] Config API
(read): assumed SNMP [0789] Config API (write):assumed SNMP [0790]
Used for:Hybrid Optical and RF link (DARPA)
[0791] BACN: Battlefield Airborne Communications Node [0792] Config
API (read): BGP (BGP-MX) & SNMP [0793] Config API (write):BGP
(BGP-MX) & SNMP [0794] Used for:Integrating legacy data links
via an airborne gateway
[0795] Other integration technologies
[0796] ONT: Quint Networking Technology [0797] Config API (read):
assumed SNMP [0798] Config API (write):assumed SNMP [0799] Used
for:UASs, munitions
[0800] ANW: Airborne Networking Waveform [0801] Config API (read):
assumed SNMP [0802] Config API (write): assumed SNMP [0803]
Background: Waveform developed for JTRS (Joint Tactical Radio
System) for airborne networking. Similar to TTNT (Tactical
Targeting Network Technology). Used for MIDS (Multi-functional
Information Distribution System) data traffic as well as other
TCP/IP network traffic
[0804] aADNS: Airborne Automated Digital Network System [0805]
Config API (read): assumed SNMP [0806] Config API (write):assumed
SNMP
[0807] ASSDL: Airborne Single Slot Data Link [0808] Config API
(read): assumed SNMP [0809] Config API (write):assumed SNMP
[0810] GBS/TGRS: Global Broadcast Service [0811] Config API (read):
assumed SNMP [0812] Config API (write):assumed SNMP [0813]
Uses:TGRS: Transportable ground receive suite for GBS
[0814] MUOS: Mobile User Objective System [0815] Config API (read):
assumed SNMP [0816] Config API (write):assumed SNMP [0817] Used
for: JTRS waveform for SatCom (BLOS)
[0818] SRW: Soldier Radio Waveform [0819] Config API (read):
assumed SNMP [0820] Config. API (write):assumed SNMP [0821] Used
for: JTRS waveform
[0822] WNW: Wideband Networking Waveform [0823] Config API (read):
assumed SNMP [0824] Config API (write):assumed SNMP [0825] Used
for: JTRS waveform [0826] Uses: OFDM, BEAM, AJ, LPI/D
[0827] CMDL: Compact Multi-band Data Link [0828] Config API (read):
assumed SNMP [0829] Config API (write):assumed SNMP
[0830] SNR: Subnet Relay [0831] Config API (read): assumed SNMP
[0832] Config API (write):assumed SNMP [0833] Used for: ad hoc for
maritime RF [0834] Uses: HFIP (IP over HF [High Frequency--3 to 30
MHz])
[0835] TSAT: Transformational Satellite Communications System
[0836] Config API (read): assumed SNMP [0837] Config API
(write):assumed SNMP [0838] Used for:Used for: HC3 (2)
[0839] HNW: Highband Network Waveform [0840] Config API (read):
assumed SNMP [0841] Config API (write):assumed SNMP [0842] Used
for:HNR: Highband Network Radio, and WIN-T Waveform. WIN-T
(Warfighter Information Network-Tactical)
[0843] NCW: Network-Centric Waveform [0844] Config API (read):
assumed SNMP [0845] Config API (write):assumed SNMP [0846] Used
for: WIN-T Waveform. WIN-T (Warfighter Information
Network-Tactical)
[0847] DISN: Defense Information System Network [0848] Config API
(read): BGP (BGP-MX) & SNMP [0849] Config API (write):BGP
(BGP-MX) & SNMP [0850] Used for:NIPRnet (Non-Classified IP
Router Network), SIPRnet (Secret IP Router Network)
[0851] Appendix B: Use cases of HART behavior
Use Case 1: Mixed HART Routers (TTNT, SatCom, and Mini-CDL)
[0852] Initial setup
[0853] Definitions of some terms in the figures: [0854] HART-Edge
Router:No Routing, endpoint node only (little/no storage) [0855]
HART-Interim Router: Routing, maintains tables for routing (to and
from), may include storage [0856] HART-Gateway Router: Routing,
Provides links to external and/or non-tactical networks (GIG,
Internet, etc.) [0857] Alias-tag Tables: [0858] H: number of hops
[0859] L: Latency [0860] B: Bandwidth Initial setup process (FIG.
48): [0861] HG1 detects GIG and declares "I'm a gateway router"
[0862] Broadcasts itself as "default route" to GIG. [0863] HG2
detects GIG and declares "I'm a gateway router" [0864] Broadcasts
itself as "default route" to GIG. [0865] HI1 detects no GIG and
declares "I'm an Interim router" (default) [0866] Receives "default
route" from JALN (HG1 and HG2, for specified subnets) [0867] HI2
detects no GIG and declares "I'm an Interim router" (default)
[0868] Receives "default route" from JALN (HG1 and HG2, for
specified subnets) [0869] Becomes "default router" for TTNT radio
subnet to link them to GIG (thru JALN) Node Joins a Mixed HART
network
When a Node Joins (FIG. 49)--Step 1:
[0869] [0870] Establish each link from that node to the existing
network [0871] Add alias-tags at various endpoints in the network
to enable label switching (faster than routing) (This is the HART
Dynamic DNS)
Step 2, FIG. 50:
[0871] [0872] SatCom connects: Uses fixed SatCom IP to connect to
the other SatCom endpoint (12.1.4.8) [0873] HE1 adds an alias-tag
for 17.0.0.0->12.1.4.8 [0874] HG1 adds an alias-tag for
17.1.2.3->12.1.3.1
Step 3, FIG. 51:
[0874] [0875] First Mini-CDL Radio connects [0876] Establishes
"link local" addresses on both endpoints
Step 4, FIG. 52:
[0876] [0877] HE1 adds an alias-tag for 17.0.0.0->224.1.1.1
[0878] HG1 adds an alias-tag for 17.1.2.3->224.1.1.2
Step 5, FIG. 53:
[0878] [0879] Second Mini-CDL Radio connects [0880] Establishes
"link local" addresses on both endpoints [0881] HI1 requests
224.1.1.2 first, but HE1 already is using, so responds with
224.1.1.3
Step 6, FIG. 54:
[0881] [0882] HE1 adds an alias-tag for 17.0.0.0->224.1.1.1
[0883] HI1 adds an alias-tag for 17.1.2.3->224.1.1.3
Step 7, FIG. 55:
[0883] [0884] HI1 forwards alias-tag to HG1, HG1 adds it as:
17.1.2.3->10.1.2.1 [0885] HI1 adds reverse alias-tag
Step 8, FIG. 56:
[0885] [0886] TTNT Radio connects [0887] Uses fixed TTNT IP to
connect to the TTNT cloud [0888] Finds default router in TTNT cloud
(HI2)->sets default router to HI2
Step 9, FIG. 57:
[0888] [0889] HE1 adds an alias-tag for 17.0.0.0->10.71.103.2
[0890] HI2 adds an alias-tag for 17.1.2.3->10.77.91.1
Step 10, FIG. 58:
[0890] [0891] HI2 forwards alias-tag to HG1: 17.1.2.3->10.2.2.1
[0892] HI2 adds reverse alias-tag Completed Table Entries are
illustrated in FIG. 59:
Packet Routing, Rerouting and Return
Routing Step 1 (FIG. 60):
[0892] [0893] Packet leaves H2 [0894] Packet: Src=19.1.2.3;
Dst=17.1.2.3; Data [0895] H2.fwdarw.GIG.fwdarw.HG1
Routing Step 2 (FIG. 61):
[0895] [0896] HG1 looks up H1 [0897] Next hop=224.1.1.2(M-CDL):
<=1 hop.fwdarw.no label switching [0898] Packet: Src=19.1.2.3;
Dst=17.1.2.3; Data [0899] HG1.fwdarw.M-CDL.fwdarw.HE1
Routing Step 3 (FIG. 62):
[0900] HE1 receives [0901] Packet: Src=19.1.2.3; Dst=17.1.2.3; Data
[0902] HE1.fwdarw.H1 Error occurs in M-CDL1 link (FIG. 63):
Re-Routing Step 1 (FIG. 64):
[0903] HG1 looks up H1 [0904] Next hop=10.1.2.1(JALN): 2
hop.fwdarw.label switching [0905] Packet: Src=19.1.2.3;
Dst=10.1.2.1; (JALN); Dst`=17.1.2.3; Data [0906]
HG1.fwdarw.JALN.fwdarw.HI1
Re-Routing Step 2 (FIG. 65):
[0907] HI1 looks up H1 [0908] Next hop=224.1.1.3(M-CDL): <=1
hop.fwdarw.no label switching [0909] Restore packet: Src=19.1.2.3;
Dst=17.1.2.3; Data [0910] HG1.fwdarw.M-CDL.fwdarw.HE1
Re-Routing Step 3 (FIG. 66):
[0911] HE1 receives [0912] Packet: Src=19.1.2.3; Dst=17.1.2.3; Data
[0913] HE1.fwdarw.H1
Return Path (FIG. 67):
[0914] Packet leaves H1 [0915] Packet: Src=19.1.2.3; Dst=17.1.2.3;
Data [0916] H1.fwdarw.HE1
[0917] HE1 looks up H2 [0918] Next hop=224.1.1.1(M-CDL2) [0919] 2
hop.fwdarw.label switching [0920] Packet: Src=19.1.2.3;
Dst=10.1.2.1; (JALN); Dst`=17.1.2.3; Data [0921]
HE1.fwdarw.M-CDL2.fwdarw.HI1
[0922] HI1 looks up H2 [0923] Next hop=10.3.2.1(JALN) [0924] <=1
hop.fwdarw.no label switching [0925] Restore Packet: Src=19.1.2.3;
Dst=17.1.2.3; Data [0926] HI1 4 JALN.fwdarw.HG1
[0927] HG1 receives [0928] Packet: Src=19.1.2.3; Dst=17.1.2.3; Data
[0929] HG1.fwdarw.GIG.fwdarw.H2 Use Case 2: Node moves from one
TTNT subnet to another TTNT subnet Initial setup (FIG. 68):
[0930] All nodes start at "I'm an Interim router" (default)
[0931] Platforms 1,3 &5 all detect no GIG, so keep: "I'm an
Interim router" [0932] TTNT addresses are fixed, use in name
exchanges. . . [0933] Start exchanging name with other HART routers
to build alias tables [0934] Platform 1 alias table: [0935] To get
to "Platform 3" send out local-TTNT to "TTNT-P3" [0936] To get to
"Platform 2" send out local-TTNT to "TTNT-P2" [0937] Platforms 3
and 5 follow similar table entries [0938] Platforms 2 & 4
detect GIG and change from "interim" to: "I'm a gateway router"
[0939] GIG found through SatCom [0940] Platform 2 becomes TTNT
gateway for top TTNT cloud (1,2,3,others) (FIG. B-21) [0941]
Platform 4 becomes TTNT gateway for bottom TTNT cloud (4,5, others)
(FIG. 68) [0942] Set Default route entry to: [0943] To get to
"GIG/Default" send out local-SatCom to "remote-SatCom-Hub" [0944]
TTNT addresses are fixed, use in name exchanges (include "I'm
default router") [0945] Start exchanging name with other HART
routers to build alias tables [0946] Platform 1 alias table
becomes: [0947] To get to "Platform 3" send out local-TTNT to
"TTNT-P3" [0948] To get to "Platform 2" send out local-TTNT to
"TTNT-P2" [0949] To get to "GIG/Default" send out local-TTNT to
"TTNT-P2" [0950] Platforms 3 and 5 follow similar table entries
[0951] Platform 2 alias table: [0952] To get to "Platform 3" send
out local-TTNT to "TTNT-P3" [0953] To get to "Platform 1" send out
local-TTNT to "TTNT-P1" [0954] To get to "GIG/Default" send out
local-SatCom to "remote-SatCom-Hub" [0955] Platforms 4 follows
similar table entries [0956] HART gateways exchange alias tables
[0957] Platform 2 alias table adds: [0958] To get to "Platform 4"
send out local-SatCom to "SatCom-P4" [0959] To get to "Platform 5"
send out local-SatCom to "SatCom-P4" [0960] Platform 4 alias table
adds: [0961] To get to "Platform 1" send out local-SatCom to
"SatCom-P2" [0962] To get to "Platform 2" send out local-SatCom to
"SatCom-P2" [0963] To get to "Platform 3" send out local-SatCom to
"SatCom-P2" Route a Packet from Platform 1 to Platform 3 (FIG. 68):
[0964] 1. Packet leaves P1: Src=P1; Dst=P3; Data [0965] Send to
P1-HART [0966] 2. @P 1-HART [0967] Table lookup, P3 goes thru TTNT
to address TTNT-P3 [0968] Change packet: Src=P1; Dst=TTNT-P3;
DstSv=P3;Data--swaps Dst to DstSv and sets next hop to TTNT-P3
[0969] Goes out TTNT to TTNT-P3 [0970] 3. TTNT-P3 forwards up to
local Router (P3-HART) [0971] 4. @P3-HART: Src=P1; Dst=P3; Data
--swaps DstSv back to Dst, forwards to P3 [0972] 5. @P3: Src=P1;
Dst=P3; Data Route a Packet from Platform 1 to Platform 5 (FIG.
68): [0973] 1. Packet leaves P1: Src=P1; Dst=P3; Data [0974] Send
to P1-HART [0975] 2. @P1-HART [0976] Table lookup, P5 --not found,
send to default [0977] Default goes thru TTNT to address TTNT-P2
[0978] Change packet: Src=P1; Dst=TTNT-P2; DstSv=P5;Data --swaps
Dst to DstSv and sets next hop to TTNT-P2 [0979] Goes out TTNT to
TTNT-P2 [0980] 3. TTNT-P2 forwards up to local Router (P2-HART)
[0981] 4. @P2-HART [0982] Table lookup, P5--found, send out SatCom
to P4 [0983] Change packet: Src=P1; Dst=SatCom-P4; DstSv=P5;Data
--sets Dst to next hop to P4 [0984] Goes out SatCom to Satcom-P4
[0985] 5. Satcom-P4 forwards up to local Router (P4-HART) [0986] 6.
@P4-HART [0987] Table lookup, P5 goes thru TTNT to address TTNT-P5
[0988] Change packet: Src=P1; Dst=TTNT-P5; DstSv=P5;Data --swaps
Dst to DstSv and sets next hop to TTNT-P5 [0989] Goes out TTNT to
TTNT-P5 [0990] 7. TTNT-P5 forwards up to local Router (P5-HART)
[0991] 8. @P5-HART: Src=P1; Dst=P5; Data --swaps DstSv back to Dst,
forwards to P3 [0992] 9. @P5: Src=P1; Dst=P5; Data Use Case 3:
Routing Data on a HART network [0993] Using Replication and Load
Balancing [0994] Steps 1-3 (FIG. 69): [0995] Data Packet stream
leaves FS2 [0996] Src=17.1.2.12; Dst=19.1.7.11 [0997] FS2.fwdarw.H1
[0998] H1 looks up policy for data traffic: [0999] Load balance
across 3 (M-CDL1, M-CDL2, A210) [1000] Redundant x2 [1001] H1 sends
to H2 [1002] Src=17.1.2.12; Dst=19.1.7.11 [1003] Copy 1.sup.st
packet to M-CDL1 and M-CDL2 [1004] Copy 2.sup.nd packet to A210 and
M-CDL 1 [1005] Copy 3.sup.rd packet to M-CDL2 and A210 [1006]
Repeat . . . [1007] H1.fwdarw.M-CDL(1,2) & A210.fwdarw.H2
[1008] H2 receives [1009] Src=17.1.2.12; Dst=19.1.7.11 [1010]
H2.fwdarw.GCS1 [1011] GCS1 receives Link Fault while Routing Data
across the HART network
[1012] In case of link failure, other links should pick up "slack"
and connection will continue as is with no interruption (FIG.
70).
[1013] Routing Control Messages across the HART network
[1014] Steps 1-4 (FIG. 71): [1015] Control message Packets leave FS
1 [1016] Src=17.1.2.11; Dst=19.1.7.12 [1017] FS1.fwdarw.H1 [1018]
H1 looks up policy (control message traffic): [1019] Send all
packets across A210 [1020] Replicate control packets across
alternating SatCom links (WGS and Inmarsat). [1021] H1.fwdarw.sends
to H2 [1022] Src=17.1.2.11; Dst=19.1.7.12 [1023] Send all packets
across A210 [1024] Copy 1.sup.st packet to WGS [1025] Copy 2.sup.rd
packet to Inmarsat [1026] Repeat . . . [1027]
H1.fwdarw.A210/SatCom.fwdarw.H2 [1028] H2 receives [1029]
Src=17.1.2.11; Dst=19.1.7.12 [1030] H2.fwdarw.GCS2 [1031] GCS2
receives Link Fault while Routing Control Messages across the HART
network
[1032] In case of link failure, other links should pick up "slack"
and the connection will continue as is, with no interruption (FIG.
72).
[1033] Glossary [1034] AN: Airborne Network [1035] API: Application
Programming Interface [1036] ARC: HARTs Autonomous Router
Configuration [1037] AS(es): Autonomous System(s)--a collection of
connected Internet Protocol (IP) routing devices under one common
network controller [1038] AXP: Cisco Application Extension
Platform, an add-on card installed in a Cisco router [1039] BACN:
Battlefield Airborne Communications Node [1040] BGP: Border Gateway
Protocol--Protocol used to route between Autonomous Systems [1041]
BGP-MX: Border Gateway Protocol [1042] COTS: Commercial
Off-the-Shelf [1043] CT: Ciphertext--Black side of red/black
network [1044] DCI: HARTs Device Configuration Interface [1045]
DISN: Defense Information System Network [1046] DNS: Domain Name
Service [1047] DSCP: DiffSery Code Point [1048] FSOC: Free Space
Optics Communications [1049] GCS:Ground Control Station [1050] GIG:
Global Information Grid [1051] GUI: Graphical User Interface [1052]
HAIPE(s): High Assurance Internet Protocol Encryptor--a Type 1
encryption device that complies with the National Security Agency's
HAIPE IS (High Assurance Internet Protocol Interoperability
Specification). [1053] HART: Hybrid Autonomous Router for Tactical
Networks [1054] HART-E: HART-Edge Router [1055] HART-G:
HART-Gateway Router [1056] HART-I: HART-Interior Router [1057]
HAS-flag: HART Address Switching flag [1058] INE: Inline Network
Encryptor--broad term for HAIPE-like devices [1059] IP: Internet
Protocol [1060] JALN: Joint Aerial Layer Network [1061] JCAN: Joint
Capability for Airborne Networking subnets [1062] LQM: Link Quality
Metric [1063] M-CDL: Mini-CDL [1064] Mini-CDL: Miniature Common
Data Link Radio [1065] OSPF: Open Shortest Path First (OSPF) is an
adaptive routing protocol for Internet Protocol (IP) networks.
[1066] PT: Plaintext--Red side of red/black network [1067] QNT:
Quint Networking Technology (QNT) program is a Defense Advanced
Research Projects Agency-led (DARPA) technology program to produce
a very small and modular digital communications system for a
variety of ground and airborne applications. [1068] RIP: Routing
Information Protocol (RIP) is a distance-vector routing protocol,
which employs the hop count as a routing metric. [1069] SatCom: A
Satellite Communication link [1070] SNMP: Simple Network Management
Protocol [1071] TCP/IP: Transport Control Protocol for Internet
Protocol, a collection of protocols for routing data on an IP
network. [1072] TAT:HART Topology and Address Tracking [1073] TTNT:
Tactical Targeting Network Technology [1074] TRL: Technology
Readiness Level [1075] UAV: Unmanned Aerial Vehicle [1076] UAS:
Unmanned Aircraft Systems [1077] WNW: Wideband Networking
Waveform
* * * * *