U.S. patent application number 16/537611 was filed with the patent office on 2020-05-21 for system and method for semiotic digital encoding.
This patent application is currently assigned to Bevilacqua Research Corporation, Inc. The applicant listed for this patent is Bevilacqua Research Corporation, Inc. Invention is credited to Andy Bevilacqua, Roy Brown, Glenn Hembree.
Application Number | 20200159996 16/537611 |
Document ID | / |
Family ID | 70612933 |
Filed Date | 2020-05-21 |
![](/patent/app/20200159996/US20200159996A1-20200521-D00000.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00001.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00002.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00003.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00004.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00005.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00006.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00007.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00008.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00009.png)
![](/patent/app/20200159996/US20200159996A1-20200521-D00010.png)
View All Diagrams
United States Patent
Application |
20200159996 |
Kind Code |
A1 |
Bevilacqua; Andy ; et
al. |
May 21, 2020 |
System and Method for Semiotic Digital Encoding
Abstract
A system and method for lowering probability that low data
rates, high bit error rates, and susceptibility to signal
degradation in communication environments during storms, solar
activity, and adversarial activity is disclosed. The system and
method affect communications by minimizing the amount of
data/information that needs to be transmitted, partly by creating
an algorithmic process/method that moves knowledge and not data and
information. The system and method is based upon the realization
that structured communications often possess a similar level of
context that can be exploited to communicate full meaning
(knowledge), even when only a small fraction of the message is
transmitted to the receiver. Reducing the number of bytes
transmitted significantly reduces the probability that a
transmission will be affected by either naturally occurring or
human supplied factors present in modern communication
environments.
Inventors: |
Bevilacqua; Andy;
(Huntsville, AL) ; Brown; Roy; (Huntsville,
AL) ; Hembree; Glenn; (Huntsville, AL) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Bevilacqua Research Corporation, Inc |
Huntsville |
KY |
US |
|
|
Assignee: |
Bevilacqua Research Corporation,
Inc
Huntsville
AL
|
Family ID: |
70612933 |
Appl. No.: |
16/537611 |
Filed: |
August 11, 2019 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15874838 |
Jan 18, 2018 |
10437930 |
|
|
16537611 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 40/149 20200101;
G06F 16/9024 20190101; G06F 40/30 20200101; G06F 8/65 20130101 |
International
Class: |
G06F 40/30 20060101
G06F040/30; G06F 40/149 20060101 G06F040/149; G06F 16/901 20060101
G06F016/901; G06F 8/65 20060101 G06F008/65 |
Claims
1. A method of structuring a Semiotic Digital Encoding (SDE)
environment, comprising: configuring a base transmitting module for
sending structured text messages; pairing a remote receiving module
to be in communication with the base transmitting module; the
structured text messages comprising both permanent unchanging data
elements as well as changing data elements; converting the
structured text message into a conceptual graph representation;
forming the conceptual graph representation with individual text
fields having connective fields for the structured text; performing
a semiotic encoding of the structured text message by converting
the structured text message into transferable data, such as machine
code, and compiling the transferable data into a state transition
table; the state transition table including individual values for
the portions of the structured text that can be transferred via a
low bandwidth communication system; for a given point in time,
giving the encoded graph nodes values based on true or false
conditions; transmitting the unchanging data elements in the form
of software keys or playbooks which, when added to the changing
data elements, represent a full message; matching the encoded graph
nodes to the individual text fields of the conceptual graph
representation; thereby once the state transition table and graph
node output are established, the base transmitting module
communicating the state table and the graph node output to the
remote receiving module; the remote receiving module receiving the
state table and graph node output; the remote receiving module
recreating the conceptual graph representation; the remote
receiving using the graph node output to fill the individual text
fields of the conceptual graph representation; thereby recreating
the original structured text message.
2. The method of claim 1, further comprising: in situations with an
oft-repeated baseline message, the base transmitting device sending
only those delta values that vary from the baseline message.
3. The method of claim 1, further comprising: the base transmitting
module and the remote receiving module each comprising a laptop and
a router.
4. The method of claim 1, further comprising: a "select message
set" function within the base transmitting module selecting which
message set to use; an "encode" function within the base
transmitting module taking a raw message and outputting a SDE
encoded message; a "decode" function within the remote receiving
module taking the SDE encoded message and outputting the
reconstructed raw message.
5. The method of claim 1, further comprising: the remote receiving
module providing an SDE acknowledgement message back to the base
transmitting module.
6. The method of claim 4, further comprising: the encoding process
using a code embedded conceptual graph structure to process each
field using the encode and decode functions.
7. The method of claim 1, further comprising: a system-transparent
software monitor observing and recording all SDE messages sent in
both directions of the communication path.
8. The method of claim 1, further comprising: implementing the
encoding process as a separate hardware box within the base
transmitting module so that any legacy software present in the
communication remains compatible and is not disturbed.
9. The method of claim 1, further comprising: exploiting
communications having a permanently structured message format; the
permanent message structure being part of a known communications
standard; the permanent message structure comprising up a majority
of the information that is communicated between the base
transmitting module and the remote receiving module; once the
permanent structure is understood by both parties, removing the
permanent structure from the communications stream, leaving only
the changing data elements themselves to be transmitted and
received; the remote receiving module adding the changed data
elements to the permanent structure of the message thereby
achieving a fully lossless representation of the original
communication.
10. The method of claim 1, further comprising: graphically
rendering complex logic streams and systems designs needed for
processing cognitive knowledge into conceptual graphs.
11. The method of claim 1, further comprising: the base
transmitting module transmitting a digital state table using less
bandwidth than an ASCII transmissions of the same digital state
table.
12. The method of claim 1, further comprising: any entity
intercepting the communication being required to know both the
conceptual graph representation and the order of transmission of
the elements of the state table to reconstruct the structured text
message, otherwise the entity lacking ability to reconstruct the
structured text message.
13. The method of claim 10, further comprising: defining a number
of states in a conceptual graph go as 2.sup.N where N is the number
of nodes in the graph such that with a graph with 10 nodes, 1028
separate decision states can be defined in only 10 bytes.
14. The method of claim 1, further comprising: the encoding process
being agnostic to whether the data being encoded contains messages
or payload data, the encoding process requiring only that the
messages are structured in some way.
15. The method of claim 1, further comprising: in the event that
the base transmitting module and the remote receiving module get
out of sync, the remote receiving device sending back an
acknowledgement saying that remote receiving module is going to
sync signal to the base transmitting module therefore please send a
regular message; arranging that the next message comes through will
be reset to be the total values, with no deltas involved and not
using SDE; the base transmitting module requesting to re-sync; and
the remote receiving module then saying "yes, I received that
message".
16. The method of claim 1, further comprising: arranging that each
communication of the paired modules to be individually configurable
with a key.
17. The method of claim 16, further comprising: incorporating the
key into the communication link between the two modules.
18. The method of claim 16, further comprising: incorporating the
key as another layer in the SDE libraries.
19. The method of claim 1, further comprising: implementing the SDE
software as a library only, where only the library of encoded
messages is supplied and the base transmitting module is
responsible for monitoring message traffic and retrieving the
corresponding message from the SDE library.
20. The method of claim 1, further comprising: implementing the SDE
software as a full software implementation, in which only a
start/stop signal is required from the base transmitting module.
Description
BACKGROUND OF THE INVENTION
[0001] Future conflicts will undoubtedly involve the need to
communicate large amounts of information into, out of and within
anti-access, area denial (A2AD) environments. These environments
pose a special challenge for the Navy because of the large
geographical areas over which they must operate and their resulting
dependence on vulnerable satellite data links. High Frequency (HF)
radio communication systems have long been valued for beyond
line-of-sight links, but over the last few decades they have slowly
been replaced with more robust communications via an expanding
array of satellite networks. The saturation of the satellite
communication spectrum, the selling of parts of the spectrum to
commercial entities, and the ease with which these assets can be
electronically jammed has caused a renewed interest in the use of
the HF bands.
[0002] Commercially available HF radios suffer from low data rates,
high bit error rates, and susceptibility to signal degradation
during storms and solar activity. Not only must the information
communicated be complete while operating under difficult
environmental conditions, it must be secure to avoid man-made
interference such as jamming, electromagnetic pulses (EMP) and
interception and decoding by adversaries within the A2AD
operational area.
[0003] In response, a mechanism to lower the probability that any
of the factors mentioned above would effect communications by
minimizing the amount of data/information that needs to be
transmitted is desired. This is achieved by creating an algorithmic
process/method that moves knowledge and not data and
information.
BRIEF DESCRIPTION OF DRAWINGS
[0004] FIGS. 1A, 1B, 1C, and 1D show Prior Art devices having
communication arrangements that can benefit from the embodiments
herein;
[0005] FIGS. 2A, 2B, 2C, 2D, and 2E show conceptual graphs;
[0006] FIG. 3A is a non-limiting example of a structured
message;
[0007] FIG. 3B shows an embodiment of a Semiotic Digital Encoding
(SDE) processes described herein acting on the structured message
of FIG. 3A;
[0008] FIG. 4A shows potential hardware implementations clarifying
the embodiment of FIG. 3B;
[0009] FIG. 5 are examples of simple and complex graphical
representations created for different STANAG messages;
[0010] FIG. 6 shows an example SDE encoding process separating a
permanent structure of a message from the temporal data contained
therein;
[0011] FIG. 7 shows a conceptual graph (symbolic) representation of
the structured (permanent) part of the STANAG message shown in FIG.
6;
[0012] FIG. 8A shows data for an assortment of STANAG messages
using the embodiments herein, showing significantly reduced
bandwidth;
[0013] FIG. 8B shows a visual contrast of an message structure
prior to use of the embodiments herein, and that same message
structure after using the embodiments herein;
[0014] FIG. 9 shows SDE test software allowing individual or
streamed SDE message sets to be timed, compared and tested;
[0015] FIG. 10 shows an example SDE module employed in an unmanned
systems control scenario;
[0016] FIG. 11 shows an SDE system module design and
functionality;
[0017] FIG. 12 shows an example SDE library generation block
diagram;
[0018] FIG. 13 shows an example SDE software block diagram;
[0019] FIG. 14 shows a top-level hardware implementation of an
embodiment of a ground control unit side;
[0020] FIG. 15 shows a top-level hardware block diagram of an
embodiment of a unmanned platform side; and
[0021] FIG. 16 shows an embodiment of an example hardware design
diagram.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] It is well-known in the field of psychology that the human
brain enhances its ability to maximize human survival by adopting
ultra-efficient operating strategies. Many of these efficiencies
involve what appear to be mental short cuts that provide the
minimum required knowledge necessary to allow humans to quickly and
efficiently solve problems ultimately related to their need to
respond quickly to threats. A well-known example is the ability of
humans to gain a full understanding of written communications when
they read cryptic sentences with missing words or jumbled letters.
In this specific case the brain provides understanding needed to
the human by using the information naturally present in the context
between the letters and words being read. In other words, the
logical structure in a sentence carries a great deal of information
about the meaning being communicated by that sentence.
[0023] The embodiments herein are based upon the realization that
structured communications possess a similar level of context that
can be exploited to communicate full meaning (knowledge) even when
only a small fraction of the message is transmitted to the
receiver. Reducing the number of bytes transmitted significantly
reduces the probability that a transmission will be affected by
either naturally occurring or human supplied factors present in
modern A2AD environments.
[0024] FIGS. 1A, 1B, 1C, and 1D show non-limiting Prior Art example
devices having communication arrangements that can benefit from the
embodiments herein. While FIGS. 1A-1D show Unmanned Aerial Vehicles
(UAVs), the embodiments herein should not be considered as limited
to UAVs.
[0025] Some UAVs communicate in known, specific protocols, which
can include but are not limited to STANAG4586 and JAUS. STANAG 4586
(NATO Standardization Agreement 4586) is a NATO Standard Interface
of the Unmanned Control System (UCS) Unmanned Aerial Vehicle (UAV)
interoperability. STANAG4586 defines architectures, interfaces,
communication protocols, data elements and message formats. Another
communication protocol is Joint Architecture for Unmanned Systems
(JAUS).
[0026] Communication devices working with STANAG or other
data-configuration may have several one byte words which they are
only using first bit value, e.g. "yes" or "no", or first 3 bit
values, e.g. numbers from 0-8. From a typical, conventional
computing-efficiency standpoint, it is cheaper to just store the
entire byte even though there are only 3 bits of actual value,
rather try to store only 3 bits of a much larger byte. Thus, for
most contexts, it is more efficient to transfer data using such
standard, easily transportable data-size-units (bytes), somewhat
comparable to container-loads on a Long Beach CA shipping dock.
Thus, one reason the embodiments described herein are not used on
typical computers is because it's easier to retrieve an entire byte
or bring up a word and put back a word, that is to try to parse,
mask, and manipulate the individual bits within a byte.
[0027] The embodiments herein reduce the amount of information that
needs to go through the pipeline in order to be successful
transmitted from point A and successfully received by point B. It's
easier to successfully transmit/receive a smaller amount through
the communication spread frequencies than larger amounts of data. A
hardware communication device that was doing a lot of
frequency-skipping, e.g. spread spectrum or skipping spectrum,
would work well with the embodiments herein.
[0028] It is important to note that the SDE embodiments described
herein are not designed to obscure or add a layer of security.
Instead, the embodiments herein act to reduce data loss. However,
it is understandable if persons working with the embodiments herein
use expressions such as data compression, encryption, as the
embodiments herein can sometimes be confused with such elements.
But make no mistake: the embodiments herein are neither data
compression nor encryption. A better expression might be "bandwidth
enhancement" and/or "improved communication efficiency".
Real-World Practical Usages
[0029] Unmanned aerial vehicles are a clear and obvious example of
usage of the embodiments herein because of their need to send
signals way up in the sky. However, the embodiments herein are not
restricted to UAVs, but instead pretty much any usage which could
benefit from a reduction in data stream or storage.
[0030] One possible usage is a military context, for command and
control of UAVs or other remote vehicles (submarines, tanks,
robots). The embodiments can make it harder for an adverse party to
achieve an aerial turn off, or where the there's actively people
trying to reduce the amount of data that can be sent between
devices through jamming or interference.
[0031] Some large UAVs may cost the federal government as much as
$10M apiece, although most of the devices illustrated in FIGS.
1A-1D cost considerably less than that. Meanwhile, the combination
of a single laptop and single panda latte may cost e.g. $150
apiece. As stated, two such combinations are needed for the
embodiments herein to be effective, totaling e.g. $300. Thus, a
$300 investment can potentially save the US government a
$10,000,000 UAV.
[0032] Using the example of UAVs, an adverse party's primary goal
might be to either disrupt communication, or to try to take over
communication. Once the records in SDE system herein are in place
and the two ends are connected, the SDE system then starts sending
only the Delta values. An adverse party would need the original
message in order to receive the entire message, because part of the
data savings is to only send the Delta value. This data savings has
the added advantage that unless an adverse party was in on the
communications from the beginning, if they divert, intercept, or
copy the signal mid-stream, they won't know what to do with the
data.
[0033] So, even with a data scope, an adverse party can see a large
stream of data, but that adverse party might not know where that
data is going or what the data means.
[0034] Instead, the primary reason would be to either disrupt its
communication or to try to take over its communication and do they
do that they have in the past. Once a record is confirmed at both
ends, the system then starts sending only the Delta values. In this
way, an adverse party would need the original message in order to
receive the entire message because part of the data savings is to
only send the change in the values, which eats less bandwidth than
the original value.
[0035] Next, a non-UAV example might include the Internet of
things, such as municipal streetlights set up as Internet-capable.
Each streetlight produces a little bit of data that has to be
streamed to municipal servers or storage or something like that.
Multiply the huge amount of streetlights. Not a huge amount of data
for each streetlight, but after adding 1000 or 10,000 streetlights
that becomes a large amount of data, which can in turn increase the
necessary bandwidth for maintaining the lights. There is also a
problem that hackers are sometimes taking municipalities hostage,
ransomware, etc.
[0036] Now add in voice data to these streetlights or traffic
lights, e.g. for blind people to know when to cross a street, and
when to not go across. Also, some municipalities implant
microphones in the light poles, for the purpose of public safety.
For example, gunshots are another area that some municipalities
monitor, sometimes from light poles, in that each gunshot may have
a unique signature and can sometimes be admissible evidence in
criminal court. So let's say it's a gunshot. Normally that sonic
signature would take up a large wide spread of data words. But
gunshots seldom occur, so the streetlights should avoid sending
such a large spread of data words un-necessarily.
[0037] The embodiments herein strive to reduce each data field
independently, and then group those fields back into a main message
which is sent often. Doing so can make the main message much
smaller, especially with an oft-repeated baseline message, and then
from there just send Delta values that vary from the baseline. This
variation can be a small number depending how the number of
alterations in the data occur. Sometimes the variations will
incrementally increase overtime, sometimes will decrease, sometimes
they'll wait around a certain value, sometimes they don't change it
all, where they are pretty much set for the entire mission, and
sometimes they're all over the board.
[0038] To address these issues requires discussing semiotics for a
brief period. Semiotics deals with the function of those of signs
and symbols in both artificially constructed and natural languages
and comprises syntactics, semantics, and pragmatics. For example,
one aspect of semiotics includes the study of languages and how
humans communicate. Encoding, in semiotics, is the process of
creating a message for transmission by an addresser to an
addressee. The complementary process--interpreting a message
received from an addresser--is called decoding.
[0039] Semiotic Digital Encoding (SDE) is an algorithmic approach
that uses a completely new paradigm for reducing the number of
bytes required to send data between entities such as ground
stations and unmanned vehicles or between manned and unmanned
platforms. In fact, an SDE communications scheme can be completely
independent of the type of platform (manned or unmanned) or type of
network being used. The fact that it is lossless means that this
algorithm can be used on both new and legacy systems. Another side
effect of this new scheme is that the encoding process is also
inherently secure. Once certified this will allow bandwidth to be
further reduced by removing the overhead associated with encryption
from the data stream.
[0040] To make the embodiments herein work for communications
between non-human entities, a method had to be found to preserve
that context/structure that was non-semantic. The solution was to
use the context inherent in structure provided in the messages
themselves. That context, which is a permanent artifact of
structured messaging standards such as STANAG4586 and JAUS, is
separated from the changing data that those permanent elements
contain. In this way the permanent elements can be shared
beforehand in the form of software keys or `playbooks` which, when
added to the changing data elements, represent a full message
representation that meets current military communications
interoperability standards (IOP's).
[0041] The SDE embodiments described herein contain several
functions, including but not limited to the following: [0042] A
"select message set" function, which selects which message set to
use. [0043] An "encode" function, which takes the raw message and
outputs the SDE encoded message. [0044] A "decode" function, which
takes the SDE message and outputs the reconstructed raw message and
possibly an optional SDE acknowledgement message.
[0045] The SDE products described herein utilize conceptual graph
structures to map out and efficiently encode a structured message
into a compact bit stream that removes unneeded information bits.
Structured field values are dynamically examined and repackaged
into smaller byte counts depending on the delta from the expected
value. This takes advantage of the fact that much of the control
data is static, fluctuates within a limited range, and increments
and/or decrements slowly. In the future if other data fields that
contain a limited enumerated set of values are experienced when
implementing the algorithm, it may be more efficient at times to
send just the bits used. This can be decided on a case-by-case
basis.
[0046] In an embodiment, the encoding process uses a code embedded
conceptual graph structure to process each field using the encode
and decode functions mentioned above.
[0047] Various of the embodiments herein operate simply as a
system-transparent software monitor that watches messages coming
across the network to the transmitter and replaces them with
encoded versions pulled from a library. However, the encoder can be
implemented as a separate hardware box so that legacy software is
not disturbed in any way.
[0048] Either way, an SDE module (hardware or software) must be
present at both the transmitting and receiving ends to allow for
encoding and decoding of the original messages to/from the SDE
format. That format is simply a bit stream of ones and zeros that
tell the keys containing the permanent structure supplied by the
standard being used how to reconstruct the message into the format
required by legacy systems.
[0049] Further, a set of software libraries for multiple
communications standards are located on a single chip that can be
easily inserted into a communications stream in front of radios on
ground stations and platforms. When communications are initiated
between the ground and an air platform, or between two air
platforms, an initial handshaking message identifies which language
(standard) each is speaking, thereby setting up the communications
between the entities from that point forward. This enables the SDE
embodiments described herein to act as a universal translator for
multiple platforms across the services. Maintenance would only
require libraries to be modified and uploaded when the
communications standards themselves are changed.
[0050] The embodiments herein create a knowledge-handling software
toolkit from existing tools that develop and edit conceptual
graphs. Various of the tools herein will convert these graphs into
either the Conceptual Graph Interchange Format or CXML. Both of
these formats are part of the International Standards Organization
(ISO) new standard for Common Logic (CL) number 24707 that is
designed to standardize formats for sharing knowledge.
[0051] Conceptual graphs represent a complete first order predicate
logic system and can therefore be `processed` in ways that make
cognitive decision aiding or contextual knowledge understanding
possible. To achieve this goal requires processing "cognitive"
knowledge, as discussed with respect to FIGS. 2A-2E. A state table
translator encodes the knowledge in the conceptual graphs into ones
and zeros for secure ultra low bandwidth transmission between
systems. An observer module that decodes the state tables back into
representational knowledge for the receiver.
[0052] While numerous non-military applications exist, the
embodiments herein take advantage of the fact that many military
communications are permanently structured in some way. This
structure is usually documented as part of a communications
standard that is used by anyone that wants to communicate using a
military system. This standardized (permanent) structure makes up
the bulk of the information that is communicated between the sender
and receiver. Once that permanent structure is understood by both
parties it can be removed from the communications stream, leaving
only the changing data elements themselves to be transmitted. The
receiver can add that data with the permanent context of the
message on the receiving end to gain a full lossless representation
of the original communication.
[0053] Conceptual Graphs (CGs) have a foundation in logic
representation, and also have first-order and predicate logic
operations defined, allowing for inference and theorem proving
procedures. Visually, a Conceptual Graph (CG) mimics the knowledge
representation ability of diagrams used in discussions using
whiteboards, slides, and on the backs of napkins. As shown in FIGS.
2A-2E, the CGs are often text snippets, typically enclosed in
squares or ovals and lines (possibly with a label) connecting one
text snippet to another. Experts often use these visual aids to
quickly and effectively communicate complicated, technical details
during brainstorming sessions, a feature that makes them a good
candidate for use by systems in sharing knowledge. In CGs, text
snippets are called Concepts, and line connections are called
Conceptual Relations. A Concept may also contain another CG to
provide contextual or nested information.
[0054] For example, the conceptual graph to describe the situation
that a person (concept) is between a rock (concept) and a hard
(modifying concept) place (concept) would be represented as shown
in FIG. 2A. Although the example shown in FIG. 2A is intentionally
simple, it still demonstrates how the extremely complex logic
streams and systems designs needed for processing cognitive
knowledge can be developed graphically using conceptual graphs
(CGs).
[0055] Referring now generally to FIGS. 2B-2D, one embodiment
provides a system and method made used for semiotic digital
encoding decoding and transfer and is generally designated by the
numeral 10. Referring now to FIG. 2B, the system 10 starts with
structured text 12. This structured text can be a text or message
as desired, including complete sentences and normal expressions.
The system 10 starts by converting the structured text 12 into a
conceptual graph representation 24 at step 14, and as further
describe below.
[0056] Next, the system 10 performs a semiotic encoding of the
structured text 12 by converting the structured text 12 into easily
transferable data, such as machine code, as compiled in a state
transition table 28, or simply state table 28. As further described
below, the state table 28 includes individual values 30 for the
portions of the structured text 12 that can easily be transferred
via a low bandwidth system. Next, the system 10 allows
communication of the state table 28 by a communication step 34 to a
receiver having the conceptual graph representation 24. The
receiver then uses the state table 28 to complete the conceptual
graph representation 24 to extract the meaning of the original
structured text 12.
[0057] FIG. 2C shows an example of more detail of a preferred
embodiment of the conversion step 14 of the system 10. In this
step, a conceptual graph representation 24 is formed from the
structured text 12 at step 23. This formation can be
contemporaneous to some of the other steps of the system, for
example in the step of the creation of the state table 28, or the
formation 23 of the conceptual graph representation 24 can
preferably be created in advance. The conceptual graph
representation 24 can take various form that can help establish the
concept of the structured text 12 that is being transfer and
communicated. For example, the conceptual graph representation 24
can use syntax to establish the generally grammatical form of the
message being delivered. In FIG. 2C, this is shown by the
conceptual graph representation 24 with individual text fields 26
having connective fields 25 for the structured text 12.
[0058] After the conceptual graph representation 24 is established
for the structured text 12, a state table 28 can be formed by step
27, as exampled in FIG. 2D, which shows an example of more detail
of the formation of a state table 28. In this step 27, for a given
point in time an encoded graph nodes 30, or individual values 30,
are given values base on true or false values: 1=true and 0=false.
The encoded graph nodes 30 match up to the individual text fields
26 of the conceptual graph representation 24. This creates the
state table 28 as exampled by the graph node output 32.
[0059] As seen in FIG. 2E, once the state table 28 and graph node
output 32 are established, the state table 28, and the graph node
output 32, can be efficiently transferred/communicated in step 34
over a low bandwidth system. At the other end of the transmission,
a user/receiver can receive the state table 28 and graph node
output 32. That receiver will have the original conceptual graph
representation 24 and then will begin recreate the conceptual graph
representation 24 in step 36. The receiver will use the graph node
output 32 to fill the individual text fields 26 of the conceptual
graph representation 24 to recreate the original structured text
12. In this manner the message can be known to the receiver.
[0060] In the embodiments herein, preferably, and in some instances
necessarily, the starting point is the structured message/text 12
that establishes the relative positions of each word or piece of
information. This facilitates the structure of the message and
allows the binary nature of the transferred data to carry the
desired message. Further, the receiving entity should have the same
conceptual graph representation 24 as the sending entity to allow
for the proper decoding. Further, knowing the transmission order of
the graph node output 32 can be important to allow the receiver to
reconstruct the conceptual graph representation 24 from the state
table 28. Since the receiver has the knowledge of the relationship
of state table 28 to the conceptual graph representation 24 created
from the original structured text 12, the receiver is able to
reconstruct the original structured text 12.
[0061] The embodiments herein have many benefits. This can include
the fact that the digital state table requires less bandwidth for
transmission that typical ASCII transmissions. Further, this system
and method is inherently secure. In order to reconstruct the
structured text 12, anyone intercepting the communication would
have to know both the conceptual graph representation 24 and the
order of transmission of the elements of the state table 28. Thus,
the system and methods described herein can be used to efficiently
and securely transfer data.
[0062] In addition to their ability to visualize knowledge, CGs
have many advantages over other classic logic processing systems
such as neural networks, Bayesian networks, inferencing, etc.
including: [0063] The system is graphical, which allows information
to be translated into understanding in the mind of the human much
more quickly. [0064] The graphs are semantic (human language),
which allows for better understanding of complex concepts. [0065]
The graphs are easily manipulated/modified using COTS/GOTS
(Commercially Off The Shelf/Government Off the Shelf) tools. [0066]
As a parallel, object-oriented schema, the representation is
completely non-brittle, allowing analysts to easily change logical
connections in the graph as things change, without completely
re-doing the graphs. [0067] The number of states that can be
defined in a conceptual graph go as 2.sup.N where N is the number
of nodes in the graph. That means that with a graph with as little
as 10 nodes, 1028 separate decision states can be defined in only
10 bytes.
[0068] Through the use of conceptual graphs as a schema for
handling cognitive knowledge, the embodiments herein overcome the
limitations imposed by older approaches and provide a stable,
easy-to-use baseline for the creation of an automated knowledge
capture, storage and processing environment to share secure
communications at ultra-low bandwidths.
[0069] Within the embodiments herein, expert knowledge is input
directly into the toolkit by SMEs or analysts in graphical form by
dragging icons down onto the screen. The icons are then connected
by the user into logical sequences. The toolkit automatically
generates the ISO standard or XML. A specially designed parser
turns this standard language into executable conceptual graphs that
can be stored as objects, XML streams, or text files.
[0070] Although any structured messages or data can be encoded
using the embodiments herein, for illustration and example within
this disclosure, the message sets commonly used for command and
control of unmanned assets within the DOD will be used,
specifically the STANAG4586 and JAUS message sets. The encoding
process does not care if the data being encoded contains C2
messages or payload data. As long as those messages are structured
in some way, the embodiments herein can be applied. This means,
however that raw payload data, such as video data is not a good
candidate for the encoding process because this data is usually
sent as an unstructured bit stream.
[0071] To illustrate how the SDE encoding process is accomplished,
the STANAG4586 message set is used, although JAUS messages are
encoded much the same way. FIG. 3A shows an example of the
STANAG4586 C2 message ID number 3003 (Mode Preference Report). This
example shows that structured messages come as a collection of
pre-defined fields that specify the length, format and precision of
each field element.
[0072] Using the structured definition from FIG. 3A, an example SDE
system 10 shown in FIG. 3B effectively separates the permanent
structured context of the message from the actual data being
transmitted. The permanent context becomes a key that allows the
full original message to be reconstructed without any loss on the
receiving end. Normally, to ensure that the communications are
secure, these keys would have to be protected. However, without
knowing the order of the bits in the transmitted data it would be
extremely difficult to decode the message meaning. Therefore by
periodically changing the byte order of the transmitted data, full
communications security is achieved.
[0073] FIG. 3B illustrates how Semiotic Digital Encoding works by
symbolically encoding the permanent parts of structured data
standards, using the elements of FIG. 3A as an example. Only those
data elements within FIG. 3A that change are then transmitted, and
added back to the permanent context again at the receiving end
without any loss. Next, FIG. 4A shows potential hardware
implementations of the SDE system 10, clarifying the embodiment of
FIG. 3B, showing how the SDE modules 400 can comprise, in an
embodiment, a laptop and a router. In FIG. 4A, the specific SDE
hardware modules 400.sub.base and 400.sub.remote look the same, but
that is merely an example and the embodiments herein are not
limited as such.
[0074] As mentioned previously the SDE library creation process
essentially separates the context contained in a structured message
from the knowledge that resides within that structure. The first
step in this process is therefore to decompose each structured
message in a NIOP standard such as STANAG4586 into a symbolic
representation of itself using conceptual graphs. This is a
manually intensive process as each field in the message is
represented by a separate concept node. Also, every message graph
will be different. Depending on how much information is contained
in a message's fields, these representations can be small and
simple or large and complex.
[0075] An example of actual graphical representations for separate
STANAG4586 messages are shown in FIG. 5. Within FIGS. 5, 6, 7 and
8B, the various shapes e.g. (oval, rectangle, square, circle)
represent types of message-contents, e.g. data, instructions,
positions, status or other. For simplicity and ease of
understanding, almost all text identifiers and markings associated
with the shapes within FIGS. 5, 6, 7 and 8B are removed, except for
a small portion of FIG. 7. This removal is first because the
markings would be so small they would be impossible to read, and
second, because FIGS. 5, 6, 7 and 8B are meant to convey principles
of the invention using shapes and visual comparisons, such that
specific data or instructions are not needed.
[0076] FIGS. 6, 7 and 8B are partly meant to convey that there
exist some nodes required for the vehicle operating mode message
therein that would result in the need for only a few bytes to be
sent. However, in sharp contrast, FIG. 5 shows that the Vehicle
Steering Command message requires a large number of nodes to
represent the knowledge contained in the message. Since every
STANAG4586 structured message requires up to 640 Bytes in its
native mode, the fewer nodes needed to graphically (symbolically)
represent the knowledge contained in the message, the more
bandwidth is saved.
[0077] Using the graphical representation of the message from FIG.
5, one can now use that information to encode a bit stream based on
the status and value of each node. Knowledge is encoded based on
the status of each node is (i.e. true or false). Node values are
encoded into the bit stream using the least number of bits to
represent the value or change in value from the previous
acknowledged checkpoint message. The bit stream created for
transmission is simply the collection of bits representing the
state of each node where one (1) is used if that node is true and a
zero (0) is used if that node is false.
[0078] FIG. 6 shows this process from end to end, and FIG. 7 shows
a visual arrangement of some portions of FIG. 6, including at least
altitude, speed, and headings.
[0079] Although the initial decomposition of the message structure
(e.g. FIG. 6) into symbolic format (e.g. FIG. 7) is labor
intensive, once all messages have been translated into a computer
readable format, they are run through an application that generates
the individual message handling code for each message structure to
be stored in the SDE software library (see e.g. FIGS. 11, 12, and
13). They do not need to be changed again unless the standard
itself changes. The SDE library contains the equivalent graphical
key function embedded in machine code. By using the graphical key
embedded library on both ends of the communications link (sender
and receiver), the SDE embodiments described herein becomes what is
referred to as a symmetric key algorithm. Once the bit stream that
represents the knowledge contained in the message is encoded by the
sender, those bits are used to fill in on the receiver's side and
the process is reversed to re-create the original STANAG4586
message. In this way SDE is a completely lossless process.
[0080] In the example shown in FIG. 7, the mode attributes of
Altitude, Heading and Speed would normally all be transmitted as 8
bit words. To report the status of each mode would require 24 bits.
By separating out the structure, the system 10 reports each mode as
a 1 if it is true and a 0 if it is false relying on the relative
position in the conceptual graph to tell the receiving entity which
mode is active at any point in time. This only requires 3 bits, an
almost 90% reduction in that part of the size of the message on the
receiving end. Because each message has a different compression
ratio the total compression achieved through the encoding process
depends on which messages are used and how often they are sent.
[0081] The secure nature of this process results from the fact that
the bit stream of one's and zero's has no inherent meaning unless
one possesses the graphical key being used. Even if an adversary
had the key, changing the order of the bits being transmitted would
require all combinations of bit orders to be created to decipher
the message. The node preference report message from FIG. 7
contains 24 nodes, so there are 2.sup.24 different possible
combinations (over 16 million). Although not impossible, to
decipher the bit stream quickly becomes impractical because of the
magnitude of the stream. By the time a hacker or unauthorized
person de-constructed and re-constructed the stream, the
information would already be stale.
[0082] FIG. 8A shows objective tests run on the bandwidth
differences between the new encoding technique and two other
well-known mathematical compression techniques. The results showed
that SDE was several orders of magnitude more efficient than the
mathematical compression techniques.
[0083] Extrapolating from FIG. 8A, the embodiments herein achieved
greater than 80% reduction in bandwidth for all scenarios exercised
using STANAG4586 and approximately a 50% overall reduction for
JAUS. The smaller compression results for JAUS result from the fact
that unlike STANAG4586, which is a message set, JAUS is actually a
communications protocol over which JAUS messages are sent. For that
reason JAUS messages are generally smaller to begin with, making it
more difficult to achieve a reduction in size. Having said that 50%
reduction is still remarkable compared to most known compression
techniques commonly available today. Again, the embodiments herein
do not perform compression, but are sometimes compared to
compression.
[0084] FIG. 8B shows an easy, non-verbal, entirely visual
comparison of how the SDE embodiments herein de-complexify a
typical message.
Review of Test\Emulation GUI (FIG. 9)
[0085] The embodiments herein further comprise a software test
suite that allows sending and receive both STANAG4586 and JAUS
messages. As shown in FIG. 9, a dashboard 904 within the test suite
allow developers to input both message types and compare SDE timing
and bandwidth usage against existing standard message sets. Within
FIG. 9, CUCS stands for Computer User Control System, while VSM
stands for Vehicle System Management.
[0086] Within FIG. 9, the size ratios 908 of the messages are shown
in the form of <somewhat> hot dog and bun shapes. These are
explained as follows. The small bar (hot dog) in the middle is the
original size (meat) of a message. The surrounding (bun) part is
how much that message was reduced to. The "count" field 928 shows
an example number of messages of that type which have been sent.
From FIG. 9A it is apparent that a 42% reduction in message size
has been achieved, completely lossless, without compression.
[0087] Additionally, the SDE embodiments perform pre-processing on
the various messages, including having the ability to convert from
one type to another type, e.g. fixed integer to float. The SDE
system 10 thus permits conversion from a float to a fixed integer
of that size, and then back, with relatively little loss.
[0088] The embodiments herein separate out the structured and
unstructured parts of a communications stream. The major difference
between structured messages and image-based payload data is that
many times the payload data is transmitted in a raw or
semi-structured format. There are exceptions, however. Even raw
imagery data contains structure that can be encoded using the
semiotic digital encoding (SDE) techniques described herein, of
which the SDE system 10 is but one example. Each data point in a
raw imagery data stream contains context that identifies its
location relative to other data points and its relative intensity
and point in time (for video).
[0089] For example, to save time and bandwidth, payload data is
sometimes pre-processed on the sensing platform before it is
transmitted to the ground, using an imagery standard such as RS170,
RGB or SMPTE-294M. In these cases, like the STANAG4586 and JAUS
standards encoded with C2 messages, the embodiments herein can be
configured to an identical approach, but only minimal compression
gains would be realized.
Problem Solving, Error Correction, Debugging, Error Recovery
[0090] There are scenarios in which the sending/receiving SDE
modules 400 can have trouble, problems, battery failure,
transmission problems, gun shots, bullet impact, or need to be
rebooted. All of these conditions have in common that the two
modules 400.sub.base and 400.sub.remote can get out of sync. Now
suppose, at that same instant, a transmission said "lets change
from float to fixed-integer". A problem could occur where the
receiving SDE module 400 on the other side could say "what? you
didn't tell me you went from fixed integer to float". If one side
forgets the agreed-upon face value of the message which can happen
say, need to reboot it for whatever reason, there is still a part
of the header that says " . . . sending this message type using
agreed-upon ID number". This in turn assists in error-recovery.
[0091] Another condition is where an SDE module 400 sends back
acknowledge saying that SDE module 400A is going to sync signal to
SDE module 400B, so send a regular message. If everything goes
properly, the next message that comes through will be reset to be
the total values, with no deltas involved, thus taking a temporary
break from the SDE system 10 described herein. SDE module 400B will
ask to re-sync at that time. The remote side (e.g. SDE module 400A)
will then say "yes, I received that message" and once the original
side (e.g. SDE module 400B) gets the acknowledge, it starts using
that base value at that point.
[0092] As such, an adverse party would have to possess a certain
key, which is easily reconfigurable and would not be the same key
for all implementations of the embodiments herein.
[0093] Each SDE module 400 would differ and be individually
configurable to be set up with a key, either as part of the
communication link either on the radio that's communicating between
the two devices, or a key that is added as another layer in the
software libraries (described in more detailed elsewhere
herein).
Design Specs
[0094] This following sections discuss and illustrated some
potential non-limiting hardware and software design specifications
for the embodiments of the SDE system 10. The software design
specification (e.g. FIGS. 12-13) describes the software structure,
functions and intended operational modes of the SDE system 10.
Meanwhile, the hardware design section (e.g. FIGS. 11, 14-16)
presents a draft design for a potential, non-limiting hardware
implementation of the SDE system 10.
[0095] For purposes of example, the following example hardware and
software designs will illustrate digitally encoding and decoding a
NATO Standardization Agreement (STANAG) 4586 message set, as a way
of providing a specific example implementation of that process. The
4586 message set was developed to be an open standard for
interoperable communications between ground stations and unmanned
air vehicles. As a standard, the STANAG messages are considered to
be "structured messages" because their size, structure, fields,
content, etc. are pre-defined.
[0096] Before moving to the examples however, it is important to
note that this specification details both non-limiting example
hardware and software designs that could be used to build a
specific implementation of a product that could be implemented
within existing software, or built as a stand-alone hardware
implementation for legacy systems or systems where a straight
software implementation is not feasible or desired. These designs
are for illustration and example only, so that the embodiments
herein should not be considered as limited exclusively thereto
[0097] FIG. 10 shows an overview of how the system will be used to
process structured command and control messages using an example of
unmanned vehicle control. As stated, the SDE principles described
herein, including but not limited to the SDE system 10, can be used
in a variety of other contexts not related to unmanned vehicle
control. In this context the "system" refers to the SDE module
itself and not the other associated elements shown in the FIG. 10.
Note that the physical configuration of the SDE unit can be
different depending on the requirements of the host platform or
system to which it is connected. For example, in an unmanned system
control system, the size, weight, form factor, and power for a
particular SDE module on the air platform side are all going to be
of paramount importance, whereas those factors are probably not
important on the ground station side. Thus, even though the
physical parameters of the paired SDE modules may be different,
their functionality will be identical.
[0098] As shown in FIG. 11, the function of an SDE module 400 in
the "OFF" mode is to allow outgoing STANAG4586 messages from the
local base to continue through the router to their original IP
address and to allow incoming STANAG 4586 messages from a remote
host to pass through the router to their original address.
Conversely, the function of an SDE module 400 in the "ON" mode is
to intercept the outgoing STANAG 4586 messages and replace them
with encoded SDE compressed format and to intercept SDE message
format from an outside host system and encode those messages into
STANAG 4586 messages before they are delivered to their original
addressed recipients.
[0099] Within the embodiments herein, there exist at least three
configurations in which the SDE software can be implemented, as
follows: [0100] LIBRARY ONLY: In this configuration, only the
library of encoded messages is supplies and the host is responsible
for monitoring message traffic and retrieving the corresponding
message from the library. [0101] FULL SOFTWARE IMPLEMENTATION: In
this configuration, only a start/stop signal is required from the
host. The bridge module then handles the tasks of monitoring
message traffic and retrieving the corresponding message from the
library. [0102] HARDWARE IMPLEMENTATION: The hardware
implementation hosts the full SDE software in a separate box that
is physically inserted between the message generator and the radio
on either or both ends.
[0103] Note that these implementations can be mixed. For example, a
LIBRARY ONLY implementation on the ground side can work with a
HARDWARE implementation on the platform side.
[0104] To implement SDE as a software function within an existing
processor (either on the ground or platform side) it is assumed
that enough spare memory and processing power exists to implement
the software and that the necessary signals to start/stop that
function can be added. It is also assumed that STANAG4586 messages
from the host system can be re-routed to the SDE module and that a
small amount of latency in the final transmission of the encoded
digital stream is acceptable.
[0105] If implemented as a "plug-in" software function that is
inserted between a ground station output and a radio on the ground
(local) side and between the radio and navigation and control unit
on the UAV (remote) side, the SDE functionality can either be
constantly encoding/decoding signals or turned on/off using
software logic built into the existing transmitting/receiving
software modules.
[0106] Once initiated, the SDE software module reads each
structured message being transmitted and compares that message to
its library of e.g. STANAG4586 messages. If the message is found in
the library, it is replaced with the encoded format. If it is not
found, the original message is released to be transmitted in its
original format. On the receiving side a similar process reads each
message, recognizes those that are in the known encoded format, and
decodes them and passes un-encoded messages without
interference.
[0107] The SDE library contains the encoded forms of all existing
STANAG4586 messages. Other types of encoded messages can also be
added to the library or separate libraries based upon functional
message types.
[0108] The SDE software comprises the interface bridge module 1104
and library module 1108. The SDE library module 1108 is a
procedurally generated code utility for converting a fixed stream
message format such as STANAG-4586 into an encoded stream message.
SDE encoding takes advantage of the knowledge inherent in a fixed
stream message format and the reduction of redundant data to
produce a smaller compact message footprint. The SDE interface
bridge 1104 is an application to intercept a fixed message format
stream on both sides of a communication link either being called
implicitly by the message applications or by being configured as
the message targeted port on both ends of a communication link. The
SDE interface bridge 1104 uses the SDE library 1108 to encode and
decode the messages on both ends of the communication link. The
interface bridge module 1104 is a utility to retrofit communication
applications that designs do not call the library module 1108
directly.
[0109] The SDE library module 1108 can be written in C, and is
designed to be OS agnostic. In its simplest implementation the
library module 1108 is supplied and the host handles all of the
bridge functions of retrieving encoded messages from the library.
Although each message library will require different amounts of
memory depending on the length and complexity of the message set
being encoded, the current SDE library module 1108 requires 347 kb
of memory space while the SDE test bridge module 1104 requires 83
kb.
[0110] When the full implementation of library and bridge is
implemented, the software only requires a single interface signal
that notifies the bridge software that it should begin the
encoding/decoding process.
[0111] FIG. 12 shows a block diagram of the modules required to
allow BRC to generate a new library module or edit or add to an
existing executable library module 1108. Once the library module is
complete it can be integrated into an existing communications
stream as shown in FIG. 13.
[0112] Note that if the existing BRC bridge module 1104 is used,
only one interface signal is needed to start/stop the encoding
process. If a base prefers to handle the replacement of STANAG4586
messages with the encoded format from the library (ground control
base side) or the reconstruction of the encoded stream back into
STANAG4586 messages (remote platform side), the embodiments herein
can accommodate such a condition for but purposes of brevity that
is not described in more detail herein.
Suggested, Non-Limiting Potential Hardware Embodiments
[0113] As shown at least within FIG. 4A, the SDE modules 400 can
comprise e.g. a conventional laptop and a router in the form of
e.g. a panda latte, which is a small pi laptop similar to raspberry
pi, except that a panda latte is basically a Windows device rather
than a Linux device. Still, there no reason to not modify it to run
on Linux, or other custom hardware.
[0114] The panda latte is not a true router in the traditional
sense, but is being used as a router for illustration purposes
within this disclosure. The key principle is to run the SDE
embodiments on a small device configuration that does not go out of
the production, and panda lattes, raspberry pi's, Windows, and
Linux are all likely to be around for a while and not become
obsolete. Ideally, the SDE embodiments are embedded into a device
which would have at least 2 network feeds, and routing capability,
which is fulfilled by e.g. the laptop and panda latte, but can also
be fulfilled by other configurations.
[0115] However, supposing software implementation within an
existing processing module is impractical because of issues such as
legacy qualifications, size, weight, power or other issues, the SDE
software can be implemented in its own hardware container on its
own processor and inserted into the signal communications path at
the appropriate place.
[0116] FIGS. 14-15 show example hardware implementations. The
following is an example-only, non-limiting list of potential
components suitable for implementing an embodiment of SDE as a
hardware system add-on: 12 V, 1 A power supply, with a DC Jack
Output; and 4 BNC cables for e.g. 1) 4586 Messages In, 2) 4586
Messages Out, 3) Radio Receive, and 4) Radio Transmit. In an
embodiment, additional requirements for loading a new program into
the SDE hardware module might include RJ45 Ethernet Cable and
computer, HDMI video cable, HDMI monitor, USB for a mouse, and USB
for keyboard.
[0117] In an embodiment, an example SDE Board comprises 2 GB DDR3
RAM, an AMD embedded G-Series System-on-a-Chip (SoC), a GX210HA
with 1 GHz dual core processor, and a 16 or 32 GB MicroSD Card.
[0118] For clarification, FIG. 16 shows an example hardware design
diagram.
* * * * *