U.S. patent application number 12/392975 was filed with the patent office on 2009-09-10 for method and apparatus of duplicate packet detection and discard.
This patent application is currently assigned to FLUKE CORPORATION. Invention is credited to James W. Kisela, Steven Koller.
Application Number | 20090225676 12/392975 |
Document ID | / |
Family ID | 41053476 |
Filed Date | 2009-09-10 |
United States Patent
Application |
20090225676 |
Kind Code |
A1 |
Kisela; James W. ; et
al. |
September 10, 2009 |
METHOD AND APPARATUS OF DUPLICATE PACKET DETECTION AND DISCARD
Abstract
Duplicate packet detection and discard employs hash values and
time stamps of received packet data, wherein the hash value is
employed as an index to a table, and if data at a table entry has a
time stamp near a time of the packet data, a duplicate is
determined.
Inventors: |
Kisela; James W.;
(Snohomish, WA) ; Koller; Steven; (Yorktown Hts,
NY) |
Correspondence
Address: |
PATENTTM.US
P. O. BOX 82788
PORTLAND
OR
97282-0788
US
|
Assignee: |
FLUKE CORPORATION
Everett
WA
|
Family ID: |
41053476 |
Appl. No.: |
12/392975 |
Filed: |
February 25, 2009 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61035018 |
Mar 9, 2008 |
|
|
|
61035021 |
Mar 9, 2008 |
|
|
|
Current U.S.
Class: |
370/252 |
Current CPC
Class: |
H04L 1/24 20130101 |
Class at
Publication: |
370/252 |
International
Class: |
G06F 11/30 20060101
G06F011/30 |
Claims
1. A method for duplicate packet detection and discard, comprising:
for multiple incoming packet data, determining a hash value of at
least a portion of a packet and a time stamp; and in response to a
request for packet data, employing at least a portion of the hash
value as an index to a table and comparing the hash value of any
data stored at the table entry; if no match is found, storing the
hash and timestamp and packet data at the table entry, but if a
match is found, comparing the timestamp of the table entry and the
requested packet data, and if the timestamps are within a time
value of each other, determining that the packet data is a
duplicate, otherwise, if the timestamps are not within the time
value, determining that the packet data is not a duplicate, and
storing the time stamp at the table entry and passing the requested
packet data on for further use.
2. The method according to claim 1, wherein said determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet other than a MAC frame.
3. The method according to claim 1, wherein said determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet other than a CRC portion of
the packet.
4. The method according to claim 1, wherein said determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet other than a MAC frame and a
CRC portion of the packet.
5. The method according to claim 1, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet sufficient to provide a
unique identification of the packet.
6. The method according to claim 1, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a sufficient number of octets of the packet
sufficient to provide a unique identification of the packet.
7. An apparatus for duplicate packet detection and discard,
comprising: a network interface for connecting to and observing
traffic on a network; a processor; a storage medium for storing
data; wherein said processor observes multiple incoming packet data
via said network interface, determines a hash value of a packet and
a time stamp and stores said hash value, time stamp and packet
data; and a device driver, wherein, in response to a request for
packet data, said device driver obtains a packet from said storage
medium, employs at least a portion of the hash value as an index to
a table and comparing the hash value of any data stored at the
table entry; if no match is found, storing the hash and timestamp
and packet data at the table entry, but if a match is found,
comparing the timestamp of the table entry and the requested packet
data, and if the timestamps are within a time value of each other,
determining that the packet data is a duplicate, otherwise, if the
timestamps are not within the time value, determining that the
packet data is not a duplicate, and storing the time stamp at the
table entry and passing the requested packet data on for further
use.
8. The apparatus according to claim 7, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a MAC frame.
9. The apparatus according to claim 7, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a CRC portion of
the packet.
10. The apparatus according to claim 7, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a MAC frame and
a CRC portion of the packet.
11. The apparatus according to claim 7, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet sufficient to provide a
unique identification of the packet.
12. The apparatus according to claim 7, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a sufficient number of octets of the packet
sufficient to provide a unique identification of the packet.
13. The apparatus according to claim 7, wherein said apparatus is
implemented as a network monitoring card.
14. The apparatus according to claim 13, wherein network monitoring
card comprises a PCIe interface.
15. An apparatus for duplicate packet detection and discard,
comprising: a server computer, said server comprising a network
interface for connecting to and observing traffic on a network; a
processor; a storage medium for storing data; wherein said
processor observes multiple incoming packet data via said network
interface, determines a hash value of a packet and a time stamp and
stores said hash value, time stamp and packet data; and a device
driver, wherein, in response to a request for packet data, said
device driver obtains a packet from said storage medium, employs at
least a portion of the hash value as an index to a table and
comparing the hash value of any data stored at the table entry; if
no match is found, storing the hash and timestamp and packet data
at the table entry, but if a match is found, comparing the
timestamp of the table entry and the requested packet data, and if
the timestamps are within a time value of each other, determining
that the packet data is a duplicate, otherwise, if the timestamps
are not within the time value, determining that the packet data is
not a duplicate, and storing the time stamp at the table entry and
passing the requested packet data on for further use.
16. The apparatus according to claim 15, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a MAC frame.
17. The apparatus according to claim 15, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a CRC portion of
the packet.
18. The apparatus according to claim 15, wherein said determining a
hash value of at least a portion of a packet comprises determining
a hash value of a portion of the packet other than a MAC frame and
a CRC portion of the packet.
19. The apparatus according to claim 15, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a portion of the packet sufficient to provide a
unique identification of the packet.
20. The apparatus according to claim 15, wherein determining a hash
value of at least a portion of a packet comprises determining a
hash value of a sufficient number of octets of the packet
sufficient to provide a unique identification of the packet.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority of U.S. provisional patent
application 61/035,018, filed Mar. 9, 2008, and U.S. provisional
patent application 61/035,021, filed Mar. 9, 2008.
BACKGROUND OF THE INVENTION
[0002] This invention relates to network test and measurement, and
more particularly to an apparatus and method for detecting
duplicate packets in network traffic.
[0003] In operation and maintenance of networks, determination of
where issues or problem points arise can be complex. A network
engineer or technician looking to resolve problems would be
interested in having accurate network protocol statistics. Network
analyzers that employ multiple ports may be connected to a network
such that packets on a network may be seen at two or more ports of
the analyzer. Misinterpreting these duplicate packets in protocol
analysis can lead to erroneous statistics and false
conclusions.
[0004] While it might be considered to store copies of each packet
seen on a network and compare the contents thereof to determine
duplicates, this would require large amounts of memory to store the
packets and many processing cycles to perform the comparison. The
requirements become greater as network speeds increase, making such
method undesirable.
[0005] A software-calculated hash over the contents of each packet
could reduce the memory requirements to perform such comparison,
but would but do little to reduce the processing cycles
required.
SUMMARY OF THE INVENTION
[0006] In accordance with the invention, an apparatus and method of
detecting duplicate packets is provided, whereby the duplicate
packets may be discarded.
[0007] Accordingly, it is an object of the present invention to
provide an improved method and apparatus for network test and
measurement.
[0008] It is a further object of the present invention to provide
an improved method and apparatus for detecting and discarding
duplicate packets seen on multiple ports.
[0009] It is yet another object of the invention to eliminate these
"duplicate" packets seen on multiple ports of a network analyzer in
a manner that minimizes execution time.
[0010] The subject matter of the present invention is particularly
pointed out and distinctly claimed in the concluding portion of
this specification. However, both the organization and method of
operation, together with further advantages and objects thereof,
may best be understood by reference to the following description
taken in connection with accompanying drawings wherein like
reference characters refer to like elements.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG. 1 is a block diagram of a network with a test
instrument installed thereon;
[0012] FIG. 2 is a block diagram of a test instrument; and
[0013] FIG. 3 is a block diagram of an implementation of the
apparatus and method.
DETAILED DESCRIPTION
[0014] The system according to a preferred embodiment of the
present invention comprises a method and apparatus to detect and
discard duplicate packets in a network test instrument.
[0015] Referring to FIG. 1, a block diagram of a network with an
apparatus in accordance with the disclosure herein, a network may
comprise plural network devices 10, 10', etc., which communicate
over a network 12 by sending and receiving network traffic 18. The
traffic may be sent in packet form, with varying protocols and
formatting thereof.
[0016] A network analyzer 14 is also connected to the network, and
may include a remote network analyzer interface 16 that enables a
user to interact with the network analyzer to operate the analyzer
and obtain data therefrom remotely from the physical location of
the analyzer.
[0017] The network analyzer comprises hardware and software, CPU,
memory, interfaces and the like to operate to connect to and
monitor traffic on the network, as well as performing various
testing and measurement operations, transmitting and receiving data
and the like. The remote network analyzer typically is operated by
running on a computer or workstation interfaced with the
network.
[0018] FIG. 2 is a block diagram of a test instrument/analyzer 40,
wherein the instrument may include network interfaces 22 which
attaches the device to a network 12 via multiple ports, one or more
processors 23 for operating the instrument, memory such as RAM/ROM
24 or persistent storage 26, display 28, user input devices 30
(such as, for example, keyboard, mouse or other pointing devices,
touch screen, etc.), power supply 32 which may include battery or
AC power supplies, other interface 34 which attaches the device to
a network or other external devices (storage, other computer,
etc.). Packet processing module 25 provides processing of packets
to assist in the duplicate detection and discard, as discussed
further hereinbelow. Memory 24 is accessible to module 25 for
transfer of data.
[0019] In operation, the network test instrument is attached to the
network, and observes transmissions on the network to collect
statistics thereon.
[0020] Packet processor module 25, which may be implemented as a
FPGA, for example, observes incoming packet data received on the
network, and calculates two hashes (`hashA` and `hashB`) on octets
contained in the packets that would be sufficient to determine
uniqueness, and maintains a high-resolution timestamp relative to
the packet data. In a particular embodiment, determination of the
hashes comprise discarding the MAC frame and CRC of each packet,
and determining the hashes on data portions that include packet
length and content. As each packet arrives on the network, module
25 writes the hashes, the packet data, and its arrival timestamp to
memory.
[0021] Meanwhile, a software module is operating and maintains a
fast lookup table whose entries contain the hashes with their
corresponding packet timestamps. For each packet, software uses
`hashA` as an index into the table. Entries at the table index are
compared against the `hashB`. If not found, `hashB` is added to an
available entry at the table index with its corresponding timestamp
and the packet is passed for further analysis. If an entry is found
for `hashB`, the corresponding timestamps are compared against a
pre-determined period. If within the period, the packet is deemed
duplicate and discarded, otherwise `hashB` entry's corresponding
timestamp is updated and the packet is passed for further
analysis.
[0022] The hash comprises a unique signature for each packet, for
example, a 64 bit value. For an incoming packet in accordance with
a particular embodiment, the source and destination MAC (12 bytes)
at the beginning of the packet is discarded. Also, the end CRC data
(4 bytes) is discarded as well. A rolling hash value is calculated
on the remaining data, and the resultant value, data and time stamp
are placed into memory.
[0023] Software maintains a fast lookup table whose entries contain
the hardware-calculated hashes with their corresponding packet
timestamps. For each packet, the software uses the
least-significant 16-bits of the hardware-calculated hash as an
index into this table. Entries at the table index are compared
against the full hardware-calculated hash for a match.
[0024] If no match found, the packet is deemed not a duplicate. The
hash and its corresponding timestamp are added to an available
entry at the table index. The packet passes for further
analysis.
[0025] If a match is found, the corresponding timestamps are then
compared. If the timestamps are within a given time, for example,
50 ms, the packet is deemed duplicate and discarded. Otherwise, the
packet is deemed not a duplicate, the entry's corresponding
timestamp is updated and the packet is passed for further
analysis.
[0026] When a duplicate packet is discarded, the next packet from
memory is tested in the same manner for duplicate. This continues
until a non-duplicate packet is encountered or no more packets are
available from server memory.
[0027] Duplicate packet detection is performed across all physical
Ethernet interfaces within a logical channel, but not across
logical channels. This is to maintain correct flow/transaction
layer analysis in the event that the user splits the same logical
traffic across the configured logical interfaces.
[0028] Referring to FIG. 3, the method and apparatus may suitably
be implemented as a network monitoring card 42 that plugs into a
PCIe slot in a high-performance server 44, to filter, aggregate,
and buffer Ethernet traffic from network 12 over multiple ports at
line rates, and a device driver. Packet, hash and time stamp data
from the card 42 is supplied via DMA to the server memory 46. The
device driver operating on the server is called to obtain network
packet data, wherein when called to read a next packet data from
the server memory, the device driver implements the above-discussed
checking to determine if a packet is duplicate, discarding
duplicates and passing non-duplicates on to the requesting process
or devices in response to the read request.
[0029] Thus, in accordance with the method and apparatus, detection
and discard of duplicate data packets is enabled, to provide more
accurate network analysis by a network test instrument.
[0030] While a preferred embodiment of the present invention has
been shown and described, it will be apparent to those skilled in
the art that many changes and modifications may be made without
departing from the invention in its broader aspects. The appended
claims are therefore intended to cover all such changes and
modifications as fall within the true spirit and scope of the
invention.
* * * * *