U.S. patent application number 10/662778 was filed with the patent office on 2005-03-17 for methods and systems for the rapid detection of concealed objects.
Invention is credited to Peschmann, Kristian R..
Application Number | 20050058242 10/662778 |
Document ID | / |
Family ID | 34274205 |
Filed Date | 2005-03-17 |
United States Patent
Application |
20050058242 |
Kind Code |
A1 |
Peschmann, Kristian R. |
March 17, 2005 |
Methods and systems for the rapid detection of concealed
objects
Abstract
The present invention provides for an improved scanning process
having a first stage to rapidly identify a threat location and a
second stage to accurately identify the nature of the threat. The
improved scanning process maintains a high degree of accuracy while
still providing an operationally desirable high throughput. The
present invention also uses improved processing techniques that
enable the substantially automated detection of threats and
decrease the dependence on operator accuracy. One embodiment of the
present invention provides an apparatus for identifying an object
concealed within a container. It comprises a first stage inspection
system having at least two X-ray projection systems to generate a
first set of data and a plurality of processors in data
communication with the first stage inspection system. The
processors process the first set of data to generate at least two
images. The two images are used to identify at least one target
region from the two images. A second stage inspection system is
then used to generate an inspection region which is then positioned
relative to the target region and made to at least partially
physically coincide with the target region. A second set of data is
produced specifically from the inspection region, data which have a
high degree of specificity for the material in the inspection
region.
Inventors: |
Peschmann, Kristian R.;
(Torrance, CA) |
Correspondence
Address: |
PATENTMETRIX
14252 CULVER DR. BOX 914
IRVINE
CA
92604
US
|
Family ID: |
34274205 |
Appl. No.: |
10/662778 |
Filed: |
September 15, 2003 |
Current U.S.
Class: |
378/57 |
Current CPC
Class: |
G01F 23/284 20130101;
G01R 27/06 20130101; G01N 22/00 20130101; G01V 5/0016 20130101;
G01V 5/0025 20130101 |
Class at
Publication: |
378/057 |
International
Class: |
G01N 023/04 |
Claims
We claim:
1. An apparatus for identifying an object concealed within a
container, comprising: a first stage inspection system having at
least two X-ray projection systems to generate a first set of data;
a plurality of processors in data communication with the first
stage inspection system wherein the processors process said first
set of data to generate at least two images; a means for
identifying at least one target region from the two images; a means
for positioning an inspection region relative to the target region
wherein the inspection region at least partially physically
coincides with the target region; and a second stage inspection
system for generating the inspection region wherein the second
stage inspection system produces a second set of data having an
X-ray signature characteristic of the material in said inspection
region.
2. The apparatus of claim 1 wherein said object is a threat.
3. The apparatus of claim 2 wherein said threat is at least one of
an illegal drug, an explosive material, or a weapon.
4. The apparatus of claim 1 wherein the means for identifying at
least one target region comprises an operator selecting a region
associated with each of the images.
5. The apparatus of claim 4 wherein the operator selects a region
based upon an X-ray image characteristic.
6. The apparatus of claim 5 wherein the X-ray image characteristic
is at least one of mass, degree of attenuation, area, atomic
number, size, shape, pattern, or context.
7. The apparatus of claim 5 wherein the operator identifies a
region in a first image as likely to be the same, or closely
located to it, in a second image.
8. The apparatus of claim 1 wherein the means for identifying at
least one target region comprises a processor executing an
algorithm to select a region associated with the images.
9. The apparatus of claim 8 wherein the region associated with the
images is selected based upon an X-ray image characteristic.
10. The apparatus of claim 9 wherein the X-ray image characteristic
is at least one of mass, degree of attenuation, area, atomic
number, size, shape, pattern, or context.
11. The apparatus of claim 1 wherein a plurality of X-ray beam
projections from the X-ray projection systems intersects the target
region at an intersection area, said target region having a
location.
12. The apparatus of claim 11 wherein the location of the target
region is determined by identifying a set of coordinates for the
intersection area.
13. The apparatus of claim 12 wherein a plurality of control
commands is produced in response to the determination of said
location of the target region.
14. The apparatus of claim 12 wherein the inspection region is
positioned relative to the target region in response to the
plurality of control commands using a three-axis control
system.
15. The apparatus of claim 1 wherein the means for positioning said
inspection region relative to the target region includes a
plurality of adjustable apertures.
16. The apparatus of claim 15 wherein the apertures can be
physically moved in the direction of the main beam axis.
17. The apparatus of claim 16 wherein the aperture is a ring
aperture having an adjustable diameter.
18. The apparatus of claim 1 wherein the means for positioning said
inspection region relative to the target region comprises a
conveyor operable to move in elevation relative to the second stage
inspection system.
19. The apparatus of claim 1 wherein the means for positioning said
inspection region relative to the target region comprises an
aperture and ring aperture.
20. The apparatus of claim 1 wherein the second stage inspection
system comprises an inspection region generation system.
21. The apparatus of claim 20 wherein the inspection region
generation system comprises a source of X-ray radiation.
22. The apparatus of claim 21 wherein the inspection region
generation system comprises an energy dispersive detector.
23. The apparatus of claim 20 wherein the inspection region
generation system comprises an array of transmission detectors.
24. The apparatus of claim 20 wherein the inspection region
generation system comprises an energy dispersive detector and an
array of transmission detectors.
25. The apparatus of claim 24 wherein the energy dispersive
detector is used to produce a signature of the material in the
inspection region and the array of transmission detectors is used
to produce data defining at least one of mass, degree of
attenuation, area, average atomic number, of the material in a
beampath.
26. The apparatus of claim 23 or 24 wherein the array of
transmission detectors is in a ring formation.
27. The apparatus of claim 24 wherein the array of transmission
detectors comprises high energy and low energy detectors.
28. The apparatus of claim 27 wherein data generated from the
transmission detectors is used to identify a reference
spectrum.
29. The apparatus of claim 28 wherein said identification of a
reference spectrum is achieved by identifying a spectrum associated
with data generated from both the high energy detectors and the low
energy detectors.
30. The apparatus of claims 27, 28 or 29 wherein said second set of
data comprises high energy and low energy transmission data
characteristic of the X-ray properties of the material in a
beampath.
31. The apparatus of claim 28 wherein the reference spectrum is
used to correct a diffraction spectrum.
32. The apparatus of claim 28 wherein the reference spectrum is
used to correct for beam hardening.
33. The apparatus of claim 23 wherein data generated from the
transmission detectors is used to identify a boundary of the
container.
34. The apparatus of claim 23 wherein data generated from the
transmission detectors is used to generate an image.
35. The apparatus of claim 1 wherein the X-ray signature
characteristic is a diffraction pattern.
36. The apparatus of claim 1 wherein the X-ray signature
characteristic is a scatter spectrum.
37. The apparatus of claim 1 wherein the X-ray signature
characteristic is an electronic response signal.
38. The apparatus of claim 1 further comprising a processor in data
communication with at least one of the first stage inspection
system or the second stage inspection system wherein the processor
is capable of executing a neural network to process at least one of
the first set of data or the second set of data to determine the
existence of a threat.
39. The apparatus of claim 38 wherein the neural network operates
as a back-propagation network having a plurality of nodes and
wherein said nodes are organized in a series of successive layers,
each layer comprising at least one node that receives inputs from
nodes in a prior layer and transmits outputs to nodes in a
subsequent layer.
40. The apparatus of claim 39 wherein nodes in a first layer are
weighted in accordance with their distance from at least one node
in a second layer.
41. The apparatus of claim 38 wherein the neural network is trained
to determine the existence of the threat using a library of known
threats.
42. The apparatus of claim 1 wherein the inspection region
encompasses portions of the container, volume within the container
and volume external to the container and wherein a composite signal
is produced when the portions of the container, volume within the
container and volume external to the container are exposed to said
X-ray beams.
43. The apparatus of claim 42 further comprising a processor to
correct the composite signal by substantially removing a signal
produced by exposing the volume external to the container to said
X-ray beams.
44. The apparatus of claim 43 wherein the volume external to the
container includes air.
45. The apparatus of claim 43 wherein the volume external to the
container includes metal.
46. The apparatus of claim 43 wherein said correction of the
composite signal includes correcting for a plurality of attenuation
affects caused by exposing the volume external to the container to
said X-ray beams.
47. The apparatus of claim 46 wherein one of said attenuation
affects is beam hardening.
48. The apparatus of claim 1 wherein the second set of data
comprises a composite signal produced when at least two of the
container, volume within the container, or volume external to the
container are exposed to said X-ray beams.
49. The apparatus of claim 1 wherein the second stage inspection
system comprises at least two energy dispersive detectors.
50. The apparatus of claim 49 wherein the energy dispersive
detectors are separated by a plurality of vanes.
51. The apparatus of claim 50 further comprising four energy
dispersive detectors.
52. The apparatus of claim 51 wherein the energy dispersive
detectors are arranged into four quadrants.
53. A method for identifying an object concealed within a
container, comprising: generating a first set of data using a first
stage inspection system having at least two X-ray projection
systems; processing said first set of data to generate at least two
images using a plurality of processors in data communication with
the first stage inspection system; identifying at least one target
region from the two images; positioning an inspection region
relative to the target region wherein the inspection region at
least partially physically coincides with the target region;
generating the inspection region through a second stage inspection
system; and producing a second set of data having a X-ray signature
characteristic of the material in the inspection region.
54. The method of claim 53 wherein an operator identifies at least
one target region by selecting a region associated with the images,
said region being selected based upon an X-ray image
characteristic.
55. The method of claim 54 wherein the X-ray image characteristic
is at least one of mass, degree of attenuation, total area, atomic
number, size, shape, or organic to inorganic ratio.
56. The method of claim 53 wherein an operator identifies at least
two target regions by selecting regions associated with each of
said images.
57. The method of claim 56 wherein the operator identifies a region
in a first image as being similar to a region in a second
image.
58. The method of claim 57 wherein the operator identifies the
regions as being similar based upon at least one of mass, degree of
attenuation, total area, atomic number, size, shape, or organic to
inorganic ratio.
59. The method of claim 53 wherein the at least one target region
is identified by a processor executing an algorithm to select a
region associated with the images.
60. The method of claim 59 wherein the region associated with the
images is selected based upon an X-ray image characteristic.
61. The method of claim 60 wherein the X-ray image characteristic
is at least one of mass, degree of attenuation, total area, atomic
number, size, shape, or organic to inorganic ratio.
62. The method of claim 53 wherein the at least one target region
is identified by a processor executing an algorithm to select at
least two regions associated with each of said images.
63. The method of claim 62 wherein the processor identifies a
region in a first image as being similar to a region in a second
image.
64. The method of claim 63 wherein the processor identifies the
regions as being similar based upon at least one of mass, degree of
attenuation, total area, atomic number, size, shape, or organic to
inorganic ratio.
65. The method of claim 53 wherein a plurality of X-ray beam
projections from the X-ray projection systems intersect the target
region at an intersection area, said target region having a
location.
66. The method of claim 65 wherein the location of the target
region is determined by identifying a set of coordinates for the
intersection area.
67. The method of claim 66 wherein a plurality of control commands
is produced in response to the determination of said location of
the target region.
68. The method of claim 67 wherein the inspection region is
positioned relative to the target region in response to the
plurality of control commands using a three-axis control
system.
69. The method of claim 53 wherein the positioning of the
inspection region relative to the target region is achieved using a
plurality of adjustable apertures.
70. The method of claim 69 wherein the aperture can be physically
moved horizontally or vertically.
71. The method of claim 69 wherein the aperture is a ring aperture
having an adjustable diameter.
72. The method of claim 53 wherein the positioning of the
inspection region relative to the target region is achieved using a
conveyor operable to move in elevation relative to the second stage
inspection system.
73. The method of claim 53 wherein the positioning of the
inspection region relative to the target region is achieved using
an aperture and ring aperture.
74. The method of claim 53 wherein the second stage inspection
system comprises an energy dispersive detector.
75. The method of claim 53 wherein the second stage inspection
system comprises an array of transmission detectors.
76. The method of claim 53 wherein the second stage inspection
system comprises an energy dispersive detector and an array of
transmission detectors.
77. The method of claim 76 wherein the energy dispersive detector
is used to produce a signature of the material in the inspection
region and the array of transmission detectors is used to produce
data defining at least one of mass, degree of attenuation, area,
average atomic number, of the material in a beampath.
78. The method of claim 75 or 76 wherein the array of transmission
detectors is in a ring formation.
79. The method of claim 75 wherein the array of transmission
detectors comprises high energy and low energy detectors.
80. The method of claim 79 wherein a reference spectrum is
determined by identifying a spectrum associated with data generated
from both the high energy detectors and the low energy
detectors.
81. The method of claim 80 wherein the reference spectrum is used
to correct a diffraction spectrum.
82. The method of claim 80 wherein the reference spectrum is used
to correct for beam hardening.
83. The method of claim 75 wherein data generated from the
transmission detectors is used to identify a boundary of the
container.
84. The method of claim 53 wherein the X-ray signature
characteristic is a diffraction pattern.
85. The method of claim 53 wherein the X-ray signature
characteristic is a scatter spectrum.
86. The method of claim 53 wherein the X-ray signature
characteristic is an electronic response signal.
87. The method of claim 53 further comprising the step of executing
a neural network to process at least one of the first set of data
or the second set of data to determine the existence of a
threat.
88. The method of claim 87 wherein the neural network operates as a
back-propagation network having a plurality of nodes and wherein
said nodes are organized in a series of successive layers, each
layer comprising at least one node that receives inputs from nodes
in a prior layer and transmits outputs to nodes in a subsequent
layer.
89. The method of claim 88 wherein nodes in a first layer are
weighted in accordance with their distance from at least one node
in a second layer.
90. The method of claim 87 wherein the neural network is trained to
determine the existence of the threat using a plurality of
libraries.
91. The method of claim 90 wherein the plurality of libraries are
accessible via a network.
92. The method of claim 91 wherein the plurality of libraries
comprise at least one library of threats and at least one library
non-threats.
93. The method of claim 92 wherein the plurality of libraries
further comprises at least one buffer library.
94. The method of claim 53 wherein the second stage inspection
system comprises at least two energy dispersive detectors.
95. The method of claim 94 wherein the energy dispersive detectors
are separated by a plurality of vanes.
96. The method of claim 53 wherein the second stage inspection
system comprises four energy dispersive detectors.
97. The method of claim 96 wherein the energy dispersive detectors
are arranged into four quadrants.
98. The method of claim 97 wherein the energy dispersive detectors
are separated by a plurality of vanes.
99. An apparatus for identifying an object concealed within a
container, comprising: a first stage inspection system having at
least two X-ray projection systems to generate a first set of data;
a plurality of processors in data communication with the first
stage inspection system wherein the processors process said first
set of data to generate at least two images; a processor executing
an algorithm for selecting a region associated with each image; a
means for positioning an inspection region relative to the target
region wherein the inspection region at least partially physically
coincides with the target region; and a second stage inspection
system for generating the inspection region wherein the second
stage inspection system produces a second set of data having an
X-ray signature characteristic of the material in said inspection
region.
100. The apparatus of claim 99 wherein the region associated with
the images is selected based upon an X-ray image
characteristic.
101. The apparatus of claim 100 wherein the X-ray image
characteristic is at least one of mass, degree of attenuation,
area, atomic number, size, shape, pattern, or context.
102. An apparatus for identifying an object concealed within a
container, comprising: a first stage inspection system having at
least two X-ray projection systems to generate a first set of data;
a plurality of processors in data communication with the first
stage inspection system wherein the processors process said first
set of data to generate at least two images; a processor executing
an algorithm for selecting a region associated with each image; a
means for positioning an inspection region relative to the target
region wherein the inspection region at least partially physically
coincides with the target region; and a second stage inspection
system for generating the inspection region wherein the second
stage inspection system produces a second set of data having an
X-ray signature characteristic of the material in said inspection
region and comprises an array of transmission detectors.
103. The apparatus of claim 102 wherein the second stage inspection
system further comprises an energy dispersive detector.
104. The apparatus of claim 103 wherein the energy dispersive
detector is used to produce a signature of the material in the
inspection region and the array of transmission detectors is used
to produce data defining at least one of mass, degree of
attenuation, area, average atomic number, of the material in a
beampath.
105. The apparatus of claim 102 wherein the array of transmission
detectors comprises high energy and low energy detectors.
106. The apparatus of claim 105 wherein a reference spectrum is
determined by identifying a spectrum associated with data generated
from both the high energy detectors and the low energy
detectors.
107. The apparatus of claim 106 wherein the reference spectrum is
used to correct a diffraction spectrum.
108. The apparatus of claim 106 wherein the reference spectrum is
used to correct for beam hardening.
109. An apparatus for identifying an object concealed within a
container, comprising: a first stage inspection system having a
X-ray projection system to generate a first set of data; a
processor in data communication with the first stage inspection
system wherein the processor processes said first set of data, said
first set of data being indicative of a target region; a second
stage inspection system for generating an inspection region
proximate to the target region wherein the second stage inspection
system produces a second set of data; and a processor capable of
executing a neural network to process the second set of data to
determine the existence of a threat.
110. The apparatus of claim 109 wherein the neural network operates
as a back-propagation network having a plurality of nodes and
wherein said nodes are organized in a series of successive layers,
each layer comprising at least one node that receives inputs from
nodes in a prior layer and transmits outputs to nodes in a
subsequent layer.
111. The apparatus of claim 110 wherein nodes in a first layer are
weighted in accordance with their distance from at least one node
in a second layer.
112. The apparatus of claim 111 wherein the neural network is
trained to determine the existence of the threat using a plurality
of libraries.
113. The apparatus of claim 112 wherein the libraries are
accessible via a network.
114. The apparatus of claim 113 the libraries comprise at least one
of a threat library, a non-threat library, and a buffer library.
Description
FIELD OF THE INVENTION
[0001] The present invention relates generally to X-ray based
methods and systems for detection of concealed threats, and threat
resolution, and more specifically to improved methods and systems,
using dual stage scanning to process luggage for faster inspection
with reduced false alarm rate.
BACKGROUND OF THE INVENTION
[0002] Conventional X-ray systems produce radiographic projection
images, which are then interpreted by an operator. These
radiographs are often difficult to interpret because objects are
superimposed. A trained operator must study and interpret each
image to render an opinion on whether or not a target of interest,
a threat, is present. With a large number of such radiographs to be
interpreted, and with the implied requirement to keep the number of
false alarms low, operator fatigue and distraction can compromise
detection performance.
[0003] Advanced technologies, such as dual-energy projection
imaging and Computed Tomography (CT), are being used for contraband
detection, beyond conventional X-ray systems. In dual-energy
imaging it is attempted to measure the effective atomic numbers of
materials in containers such as luggage. However, the dual-energy
method does not readily allow for the calculation of the actual
atomic number of the concealed `threat` itself, but rather yields
only an average atomic number that represents the mix of the
various items falling within the X-ray beam path, as the contents
of an actual luggage is composed of different items and rarely
conveniently separated. Thus dual-energy analysis is often
confounded. Even if the atomic number of an item could be measured,
the precision of this measurement would be compromised by X-ray
photon noise to the extent that many innocuous items would show the
"same" atomic number as many threat substances, and therefore the
atomic number in principle cannot serve as a sufficiently specific
classifier for threat versus no threat.
[0004] In X-ray CT cross-sectional images of slices of an object
are reconstructed by processing multiple attenuation measurements
taken at various angles around an object. CT images do not suffer
much from the super-positioning problem present in standard
radiographs. However, conventional CT systems take considerable
time to perform multiple scans, to capture data, and to reconstruct
the images. The throughput of CT systems is generally low. Coupled
with the size and expense of CT systems this limitation has
hindered CT use in applications such as baggage inspection where
baggage throughput is an important concern. In addition, CT alarms
on critical mass and density of a threat, but such properties are
not unique to explosives. CT based systems suffer from high false
alarm rate. Any such alarm is then to be cleared or confirmed by an
operator, again interpreting images, or hand searching.
[0005] Apart from X-ray imaging systems, detection systems based on
X-ray diffraction, or coherent scatter are also known. Their
primary purpose is not to acquire images but to obtain information
about the molecular structure of the substances an object is
composed of. The so-called diffraction or coherent scatter
signature is based on BRAGG reflection, that is the interference
pattern of X-ray light, which develops when X-rays are reflected by
the molecular structure or electron density distribution of a
substance.
[0006] Various inspection region geometries have been developed and
disclosed. Kratky, in Austrian Patent No. 2003753 publishes a
refined arrangement of circular concentric apertures combined with
an X-ray source and a point detector, to gain the small angle
diffraction signature of an object placed between the apertures.
More recently Harding in U.S. Pat. No. 5,265,144, uses a similar
geometry but replaces the point shaped detector aperture with an
annular detector configurations. Both patents are incorporated
herein by reference.
[0007] The resulting diffraction spectra can be analyzed to
determine the molecular structure of the diffracting object, or at
least to recognize similarity with any one of a number of spectra,
which have previously been obtained from dangerous substances.
[0008] One approach to detecting explosives in luggage was
disclosed in British patent No. 2,299,251 in which a device uses
Bragg reflection from crystal structures to identify crystalline
and poly-crystalline substances. Substances can be identified
because the energy spectrum distribution of the polychromatic
radiation reflected at selected angles is characteristic of the
crystal structure of the substance reflecting the radiation.
[0009] U.S. Pat. Nos. 4,754,469, 4,956,856, 5,008,911, 5,265,144,
5,600,700 and 6,054,712 describe methods and devices for examining
substances, from biological tissues to explosives in luggage, by
recording the spectra of coherent radiation scattered at various
angles relative to an incident beam direction. U.S. Pat. No.
5,265,144 describes a device using concentric detecting rings for
recording the radiation scattered at particular angles. Each of the
prior art systems and methods, however, suffer from low processing
rates because the scatter interaction cross sections are relatively
small and the exposure times required to obtain useful diffraction
spectra are long, in the range of seconds and minutes. For security
inspections, equipment performance has to combine high detection
sensitivity and high threat specificity with high throughput, at
the order of hundreds of bags per hour.
[0010] U.S. Pat. No. 5,182,764 discloses an apparatus for detecting
concealed objects, such as explosives, drugs, or other contraband,
using CT scanning. To reduce the amount of CT scanning required, a
pre-scanning approach is disclosed. Based upon the pre-scan data,
selected locations for CT scanning are identified and CT scanning
is undertaken at the selected locations. The inventors claim the
pre-scan step reduces the scanning time required for each scanned
item, therefore increasing throughput. However, the use of CT
scanning is still inefficient, not threat specific, and does not
allow for rapid scanning of objects.
[0011] U.S. Pat. No. 5,642,393 discloses a multi-view X-ray
inspection probe that employs X-ray radiation transmitted through
or scattered from an examined item to identify a suspicious region
inside the item. An interface is used to receive X-ray data
providing spatial information about the suspicious region and to
provide this information to a selected material sensitive probe.
The material sensitive probe, such as a coherent scatter probe,
then acquires material specific information about the previously
identified suspicious region and provides it to a computer. The
disclosed system does not, however, address critical problems that
arise in the course of the frequency of false alarms by applying
threat specific analysis.
[0012] Another object of the invention is to provide for improved
processing techniques performed in association with various
scanning systems. The improved processing techniques enable the
substantially automated detection of threats and decrease the
dependence on operator skill and performance.
[0013] Another object of the invention is to provide for a method
and system to screen for relatively small amounts of threat
material.
[0014] Another object of the invention is to provide for an
improved method and system for screening for explosives in the form
of thin sheets.
[0015] Another object of the invention is to provide a screening
solution at low cost by utilizing standard industrial components,
including relatively low cost and rugged industrial X-ray systems
and detector systems.
[0016] Accordingly, one embodiment of the present invention
provides an apparatus for identifying an object concealed within a
container. These objects may be considered threats, such as an
illegal drug, an explosive material, or a weapon. The apparatus
comprises a first stage inspection system having at least two X-ray
projection systems projecting in different directions to generate a
first set of data, and a data applying a scatter probe to a
selected suspicious region, including the accurate identification
of a suspicious region, correction of detected data, and the nature
of processing algorithms used.
[0017] Accordingly, there is need for an improved automatic threat
detection and resolution system that captures data through an X-ray
system and utilizes this data to identify threat items in a rapid,
yet accurate, manner. Preferably, the system is highly threat
specific in order to reliably and automatically discern threats
from innocuous materials and items while still being able to
process in excess of 100 bags per hour. Preferably the system
utilizes relatively inexpensive industrial components, and does not
need special support facilities. Additionally, the system should
provide for greater accuracy in utilizing pre-scan data to identify
an inspection region and in processing scan data.
SUMMARY OF THE INVENTION
[0018] One object of the present invention is to provide for an
improved scanning process having a first stage to pre-select the
locations of potential threats and a second stage to accurately
identify the nature of the threat. The improved scanning process
increases throughput by limiting the detailed inspection to a small
fraction of the total bag volume, and it decreases processing
system in data communication with the first stage inspection
system. The processors process the first set of data to generate at
least two images or image data files. The images are then subjected
to a set of image interpretation algorithms, also processed by the
processors, to identify target regions in the two images. Since the
images are projected in different direction it is possible to
back-project identified target regions and to locate those targets
in system coordinates.
[0019] The first stage inspection system locates potential threat
items, regions, and/or areas, based on X-ray images, manual or
automatic detection algorithms, and triangulation. A second stage
inspection system, discussed below, then focuses on the identified
items, regions, and/or areas to produce characteristic signatures
which are then used to determine whether a threat is, in fact,
present.
[0020] In one embodiment, a target region is identified from the
two images generated in the first stage inspection system by having
an operator select a particular region displayed in the images. In
a preferred embodiment, the operator directs a cursor, using an
interface, such as a mouse, to position crosshairs on each of the
two images. The two crosshairs determine a certain location in
system coordinates. The selection for the cross hair location may
occur based upon an X-ray image characteristic, such as the X-ray
shadow of an object, seen in both images. Optionally, the target
selection process may be performed electronically. Target regions
are identified from the two images by having a processor execute an
algorithm to select regions in the images, which correspond to
objects or mass accumulations. With the locations of each of the
two images determined, the coordinates corresponding to the
physical locations of the target-region can be determined and used
to direct the system, and, in particular, the conveyor.
[0021] Once the target region is determined, either through
automatic or operator processing means, a plurality of control
commands is produced and used to position, by a multiple-axis
motion control system, the second stage inspection system such that
an inspection region, at least partially, coincides with the
determined target coordinates. In one embodiment, the inspection
region is positioned relative to the target region using a
plurality of adjustable apertures that can be physically moved. The
apertures can be ring-shaped with an adjustable diameter.
Optionally, the means for positioning the inspection volume
relative to the target region comprises a motion-controlled
conveyor operable to move in elevation as well as back and forth
relative to the second stage inspection system. Optionally, the
inspection region can be moved across the conveyor by mounting the
second stage inspection system on a C-arm which is motion
controlled to move back and forth across the conveyor, or
alternatively by employing a parallel set of fixed linear bearings
and synchronized linear motion to effect the same relative
movement.
[0022] The second stage inspection system generates an inspection
volume in space and produces a second set of data having an X-ray
signature characteristic of the material in that inspection volume.
The X-ray signature characteristic is a diffraction pattern, also
called scatter spectrum, and an intensity level associated with
that spectrum, and, in addition, a set of dual energy transmission
measurements in close proximity to the ray path of the diffraction
measurement.
[0023] The second stage inspection system comprises a source of
X-ray radiation. In one embodiment, it comprises an energy
dispersive detector. In another embodiment, it comprises an array
of transmission detectors. In a preferred embodiment, it comprises
both an energy dispersive detector and an array of transmission
detectors. The energy dispersive detector is used to produce a
signature of the material in the inspection region and the array of
transmission detectors is used to produce data defining at least
one of mass, degree of attenuation, area, and average atomic
number, of the material in a beampath. Optionally, the array of
transmission detectors is in a ring formation. In a preferred
embodiment, the array of transmission detectors comprises high
energy and low energy detectors. Data generated from the
transmission detectors is used to determine a reference spectrum by
identifying a spectrum associated with data generated from both the
high energy detectors and the low energy detectors. The reference
spectrum can be used to correct a diffraction spectrum or to
correct for beam hardening.
[0024] It is preferred that the present invention includes a method
of automatically determining the existence of a threat based upon
the second set of data. While a variety of algorithms can be
employed for this task, a preferred approach is to use a neural
network to process at least one of the first set of data or the
second set of data to determine the existence of a threat. In one
embodiment, the neural network operates as a back-propagation
network having a plurality of nodes and wherein the nodes are
organized in a series of successive layers, each layer comprising
at least one node that receives inputs from nodes in a prior layer
and transmits outputs to nodes in a subsequent layer. Optionally,
the nodes in a first layer are weighted in accordance with their
distance from at least one node in a second layer. Optionally, the
neural network is trained to determine the existence of the threat
using a plurality of libraries, such as network accessible
libraries, threat libraries, non-threat libraries, and/or other
libraries.
[0025] These and other embodiments of the present invention will be
disclosed in greater detail with reference to the drawings and
detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
[0026] These and other features and advantages of the present
invention will be appreciated, as they become better understood by
reference to the following Detailed Description when considered in
connection with the accompanying drawings, wherein:
[0027] FIG. 1 is a schematic view of one embodiment of the dual
stage X-ray scanning system;
[0028] FIG. 2 is a schematic view of one embodiment of an X-ray
scanning system for the first stage scanning system;
[0029] FIG. 3 is a schematic view of one embodiment of the first
stage of the X-ray scanning system for identifying a target
region;
[0030] FIG. 3a depicts exemplary images for identifying the
location of an item within a container;
[0031] FIG. 4 is a schematic diagram of a cross-section of one
embodiment of a preferred beam delivery system for use in a second
stage scanning system;
[0032] FIG. 5 is a schematic diagram of one embodiment of the beam
delivery system of the second stage scanning system;
[0033] FIG. 6 is a schematic diagram of an exemplary look up source
for transmission spectra;
[0034] FIG. 7 is a schematic representation of a beam delivery
system having multiple energy dispersive detectors;
[0035] FIG. 8 is a graphical representation of an artificial neural
network;
[0036] FIG. 9 is a flow diagram describing a plurality of steps for
practicing one embodiment of the present invention; and
[0037] FIG. 10 is a flowchart depicting a process of training the
neural network.
DETAILED DESCRIPTION OF THE INVENTION
[0038] The methods and systems described herein are directed
towards finding, locating, and confirming threat items and
substances. Such threats may comprise explosives such as C4, RDX,
Semtex, Seismoplast, PE4, TNT, dynamite, PETN, ANFO among others,
as well as other contraband such as drugs. Although the embodiments
have been described in the context of a baggage inspection system,
it should be evident to persons of ordinary skill in the art that
items other than luggage such as other packages, mail, and
cargo-containers, or even processed food stuffs, can also be
analyzed and screened or graded and that the descriptions are
exemplary and are not restrictive of the invention. Further, while
the invention is described as a dual-stage system and method, the
processing techniques discussed herein can be applied to each of
the individual scanning stages.
[0039] Referring to FIG. 1, a dual stage scanning system 100
comprises a housing 130, which encompasses a conveyor system 115
for moving containers, baggage, luggage, or similar object 110
through a plurality of scanning stages 150, 155. A sensor system
165 is connected at the entrance to determine when an object being
scanned 110 enters the scan field and communicates with a
controller [not shown] to activate or deactivate an X-ray radiation
source, 170, 172, as needed. A lead lined tunnel 180 surrounds the
conveyor to reduce radiation leakage outside the equipment. At
least one radiation source is not expressly depicted in FIG. 1 and
would be visible if the system were viewed from the opposite
side.
[0040] Referring to FIG. 2, the first stage 150, comprises two
X-ray cameras held together by a support structure 220, such as a
frame or yoke, for stability. Each camera consists of an X-ray
source 170, 171, a X-ray focusing means, such as a collimating slit
comprised of a radio-opaque material, for example lead (not shown),
and an array of detectors, 200, 201. In one embodiment, it is
preferred that the detectors are configured into a L-shape in order
to save space. One of ordinary skill in the art would appreciate
that other folded configurations may be acceptable, provided that
the detectors are appropriately positioned relative to the
inspection region and X-ray source.
[0041] Behind each slit collimator, a thin sheet of X-rays 210 is
formed. Within the sheet, a fan of pencil beams can be defined,
shown as dashed lines in FIG. 2, by connecting lines between the
stationary focus, not shown, and channels in the detector array.
Between focus and detector is a tunnel 180 through which the
luggage is transported or moved using any means known in the art,
including, for example, a conveyor 115, the surface of which is
depicted in FIG. 2. Wherever in the system radiation has to be
transmitted from X-ray sources 170, 171 and through the region
defined by tunnel 180, the conveyor belt support structure as well
as the tunnel has windows constructed from materials essentially
translucent to X-rays. The collimating slits and detector arrays
are oriented so that the radiation-fans intersect the main conveyor
surface within a few degrees of perpendicular relative to the
conveyor surface. The two X-ray sources and their fans point in
different directions.
[0042] In one preferred embodiment, the detector arrays are mounted
on printed circuit boards with a vector positioned normal to their
surfaces directed to the X-ray focus. An exemplary printed circuit
board has a capacity of 64 channels, and the boards are physically
arranged in Venetian blind configuration. The detector arrays
consist of linear arrays of silicon photodiodes that are covered
with scintillation material, which produces light when exposed to
X-rays. The light is detected by the photodiodes that produce
corresponding photo current signals. The detectors measure to what
degree the X-ray signal has attenuated due to passing through a
defined inspection volume. Specifically, the detected data are
converted to digital format, corrected for detector gain and
offset, and then stored. The required processor means may comprise
computing hardware, firmware and/or software known to persons of
ordinary skill in the art. When a container under inspection is
moving through the tunnel and passing through the X-ray
projections, both detector arrays are being sampled repetitively
between 50 and 500 times per second. Displaying the line
projections on a monitor renders the projection X-ray image.
[0043] While a conventional line scan system could be used as the
first stage scanning system, it is preferred to use the system as
described herein. More specifically, the present invention provides
for the placement of at least two X-ray sources such that the
directions of the X-ray projections emanating from the sources are
mirrored relative to the central vertical plane. Therefore, from
the perspective of a view along the path of conveyance through the
first stage scanning system, at least one X-ray generator is
mounted at a five o'clock position and at least one X-ray generator
is mounted at the 7 o'clock position.
[0044] One of ordinary skill in the art would appreciate that the
first stage scanning system is not limited to the specific
embodiments described above and that other variations are included
within the scope of this invention. In one alternative embodiment,
detector arrays are expanded from a single array to multiple
parallel arrays of detectors. In a second alternative embodiment,
X-ray projections are taken using two-dimensional pixellated
detector planes, without requiring the use of a conveyance means.
It should be appreciated that, while the present invention will be
further described using a description of the invention based on
using the line scan configuration of single stationary foci and
single line detector arrays in conjunction with a means of
conveyance, the present invention includes other systems and
methods that generate X-ray projection images and that such systems
and methods can be used in the novel dual stage scanning system
disclosed herein.
[0045] An alternative embodiment uses dual energy imaging. Dual
energy imaging can be utilized to display an image where materials
of a metallic constituency are suppressed (not displayed) or
materials of an organic constituency are suppressed. Having the
ability to selectively display certain materials within images
helps reduce image clutter. For example, when inspecting containers
for masses or explosives, which have little or no metallic
component, the "organic materials only" display is preferred. The
dual energy approach can be further refined to automatically
discriminate between similar materials of higher and lower relative
atomic numbers, such as between a plastic comprised of more lower
atomic number atoms like hydrogen and carbon and a plastic
comprised of more higher atomic number elements like oxygen and
nitrogen; or between aluminum (atomic number 13) and steel (atomic
number 26).
[0046] In one embodiment, dual energy data is generated by using an
X-ray tube with extended spectral emission, which is standard, in
conjunction with arrays of stacked detectors, where the first
detector is positioned to detect more of the lower energy, or
so-called softer X-ray photons, and the second detector is
positioned to detect the balance of the energy, namely the higher
energy, or so-called harder, photons. The second detector is
typically positioned behind the first detector. The low energy and
high energy measurements are combined in a suitable way using a
series of calibration measurements derived from dual energy
measurements taken of identified organic and metallic materials of
known thicknesses and result in the display of images, including
organic only or metal only images. One of ordinary skill in the art
would appreciate that various dual energy line scan systems are
commercially available.
[0047] It is preferred to use projection imaging as the first stage
scanning step in this invention. Features shown in the projection
images can be used by an operator to make a final decision on
whether items identified in a container represent a threat of some
type. Additionally, by taking projections from at least two
different angles, it is possible to triangulate the location of a
potential threat relative to the physical coordinates of the system
and use those coordinates to perform a more specific and focused
second stage scan. The triangulation process localizes certain
items that generate features of interest in the images and
identifies their location in the form of system coordinates.
[0048] To perform the triangulation process, the images that form
the basis of the triangulation process and that are used to
identify a target region are first identified. In one embodiment,
the images are analyzed by an operator who visually and
approximately determines a plurality of X-ray image
characteristics, such as degree of attenuation and projected area,
associated with mass, atomic number (identified using image color
coding), and shape. Operators also use contextual information, such
as an X-ray opaque organic mass in a transistor radio or a
suspiciously thick suitcase wall. The analytical process is known
to those of ordinary skill in the art and includes the
interpretation of X-ray image characteristics.
[0049] In another embodiment, images are identified by determining
the target regions automatically. For example, where the screening
target is a mass of plastic explosive, known algorithms, working on
dual energy X-ray projection image data, can be combined to
automatically find such target. Examples for such algorithm
components include, but are not limited to, edge detection,
watershed, and connected component labeling.
[0050] Referring to FIG. 3, a container 110 is moved on a conveyor
115 through a tunnel 180 in x-direction, perpendicular to the plane
of the Figure. A first X-ray generator 170, C1, with an X-ray
emitting focus projects a fan of X-rays 300 through a slit
collimator onto an array of detectors mounted on printed circuit
boards 200. One of ordinary skill in the art would appreciate that
only a small sampling of detectors are shown in FIG. 3 and that a
typical system would have a far greater number of detectors,
preferably 700 to 800, more preferably 740. As shown, the
orientation of the fan plane is perpendicular to the conveyor
surface. While a container is being moved along the conveyor
surface, the detectors are read out repeatedly, and their signals
are converted into digital format by detector electronics that are
also mounted on the detector boards 200. The data are being
processed and sorted further and stored in a computer [not shown]
for display on a monitor [not shown]. Each horizontal line on the
monitor corresponds to one particular detector in the array.
Therefore, in a system using 740 detectors, the full image is
composed of 740 lines.
[0051] A second X-ray camera, C2, consisting of X-ray generator
171, slit collimator (not shown) and detector array 201 is mounted
in a different orientation, and offset in conveyor direction, by
typically 100 mm. The detectors aligned with this camera are
sampled essentially simultaneously with the detectors of the first
camera and produce a second image displayed on a monitor.
[0052] Operationally, an item 340 located within the container 110
is recognized in the course of the first stage scan using a
detection algorithm or by operator analysis, depending upon the
system mode chosen. With the item 340 identified, the approximate
centerline X-ray projections 330, 331 that pass through the object
can be determined. Each of the centerlines 330, 331 is associated
with a certain detector channel, 310 and 311 respectively in each
view.
[0053] Referring to FIG. 3a, once the detector channels have been
determined, the location of the associated item 340 can be found in
the y-z coordinate system. Two images 380, 381 corresponding to the
two views are shown. With knowledge of the detectors associated
with the centerlines 331, 330 and the range of detectors, 308 to
314, defined, the y and z coordinates of the item 340 can be
derived. The x-coordinate is defined by the direction of conveyor
motion and is known because the conveyor motion control system,
timing of X-ray exposure, and the fixed offset of the two scan
planes are known. The x-coordinate can, for example, be referenced
to the beginning, or leading edge of the container, which can be
detected by a light curtain or similar position-detecting device.
In particular, the two images are referenced to each other
precisely in the x-coordinate direction.
[0054] The purpose of this triangulation or localization of
identified items in a container is to generate control commands
that can be used to position and focus the inspection region or
inspection volume of the second stage scanning system on the
identified item. Therefore, the first inspection stage quickly
locates potential threats and determines their coordinates, as
referenced to the system, while the second stage focuses on better
determining the nature of the identified potential threat. It
should be appreciated that, because the first stage
characterization of a threat is loosely based on features in X-ray
images, it will locate, find, and label, as a potential threat,
items which are innocuous, in addition to real threats. Therefore,
the performance of a detection system based only on the first
stage, as described, suffers from a high false alarm rate.
[0055] One of ordinary skill in the art would also appreciate that
other elements of the first stage scanning system are not depicted
in FIG. 1 but would be included in an implementation of the system.
For example, a shielding curtain is positioned at both the entrance
and exit of the system 100 to protect against radiation leakage to
the surrounding environment. The system 100 is controlled by a data
interface system and computer system that is capable of rapid, high
data rate processing, is in data communication with storage media
for the storage of scan data and retrieval of reference libraries,
and outputs to a monitor having a graphics card capable of
presenting images.
[0056] It should also be appreciated that a second stage scan may
not be required. In one embodiment, radiographic images from the
first stage scan are displayed on a computer monitor for visual
inspection with target regions or potential threats identified. An
operator may dismiss some of the identified regions or threats
based on context, observation, or other analytical tools. If no
threats are identified, the container is cleared to exit the
inspection system without subjecting it to the second stage of
scanning. However, if the operator is unable to resolve an area as
being a non-threat, the area is identified as a target region.
[0057] The second stage inspection or scanning system closely
inspects the identified target locations by deriving more specific
information, or a signature, and confirming the first stage threat
alarm only if the obtained signature matches the signature of a
threat substance or threat item. An alarm confirmed by the second
stage system are then taken seriously by operators and indicate the
need for further inspection, including, but not limited to,
operator image interpretation, additional scanning, and/or hand
searching the container.
[0058] In a preferred embodiment, the second stage scanning system
uses diffracted or scattered radiation to determine the properties
of a material, obtain a signature, and, accordingly, identify a
threat. Diffracted or scattered radiation comprises photons that
have experienced an interaction with the object under
investigation. In the special case of small angle scattering, the
majority of interactions are elastic or energy-conserving;
specifically, the diffracted photon has the same energy as it had
before the interaction, just its direction of propagation has
changed. If the energy distribution of the scattered photons is
being analyzed by an energy-dispersive detector system, which is
commercially available, certain properties of the material causing
the scatter are being encoded in the signature. Photons scattered
under small angles are scattered selectively due to interference
effects. Since the process does not change the energy of the
photons the signal also contains the distribution of the primary
radiation in a simply multiplicative way. The incoming primary
radiation, as well as the scattered radiation, encounter further
spectral modifications due to other types of interactions, such as
Compton scatter and photoelectric absorption, which are not energy
preserving. If one wants to view the characteristics of the
scattering material, other distracting spectral effects have to be
removed.
[0059] The detected signature of a threat is therefore a
combination of X-ray properties. One important property is a BRAGG
diffraction spectrum, observed at small diffraction angles between
2 and 8 degrees, with a preferred value around 3 degrees.
[0060] FIG. 4 shows schematically a cross section of a preferred
beam delivery system used to obtain BRAGG spectra at small angles.
Other beam delivery systems can also be used in the present
invention, including those disclosed by Kratky, et al. in Austrian
Patent No. 2003753 and Harding in U.S. Pat. No. 5,265,144. The
preferred system depicted in FIG. 4 further includes a transmission
detector.
[0061] A beam delivery system separates the photon radiation
emitted by the focus 400 of the X-ray source 404 into a plurality
of beams. A beam 401 is formed by passing through apertures 410 and
is directly detected by detectors 402, which are within the beam's
direct line-of sight. These beams are referred to as transmission
beams. Scatter interactions are detected by blocking direct
line-of-sight detection through the use of ring apertures 410, 411
and exposing the associated detector 420 only to scattered
radiation 492. Therefore, scatter radiation, generated when certain
beams interact with an inspection region or volume 445, can be
detected in the same apparatus as transmission radiation.
[0062] The choice of ring aperture diameters, distance to focus,
and distance to detector determines the effective scatter angle 430
of the photons falling on the detector. In one embodiment, the
scatter angle 430 is approximately the same for substantially all
photons detected by the detector of the scattered radiation. It is
preferred to configure the beam delivery system to establish an
effective scatter angle of between two and 8 degrees. It is more
preferable to have a scatter angle at or about 3 degrees. Using a
beam delivery system having a circular symmetry has the advantage
of obtaining a scatter contribution from a larger volume of the
material being inspected, thereby increasing the inherently weak
scatter signal. Additionally, the scatter spectrum can be cost
efficiently detected using only a single detector channel 420 with
an entrance aperture in the shape of a hole 421.
[0063] The scatter signal is generated by positioning the target
region 445, identified in the first stage scan, between the beam
forming apertures, irradiating that region 445 using the conical
beam 442, and making sure scatter radiation from the target region
445can be detected by the scatter detector. The target region 445,
often contained within a container 450, is in the shape of a tube
or ring 445 and is referred to as the inspection volume or
inspection region. The length, diameter, and wall thickness of the
inspection volume depends on the particular shape of the elements
of the beam delivery system, including focus size, ring aperture
diameter and width, detector opening and overall distance. In a
preferred embodiment for the inspection of large luggage, the
inspection volume is at or about 60 cubic centimeters.
[0064] In a preferred embodiment, as shown in FIG. 5, the
components of the beam delivery system are mounted to the open ends
of a rigid support structure 500 formed in the shape of a C
(referred to herein as a C-arm) and aligned with a tolerance of at
or about 0.1 millimeters. A first arm of the C-arm comprises a
X-ray tube with X-ray focus 172, a beam limiting aperture hole
mounted to the tube head 401, and a ring-shaped aperture 410. A
second arm holds comprises a transmission detector array 402, a
second ring aperture 411, and an energy dispersive detector 420,
equipped with an aperture hole.
[0065] The energy dispersive detector 420 is positioned to receive
scattered radiation from a target object placed on the conveyor
running between the arms of the C-arm support structure where a
first arm is above the conveyor and a second arm is below the
conveyor. The transmission detector is positioned to receive
radiation attenuated by the same target object. It is preferable
for the C-arm to be mobile and capable of moving in the x-direction
along the length of the conveyor. Therefore, the C-arm with tube
and detectors can be re-positioned along the length of the
conveyor.
[0066] In a preferred embodiment, the scatter detector 420 is
comprised of cadmium telluride or cadmium zinc telluride and is
operated at room temperature, or approximate to room temperature,
An exemplary embodiment is available from the e-V Products Company,
Saxonburg, Pa. This type of detector has a spectral resolution
performance that is well matched to the limited angular
requirements of this application, and therefore the limited
spectral resolution of the beam delivery system.
[0067] In one mode of operation, the potential threat locations
inside a container are found automatically by the first stage, and,
based upon the physical coordinates obtained through triangulation,
the second stage scanning system is automatically positioned to
generate an inspection region that substantially overlaps with the
identified target region. Where multiple threat locations are
identified, the second stage scanning system is sequentially
repositioned to focus on each subsequent target region. To scan
each target region, the second stage X-ray source is activated and
the scatter detector and transmission detector are sampled
simultaneously. In a preferred embodiment, a transmission spectrum
associated with the detected transmission data is characterized
using a look up reference, figure, table, or chart, and the scatter
spectrum is normalized using that identified transmission
spectrum.
[0068] In another mode of operation, an operator actively
identifies images that he or she believes corresponds to a
potential threat. X-ray images from the first inspection stage are
displayed to the operator, and the operator points to a suspicious
object as it appears in both views. To support this functionality,
operators use a computer system, comprising a mouse and monitor, to
position cross hairs over the areas of interest on each of the
images. Using coordinate data generated through triangulation, the
second stage scanning system automatically positions itself such
that an inspection region overlaps with the target region,
activates the X-ray source and simultaneously samples the scatter
detector and transmission detector. In a preferred embodiment, a
transmission spectrum associated with the detected transmission
data is characterized using a look up reference, figure, table, or
chart, and the scatter spectrum is normalized using that identified
transmission spectrum.
[0069] As discussed above, a transmission detector is integrally
formed with the beam delivery system, as shown in FIGS. 4 and 5. A
preferred transmission detector comprises a 16 channel array of
dual energy detectors. The detector array further comprises pairs
of detectors, including a low energy channel that receives and
measures a first amount of radiation first (low energy) and a high
energy channel that receives and measures a substantial portion of
the balance of radiation (high energy). Dual energy detection has
been described in connection with the linear scan arrays of the
first inspection stage and is known to persons of ordinary skill in
the art.
[0070] The low energy and high energy detectors measure a plurality
of low energy and high energy values that can be used to
characterize the material being scanned. In a preferred embodiment,
low energy and high energy data are used to reference a look up
reference, figure, table, or chart (referred to as a look up
source) which contains transmission spectra arranged in accordance
with corresponding high and low energy values. The look up source
is constructed with high energy values on one axis (i.e. the
x-axis), and low energy values on a second axis (i.e. the y-axis).
Referring to FIG. 6, an exemplary look up source 600 is shown. The
source 600 is a graph with high energy values on the x-axis 605 and
low energy values on the y-axis 610. Points 615 corresponding to
measured spectra 620 are positioned on the graph according to
certain linear combinations of the measured high and low dual
energy detector signals on the x and y axis.
[0071] The transmission spectra used to normalize scatter data is
therefore identified by obtaining high energy and low energy data
values, identifying the point on the graph corresponding to the
detected high and low energy values, and looking up the spectrum
associated with that point. Where the detected high and low energy
values yield a point on a graph that corresponds to an intermediate
point 630 proximate to pre-established points 635, 615, a
corresponding transmission spectra 645 can be calculated by
performing a two-dimensional interpolation of the spectra 640, 620
associated with the pre-established points 635, 615.
[0072] To create the look up source, an exemplary approach places
various materials of known composition and thickness, exposes them
to X-ray sources, measures the resulting high and low energy data
values, and uses the scatter detector to measure the corresponding
transmission spectrum. More specifically, the beam path of the beam
delivery system is modified to allow a direct beam from the focus
through the pinhole to fall on the energy dispersive scatter
detector. To further reduce the photon flux into a range that can
be tolerated for energy-dispersive measurement, the current of the
X-ray source is preferably reduced by a large factor, e.g. 100.
Under these parameters, the scatter detector can be used to measure
the transmission spectrum. Materials of known composition and
thickness are placed in the beam path. The materials are exposed to
X-ray radiation. Dual energy measurements are made using the dual
energy detectors and a transmission spectrum is obtained using the
scatter detector. Through this approach, for each material
composition and thickness, a transmission spectrum is obtained and
correlated with discrete pairs of dual energy transmission detector
readings. This information is then arranged on a chart with the
high energy value of the dual energy detector measurement on the
x-axis, and the low energy value on the y-axis.
[0073] It should be appreciated that, in the disclosed embodiment,
the spectra are the looked-up objects of the look up source.
Instead of the spectra, however, the look up source can
alternatively consist of spectral attenuation functions related to
the attenuation of the materials placed in the beam when the look
up source is being generated. The spectrum can then be obtained by
multiplying one fixed spectrum, for example the spectrum measured
without the material placed into the beam, with the spectral
attenuation function retrieved from the look up source.
Alternatively, the look-up source can contain numbers that are the
parameters of analytical expressions, e.g. polynomials, which are
formed to describe the attenuation functions in a parametric
way.
[0074] The presently described approach is preferred because it
enables the construction of a transmission detector array from
lower cost materials, as opposed to constructing the array using
more expensive energy dispersive detectors and support electronics.
Moreover, it also addresses the difficult problem of using energy
dispersive detectors to measure transmission spectra at the high
flux rates that are experienced at the location of the transmission
detector in the given configuration and at the same time at which
the scatter data are recorded. The required strong attenuation of
the transmission beams is a difficult problem that is avoided using
the present invention. The look up table is an important element
because the preferred dual energy detectors used in the
transmission detector cannot deliver spectra directly.
[0075] As discussed, transmission spectra are being used to correct
the scatter spectra that are being recorded by the energy
dispersive detector. Normalizing scatter spectra with transmission
spectra corrects for the confounding effects introduced by the
specific spectral distribution of the primary radiation, as emitted
from the X-ray source, as well as by spectrum-distorting effects
known as beam hardening. To correct the scatter spectra, the
detected scatter spectra are divided by the looked-up transmission
spectra.
[0076] A normalized scatter spectrum exhibits a plurality of
features. A first feature is that the location of the peaks and
valleys of the spectrum are determined by the molecular structure
of the materials located in the probe region. A second unrelated
feature is that the average spectral signal of the normalized
scatter signal, which can be of varying intensity, is linearly
related to the gravimetric density of the material in the probe
region. This can be used for threat discrimination since most
explosives, particularly military explosives, have a density range
above that of most other plastic or food items in suitcases.
[0077] In one embodiment, the normalized scatter signal is used to
identify a threat item by comparing the obtained normalized scatter
spectrum and/or spectral signal with a library of scatter signals
from known threat items. This comparison can occur automatically by
using a processor to compare a library of threat items, stored in a
memory, with the obtained scatter signals. Such a library is
developed by measuring the normalized scatter signatures of known
threat items. In addition to using the transmission detector to
generate data used to identify reference spectra, the transmission
detector can function in a plurality of other ways. In one
embodiment, the transmission detector acts as a position sensor.
The transmission beam is interrupted or attenuated momentarily when
an object on the conveyor crosses it. Tracking the moment of
interruption can provide information on the physical position of
the container on the conveyor and be used to appropriately position
the beam delivery system or container.
[0078] In a second embodiment, the transmission detector array
functions as an imaging detector to provide precise attenuation
data for certain areas in containers, like container wall areas,
where contraband can be hidden. When the circular beam is centered
on an edge of a container, the edge of the container can be imaged
in good detail, and can help analyze the edges for concealed
threats.
[0079] In a third embodiment, transmission detector measurements
can be used to determine whether the inspection region is, in fact,
the same target region previously identified in the first stage
scan. If the transmission data correlates with X-ray
characteristics different than those obtained in the first stage
scan, the relative positioning of the second stage scanning system
and the object under inspection may be modified until the
transmission data correlates with the same material characteristics
that was identified in the first stage scan.
[0080] In a fourth embodiment, transmission detector data are also
being used to simplify the algorithm-training procedure of the
system, as described below, in particular the collection of threat
material properties with irregularly shaped threat samples, like
sticks of dynamite.
[0081] It should be noted that it would appear because the scatter
radiation path and transmission path differ downstream from the
scatter volume, there would be inconsistencies in the data when
scatter and transmission data are combined. This inconsistency is
one example of a number of partial volume effects, solutions for
which are addressed herein. However, the inconsistencies are not
significant and can be tolerated without encountering significant
performance degradation of the system as a whole. As shown, FIG. 4
is not an isometric schematic and, in reality, the scatter angle is
preferably about 3 degrees, and the real path differences are
comparatively smaller.
[0082] As previously discussed, the second stage scanning system
positions an inspection region to physically coincide with the
target region identified in the first stage scan. The positioning
means may be achieved using any method known in the art. In one
embodiment, a plurality of control commands is produced in response
to the determination of the location of the target region. The
control commands are generated by at least one processor in data
communication with a plurality of processors capable of executing
the aforementioned triangulation techniques and/or determining the
intersection of projection lines to identify the location of the
target region in three dimensional system coordinates.
[0083] The control commands comprise data signals that drive a
three-axis control system. The vertical position of the
second-stage inspection volume can be adjusted to the target volume
or region of the first stage scan by moving the conveyor system up
or down. In another embodiment, the control commands comprise data
signals that drive the adjustment of the beam delivery system in
the second stage scanning system. The beam delivery system
adjustment can include any type of adjustment to the collimation or
beam focus, including the physical movement of a plurality of
apertures horizontally, vertically, or diagonally, the physical
modification of the diameter of the ring aperture by, for example,
increasing or decreasing the aperture size. In another embodiment,
the position of the support structure, or C-arm, can be modified
along the conveyor direction to appropriately position the beam
delivery system.
[0084] The second stage scan may be compromised when the volume of
the target region is smaller than the inspection region of the
second stage. In such cases, extraneous material, other than the
material identified as being a potential threat, such as air,
metal, or container edges, may be included. The resulting scatter
radiation is therefore a function of multiple material types and
may not be readily identifiable as being the signature of a single
substance.
[0085] In one embodiment, the present invention comprises a threat
recognition process that incorporates a training methodology which
relies on libraries in which threat signatures are obtained by
combining the threat with other common materials, such as clothing,
plastic, air, and metals. Specifically, the data used in training
and developing the detection process are chosen to include data,
which are corrupted by errors based on partial volume data from
statistically varying containers and threat and non-threat material
combinations. When the inspection volume is partially filled with a
threat substance and partially filled with a second innocuous
substance, a combination signal will be detected by the second
scanning stage. The automatic threat recognition methodology
recognizes the threat from the combination signal based upon the
aforementioned training. An exemplary automatic threat recognition
methodology, based on neural networks, is described below.
[0086] In a second embodiment, the detected scatter data is
corrected for the effects of extraneous materials by pre-processing
the data. The motion control system tracks where the inspection
volume or region is located in relative to a specific reference
point, such as the approximate outlines of the container, and
relative to the conveyor system. Because of the ability to measure
and track these reference points, the amount and portion of the
inspection volume occupied by the conveyor structure can be
determined. The conveyor structure includes the belt material as
well as the structural member that is underneath the conveyor,
which is referred to as the slider bed.
[0087] To correct the scatter spectrum for the presence of the
conveyor in the inspection volume, the scatter spectrum of the
conveyor materials is measured and stored in a reference database.
When the scatter spectrum of the inspection region is detected and
it is determined that the conveyor occupied a portion of the
inspection region, the scatter spectrum is corrected by multiplying
the conveyor material scatter spectrum by a weighting factor to
account for the size of the inspection volume occupied and that
amount is subtracted from the measurement.
[0088] Similarly, when part of the inspection volume is filled with
air, as in cases when suitcase walls are targeted by the inspection
volume, it is known that the contribution of the air-filled portion
of the inspection volume to the scatter signal is approximately
zero, and therefore, substantially all of the scatter signal can be
attributed to the material in the remainder of the inspection
volume. By accounting for the air volume contribution, the
characterization of the material in the remaining inspection volume
is rendered more precise. Optical detectors, such as a plurality of
light-curtains, can be positioned across and within the scanning
system to generate control signals that convey information about
the height and edges of the container relative to the conveyor
system and relative to the inspection region. It therefore can be
calculated which portion of the inspection region is filled with
air.
[0089] In another embodiment, transmission values for the scatter
beam are measured by an array detector. An exemplary array
comprises 16 channels and yields transmission data for 16
subdivisions within the inspection volume. The transmission values
can be used to characterize the material distribution in the
inspection volume. Based on these transmission values, approximate
mass values can be determined for masses contained in each of the
16 subdivisions. For example, where the transmission detector value
returns a value indicating the subdivision has material with zero
thickness, it can be assumed that the subdivision is occupied by
air.
[0090] In a preferred embodiment, the inspection volume is
subdivided. By reducing the size of the inspection region, one can
ensure that fewer differing materials occupy the same region and
can therefore avoid the complex composite signals that get
generated when multiple materials fill a single inspection region.
In one embodiment, system resolution is increased by providing
multiple energy dispersive detectors, such as 2, 3, 4, 5, 6 or
more, in place of a single energy dispersive detector as shown in
FIG. 4.
[0091] Referring to FIG. 7, a schematic representation of the beam
delivery system of FIG. 4 780 is shown relative to a beam delivery
system having multiple energy dispersive detectors 785. A first
system 780 comprises single detector 700s, circular aperture 701s,
inspection volume 702s, circular aperture 703s, and X-ray focus
704s. The dark areas represent the presence of radiation blocking
material, e.g. 1/4 inch lead alloy, and the white areas represent
areas that are transparent to X-rays above 30 keV. A second system
785 comprises an X-ray focus 704q, circular aperture 703q, divided
inspection volume 702q, detector side beam shaping aperture 701q,
and quadruple detector 700q. The aperture 70lq is center-symmetric
and consists of four slits, each conforming to part of a circle.
The centers of the circular slits are chosen to be of the same
pattern as the detectors of the quadruple detector 700q. For
example, if the detector cluster consists of four channels centered
on the four corners of a 2 by 2 mm square, the centers of the
partial and circular apertures lay on a circle with diameter equal
to the square root of 2 times 2 mm. The resulting inspection region
for each individual detection region is about one quarter of the
full inspection volume. A subdivided inspection region provides a
higher spatial resolution of the second stage inspection. Clusters
of energy dispersive detectors with their supporting electronics
are commercially available from companies such as eV Products,
Saxonburg, Pa.
[0092] If more than one scatter detector is being employed, a
collimating system of vanes can be placed in front of the detector
cluster orthogonal to the surface of the detector and in line with
the plane of separation between each detector. Using a separator
705, diffracted radiation is more effectively limited to reach the
appropriate channel in the cluster and, consequently, detected
signals are more readily associated with materials from specific
areas within the inspection region. The separator 705 extends from
the surface of the detector cluster toward the surface of the
adjacent aperture. The number of separator vanes is dependent on
the number of detectors. A typical vane material and thickness is
lead alloy of 0.5 mm thickness.
[0093] Referring to FIG. 9, a flowchart summarizing the operational
process of one embodiment of the present invention is provided. A
container enters into the first stage scan 905 where it is exposed
to a plurality of projected beams 915. From that exposure, X-ray
characteristics are determined 920 and target regions containing
potential threats are identified 925, 935. If no potential threats
are identified, the container is not subjected to a second scanning
stage 940. The three dimensional coordinates of the target region
is determined 945 and, accordingly, the inspection region generated
by the second stage scanning system is coordinated to coincide with
the target region 950. The inspection region is subjected to X-ray
radiation in order to obtain transmission and spectral data 955.
The spectral data is then analyzed 960 to determine the existence
of a threat. The data collected in the second stage scan comprises
both localized dual energy transmission data and localized BRAGG
diffraction spectra, which are subject to statistical variances,
originating from photon signal fluctuations, partial volume
limitations, or variations of the type of luggage and their
contents, among other causes. As such, it is preferred to have a
processing methodology that accounts for the fact that the raw data
is not sufficiently sensitive to detect threats with sufficiently
low false alarm rate.
[0094] In a preferred embodiment, the automatic threat resolution
is performed by a probabilistic technique in which a plurality of
input data points, obtained from the raw spectral scan data,
contribute to the probability that the corresponding spectrum
belongs to a particular class of threat or non-threat items. Such a
probabilistic technique relies on the plurality of input data
points as a whole rather than on individual data points. Although
probabilistic classification techniques can include explicit,
identifiable rules created by a programmer, the preferred
techniques utilize a classification procedure that incorporates the
results of training. For example, the classification algorithm can
be used to process a training set consisting of patterns for
structures of known classification. The results of this processing
are used to adjust the algorithm, so that the classification
accuracy improves as the algorithm learns by processing the
training sets.
[0095] One type of trainable classifier that can be employed is an
artificial neural network. Artificial neural networks attempt to
model human biological neural networks to perform pattern
recognition and data classification tasks. Neural networks are fine
grain parallel processing architectures composed of non-linear
processing units, known as neurons or nodes, which attempt to
replicate the synaptic-dendritic interconnections found in the
human brain.
[0096] Different types of neural networks exist. One type of
network is a multi-player, feed-forward network. A feed forward
network passes a signal by links from input nodes to output nodes,
in one direction only. In most implementations, the nodes are
organized into multiple layers: the input layer, output layer, and
several "hidden layers" in between. The adjacent layers are
normally fully interconnected.
[0097] FIG. 8 depicts a schematic representation of a preferred
type of artificial neural network 800 known as a hidden-layer
feed-forward network consisting of an input layer 810 of neurons or
nodes, at least one hidden layer 820, and an output layer 830. The
neuron layers are linked via a set of synaptic interconnections.
Each neuron in the input layer is typically connected to each
neuron in the hidden layer, and each neuron in the hidden layer is
typically connected to each neuron in the output layer, via a
synaptic connection; these may be physical, electronic connections,
or they may be embodied in software, as may be the neurons
themselves, which software operates on conventional digital
computers.
[0098] The neurons or nodes typically accept several inputs and
create a weighted sum (a vector dot product). This sum is then
tested against an activation rule (typically a threshold) and then
processed through an output function. The output function could be
a non-linear function such as a hard-limiter; a sigmoid function; a
sine-function or any other suitable function known to a person of
ordinary skill in the art. The threshold determines how high the
input to that neuron must be in order to generate a positive output
of that neuron. A neuron may be considered to be turned on, for
instance, whenever its value is above a predetermined value such
as, for instance, 0.9 and turned off with a value of less than
another value such as 0.1, and has an undefined "maybe" state
between those values. The connectivity pattern defines which node
receives the output value of a previous node as their input. The
connection between two neurons is realized in mathematical terms by
multiplying the output of the lower level neuron by the strength of
that connection (weight). At each instant of propagation, the
values for the inputs define an activity state. The initial
activity state is defined upon presentation of the inputs to the
network.
[0099] The output response of any hidden layer neuron (o.sub.j) and
any output layer neuron is a function of the network input to that
neuron defined by the difference of that neuron's threshold
(.theta.) and the input to it. The value of the input into each
hidden or output layer neuron is weighted with the weight currently
stored for the connection strengths between each of the input and
hidden layer neurons, and the hidden and output layer neurons,
respectively. Summing over all connections into a particular neuron
and subtracting this sum from the threshold value may be performed
according to the following sigmoid-type Fermi function:
o.sub.j=[1+exp(.theta..sub.j-.SIGMA..sub.i.multidot.w.sub.ji*o.sub.i)].sup-
.-1
[0100] where i and j represent neurons of two different layers with
j representing the higher layer; .theta..sub.j represents the bias
value for j layer neuron; and w.sub.ji represents the strength of
the connection between neuron i and neuron j. Alternatively,
sine-type functions, or any other suitable function known in the
art, may be used to obtain the desired type of response function
for the output of a neuron. The weights are chosen so as to
minimize the error between the produced result and the correct
result. A learning rule defines how to choose the weight values.
Several commonly used learning rules are back-propagation,
competitive learning, adaptive resonance, and
self-organization.
[0101] In a preferred embodiment, the artificial neural network
uses back-propagation learning. The back-propagation learning
algorithm, derived from the chain rule for partial derivatives,
provides a gradient descent learning method in the space of weights
and can be further understood by reference to D. E. Rumelhart, et
al., Parallel Distributed Processing, ch. 8, pp. 322-28 (MIT Press,
1986) and Haykin, Simon (1999), "Neural Networks", Prentice Hall,
both of which are incorporated herein by reference.
[0102] Back-propagation learning involves a set of pairs of input
and output vectors. The network uses an input vector to generate
its own, or actual, output vector. The actual output vector is
compared with a desired output, or target, vector that may be
defined usually in the course of training. The weights are changed
to obtain a match between the target vector and the actual output
vector. The conventional delta rule may be used for this
calculation where the weight for a particular synapse or connection
between units is adjusted proportionally to the product of an error
signal, delta, available to the unit receiving input via the
connection and the output of the unit sending a signal via the
connection. If a unit is an output unit, the error signal is
proportional to the difference between the actual and target value
of the unit. If it is a hidden layer, it is determined recursively
in terms of the error signals of the units to which it directly
connects and the weights of those connections.
[0103] Thus, the training of a neural network is the process of
setting the connection weights so that the network produces a
desired output in response to any input that is normal for the
situation. A supervised training refers to the kind of training
that requires a training set, i.e. a set of input-output patterns.
The back-propagation algorithm is an efficient technique to train a
feed-forward network. It operates to send an error back through the
network during the training process, thereby adjusting all the link
weights in correspondence with their contribution to the error. The
weights of the network therefore gradually drift to a better set of
values. The initial weights are chosen randomly within reasonable
limits and adjustments are left to the training process.
[0104] Referring back to FIG. 8, the artificial neural network 800
is trained on a suitably large set of threat and non-threat X-ray
raw scan data, to generate an output 840, in accordance with the
error back-propagation learning method described above. As
described earlier, the required set of threat and non-threat raw
scan data for training can be obtained either from the scanning
system of the first stage or the scanning system of the second
stage or both depending upon whether artificial neural networks are
used to process scan data from the first stage or the second stage
or from both the stages. Thus, the `scan data` 805 to be used to
train the neural net 800 may comprise of raw attenuation data, raw
transmission photon counts, raw diffraction photon spectra or any
other data known to a person of ordinary skill in the art.
[0105] The purpose of the neural network processing step is to have
a processing means capable of recognizing a threat signature. A
threat signature is defined as a spectrum, i.e. an array of numbers
corresponding, on a one-to-one basis, to the discretized values of
a physical quantity, such as the energy of X-rays, and includes
unrelated, but relevant, other values, such as transmission
detector array data, bag height, and other environmental factors.
Although the spectrum may consist of any amount of data points, the
present invention preferably operates on a spectrum data set of
between 200 and 800 points and, more preferably, of approximately
500 points. Additionally, while the network may consist of any
number of layers, it is preferred that it consists of four layers,
including one input layer, two hidden layers, and one output layer.
Further, while the network can have multiple output nodes with
various indicators, it is preferred that, for the present
invention, the network comprise a single node the output of which
may be interpreted as either "yes, a threat has been recognized" or
"no, a threat has not been recognized".
[0106] In a preferred embodiment, the raw diffraction spectrum data
from the second stage is used to generate the required set of
threat and non-threat scan data for training. These spectral counts
represent raw, that is non-normalized, scan data 805 that is
subsequently used to train the neural network 800. Alternatively,
this scan data 805 may be further processed to generate a plurality
of normalized data.
[0107] The scanning process is repeated to obtain scan data of a
sufficiently large number of containers containing threat and
non-threat items packaged in a variety of permutations and
combinations to model real-world scenarios. This raw scan data,
referred to hereinafter as training data, comprise an input-set to
be used for training the neural network. Since the training data is
obtained by scanning containers containing known materials, each
output training data maybe further tagged to identify whether the
respective training data represents a defined/known threat or
non-threat item. This output training data maybe further stored in
a suitable library or database such as a file server on a digital
computer system along with the tagged identification information.
Furthermore, the library or database of training data may be
enhanced to incorporate and reflect all previously known threat
materials and their corresponding raw `scan data`.
[0108] In a preferred embodiment, two libraries are generated. A
first library has signatures of threats. A larger second library
has signatures of innocuous, or non-threat, items. The training
process utilizes the threat and non-threat signatures to introduce
into the system threat-like and non-threat-like signatures.
[0109] A threat-like signature is a linear combination of a sample
from the threat library with a plurality of samples, such as two,
from the non-threat library. The coefficients of the mix are
randomly simulated. A simulated white noise is also added to the
generated mixture, with its amplitude also randomly generated,
within an interval from zero to a given fraction of the signal. A
non-threat-like signature is a mixture of a plurality of non-threat
signatures, such as two or three. The coefficients of the mix are
randomly simulated. A simulated white noise is also added to the
generated mixture, with its amplitude also randomly generated,
within an interval from zero to a given fraction of the signal.
Using signature mixes, incorporated with noise, the system is
trained to recognize a threat or a non-threat by outputting an
appropriate recognition answer from the last output node within the
context of a reasonable level of noise and interference from
overlapping items.
[0110] FIG. 10 depicts a plurality of steps, in flow diagram
format, of one embodiment of the back-propagation training process
of the invention. One of ordinary skill in the art would appreciate
that the processing is conducted using a computer having a
plurality of processors for executing the analytical processes
described herein, embodied in at least one software program, a
plurality of storage devices for storing the requisite data,
library information, and other information necessary to conduct
these analyses, and an output device, such as monitor, among other
commonly known computing devices and peripherals.
[0111] At the beginning of the training process 1003, the synaptic
weights and thresholds of the neural net are initialized 1005 with,
for example, random numbers. After initialization 1005, the input
layer of the neural network is introduced 1010 to a first set of
training data and the net is run to receive 1015 an actual output.
The neural net makes use of the randomly assigned weights and
thresholds to generate an output on the basis of a suitable
resolving function such as a sigmoid-type Fermi equation (described
earlier), a sine function or any other function known to a person
of ordinary skill in the art. The output could be in the form of
differentiable signals such as numerals between, say, 0 and 1, in
the form of positive or negative states implied by an output
numeral of greater than or less than 0 respectively, or any other
suitable indication as evident to a person of ordinary skill in the
art.
[0112] The first set of training data is introduced into the system
and, based on the random weights and thresholds, produces an output
`x`, i.e. a numeral greater than 0. If the training data represents
a threat, this output indication is set as a benchmark to identify
a `threat` while a numeral less than 0 maybe set to identify a
`non-threat` item. Once a suitable benchmark is set, the training
process is repeated with the next set of training data and
corresponding actual outputs are received. The actual output is
compared 1020 with the desired output, defined by an operator with
knowledge as to whether input data is or is not representative of a
threat, for the corresponding set of training data that was fed to
the neural net in step 1010. If the actual output received 1015 is
commensurate with the desired or targeted output or, if the
difference between the target and actual output falls below a
predefined acceptable level, a check 1025 is made to see if the
neural net has been trained on the entire set of training data. If
not then the next set of training data is introduced to the neural
net and the foregoing steps are repeated. The training process
continues until the neural net has been trained on the entire set
of training data.
[0113] If the comparison 1020 suggests that the actual output is
not in agreement with the desired or targeted output, the ensuing
additional steps are performed. The difference between the actual
and desired outputs is used to generate 1030 an error pattern in
accordance with a suitable back-propagation rule such as the `delta
rule` or any other error estimation rule known to a person of
ordinary skill in the art. The error pattern is used to adjust 1035
the synaptic weights on the output layer such that the error
pattern would be reduced the next time if the same set of training
data were presented as the inputs. Then the weights of the hidden
layers, preceding the output layer, are modified 1040 by comparing
what outputs they actually produce with the results of
neurons/nodes in the output layer to form an error pattern for the
hidden layer.
[0114] The error can thus be propagated as far back over as many
hidden layers as constituting the neural network. Finally, the
weights for the input layer are similarly educated 1045, and the
next set of training data is introduced to the neural network to
iterate through the learning cycle again. The neural network is
therefore trained by presenting each set of training data in turn
at the inputs and propagating forwards and backwards, followed by
the next input data, and repeating this cycle a sufficient number
of times such that the neural network keeps getting closer and
closer to the required weight values each time. Thus, the network,
through the iterative back-propagation process, establishes a set
of weights and thresholds for neural connections so that a desired
output pattern is produced for the presented input information. The
learned information of a neural network is contained in the values
of the set of weights and thresholds.
[0115] In exemplary embodiments, the neural network is structured
such that, through iterative forward and backward propagation,
every node in a layer can be made to contribute to every node in a
subsequent layer, only certain nodes in a layer maybe used to
contribute to certain nodes in a subsequent layer, or every node in
a layer contributes to every node in a subsequent layer but the
impact of certain first layer nodes on subsequent layers are
weighted relative to other first layer nodes. In a preferred
embodiment, the nodes closest to subsequent layer nodes are
weighted relative to other nodes in that same layer.
[0116] More preferably, links between the input layer and the first
hidden layer are not chosen randomly, but selected to have a
special distribution. Each hidden layer node is responsible for a
region of the spectrum. Links to each hidden layer node from this
region have higher weights. Therefore, the farther an input node is
from this region, and the less responsible it is, the weaker the
link with that input node. Together, the hidden nodes encompass the
entire input layer spectrum. By distributing a pre-assigned degree
of influence over links, a form of convolution is provided. This
embodiment is particularly preferred where preprocessing is not
reasonably effectuated because the input data size is too
large.
[0117] Because new threats may develop over time, it is desirable
to have a simple procedure that updates the network to recognize
such additional threats. In a preferred embodiment, multiple
networks are formed and trained to identify distinct threats.
Therefore, new threat recognition is done by implementing a neural
network as a set of multiple networks, each trained to identify a
specific threat.
[0118] Each network group is formed and trained to address and
recognize one threat. Thus, there is a one-to-one correspondence
between the threats (T.sub.1, T.sub.2, T.sub.3 . . . T.sub.n) and
the groups (G.sub.1, G.sub.2, G.sub.3 . . . G.sub.n). Network group
G.sub.n is trained to recognize threat T.sub.n. Where G.sub.n
recognizes any other threat, T.sub.n-2, T.sub.n-1, T.sub.n+1, it is
not considered relevant. G.sub.n is trained using threat signatures
of T.sub.n and a corresponding library of non-threat signatures. A
group consists of a plurality, such as two, three, or four,
completely separate networks, each similarly trained to recognize
the same threat. Threat recognition is achieved where the average
of the recognition results of each network indicates a threat.
Where an additional threat is identified, T.sub.new, a new group of
networks, G.sub.new, can be created, without having to retrain or
modify all existing groups. This permits the more efficient,
incremental addition of new threat recognition networks.
[0119] One of ordinary skill in the art would appreciate that the
output of these network groups can be handled in various ways.
Specifically, a system can output a threat alarm if the recognition
result, averaged over all network groups, indicates a threat. A
system can output a threat alarm if only one group of networks
indicates a threat or if a subset of network groups indicates a
threat. In a preferred embodiment, threat recognition is
effectuated by monitoring the output of substantially all groups of
networks. If the recognition result, averaged over all groups,
indicates the existence of a threat, the output results of
individual network groups are analyzed. If at least two groups
indicate the presence of a threat, then the groups are reviewed to
determine which threat has been recognized. One of ordinary skill
in the art would appreciate that various derivations of the
above-described process can be conducted without departing from the
scope of this invention. For example, the threshold analysis can be
performed even if only a portion of all network groups is monitored
and the secondary analysis can be performed if fewer than two
groups indicate a threat.
[0120] It is further preferred to regulate the balance between the
sensitivity of detection with the selectivity of detection by
incorporating an additional input node that is used to inject a
sensitivity level into the training process. Systems with higher
sensitivity will detect more threats at the expense of having
greater false alarms. Systems with higher selectivity will have
fewer false alarms, with the disadvantage of possibly not detecting
all threats. In the course of operation, it may be necessary to
change the sensitivity/selectivity balance, depending on various
circumstances, and therefore it is desirable to have a means for
doing so.
[0121] In one embodiment, a plurality of different networks is
stored in a storage device in data communication with processors
responsible for executing neural network analytical processes. Each
set of networks has a different level of sensitivity, i.e. least
sensitive, less sensitive, normal sensitivity, more sensitive, most
sensitive. Depending on the requisite level of security (versus
requisite level of throughput), the appropriate network set is
loaded into the system. Alternatively, a network set having a
standard level of sensitivity may be used in operation and a
parallel network set, having varying levels of sensitivity, may be
concurrently loaded and ready for use, when necessary. Having a
parallel network avoids downtime associated with loading new
network sets into local memory, or RAM.
[0122] Networks having varying degrees of sensitivity are developed
by incorporating a sensitivity variable in the training process.
With each recognition task, a desired level of sensitivity is
communicated to an additional input layer node, thereby inherently
incorporating it into the training process. Every training event
could further be associated with a randomly selected sensitivity
level, selected from within a reasonable range. Training is
therefore conducted with the selected level of sensitivity.
[0123] The embodiment of back-propagation learning process, as
described with reference to the flow diagram of FIG. 10, assumes
that the training data is first collected by operating the first
and/or second scanning systems offline, namely by scanning a large
number of baggage containing known threat materials camouflaged
amongst non-threat items in a variety of combinations to represent
a variety of concealment scenarios. The training data so obtained
is then used to educate the neural network that can then be used to
operate in real-world situations. Thus, in this embodiment, when
the first and/or second stage systems are online, that is
operational at real-world sites such as at airports for luggage
inspection, the associated neural network is already partially
educated (on the training data obtained through test baggage) to
discern threat from non-threat items.
[0124] However, in another embodiment of the back-propagation
learning process, the neural network need not be taught through the
scanning of test baggage prior to running the first and/or second
scanning systems online. Instead the first and/or second scanning
systems can be operated online and the scan data can then be fed
into the neural network in real-time. On the basis of this
real-time scan data, the neural net is made to classify threat and
non-threat items. At the same time, the scanned image of the
baggage is also presented to an operator in the form of a visual
display such as on a conventional computer screen, as is known in
the art. The operator compares the output of the neural net with
his own observation of the scanned baggage. In case the output of
the neural net is found to be erroneous, the operator prompts the
neural net with the correct or desired output, enabling the neural
network to adjust its weights and thresholds accordingly. Thus, in
the second embodiment when the system is first used, it will have
relatively little knowledge about threat and non-threat materials
to be identified and recognized. However, with sufficient positive
reinforcement of a relationship between the acquired scan data and
operator prompted identification, the neural network will learn how
to identify threat objects. This self-learning process enables the
neural network to learn continuously.
[0125] The on-line, self-learning training process does have
certain advantages. A company or organization that uses the present
invention may not want to share or disclose data to third parties
due to privacy or security reasons. Therefore, it may be essential
to enable self-learning. Furthermore, the flow of data may change
seasonally depending upon how containers change. Specifically,
seasonal variations do occur in airline passenger travel where
passenger bags may get larger in the winter to accommodate more
clothing or the number of total bags may increase due to larger
numbers of people traveling in the summer. To address such seasonal
variation, it is more practical to allow on-line autonomous
adaptation.
[0126] Finally, there may be a variety of system users in different
geographical regions that experience different types of threat and
non-threat items. In such cases, a standard library may not be as
helpful as self-taught systems that automatically learn in
accordance with its own unique context. More specifically, over
time, a system in the field will be trained on containers that have
threat and non-threat items unique to their geographic context. Due
to operator training and interaction, a particular system would
therefore develop a trained processing system tailored to their
geographical context. It is preferred, however, that, irrespective
of the geographical location, systems get periodically trained
using threats that are new or infrequently seen to ensure that the
system does not forget the identity of such threats. This update
could be performed by the statistically controlled re-injection of
threats from existing threat databases.
[0127] Although this online self-learning process has been
described separately as an embodiment, this continuous
self-learning process can be used in conjunction or combination
with the offline teaching process of the first embodiment, using
test baggage. In a preferred embodiment, the neural network is
first trained on scan data obtained by running the X-ray system
offline on test baggage and then through operator prompts in
real-time operations as well, so that the ability of the neural
net, in identifying objects, continuously improves through
self-learning. Nevertheless, the system may undergo retraining
off-line using data from multiple site locations, thereby taking
full advantage of the sum total of learning being generated by the
operation of multiple systems.
[0128] Operationally, acquired scan data is fed into the neural
network for identification. If the object is identified with a high
degree of confidence, the identification and scan image is conveyed
to an operator, along with an indication of what the object may be,
to enable the operator to take an action, including conducting a
hand search, questioning the container owner, permitting the
container to pass, or calling in additional personnel. In one
embodiment, the operator provides feedback to the system based upon
the identity of the threat/non-threat and action taken. For
example, if the system identifies the existence of a threat, the
operator can check the container to determine if a threat exists
and then inform the system whether it was or was not correct. If
correct, the neural network increases its confidence factor for
that object's scan data and stores the scan data in a suitable
database as an exemplar for retraining. If incorrect, the neural
network implements the error back-propagation process to suitably
adjust its weights and thresholds and stores the scan data in a
suitable database as an exemplar for retraining.
[0129] This on-line adaptation process using incoming data requires
certain precautions, however. It is preferred that the system
utilizes groups of networks, which are accompanied by libraries of
threat patterns and innocuous patterns, and that the system is not
authorized to modify these libraries. It is further preferred to
include an additional library, a buffer-library, that is available
for modification based upon incoming data. Specifically, the buffer
library comprises incoming new data, and is preferably periodically
cleansed of older data. Consequently, the networks are being
re-trained using the buffer-library and the two stable libraries,
with a proper adaptation time scale. Several previous versions of
network groups are stored as a back up and a comparison of newly
adapted system with its older versions can be conducted and
produced in the form of a report. As described, on-site training
can be set up as an automatic feature, but operator input may be
required in the rare case of a real alarm.
[0130] It is the intention of the inventors to embody within the
scope of this patent all embodiments that reasonably and properly
come within the scope of this specification. For example, while the
dual-stage scanning system has been described with reference to a
first stage scanning system comprising a dual-view line scanner and
a second stage scanning system, comprising a transmission and
scatter scan, other modifications and changes can be made by those
of ordinary skill in the art on the basis of the number of beam
projections and the singularity or duality of incident beam
energies used. Additionally, while the details of processing the
scan data using trained classifiers has been used with the second
stage scanning unit, one of ordinary skill in the art would
appreciate that it could be used with the first stage unit to
better help identify threats or target regions
* * * * *