U.S. patent application number 12/129410 was filed with the patent office on 2009-01-01 for user guided object segmentation recognition.
Invention is credited to Peter Dugan, Michael Riess.
Application Number | 20090003699 12/129410 |
Document ID | / |
Family ID | 40088192 |
Filed Date | 2009-01-01 |
United States Patent
Application |
20090003699 |
Kind Code |
A1 |
Dugan; Peter ; et
al. |
January 1, 2009 |
USER GUIDED OBJECT SEGMENTATION RECOGNITION
Abstract
A system for segmenting radiographic images of a cargo container
can include an object segmentation recognition module adapted to
perform a series of functions. The functions can include receiving
a plurality of radiographic images of a cargo container, each image
generated using a different energy level and segmenting each of the
radiographic images using one or more segmentation modules to
generate segmentation data representing one or more image segments.
The functions can also include identifying image layers within the
radiographic images using a plurality of layer analysis modules by
providing the plurality of radiographic images and the segmentation
data as input to the layer analysis modules, and determining
adjusted atomic number values for an atomic number image based on
the image layers. The functions can include adjusting the atomic
number image based on the adjusted atomic number values for the
regions of interest to generate an adjusted atomic number image and
identifying regions of interest within the adjusted atomic number
image based on an image characteristic. The functions can also
include providing coordinates of each region of interest and the
adjusted atomic number image as output.
Inventors: |
Dugan; Peter; (Ithaca,
NY) ; Riess; Michael; (Endwell, NY) |
Correspondence
Address: |
MILES & STOCKBRIDGE PC
1751 PINNACLE DRIVE, SUITE 500
MCLEAN
VA
22102-3833
US
|
Family ID: |
40088192 |
Appl. No.: |
12/129410 |
Filed: |
May 29, 2008 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60940632 |
May 29, 2007 |
|
|
|
Current U.S.
Class: |
382/173 |
Current CPC
Class: |
G06K 9/6263 20130101;
G01N 23/06 20130101 |
Class at
Publication: |
382/173 |
International
Class: |
G06K 9/34 20060101
G06K009/34 |
Claims
1. A system for user guided segmentation of radiographic images of
a cargo container, the system comprising: an object segmentation
recognition module; and an operator terminal including a display
device and an input device, the operator terminal coupled to the
object segmentation module, the object segmentation recognition
module having instructions stored in a memory that when executed
cause the object segmentation recognition module to perform a
series of functions including: segmenting a plurality of
radiographic images of a cargo conveyance; outputting region of
interest coordinates and a corrected atomic number image as output;
providing a first image to the operator terminal for display, the
first image being based on the corrected atomic number image;
receiving input from the operator terminal, the input including an
indication of a selected region of the first image; performing an
additional segmentation process on only the selected region of the
first image; and providing a second image to the operator terminal
for display, the second image based on the selected region of the
first image and results of the additional segmentation process.
2. The system of claim 1, wherein the plurality of radiographic
images are generated using four energy levels.
3. The system of claim 1, wherein the region of interest
coordinates are determined by comparing an estimated atomic value
of each image segment to a threshold value.
4. The system of claim 1, wherein segmenting the plurality of
radiographic images includes: segmenting each of the radiographic
images using one or more segmentation modules to generate
segmentation data; identifying regions of interest and image layers
containing regions of interest; analyzing the image layers using
the plurality of radiographic images and the segmentation data in
order to determine corrected atomic number values for the regions
of interest; and correcting an atomic number image based on the
corrected atomic number values for the regions of interest.
5. The system of claim 4, wherein regions of interest are
identified by an image characteristic.
6. The system of claim 1, wherein the radiographic images include
only radiographic image data.
7. The system of claim 1, wherein the radiographic images include
effective atomic number data and radiographic image data.
8. The system of claim 1, wherein the function of providing the
first image includes providing coordinates of each region of
interest and the corrected atomic number image to an operator
station.
9. The system of claim 1, wherein the function of providing the
first image includes providing the coordinates of each region of
interest and the corrected atomic number image to another
system.
10. A method for user guided segmentation of radiographic images,
the method comprising: segmenting a plurality of radiographic
images in order to determine regions of interest and corrected
atomic number values; correcting an atomic number image to generate
a corrected atomic number image; providing a first image to an
operator terminal for display, the first image being based on the
corrected atomic number image; receiving input from the operator
terminal, the input including an indication of a selected region of
the first image; performing an additional segmentation process on
the selected region of the first image; providing a second image to
the operator terminal for display, the second image based on the
selected region of the first image and results of the additional
segmentation process; and providing updated regions of interest and
corrected atomic number values based on the additional segmentation
process as output.
11. The method of claim 10, wherein the plurality of radiographic
images are images of a cargo container.
12. The method of claim 10, wherein the plurality of radiographic
images includes four images each image being generated using a
different energy level.
13. The method of claim 10, wherein identifying regions of interest
includes comparing an estimated atomic value of each image object
to a threshold value.
14. The method of claim 10, wherein the radiographic images include
only radiographic image data.
15. The method of claim 10, wherein the radiographic images include
effective atomic number data and radiographic image data.
16. A radiographic image segmentation apparatus comprising: means
for segmenting a radiographic image in order to determine regions
of interest and corrected atomic number values; means for providing
a first image to an operator terminal for display; means for
receiving input from the operator terminal, the input including an
indication of a selected region of the first image; means for
performing an additional segmentation process on the selected
region of the first image; means for providing a second image to
the operator terminal for display, the second image being based on
the selected region of the first image and results of the
additional segmentation process; and means for providing updated
regions of interest and corrected atomic number values based on the
additional segmentation process as output.
17. The apparatus of claim 16, wherein the radiographic image is an
image of a cargo container.
18. The apparatus of claim 16, further comprising means for
correcting an atomic number image to generate a corrected atomic
number image, wherein the first image is based on the corrected
atomic number image
19. The apparatus of claim 16, wherein the means for receiving a
radiographic image further includes means for receiving a plurality
of radiographic images.
20. The apparatus of claim 19, wherein each radiographic image is
generated using a different energy level.
Description
[0001] The present application claims the benefit of U.S.
Provisional Patent Application No. 60/940,632, entitled "Threat
Detection System", filed May 29, 2007, which is incorporated herein
by reference in its entirety.
[0002] Embodiments of the present invention relate generally to
image segmentation and, more particularly, to computer systems and
methods for user guided image segmentation of radiographic
images.
[0003] Image segmentation, the process of separating objects of
interest from the background (or from other objects) in an image,
is typically a difficult task for a computer to perform. If an
image scene is simple and the contrast between objects in the scene
and the background is high, then the task may be somewhat easier.
However, if an image scene is cluttered and the contrast between
objects in the scene and the background (or other objects) is low,
image segmentation can be a particularly difficult problem. For
example, in a radiographic image of a three-dimensional object such
as a cargo container there can be numerous layers of objects and
contrast may be low between the objects and the background. In
addition to the difficulties often associated with low contrast and
cluttered scenes, radiographic images of objects having layers may
also present a need to segment the image in two ways: in the x-y
plane (i.e., the plane the image was produced on) and by layer of
depth in order to correct for layer effects such as
overlapping.
[0004] User guided object segmentation recognition can be
considered a type of semi-automatic image segmentation. Due to the
present difficulty of image segmentation discussed above and the
potentially safety critical nature of image processing, such as for
security screening, it may be desirable to use a human operator to
aid a computer in segmentation of readiographic images. In user
guided segmentation, an image is presented to an operator. This
image can be a raw image or a processed image. For example, the
image can be a radiographic image on which an automatic
segmentation process has been performed.
[0005] Embodiments of the present invention can be used in an
imaging system, such as a nuclear material detection system, that
includes a capability of producing images using a plurality of
different energy levels. Each energy level provides a different
imaging characteristic such as energy penetration of the object
being scanned. Different images produced using different energy
levels can be used in conjunction with each other to better
identify layers within the object being scanned.
[0006] In an exemplary embodiment, an operator (or user) of a
system can view an image and outline or select a region of interest
(ROI) using an input device such as a mouse or graphics tablet. For
example, an operator can outline a region of interest using mouse
clicks to define a path around the ROI. Alternatively, the operator
can select a region of the image using a click and drag approach
common in many Windows applications. Once the region of interest
has been selected by the operator, then an automatic recognition
(e.g., an Z effective estimation processes) process can be
performed on the region. By narrowing the processing region, the
automatic image segmentation algorithms may be able to achieve an
improved Z effective result, or a different result. It is also
possible that even with user guidance the automatic segmentation
routines or modules produce the same result as was initially
determined. In any case, the result of the automatic segmentation
on the user selected region can be displayed for the operator to
view.
[0007] One exemplary embodiment can include a system for user
guided segmentation of radiographic images of a cargo container.
The system can include an object segmentation recognition module
and an operator terminal including a display device and an input
device, the operator terminal coupled to the object segmentation
module. The object segmentation recognition module can have
instructions stored in a memory that when executed cause the object
segmentation recognition module to perform a series of functions.
The functions can include segmenting a plurality of radiographic
images of a cargo conveyance and outputting region of interest
coordinates and a corrected atomic number image as output. The
functions can also include providing a first image to the operator
terminal for display, the first image being based on the corrected
atomic number image. The functions can include receiving input from
the operator terminal, the input including an indication of a
selected region of the first image; and performing an additional
segmentation process on only the selected region of the first
image. The functions can include providing a second image to the
operator terminal for display, the second image based on the
selected region of the first image and results of the additional
segmentation process.
[0008] Another embodiment includes a method for user guided
segmentation of radiographic images. The method includes segmenting
a plurality of radiographic images in order to determine regions of
interest and corrected atomic number values, and correcting an
atomic number image to generate a corrected atomic number image.
The method can also include providing a first image to an operator
terminal for display, the first image being based on the corrected
atomic number image, and receiving input from the operator
terminal, the input including an indication of a selected region of
the first image. The method can also include performing an
additional segmentation process on the selected region of the first
image and providing a second image to the operator terminal for
display, the second image based on the selected region of the first
image and results of the additional segmentation process. A goal of
user guided segmentation can be to get a better characterization of
the objects which can result in better layer extraction and a
better Z estimation for the material being studied. Providing a
human user or operator in the loop can allow interaction with areas
of an image that might be potential threats and that may not get
resolved by the automatic ROI processing. The method can include
providing updated regions of interest and corrected atomic number
values based on the additional segmentation process as output.
[0009] Another embodiment includes a radiographic image
segmentation apparatus. The apparatus can include means for
segmenting a radiographic image in order to determine regions of
interest and corrected atomic number values, and means for
providing a first image to an operator terminal for display. The
apparatus can also include means for receiving input from the
operator terminal, the input including an indication of a selected
region of the first image and means for performing an additional
segmentation process on the selected region of the first image. The
apparatus can also include means for providing a second image to
the operator terminal for display, the second image being based on
the selected region of the first image and results of the
additional segmentation process; and means for providing updated
regions of interest and corrected atomic number values based on the
additional segmentation process as output.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is a block diagram of an exemplary object
segmentation recognition processor showing inputs and outputs;
[0011] FIG. 2 is a block diagram of an exemplary object
segmentation recognition processor showing an exemplary OSR
processor in detail;
[0012] FIG. 3 is a flowchart showing an exemplary method for image
segmentation;
[0013] FIG. 4 is a block diagram of an exemplary object
segmentation recognition apparatus showing data flow and processing
modules;
[0014] FIG. 5 is a block diagram of an exemplary system for user
guided segmentation;
[0015] FIG. 6 is a flowchart showing an exemplary method for user
guided image segmentation recognition;
[0016] FIG. 7 is a diagram of an exemplary radiographic image
showing an initial automatic segmentation result;
[0017] FIG. 8 is a diagram of the exemplary radiographic image of
FIG. 7 with a user guided input shown; and
[0018] FIG. 9 is a diagram of the exemplary radiographic image of
FIG. 7 that has been subjected to an additional segmentation
process based on user guided input and shows a different result
from that of FIG. 7.
DETAILED DESCRIPTION
[0019] FIG. 1 shows a block diagram of an exemplary object
segmentation recognition processor showing inputs and outputs. In
particular, an object segmentation recognition
[0020] (OSR) processor 102 is shown receiving one or more images
104 as input and providing region of interest (ROI) or object
coordinates 106 as output.
[0021] In operation, the images 104 provided or obtained as input
to the OSR processor 102 can include radiographic images or other
images. For example, the images 104 can include radiographic images
of a cargo conveyance such as a cargo container. The images 104 can
include one or more images, for example four images can be provided
with each image being generated using a different radiographic
energy level. Also, the images 104 can include radiographic images
or other images derived from radiographic images, such as, for
example, an atomic number image representing estimated atomic
numbers associated with radiographic images.
[0022] The OSR processor 102 can obtain, request or receive the
images 104 via a wired or wireless connection, such as a network
(e.g., LAN, WAN, wireless network, Internet or the like) or direct
connection within a system. The OSR processor 102 can also receive
the images 104 via a software connection (e.g., procedure call,
standard object access protocol, remote procedure call, or the
like). In general, any known or later developed wired, wireless or
software connection suitable for transmitting data can be used to
supply the images 104 to the OSR processor 102. The OSR processor
102 can be requested to segment images by another process or
system, or can request images for segmenting from another process
or system. If the images 104 include more than one image, the
images can be registered prior to being sent for segmentation.
[0023] The OSR processor 102 processes the images 104 to segment
the images 104 and identify objects within the images 104. The OSR
processor 102 can also extract or identify layers within the images
in order to help segment the images more accurately. The layer
information can also be used to correct or adjust estimated atomic
numbers in an atomic number image or map. The atomic number image
or map can include a representation of estimated atomic numbers
determined from the images 104.
[0024] Once the images 104 have been segmented and the layer
information has been determined, regions of interest (ROIs) within
the images 104 can be located or determined. The ROIs can be
determined based on an image characteristic such as estimated
atomic number of the ROI (or object), shape of the ROI, position or
location of the ROI, or the like. The OSR processor 102 can provide
ROI/object coordinates 106 as output. The ROI/object coordinates
106 can be associated with the input images 104 or an atomic number
image. The output ROI/object coordinates 106 can be outputted via a
wired or wireless connection, such as a network (e.g., LAN, WAN,
Internet or the like) or direct connection within a system. The
output ROI/object coordinates 106 can be outputted via a software
connection (e.g., response to a procedure call, standard object
access protocol, remote procedure call, or the like).
[0025] FIG. 2 is a block diagram of an exemplary object
segmentation recognition processor showing an exemplary OSR
processor in detail. In addition to the components already
described above, the OSR processor 102 includes a segment
processing section 202 having a connected region analysis module
204, an edge analysis module 206, a ratio layer analysis module 208
and a blind source separation (BSS) layer analysis module 210. The
OSR processor 102 also includes an object ROI section 212 having a
layer analysis and segment association module 214 and an object ROI
determination module 216.
[0026] In operation, the segment processing section receives the
images 104. Once received, the images 104 can be processed using
one or more image segmentation modules (e.g., the connected region
analysis module 204, the edge analysis module 206, or a combination
of the above). It will be appreciated that the segmentation modules
shown are for illustration purposes and that any known or later
developed image segmentation processes can be used. Also, the
selection of the number and type of image segmentation modules
employed in the OSR processor 102 may depend on a contemplated use
of an embodiment and the selection may be guided by a number of
factors including, but not limited to, type of materials being
scanned, configuration of the scanning system and objects being
scanned, desired performance characteristics, time available for
processing, or the like. The images 104 can also be processed by
one or more layer analysis modules (e.g., the ratio layer analysis
module 208, the BSS layer analysis module 210, or a combination of
the above).
[0027] Once the segmentation processing (object segmentation, layer
analysis, or both) has been completed, the resulting image segment
data can be provided to the object ROI section 212. In the object
ROI section 212, the layers and segments of the image segment data
are analyzed and combined or associated to produce segment-layer
data that contains information about objects and layers within the
images 104. The segment-layer data can be in the form of an atomic
number image that represents a composite of the images 104 and has
been adjusted or corrected based on layers and segments to provide
an image suitable for identification of ROIs. The segment-layer
data can also be represented in any form suitable for transmitting
the information that may be needed to analyze the images 104. The
segment-layer data is then provided to the object ROI determination
module 216 for analysis and identification of ROIs.
[0028] The object ROI determination module 216 can use one or more
image characteristics to identify ROIs within the images 104 or the
segment-layer data. Image characteristics can include an estimated
atomic number for a portion of the image (e.g., a pixel, segment,
object, region or the like), a shape of a segment or object within
the image, or a position or location of an object or segment. In
general, any image characteristic that is suitable for identifying
an ROI can be used.
[0029] Once the ROIs have been determined, coordinate data (106)
representing each ROI can be provided as output. The output can be
provided as described above in connection with reference number 106
of FIG. 1. Also, segment-layer data or an adjusted or corrected
atomic number image can be provided in addition to, or as a
substitute for, the ROI coordinates.
[0030] FIG. 3 is a flowchart showing an exemplary computer
implemented method for image segmentation. Processing begins at
step 302 and continues to step 304.
[0031] In step 304, one or more radiographic images are obtained.
These images can be provided by an imaging system (e.g., an x-ray,
magnetic resonance imaging device, computerized tomography device,
or the like). In general, any imaging device suitable for
generating images that may require segmenting can be used.
Processing continues to step 306.
[0032] In step 306, the radiographic images are segmented. The
segmentation can be performed using one or more image segmentation
processes. Examples of segmentation methods include modules or
processes for segmentation based on clustering, histograms, edge
detection, region growing, level set, graph partitioning,
watershed, model based, and multi-scale. Processing continues to
step 308.
[0033] In step 308, any layers present in the images are
determined. The layers can be determined using one or more layer
extraction or identification processes. For example, a ratio layer
analysis process and a BSS layer analysis process can be used
together to identify layers in the images. A goal of layer
identification and extraction is to remove overlapping effects
which may be present. By removing overlapping effects, the true
gray level of a material can be determined. Using the true gray
level, a material's effective atomic number (and, optionally,
material density) can be determined. Using the effective atomic
number the composition of the material can be determined and
illicit materials, such as special nuclear materials can be
detected automatically.
[0034] The ratio method of layer identification and overlap effect
removal is known in the art as applied to dual energy and is
described in "The Utility of X-ray Dual-energy Transmission and
Scatter Technologies for Illicit Material Detection," a published
Ph.D. Dissertation by Lu Qiang, Virginia Polytechnic Institute and
State University, Blacksburg, Va., 1999, which is incorporated
herein by reference in its entirety. Generally, the ratio method
provides a process whereby a computer can solve for one image layer
and remove any overlapping effects of another layer. Thus, regions
that overlap can be separated into their constituent layers and a
true gray level can be determined for each layer.
[0035] Blind source separation (or blind signal separation) is a
technique known in the art, and refers generally to the separation
of a set of signals from a set of mixed signals, without the aid of
information (or with very little information) about the source
signals or the mixing process. However, if information about how
the signals were mixed (e.g., a mixing matrix) can be estimated, it
can then be used to determine an un-mixing matrix which can be
applied to separate the components within a mixed signal.
[0036] The BSS method may be limited by the amount of independence
between materials within the mixture. Several techniques exist for
estimating the mixing matrix; some include using an unsupervised
learning process. The process can include incrementally changing
and weighting coefficients of the mixing matrix and evaluating the
mixing matrix until optimal conditions are met. Once the mixing
matrix is estimated, un-mixing coefficients can be computed.
Examples of some BSS techniques include projection pursuit gradient
ascent, projection pursuit stepwise separation, ICA gradient
ascent, and complexity pursuit gradient ascent. In general, an
iterative hill climbing or other type of optimization process can
be used to estimate the mixing matrix and determine an optimal
matrix. Also, contemplated or desired performance levels may
require development of custom algorithms that can be tuned to a
specific empirical terrain provided by the mixing and un-mixing
matrices. Once the layers are identified and overlapping effects
are removed, processing continues to step 310.
[0037] In step 310, segments that have been identified are
associated with any layers that have been determined or identified
in step 308. Associating segments with layers can help to remove
any overlapping effects and also can improve the ability to
determine a true gray value for a segment. Processing continues to
step 312.
[0038] In step 312, ROIs are determined. The ROIs can be determined
based on an image characteristic as described above. Processing
continues to step 314.
[0039] In step 314, a gray level atomic number image is optionally
adjusted to reflect the corrections or adjustments provided by the
layer determination. The adjustments or corrections can include
changes related to removal of overlap effects or other changes.
Processing continues to step 316.
[0040] In step 316, the ROI coordinates and, optionally, the
adjusted or corrected gray level image are provided as output to an
operator or another system. The output can be in a standard format
or in a proprietary format. Processing continues to step 318 where
the method ends. It will be appreciated that steps 304-316 can be
repeated in whole or in part to perform a contemplated image
segmentation process.
[0041] FIG. 4 is a block diagram of an exemplary object
segmentation recognition apparatus showing data flow and processing
modules. In particular, four gray scale radiographic images
(402-408), each generated using a different energy level, are
provided to an effective Z-value determination module 410. The
effective Z-value determination module determines a pixel-level
Z-value gray scale image 412.
[0042] The pixel-level Z-value gray scale image 412 can be provided
to an image segmentation and layer analysis module 414. The
segmentation and layer analysis module 414 segments the image and
analyzes layers, as described above, to generate a layer corrected
image representing true gray values, ROI coordinates, or both.
[0043] As mentioned above, it may be desirable to use a human
operator to aid in segmentation. FIG. 5 shows an exemplary nuclear
detection system that provides for user guided input for object
segmentation recognition. In particular, an object screening system
500 can be used to screen an object to be scanned 502 in order to
detect contraband such as nuclear material. The object 502 is
subjected to one or more electromagnetic energies (with two, 504a
and 504b, being shown for illustration purposes) produced by the
scanner 506. The scanner 506 receives returned or radiated energy
and produces scanned images 508 that are sent to a threat detection
system 510.
[0044] The threat detection system 510 processes the scanned images
508 to detect (either automatically or with some manual input)
nuclear material or other possible threats revealed by the
radiographic imaging. Part of the detection process includes
segmenting one or more of the images using OSR module 511. The
results of the segmentation can be provided to an operator station
514 via link 512. The results can be in the form of a graphical
image suitable for display to an operator or user of the object
screening system 500.
[0045] Once the operator has viewed the image displayed on the
operator station 514, the operator can provided input to the
operator station 514. The input (or an encoded form thereof) can be
transmitted via link 512 to the threat detection system 510 and/or
the OSR module 511. The OSR module 511 can then perform an
additional segmentation process on a selected portion of the image
to produce another segmentation result. A second image, based on
the result of the additional segmentation process can be provided
to the operator station 514 for viewing by the operator. The
segmentation results can be released by the operator for additional
processing by the threat detection system 510 or additional
segmentation process can be requested. Also, the operator can
manually indicated object segmentations using an input device
associated with the operator station 514. The manually generated
segmentations, the automatic segmentation results, or both, can be
sent to the threat detection system 510.
[0046] The threat detection system can be a stand alone system or
form part of a larger security system. Link 512 can be a wired or
wireless link such as a LAN, WAN, wireless network connection,
radio link, optical link, or the like.
[0047] The energies 504a and 504b can include, for example, two or
more different energy levels of x-ray energy. It will be
appreciated that other types of electromagnetic energy can be used
to scan the object 502. It will also be appreciated that although
two energies (504a and 504b) are shown, more or less energies can
be used with an embodiment. Any type of scanner suitable for
detecting contraband such as nuclear material and capable or
producing an image (or array of values) may be used. The object
being screened (or scanned) can include a cargo container, a truck,
a tractor trailer, baggage, cargo, luggage, a vehicle, an air cargo
container, and/or any object being transported that could
potentially contain nuclear material or a portion of a threat or
weapon system, or any object for which threat or contraband
screening is contemplated or desired.
[0048] FIG. 6 is a flowchart showing an exemplary method for user
guided image segmentation recognition. Processing for the method
begins at step 602 and continues to step 604.
[0049] In step 604, radiographic images are obtained. The images
can be in the form of raw radiographic image data, gray level data
representing effective atomic number, or a hybrid image containing
both. Processing continues to step 606.
[0050] In step 606, the radiographic images are automatically
segmented by computer using one or more segmentation and/or layer
analysis algorithms as discussed above. Processing continues to
step 608.
[0051] In step 608, ROI coordinates and, optionally, adjusted gray
level image(s) are output as a first image to an operator station.
At the operator station, the output can be displayed in any
suitable means such as on a display screen or printed. Processing
continues to step 610.
[0052] In step 610, input is received from an operator using the
operator station indicating a selected region of the first image.
Processing continues to step 612.
[0053] In step 612, an additional segmentation process is performed
on the selected region. The additional segmentation process can be
performed using the same algorithms as used for the initial
automatic segmentation or may be performed using different
algorithms. Also, the same algorithms may be used with different
parameters for the segmentation of the selected region. Processing
continues to step 614.
[0054] In step 614, a second image is provided to the operator
station. The second image is based on results of the additional
segmentation process of step 612. Processing continues to step 616.
The second image could be a smaller region of the original image or
sub-image.
[0055] In step 616, ROI coordinates and, optionally, adjusted gray
level images based on the additional segmentation processing are
output. The ROI coordinates and adjusted gray level image can be
provided to another module in a threat detection system, such as
material context analysis processor. The output can be provided to
another system or another operator. The decision to release the
results of the additional segmentation processing can be made by
the operator or can be automatic. The final output can be a
combination of the segmentation results of the initial processing
and the segmentation results of the additional processing.
Processing continues to step 618 where processing ends. It will be
appreciated that steps 604-616 can be repeated in whole or in part
to perform a contemplated user guided image segmentation
process.
[0056] FIGS. 7-9 show a diagrammatic representation of a sequence
of radiographic images on an operator display screen. FIG. 7 shows
an initial automatic segmentation result. In particular, a first
object 702 has been recognized by the automatic routines of the OSR
processor. Also, a second object 704 has been recognized. However,
a portion 706 (shown by dashed line) of object 704 is obscured
behind object 702 and was not recognized as being associated with
object 704, rather was include in object 702. An operator viewing
the image of FIG. 7 may recognize that a portion of object 704
appears to be obscured and was not correctly identified using
automatic techniques. The operator can then elect to have
additional user guided segmentation performed.
[0057] FIG. 8 shows the image of FIG. 7 with a graphical user
interface (GUI) element (802) drawn around a selected portion of
the image. The GUI element can be indicated using any suitable
input means such as mouse, keyboard, stylus and tablet, touch
screen, or the like. In general, any input means suitable for
indicating a selected region of the image can be used. The GUI
element can be drawn using a "click and drag" technique that is
commonly used in many personal computer software applications.
Also, other techniques can be used such as Livewire (based on the
lowest cost path algorithm by Dijkstra) or Intelligent
Scissors.
[0058] Once the selected region or portion of the image has been
indicated by the operator, the region can be sent to the
segmentation module for an additional segmentation processing. The
additional segmentation processing can include the same or
different algorithms or parameters as used for the initial
automatic segmentation. Also, if multiple images are available, a
user guided layer extraction can be performed in which the operator
can view each image individually or view the group as a composite
and can select regions of interest from each of the images for
additional layer extraction or analysis processing. Once the
additional segmentation and/or layer correction has been performed,
a second image can be provided to the operator station. This image
can be the same or different from the initial segmentation
image.
[0059] For example, FIG. 9 shows a second image that is different
from the first image shown in FIG. 7. In the image in FIG. 9 there
are two objects (now labeled 902 and 904), however, the object 904
now includes the portion (706) that was previously obscured and
included in object 702 of FIG. 7. By performing user guided
segmentation, an operator can better analyze objects of interest
and possibly determine a more accurate gray value and shape or size
of the object.
[0060] It will be appreciated that the modules, processes, systems,
and sections described above can be implemented in hardware,
software, or both. Also, the modules, processes systems, and
sections can be implemented as a single processor or as a
distributed processor. Further, it should be appreciated that the
steps mentioned above may be performed on a single or distributed
processor. Also, the processes, modules, and sub-modules described
in the various figures of the embodiments above may be distributed
across multiple computers or systems or may be co-located in a
single processor or system. Exemplary structural embodiment
alternatives suitable for implementing the modules, sections,
systems, means, or processes described herein are provided
below.
[0061] The modules, processors or systems described above can be
implemented as a programmed general purpose computer, an electronic
device programmed with microcode, a hard-wired analog logic
circuit, software stored on a computer-readable medium or signal, a
programmed kiosk, an optical computing device, a GUI on a display,
a networked system of electronic and/or optical devices, a special
purpose computing device, an integrated circuit device, a
semiconductor chip, and a software module or object stored on a
computer-readable medium or signal, for example.
[0062] Embodiments of the method and system (or their
sub-components or modules), may be implemented on a general-purpose
computer, a special-purpose computer, a programmed microprocessor
or microcontroller and peripheral integrated circuit element, an
ASIC or other integrated circuit, a digital signal processor, a
hardwired electronic or logic circuit such as a discrete element
circuit, a programmed logic circuit such as a PLD, PLA, FPGA, PAL,
or the like. In general, any process capable of implementing the
functions or steps described herein can be used to implement
embodiments of the method, system, or a computer program product
(software program).
[0063] Furthermore, embodiments of the disclosed method, system,
and computer program product may be readily implemented, fully or
partially, in software using, for example, object or
object-oriented software development environments that provide
portable source code that can be used on a variety of computer
platforms. Alternatively, embodiments of the disclosed method,
system, and computer program product can be implemented partially
or fully in hardware using, for example, standard logic circuits or
a VLSI design. Other hardware or software can be used to implement
embodiments depending on the speed and/or efficiency requirements
of the systems, the particular function, and/or particular software
or hardware system, microprocessor, or microcomputer being
utilized. Embodiments of the method, system, and computer program
product can be implemented in hardware and/or software using any
known or later developed systems or structures, devices and/or
software by those of ordinary skill in the applicable art from the
function description provided herein and with a general basic
knowledge of the computer, image processing, radiographic, and/or
threat detection arts.
[0064] Moreover, embodiments of the disclosed method, system, and
computer program product can be implemented in software executed on
a programmed general purpose computer, a special purpose computer,
a microprocessor, or the like. Also, the method of this invention
can be implemented as a program embedded on a personal computer
such as a JAVA.RTM. or CGI script, as a resource residing on a
server or image processing workstation, as a routine embedded in a
dedicated processing system, or the like. The method and system can
also be implemented by physically incorporating the method into a
software and/or hardware system, such as the hardware and software
systems of multi-energy radiographic cargo inspection systems.
[0065] It is, therefore, apparent that there is provided, in
accordance with the present invention, a method, computer system,
and computer software program for user guided image segmentation.
While this invention has been described in conjunction with a
number of embodiments, it is evident that many alternatives,
modifications and variations would be or are apparent to those of
ordinary skill in the applicable arts. Accordingly, Applicant
intends to embrace all such alternatives, modifications,
equivalents and variations that are within the spirit and scope of
this invention.
* * * * *