U.S. patent application number 09/758889 was filed with the patent office on 2001-07-19 for joint optimization of parameters for the detection of clustered microcalcifications in digital mammograms.
This patent application is currently assigned to Qualia Computing, Inc. Invention is credited to Broussard, Randy P., Ochoa, Edward M., Rathbun, Thomas F., Resenstengel, John E., Rogers, Steven K..
Application Number | 20010008562 09/758889 |
Document ID | / |
Family ID | 27371083 |
Filed Date | 2001-07-19 |
United States Patent
Application |
20010008562 |
Kind Code |
A1 |
Rogers, Steven K. ; et
al. |
July 19, 2001 |
Joint optimization of parameters for the detection of clustered
microcalcifications in digital mammograms
Abstract
A method and system for detecting and displaying clustered
microcalcifications in a digital mammogram, wherein a single
digital mammogram is first automatically cropped to a breast area
sub-image which is then processed by means of an optimized
Difference of Gaussians filter to enhance the appearance of
potential microcalcifications in the sub-image. The potential
microcalcifications are thresholded, clusters are detected,
features are computed for the detected clusters, and the clusters
are classified as either suspicious or not suspicious by means of a
neural network. Thresholding is preferably by sloping local
thresholding, but may also be performed by global and dual-local
thresholding. The locations in the original digital mammogram of
the suspicious detected clustered microcalcifications are
indicated. Parameters for use in the detection and thresholding
portions of the system are computer-optimized by means of a genetic
algorithm. The results of the system are optimally combined with a
radiologist's observation of the original mammogram by combining
the observations with the results, after the radiologist has first
accepted or rejected individual detections reported by the
system.
Inventors: |
Rogers, Steven K.;
(Beavercreek, OH) ; Broussard, Randy P.; (Huber
Heights, OH) ; Ochoa, Edward M.; (San Antonio,
TX) ; Rathbun, Thomas F.; (Beavercreek, OH) ;
Resenstengel, John E.; (Huber Heights, OH) |
Correspondence
Address: |
BIEBEL & FRENCH
35 EAST FIRST STREET
DAYTON
OH
45402
|
Assignee: |
Qualia Computing, Inc
|
Family ID: |
27371083 |
Appl. No.: |
09/758889 |
Filed: |
January 11, 2001 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09758889 |
Jan 11, 2001 |
|
|
|
09416437 |
Oct 12, 1999 |
|
|
|
6205236 |
|
|
|
|
60057801 |
Aug 28, 1997 |
|
|
|
60066996 |
Nov 28, 1997 |
|
|
|
60076760 |
Mar 3, 1998 |
|
|
|
Current U.S.
Class: |
382/132 ;
382/168; 382/225 |
Current CPC
Class: |
B29L 2023/18 20130101;
G06V 10/443 20220101; B25J 15/04 20130101; B29C 49/0021 20130101;
G06K 2209/05 20130101; G06K 9/6254 20130101; A61B 6/502 20130101;
G06K 9/38 20130101; Y10S 128/922 20130101; B29C 2791/007 20130101;
G06T 5/20 20130101; G06V 2201/03 20220101; G06T 7/0012 20130101;
G06K 9/4609 20130101; G06V 10/28 20220101; B29C 2791/006
20130101 |
Class at
Publication: |
382/132 ;
382/225; 382/168 |
International
Class: |
G06K 009/00; G06K
009/62 |
Claims
What is claimed is:
1. A method for detecting clustered microcalcifications in a
digital mammogram image by digital processing comprising, in
sequence, the steps of: a) optimizing a value for a predetermined
filtering algorithm parameter; b) filtering a digital mammogram
image to produce a filtered image comprising essentially
microcalcifications; c) thresholding the filtered image from step
b) to produce a thresholded image; d) forming clusters of
microcalcifications in the thresholded image from step c); and e)
inputting the identified clusters of microcalcifications to a
classifier to distinguish suspicious clusters of
microcalcifications from nonsuspicious clusters of
microcalcifications; wherein said step a) comprises: i) providing
an optimizing algorithm; ii) providing a database of training
images; and iii) inputting the database of training images into
said optimizing algorithm to determine an optimum value for said
predetermined filtering algorithm parameter; and wherein said step
b) comprises: i) providing a filtering algorithm incorporating the
optimized predetermined filtering algorithm parameter; and ii)
applying the filtering algorithm to the digital mammogram image to
produce a filtered image comprising essentially
microcalcifications.
2. The method of claim 1 wherein said step a) comprises optimizing
a parameter corresponding to the size of microcalcifications in the
digital mammogram image.
3. A method for detecting clustered microcalcifications in a
digital mammogram image by digital processing comprising, in
sequence, the steps of: a) optimizing at least one value for a
predetermined threshold parameter; b) filtering a digital mammogram
image to produce a filtered image comprising essentially
microcalcifications; c) thresholding the filtered image from step
b) to produce a threshold image; d) forming clusters of
microcalcifications in the filtered image from step c); and e)
inputting the identified clusters of microcalcifications to a
classifier to distinguish suspicious clusters of
microcalcifications from nonsuspicious clusters of
microcalcifications; wherein said step a) comprises: i) providing
an optimizing algorithm; ii) providing a database of training
images; and iii) inputting the database of training images into
said optimizing algorithm to determine optimum values for said
predetermined threshold parameters; and wherein said step c)
comprises: i) providing a thresholding algorithm incorporating the
optimized predetermined threshold parameter; and ii) applying the
thresholding algorithm to the filtered image to identify potential
microcalcifications.
4. The method of claim 3 wherein said step a) comprises computing
the histogram of the filtered image and optimizing a parameter
corresponding to an upper fraction of the histogram.
5. The method of claim 4 wherein said parameter corresponding to an
upper fraction of the histogram comprises a parameter related to a
maximum gray level in the filtered image.
6. The method of claim 3 wherein said step a) comprises optimizing
a local threshold parameter set for determining a local threshold
and optimizing a local neighborhood parameter for determining a
local neighborhood size within which the local threshold is
applied.
7. The method of claim 3 wherein: said step a) further comprises;
A) computing the histogram of the filtered image and optimizing a
parameter, f, corresponding to an upper fraction of the histogram;
B) optimizing a local threshold parameter set for determining a
local threshold; and C) optimizing a local neighborhood parameter
for determining a local neighborhood size within which the local
threshold is applied; and said step c) comprises: A) producing a
globally thresholded image by retaining an upper fraction of the
pixels in the filtered image; and B) producing a locally
thresholded image by locally thresholding pixels in a window of the
filtered image, determined by the local neighborhood parameter, and
retaining the pixels in the window whose gray levels satisfy a
relation determined by the local threshold parameter set.
8. The method of claim 7 including the step of logically ANDing the
globally thresholded image and the locally thresholded image.
9. A method for detecting clustered microcalcifications in a
digital mammogram image by digital processing comprising, in
sequence, the steps of: a) optimizing a value for at least one
predetermined clustering algorithm parameter; b) filtering a
digital mammogram image to produce a filtered image comprising
essentially microcalcifications; c) thresholding the filtered image
from step b) to produce a thresholded image; d) forming clusters of
microcalcifications in the thresholded image from step c); and e)
inputting the identified clusters of microcalcifications to a
classifier to distinguish suspicious clusters of
microcalcifications from nonsuspicious clusters of
microcalcifications; wherein said step a) comprises: i) providing
an optimizing algorithm; ii) providing a database of training
images; and iii) inputting the database of training images into
said optimizing algorithm to determine optimum values for said
predetermined clustering algorithm parameters; and wherein said
step d) comprises: i) providing a clustering algorithm
incorporating the optimized predetermined clustering algorithm
parameter; and ii) applying the clustering algorithm to the
thresholded image to identify potential clusters of
microcalcifications.
10. The method of claim 9 wherein said step a) comprises optimizing
a parameter, d.sub.nn, corresponding to a maximum distance between
microcalcifications.
11. The method of claim 9 wherein said step a) comprises optimizing
a parameter, .mu.Cs.sub.min, corresponding to a minimum number of
microcalcifications for forming a cluster.
12. The method of claim 9 wherein said step a) comprises: A)
optimizing a parameter, d.sub.nn, corresponding to a maximum
distance between microcalcifications in a cluster; and B)
optimizing a parameter, .mu.Cs.sub.min, corresponding to a minimum
number of microcalcifications for forming a cluster.
13. The method of claim 12 wherein said step d) comprises
identifying groups of microcalcifications in which at least
.mu.Cs.sub.min microcalcifications are within distance d.sub.nn of
each other to define the clusters of microcalcifications.
14. A method for detecting clustered microcalcifications in a
digital mammogram image by digital processing comprising, in
sequence, the steps of: a) jointly optimizing a filtering algorithm
parameter, a thresholding algorithm parameter and a clustering
algorithm parameter; b) providing a filtering algorithm
incorporating the optimized filtering algorithm parameter and
applying the filtering algorithm to the digital mammogram image to
produce a filtered image comprising essentially
microcalcifications; c) providing a thresholding algorithm
incorporating the optimized thresholding algorithm parameter and
applying the thresholding algorithm to the filtered image to
produce a thresholded image; d) providing a clustering algorithm
incorporating the optimized clustering algorithm parameter and
applying the clustering algorithm to the thresholded image to
identify potential clusters of microcalcifications; and e)
inputting the identified clusters of microcalcifications to a
classifier to distinguish suspicious clusters of
microcalcifications from non-suspicious clusters of
microcalcifications.
15. The method of claim 14 wherein said step a) comprises: i)
providing an optimizing algorithm; ii) providing a database of
training images; and iii) inputting the database of training images
into said optimizing algorithm to determine an optimum value for
each of said algorithm parameters.
16. The method of claim 14 wherein said step of optimizing the
filtering algorithm parameter comprises optimizing a parameter
corresponding to the size of microcalcifications in the digital
mammogram image.
17. The method of claim 14 wherein said step of optimizing the
thresholding algorithm parameter comprises computing the histogram
of the filtered image and optimizing a parameter corresponding to
an upper fraction of the histogram.
18. The method of claim 17 wherein said parameter corresponding to
an upper fraction of the histogram comprises a parameter related to
a maximum gray level in the filtered image.
19. The method of claim 14 wherein said step of optimizing a
thresholding algorithm parameter comprises optimizing a local
threshold parameter set for determining a local threshold and
optimizing a local neighborhood parameter for determining a local
neighborhood size within which the local threshold is applied.
20. The method of claim 14 wherein: said step of optimizing a
thresholding algorithm parameter further comprises; i) computing
the histogram of the filtered image and optimizing a parameter, f,
corresponding to an upper fraction of the histogram; ii) optimizing
a local threshold parameter set for determining a local threshold;
and iii) optimizing a local neighborhood parameter for determining
a local neighborhood size within which the local threshold is
applied; and said step c) comprises: (i) producing a globally
thresholded image by retaining an upper fraction of the pixels in
the filtered image; and (ii) producing a locally thresholded image
by locally thresholding pixels in a window of the filtered image,
determined by the local neighborhood parameter, and retaining the
pixels in the window whose gray levels satisfy a relation
determined by the local threshold parameter set.
21. The method of claim 20 including the step of logically ANDing
the globally thresholded image and the locally thresholded
image.
22. The method of claim 14 wherein said step of optimizing the
clustering algorithm parameter comprises optimizing a parameter
d.sub.nn, corresponding to a maximum distance between
microcalcifications.
23. The method of claim 14 wherein said step of optimizing the
clustering algorithm parameter comprises optimizing a parameter,
.mu.Cs.sub.min, corresponding to a minimum number of
microcalcifications for forming a cluster.
24. The method of claim 14 wherein said step of optimizing the
clustering algorithm parameter comprises: i) optimizing a
parameter, d.sub.nn, corresponding to a maximum distance between
microcalcifications in a cluster; and ii) optimizing a parameter,
.mu.Cs.sub.min, corresponding to a minimum number of
microcalcifications for forming a cluster.
25. The method of claim 24 wherein said step d) comprises
identifying groups of microcalcifications in which at least
.mu.Cs.sub.min, microcalcifications are within distance d.sub.nn,
of each other to define the clusters of microcalcifications.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. application Ser.
No. 09/416,437 filed Oct. 12, 1999, which claims the benefit of
U.S. application Ser. No. 09/141,802, filed Aug. 28, 1998, which
claims the benefit of U.S. Provisional Application No. 60/057,801,
filed Aug. 28, 1997, U.S. Provisional Application No. 60/066,996,
filed Nov. 28, 1997, and U.S. Provisional Application No.
60/076,760, filed Mar. 3, 1998, the entire disclosures of all of
which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] This invention relates to a method and system for automated
detection of clustered microcalcifications from digital images
without reduction of radiologist sensitivity.
[0004] 2. Discussion of Background
[0005] Mammography, along with physical examination, is the current
procedure of choice for breast cancer screening. Screening
mammography has been responsible for an estimated 30 to 35 percent
reduction in breast cancer mortality rates. However, in 1996
approximately 185,700 new breast cancer cases were diagnosed and
44,300 women died from this disease. Women have about a 1 in 8
chance of being diagnosed with breast cancer, and 1 in 30 will die
of this disease in her lifetime.
[0006] Although mammography is a well-studied and standardized
methodology, for 10 to 30 percent of women diagnosed with breast
cancer, their mammograms were interpreted as negative.
Additionally, only 10 to 20 percent of patients referred for biopsy
based on mammographic findings prove to have cancer. Further,
estimates indicate the malignancies missed by radiologists are
evident in two-thirds of the mammograms retrospectively. Missed
detections may be attributed to several factors including: poor
image quality, improper patient positioning, inaccurate
interpretation, fibroglandular tissue obscuration, subtle nature of
radiographic findings, eye fatigue, or oversight.
[0007] To increase sensitivity, a double reading has been
suggested. However, the growing increase in the number of screening
mammograms makes this option unlikely. Alternatively, a
computer-aided diagnosis (CAD or CADx) system may act as a "second
reader" to assist the radiologist in detecting and diagnosing
lesions. Several investigators have attempted to analyze
mammographic abnormalities with digital computers. However, the
known studies are believed to have achieved rates of true-positive
detections versus false-positive detections that are undesirably
low.
[0008] Microcalcifications represent an ideal target for automated
detection because subtle microcalcifications are often the first
and sometimes the only radiographic findings in early, curable
breast cancers, yet individual microcalcifications in a suspicious
cluster have a fairly limited range of radiographic appearances.
Between 30 and 50 percent of breast carcinomas detected
radiographically demonstrate microcalcifications on mammograms, and
between 60 and 80 percent of breast carcinomas reveal
microcalcifications upon microscopic examination. Any increase in
the detection rate of microcalcifications by mammography will lead
to further improvements in its efficacy in the detection of early
breast cancer.
[0009] Although the promise of CAD systems is to increase the
ability of physicians to diagnose cancer, the problem is that all
CAD systems fail to detect some regions of interest that could be
found by a human interpreter. However, human interpreters also miss
regions of interest that are subsequently shown to be indicators of
cancers. Missing a region that is associated with a cancer is
termed a false negative error while associating a normal region
with a cancer is termed a false positive error.
[0010] It is not yet clear how CAD system outputs are to be
incorporated by practicing radiologists into their mammographic
analyses. No existing CAD system can claim to find all of the
suspicious regions detected by an average radiologist, and they
tend to have unacceptably high false positive error rates. However,
CAD systems are capable of finding some suspicious regions that may
be missed by radiologists.
SUMMARY OF THE INVENTION
[0011] Accordingly, an object of this invention is to provide a
method and system for automated detection of clustered
microcalcifications from digital mammograms.
[0012] These and other objects are achieved according to the
invention by providing a novel method and system for automated
detection of clustered microcalcifications from digital mammograms
in which a digital mammogram is obtained, parameters necessary for
cropping the digital mammogram image are optimized, the digital
mammogram is cropped based on the optimized cropping parameters to
select breast tissue for further analysis, parameters necessary for
detecting clustered microcalcifications are optimized, and
clustered microcalcifications in the cropped digital mammogram are
detected based on the optimized clustered microcalcification
detection parameters.
[0013] The detected clustered microcalcifications are then stored
as a detections image, the detections image is processed for
display, and a computer-aided detection image is produced for
review by a radiologist.
[0014] The radiologist first reviews the original mammograms and
reports a set of suspicious regions of interest, S1. A CAD system,
or more particularly, the CAD system of the invention, operates on
the original mammogram and reports a second set of suspicious
detections or regions of interest, S2. The radiologist then
examines the set S2, accepts or rejects members of S2 as
suspicious, thus forming a third set of suspicious detections, S3,
that is a subset of set S2. The radiologist then creates a fourth
set of suspicious detections, S4, that is the union of sets S1 and
S2, for subsequent diagnostic workups. CAD system outputs are
thereby incorporated with the radiologist's mammographic analysis
in a way that optimizes the overall sensitivity of detecting true
positive regions of interest.
[0015] Other objects and advantages of the invention will be
apparent from the following description, the accompanying drawings
and the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] FIG. 1 is a flow diagram illustrating the automated system
for the detection of clustered microcalcifications in a digital
mammogram;
[0017] FIGS. 2 and 3 are flow diagrams illustrating the
autocropping method and system of the invention;
[0018] FIGS. 4-10 are flow diagrams illustrating in more detail the
autocropping method and system of the invention;
[0019] FIG. 11 is a flow diagram illustrating in greater detail the
clustered microcalcification detector of the invention;
[0020] FIG. 12 is a schematic diagram illustrating a 3.times.3
cross-shaped median filter of the invention;
[0021] FIG. 13 is a three-dimensional plot of a Difference of
Gaussians (DoG) filter kernel;
[0022] FIG. 14 is a cross-sectional view through the center of the
DoG filter kernel of FIG. 13;
[0023] FIG. 15 is a flow diagram illustrating the global
thresholding portion of the microcalcification detection
system;
[0024] FIG. 16 is a flow diagram illustrating the dual local
thresholding of the invention;
[0025] FIG. 17 is a flow diagram illustrating combining the results
of global and dual-local thresholding;
[0026] FIG. 18 is a flow diagram illustrating the sloping local
thresholding of the invention;
[0027] FIG. 19 is a flow diagram illustrating the clustering method
of the invention;
[0028] FIG. 20 is a schematic diagram illustrating the clustering
method of the invention;
[0029] FIG. 21 is a flow diagram illustrating the feature
computation process of the invention;
[0030] FIG. 22 is a flow diagram illustrating a classifier having
one discriminant function per class;
[0031] FIG. 23 is a schematic diagram illustrating a multi-layer
perceptron neural network for a two-class classifier;
[0032] FIG. 24 is a histogram of testing results after detection
and classification;
[0033] FIG. 25 is a flow diagram illustrating the parameter
optimization method of the invention;
[0034] FIG. 26 is a plot of a free response receiver operating
characteristic curve of the invention before classifying
detections;
[0035] FIG. 27 is a plot of a free response receiver operating
characteristic curve of the invention after classifying
detections;
[0036] FIG. 28 is a plot of probability density functions showing
the relationship between the probabilities of false negative and
false positive detections;
[0037] FIG. 29 is a plot of probability density functions showing
the relationship between the probabilities of true negative and
true positive detections;
[0038] FIG. 30 is a Venn diagram showing the relationship between
radiologist and CAD system detections; and
[0039] FIG. 31 is a flow diagram illustrating a method for
incorporating computer-aided diagnosis detections with those of a
human interpreter for optimal sensitivity.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0040] Referring now to the drawings, wherein like reference
numerals designate identical or corresponding parts throughout the
several views, and more particularly to FIG. 1 thereof, there is
shown a flow diagram illustrating a sequence of steps performed in
order to detect the locations of clusters of microcalcifications
within a digital mammogram.
[0041] In a first step 100, a digital mammogram is obtained using
hardware such as digital mammography systems, or by digitizing
mammography films using laser or charge-coupled device (CCD)
digitizers. In an optimized cropping step 200, a rectangular
analysis region containing breast tissue is segmented from the
digital mammogram image and a binary mask corresponding to the
breast tissue is created for use in later processing steps to
decrease the time required for processing the mammogram image. The
binary mask is also used to limit detections to areas of the image
containing breast tissue.
[0042] Clustered microcalcifications are detected in a clustered
microcalcification detection step 300. After first filtering the
cropped image with a median filter to reduce noise, the image is
filtered using an optimized difference of Gaussians (DoG) filter to
enhance the microcalcifications. The DoG-filtered image is then
subjected to optimized threshold tests to detect potential
microcalcifications. The detected microcalcifications are shrunk to
single-pixel representations and detections outside of the breast
area are removed. The remaining microcalcifications are grouped
into clusters. Features are then computed for the clusters.
Detected clusters are classified as either suspicious or
non-suspicious in a classification step 400.
[0043] The parameters used by the autocropping, clustered
microcalcification detection, and classification steps 200, 300,
400 are optimized in a parameter-optimizing step 500. The
parameters are optimized by parameter-optimizing means that uses a
genetic algorithm (GA) so as to maximize the true-positive
detection rate while minimizing the false-positive detection rate.
Of course, other optimization schemes may be used as well.
[0044] The detected clustered microcalcifications are stored in a
list of image coordinates. The detection results are processed in a
processing step 600 by simply adding an offset to each of the
microcalcification coordinates to account for translation of the
coordinates incurred as a result of the cropping procedure.
Detected clustered microcalcifications are indicated on the digital
mammogram by means of rectangles drawn around the clustered
microcalcifications in a display step 700. Other indicators may be
used such as, for example, arrows pointing to suspected
microcalcifications, or ellipses around suspected
microcalcifications.
ACQUIRING A DIGITAL REPRESENTATION OF A MAMMOGRAM
[0045] One method of obtaining digital mammograms comprises
digitizing radiologic films by means of a laser or charge-coupled
device (CCD) scanner. Digital images obtained in this manner
typically have a sample spacing of about 100 .mu.m per pixel, with
a gray-level resolution of 10 to 12 bits per pixel. In one
embodiment of the present invention, radiologic films are scanned
using a Model CX812T digitizer manufactured by Radiographic Digital
Imaging of Compton, California, to produce digital images having 50
.mu.m spacing per pixel and 12 bits of gray-level resolution per
pixel. Another possible input source for digital images is a
digital mammography unit from Trex Medical Corporation of Danbury,
Conn., which has a spatial resolution of about 45 .mu.m per pixel
and a gray-level resolution of 14 bits per pixel.
[0046] The digital images are stored as digital representations of
the original mammogram images on computer-readable storage media.
In a preferred embodiment, the digital representations or images
are stored on a 2 GB hard drive of a general-purpose computer such
as a PC having dual Pentium II.RTM. microprocessors running at 200
MHZ, 512 MB of RAM memory, a ViewSonic PT813.RTM. monitor, a
pointing device, and a Lexmark Optra S1625.RTM. printer. The system
operates within a Windows NT.RTM. operating system.
AUTOCROPPING
[0047] As may be seen in FIGS. 2 and 3, a digital mammogram image
190 is first cropped to segment an analysis region 296 from the
image and produce a binary mask 298 corresponding to breast tissue
in the analysis region. Preferably, the cropping is performed
automatically, although it could be cropped manually. The image is
cropped as a preliminary step because the breast tissue does not
cover the whole radiographic film. Focusing the processing of the
image on only that portion of the image which contains breast
tissue reduces the time required to process the image. Also, other
items appearing on the film, such as labels and patient
information, are excluded from consideration, and false-positive
indications lying outside of the breast tissue area are
eliminated.
[0048] Referring to FIGS. 4 through 10, the autocropping process
will be described in detail. The image is first subsampled from 50
.mu.m to 400 .mu.m to reduce the amount of data to be processed in
step 202. Of course, the image may be downsampled to other
resolutions as desired. Not all of the original image data is
needed to reliably segment the breast tissue from the remainder of
the image. Subsampling every eighth pixel in both the horizontal
and vertical directions reduces the amount of data by 64 times. For
purposes of segmenting the breast tissue from the rest of the
image, the consequent loss of resolution is immaterial.
[0049] A white border twenty pixels in width is added around all
sides of the subsampled image in step 204. White corresponds to the
maximum pixel value possible given the number of bits used to
represent each pixel. For images having 12 bits of gray-scale
resolution, the maximum gray-scale value is 4095. The bordered
image is then thresholded in step 206 with a relatively high
threshold value such that most of the breast tissue is guaranteed
to be less than the threshold to produce a binary image. In one
embodiment of the invention, the threshold is set equal to a
predetermined percentage of the gray-scale value of a pixel near
the top middle portion of the image. The thresholded image is then
inverted, that is, ones become zeroes and zeroes become ones, in
step 208. The inverted image is then dilated in step 210. Dilation
is a morphological operation in which each pixel in a binary image
is turned on, that is, set to a value of one, if any of its
neighboring pixels are on. If the pixel is already on, it is left
on.
[0050] In step 212 the dilated image is cropped to the size of the
largest blob. Blobs are contiguous groups of pixels having the
value one. This step 212 removes bright borders from the subsampled
mammogram representation while ensuring that none of the breast
area is reduced. Other techniques that threshold to find the border
have a very difficult time dealing with bright areas in the breast
adjacent to the border such as, for example, when breast implants
are visible in the image. Pixels from the original image, resulting
from step 202, corresponding to the locations of the pixels in the
cropped blob, are selected for subsequent processing. Note that
this is a simple subset of pixels from the input image.
[0051] The image from step 212 is histogram equalized in step 214.
The average brightness of the image will vary widely from mammogram
to mammogram. Moreover, different digitizers having different
optical density characteristics are an additional source of
variability in brightness levels in the digital representation of
the mammogram. The breast mask that is the output of the
autocropper is mainly defined by means of a region-growing
algorithm that requires a single contrast setting to work properly.
However, it has been determined experimentally that a single
contrast setting will not work for a wide range of image inputs.
Therefore, each image is mapped into a normalized image space using
an automatic histogram enhancement process, after which a single
contrast setting works well.
[0052] First, a histogram of the image is obtained. Typically, most
of the data in the breast area will be in the lower histogram bins
(corresponding to gray-scale values of about 0-1000), with borders
and labels being in the higher bins (corresponding to gray-scale
values of about 4000-4095) for 12-bit data. The upper and lower bin
values that contain the typical breast data are determined. The
lower bin value is the first highest peak encountered when going
from the lowest gray-scale value toward the highest gray-scale
value. The upper bin is the last zero-value bin encountered when
going from the highest gray-scale level toward the lowest
gray-scale value. Then the data are reduced to an eight-bit
representation and linearly stretched over the range of the data
type. For example, values in the lower bins are set to zero. Values
of data in the upper bins are set to 255. The rest of the data are
then linearly mapped between the lower and upper bins.
[0053] After the image has been histogram equalized, the equalized
image may be considered to be a matrix. The image matrix is divided
into left and right halves, of equal size if possible, and the
brighter side is selected in a step 216. The sums of all the pixels
in the left and right halves are computed. The sum values are then
compared and the side having the greater sum is the brighter
side.
[0054] Prior to region growing the brighter side, algorithm
variables are initialized in step 218. The size of the region-grown
mask is preliminarily checked in step 220. If it is large enough,
then the mask is acceptable. Otherwise, processing continues to
find the mask. The side of the image to be region grown is selected
in step 222. In step 224 this region is searched to find its
maximum gray-scale value. This maximum value is used to find a
pixel to start a region-growing algorithm. Region growing is the
process of grouping connected pixels sharing some like
characteristic. The choice of characteristic influences the
resultant region. The input to a region growing function is a
gray-scale image and a starting point to begin growing. The output
is a binary image with ones indicating pixels within the grown
region, i.e., blobs. Region growing will create a single blob, but
that blob may have within it internal holes, that is, pixels that
are off. To grow a blob, each of the four nearest neighbors of a
pixel of interest are looked at. The contrast ratio is computed for
each nearest neighbor pixel. If the contrast ratio is less than a
contrast ratio threshold, then the neighbor pixel is set to a one
in a binary mask image. Otherwise, the neighbor pixel is set to
zero. The region growing algorithm spirals outwardly from the
starting or seed pixel, progressively looking at nearest neighbor
pixels until done. To those skilled in the art, it is clear that
other region growing algorithms may also be applied.
[0055] In step 226, region growing begins with the pixel identified
from the previous step 224 to produce a binary mask. The size of
the mask resulting from step 226 is computed in step 228 and
checked in step 230. There may be three points of failure for this
approach. First, the brightest point in the search region may be an
artifact outside the breast. Therefore, if the resulting mask is
not large enough (50 pixels), then the search region is moved
closer to the side of the image and searched again. This is
repeated three times, each time lowering the contrast value
threshold. This corresponds to the path taken through steps 232 and
234. Second, the side selection approach may be in error.
Therefore, if a valid breast mask is not found in the first side
searched, then the other side of the equalized image is searched.
This corresponds to the path taken through steps 236 and 238.
Third, if a valid breast mask is not found on either side, then the
whole breast is thresholded and the largest object is taken to be
the breast mask in step 240.
[0056] Since a constant contrast value is used in the
region-growing algorithm, some masks will be too large. Typically,
there will be "tails" along the edge of the digitized mammogram
image where extra light leaked in while the original mammogram film
was being digitized. The tails are reduced by applying a series of
erodes and then a series of dilates to the image. Erosion is a
morphological operation in which each pixel in a binary image is
turned off unless all of its neighbors are on. If the pixel is
already off, it is left off. But first, the holes in the mask must
be filled in or the multiple erodes may break the mask into
disjoint sections. Thus, holes in the mask are closed in step 242
by means of a majority operation. The majority operation is a
morphological operation in which each pixel in a binary image is
turned on if a majority of its neighboring pixels are on. If the
pixel is already on, it is left on.
[0057] However, another problem is that some smaller breast masks
can not undergo as many erodes as can larger breast masks.
Therefore, as a fail-safe measure, the sum of the breast mask is
taken before and after the erodes and dilates. If the size is
reduced too much (i.e., by more than 50%), the original mask before
the morphological operators is used. Thus, a duplicate copy of the
mask is made in step 244 before the mask is eroded and dilated in
steps 246 and 248, respectively. The size of the resultant mask is
then computed in step 250 and compared with the size of the mask
from step 242 in step 252. If the new size is less than half the
old size, then the duplicate mask, from step 244, is selected in
step 254 for subsequent processing. Otherwise, the resultant mask
from step 248 is used.
[0058] The original image (from step 202) is then cropped to the
size of the breast mask just found (either from step 242 or step
248) in step 256. In case the resulting mask is too small for
subsequent processing, a crop adjustment is always made in step
258. The adjustment comes in the form of increasing the size of the
breast mask bounding box by including additional pixels from the
original image in the cropped image.
[0059] The cropped image is then automatically histogram enhanced
in step 260 as previously described above in connection with step
214. This enhanced image is passed through a loose region growing
step 262 to produce a generous mask. This means that the image is
subjected to a lower threshold to yield more "on" pixels. This mask
is then subjected to hole-closing, eroding, and dilating in steps
264, 266, and 268, respectively, as above, but to a lesser
degree.
[0060] The same steps described above are repeated one final time
in steps 270 through 276, but the crop adjustments are less and the
contrast value is increased for a tight region growing step 276.
This tight region growing step 276 can afford the higher contrast
value since it will be region growing in just the cropped image.
This results in a parsimonious estimate of breast tissue. The
resulting mask is segmented to find the largest object in step 278
and its bounding box shrunk to just enclose the object in step 280.
There may still be some holes in the breast mask. Therefore, after
crop adjustments in step 282, the mask is inverted in step 284 and
the largest object is found in step 286. This largest object is
extracted and then inverted in step 288 to obtain the penultimate
mask.
[0061] The final mask is obtained by closing holes in the
penultimate mask with multiple majority operations and dilations in
step 290. The image is then cropped to the size of the resulting
mask and the autocropping is complete. An important result from the
autocropper is the offset of the cropped image. This is the pixel
location in the original image that corresponds to the pixel in the
upper left pixel of the cropped image. Keeping track of all the
cropping and crop adjustments determines this offset value.
[0062] The output of the autocropping process is a rectangular
array of pixels representing a binary mask wherein the pixels
corresponding to breast tissue are assigned a value of one while
the remainder of the pixels are assigned a value of zero. Put
another way, the binary mask is a silhouette of the breast made up
of ones while the background is made up of zeroes.
[0063] Parameters of the autocropper may be optimized to obtain
better breast masks. The procedure is described below in the
optimization section.
DETECTION OF CLUSTERED MICROCALCIFICATIONS
[0064] Turning now to FIG. 11, there is seen therein a flow diagram
illustrating in greater detail the clustered microcalcification
detection system 300 of the invention.
[0065] That portion of the digital representation of the mammogram
corresponding to the analysis region 296, designated a cropped
sub-image 302, produced in the cropping step 200, is first
processed to reduce noise in a noise reduction step 310 to reduce
digitization noise that contributes to false detections of
microcalcifications. The noise-reduced image is then filtered using
an optimized target-size-dependent difference of Gaussians (DoG)
spatial kernel in step 320 to enhance differences between targets
and background, thus creating global and local maxima in the
filtered image. The optimized DoG-filtered image is then
thresholded in step 340 to segment maxima that represent potential
detections of micro calcifications.
[0066] The detected maxima are converted to single-pixel coordinate
representations in a conversion step 350. The coordinate
representations of the detected maxima are compared with the binary
mask of the analysis area in a first false-positive removal step
360 to remove false detections outside the breast mask area. The
remaining coordinate representations in the analysis area are
clustered in a clustering step 370. Features are computed for the
remaining clusters in a feature computation step 380 and used to
remove non-suspicious detections in a classifying step 400 (FIG.
1). The remaining detections are outputted as detected clustered
microcalcifications in an outputting step 600 in the form of
cluster coordinates.
[0067] Turning now to a more detailed discussion of the steps in
the clustered microcalcification detection process, the digital
mammogram image is first filtered to reduce noise in the image.
Although the main limitation in image quality should be the
granularity of the film emulsion, noise is introduced from the
process of digitization. This noise may later be detected as a
pseudocalcification. In this system, a cross-shaped median filter
is used because it is well known to be extremely effective at
removing single-pixel noise. The median filter is a non-linear
spatial filter that replaces each pixel value with the median of
the pixel values within a kernel of chosen size and shape centered
at a pixel of interest. Referring to FIG. 12, it may be seen that
the cross shape is formed by the set of pixels which include the
center pixel and its four nearest neighbors. The cross shape
preserves lines and corners better than typical block-shaped median
filters and limits the possible substitution to the four nearest
neighbors, thereby reducing the potential for edge
displacement.
[0068] After noise has been reduced, the image is filtered with an
optimized DoG kernel to enhance microcalcifications. Filtering is
accomplished by convolving the noise-reduced image with the DoG
kernel. In an alternative embodiment, filtering is accomplished by
first obtaining the fast Fourier transforms (FFTs) of the
noise-reduced image and the DoG kernel, then multiplying the FFTs
together, and taking the inverse FFT of the result.
[0069] The DoG kernel was chosen because neurophysiological
experiments provide evidence that the human visual pathway includes
a set of "channels" that are spatial frequency selective.
Essentially, at each point in the visual field, there are
size-tuned filters or masks analyzing an image. The operation of
these spatial receptive fields can be approximated closely by a
DoG.
[0070] The 2-D Gaussian mask is given as:
G(x,y)=ce-(x.sup.2+y.sup.2)/2.sigma..sup.2 (1)
[0071] where c normalizes the sum of mask elements to unity, x and
y are horizontal and vertical indices, and .sigma. is the standard
deviation. Using Equation 1, the difference of two Gaussians with
different u yields:
DoG(x,y)=c.sub.1e-(x.sup.2+y.sup.2)/2.sigma..sub.1.sup.2-c.sub.2e
-(x.sup.2+y.sup.2)/2.sigma..sub.2.sup.2 (2)
[0072] It has been shown that when .sigma..sub.2=1.6.sigma..sub.1,
then the DoG filter's response closely matches the response of
human spatial receptive filters. Therefore, with motivation from
human physiology, let the ratio of the DoG standard deviation
constants be 1:1.6. Then, for a target of size (average width) t
pixels, use .sigma..sub.2=t/2 and, from the rule of thumb,
.sigma..sub.1=.sigma..sub.2/1.6.
[0073] Since microcalcifications typically range from 100 to 300
.mu.m in diameter, potential target sizes for the 50 .mu.m
digitized mammograms correspond to 2 to 6 pixels. It has been found
that a DoG kernel constructed using an optimization technique for
selecting the target size parameter, such as the GA detailed below,
has an optimized target size of t=6.01 pixels. The targetsize t
will vary depending on such factors as the resolution and scale of
the image to be processed. The impulse response of a DoG filter
having t=6.01 pixels and .sigma..sub.2=1.6.sigma- ..sub.1 is shown
in FIGS. 13 and 14.
[0074] Once the noised-reduced cropped image has been DoG filtered
to enhance differences between targets and background, the
DoG-filtered subimage contains differences in gray levels between
potential microcalcifications and background. Although
microcalcifications tend to be among the brightest objects in
DoG-filtered subimages, they may exist within regions of high
average gray levels and thus prove difficult to reliably segment.
The thresholding process used in one embodiment of the invention
that generally addresses these concerns involves pair-wise pixel
"ANDing" of the results of global histogram and locally adaptive
thresholding. However, the preferred embodiment of the invention
uses sloping local thresholding.
[0075] Since targets tend to exist within an image's higher gray
levels, then the global threshold may be approximated by finding
the level which segments a preselected percentage of the
corresponding higher pixel levels in the image histogram. An
embodiment of a global thresholding method is illustrated in FIG.
15. Locally adaptive thresholding may be implemented by varying the
high and low thresholds based on the local pixel value mean and
standard deviation. An embodiment of a dual-local thresholding
method is illustrated in FIG. 16.
[0076] After computing the image histogram, p(r.sub.k), the gray
level threshold, g, used to segment a preselected upper fraction, f
of the histogram, is found using: 1 f = 1 - k = 0 g p ( r k ) ( 3
)
[0077] where r.sub.k is the k.sup.th gray level,
0.ltoreq.g.ltoreq.g.sub.m- ax, and g.sub.max is the maximum gray
level in the image.
[0078] The locally adaptive thresholds, t.sub.lo and t.sub.hi, are
found using
t.sub.lo=k.sub.lo.sigma..sub.NN(x,y)+.mu..sub.NN(x,y) (4)
[0079] and
t.sub.hi=k.sub.hi.sigma..sub.NN(x,y)+.mu..sub.NN(x,y) (4)
[0080] where k.sub.lo and k.sub.hi are used to preselect the
multiple of .sigma..sub.NN(x,y), the local standard deviation of
gray-level intensities, and .mu..sub.NN(x,y) is the local
gray-level mean of the N.times.N neighborhood centered on the pixel
at (x,y) of the DoG-filtered image. Other neighborhood shapes, such
as rectangular, circular, and ellipsoidal, may also be used. Pixels
whose brightness or gray-level value falls within the threshold
interval, that is, t.sub.lo<brightness<t.sub.hi, are set
equal to one. Optimization of f, k.sub.lo, k.sub.hi, and N is
discussed below in connection with the parameter-optimizing
process. The results of the global thresholding process may be
combined with the results of the local thresholding step by
logically ANDing them as shown in FIG. 17. Alternatively, either
thresholding method may be used alone.
[0081] The preferred thresholding means are illustrated in FIG. 18
wherein it may be seen that an N.times.N window is centered at a
pixel x,y in the input image p(x,y). The mean, .mu.(x,y), and
standard deviation, .sigma.(x,y), of the digital mammogram image
pixels under the window are computed. A local threshold value,
T(x,y), is computed as:
T(x,y)=A+B.nu.(x,y)+C.sigma.(x,y) (6)
[0082] where values for N, A, B, and C are computed during a
parameter optimization stage, discussed below. Values for T(x,y)
are computed for every x,y location in the image.
[0083] The digital mammogram has also been DoG filtered, producing
an image d(x,y). Each pixel of the DoG-filtered image d(x,y) is
compared to the threshold value T(x,y). Pixels in the locally
thresholded image l.sub.s(x,y) are set to one where values of the
DoG-filtered image are greater than the threshold, and set to zero
elsewhere.
[0084] The advantage of this novel local sloping thresholding
method over prior art thresholding methods is that the threshold is
computed from the pixels in a pre-DoG-filtered image rather than
from a post-DoG-filtered image. This eliminates the need for
background trend correction. In conventional local thresholding,
the threshold is computed as:
T(x,y)=B.mu.(x,y)+C.sigma.(x,y) (7)
[0085] from the mean and standard deviation of the DoG-filtered
image. The problem of using a local threshold computed from the
DoG-filtered image is that DoG-filtered images typically have mean
values close to zero and standard deviations significantly affected
by the presence of targets.
[0086] Local thresholds computed from the statistics of the
DoG-filtered image suffer from the following adverse effects.
First, since the mean value is close to zero, a degree of freedom
is lost in the computation of the threshold, which becomes
essentially a function of the standard deviation. Second, the
absolute brightness of the input image is lost. To keep many
spurious detections from occurring, it is desirable to have high
thresholds in bright regions. However, the information about the
local mean of the input image is not available in the DoG-filtered
image. Finally, the standard deviations of DoG-filtered images are
increased by detections of targets. This is so because when local
bright spots of proper size exist in the original image, large
gray-scale values result in the DoG-filtered image. Thus, the
presence of targets in a region increases the local standard
deviation thereby raising the threshold of that region. The higher
threshold reduces the probability of passing a bright spot to
subsequent processing stages.
[0087] The novel local thresholding method just described solves
the above problems by computing thresholds from the input image,
which are then applied to the DoG-filtered image. Additionally, the
threshold computed here includes an offset term A, which is
independent of the local image mean.
[0088] After thresholding, detections are converted to single-pixel
representations by computing the centroid or center of gravity of
groups of contiguous pixels found by the thresholding process.
Detections are thus represented as single pixels having a value of
logical one while the remaining pixels have a value of logical
zero.
[0089] False-positive detections outside of the breast area are
removed by logically ANDing the binary mask from the autocropper
with the single-pixel representations of the detections.
[0090] Calcifications associated with malignancies usually occur in
clusters and can be extensive. The cluster detection module
identifies clusters based on a clustering algorithm as depicted in
FIG. 19. Specifically, a suspicious cluster is declared when at
least .mu.Cs.sub.min or more detected signals are separated by less
than a nearest neighbor distance, d.sub.nn. Optimization of
.mu.Cs.sub.min and d.sub.nn is discussed below in connection with
the parameter optimizing process. FIG. 20 illustrates the
clustering process for the case wherein .mu.Cs.sub.min=5 and
d.sub.nn=4.
[0091] Additional false-positive clustered microcalcifications are
removed by means of a classifier, detailed below. Features are
extracted for each of the potential clustered microcalcifications
as shown in FIG. 21. The eight features computed for each of the
potential clustered microcalcifications in a preferred embodiment
are:
[0092] 1. The larger eigenvalue (.lambda..sub.1) of the covariance
matrix of the points in a cluster;
[0093] 2. The smaller eigenvalue (.lambda..sub.2) of the covariance
matrix of the points in a cluster;
[0094] 3. The ratio of the smaller eigenvalue of the covariance
matrix to the larger eigenvalue of the covariance matrix of the
points in a cluster. Equivalent to the ratio of the minor axis to
the major axis of an ellipse fitted to cover the points in a
cluster;
[0095] 4. Linear density calculated as the number of detected
microcalcifications divided by the maximum interpoint distance;
[0096] 5. Standard deviation of the distances between points in a
cluster;
[0097] 6. Mean minus median of the distances between points in a
cluster;
[0098] 7. Range of points in cluster calculated as maximum
interpoint distance minus the minimum interpoint distance; and
[0099] 8. Density of a cluster calculated as the number of
detections divided by the area of a box just large enough to
enclose the detections.
[0100] Of course, other features could be computed for the
potential microcalcification clusters, and the invention is not
limited to the number or types of features enumerated herein.
CLASSIFYING DETECTIONS
[0101] The cluster features are provided as inputs to the
classifier, which classifies each potential clustered
microcalcification as either suspicious or not suspicious. In
practice, the clustered microcalcification detector is only able to
locate regions of interest in the digital representation of the
original mammogram that may be associated with cancer. In any
detector, there is a tradeoff between locating as many potentially
suspicious regions as possible versus reducing the number of normal
regions falsely detected as being potentially suspicious. CAD
systems are designed to provide the largest detection rates
possible at the expense of detecting potentially significant
numbers of regions that are actually normal. Many of these unwanted
detections are removed from consideration by applying pattern
recognition techniques.
[0102] Pattern recognition is the process of making decisions based
on measurements. In this system, regions of interest or detections
are located by a detector, and then accepted or rejected for
display. The first step in the process is to characterize the
detected regions. Toward this end, multiple measurements are
computed from each of the detected regions. Each measurement is
referred to as a feature. A collection of measurements for a
detected region is referred to as a feature vector, wherein each
element of the vector represents a feature value. The feature
vector is input to a discriminant function.
[0103] Referring to FIG. 22, there may be seen therein a classifier
having a feature vector x applied to a set of discriminant
functions g(x). The classifier shown in FIG. 22 is designed with
one discriminant function per class. A discriminant function
computes a single value as a function of an input feature vector.
Discriminant functions may be learned from training data and
implemented in a variety of functional forms. The output of a
discriminant function is referred to as a test statistic.
Classification is selecting a class according to the discriminant
function with the greatest output value. The test statistic is
compared to a threshold value. For values of the test statistic
above the threshold, the region or detection associated with the
feature vector is retained and displayed as potentially suspicious.
When the test statistic is below the threshold, the region is not
displayed.
[0104] Many methods are available for designing discriminant
functions. One approach considered for this invention is a class of
artificial neural networks. Artificial neural networks require
training, whereby the discriminant function is formed with the
assistance of labeled training data.
[0105] In a preferred embodiment, the classification process is
implemented by means of a multi-layer perceptron (MLP) neural
network (NN). Of course, other classifier means could be used such
as, for example, a statistical quadratric classifier. Only
potential clustered microcalcifications classified as suspicious
are retained for eventual designation for a radiologist.
Alternatively, it may be desirable to iteratively loop between MLP
NN analysis of the individual microcalcification detections and the
microcalcification clusters.
[0106] Referring to FIG. 23, a schematic diagram of an MLP NN may
be seen therein. The MLP NN includes a first layer of J hidden
layer nodes or perceptrons 410, and one output node or perceptron
420 for each class. The preferred embodiment of the invention uses
two output nodes, one each for the class of suspicious detections
and the class of non-suspicious detections. Of course, more or
fewer classes could be used for classifying clusters of
microcalcifications. Each computed feature x.sub.i is first
multiplied by a weight w.sub.ij, where i is an index representing
the i.sup.th feature vector element, and j is an index representing
the j.sup.th first layer node. The output y.sub.j of each first
layer perceptron 410 is a nonlinear function of the weighted inputs
and is given by: 2 y j = f ( i = 1 d ( w i , j .times. x i ) ) ( 8
)
[0107] where d represents the total number of features x.sub.i and
f(.cndot.) is typically a saturating nonlinearity. In this
embodiment, f(.cndot.)=tan h(.cndot.). The first layer or hidden
layer node outputs y.sub.j are then multiplied by a second layer of
weights u.sub.j,k and applied to the output layer nodes 420. The
output of an output layer node 420 is a nonlinear function of the
weighted inputs and is given by: 3 z k ( y ) - f ( j = 1 J ( u j ,
k .times. y j ) ) ( 9 )
[0108] where k is an index representing the k.sup.th output
node.
[0109] The hyperbolic tangent function is used in a preferred
embodiment of the system because it allows the MLP NN to be trained
relatively faster as compared to other functions. However,
functions other than the hyperbolic tangent may be used to provide
the outputs from the perceptrons. For example, linear functions may
be used, as well as smoothly varying nonlinear functions, such as
the sigmoid function.
[0110] The weight values are obtained by training the network.
Training consists of repeatedly presenting feature vectors of known
class membership as inputs to the network. Weight values are
adjusted with a back propagation algorithm to reduce the mean
squared error between actual and desired network outputs. Desired
outputs of z.sub.1 and z.sub.2 for a suspicious input are +1 and
-1, respectively. Desired outputs of z.sub.1 and z.sub.2 for
non-suspicious inputs are -1 and +1, respectively. Other error
metrics and output values may also be used.
[0111] In this embodiment of the system, the MLP NN is implemented
by means of software running on a general-purpose computer.
Alternatively, the MLP NN could also be implemented in a hardware
configuration by means readily apparent to those with ordinary
skill in the art.
[0112] After training, each detected clustered microcalcification
is classified as either suspicious or not suspicious by means
forming the difference z.sub.1-z.sub.2, then comparing the
difference to a threshold, .theta.. For values of z.sub.1-z.sub.2
greater than or equal to the threshold .theta., i.e.,
z.sub.1-z.sub.2.gtoreq..theta., the classifier returns a value of
+1 for suspicious clustered microcalcifications, and for values of
z.sub.1-z.sub.2<.theta., the classifier returns a value of -1
for non-suspicious clustered microcalcifications.
[0113] In order to arrive at optimum values for the respective
weights, and the number of first layer nodes, the MLP NN was
trained with a training set of feature vectors derived from a
database of 978 mammogram images.
[0114] To develop and test the CAD system of the invention, truth
data was first generated. Truth data provides a categorization of
the tissue in the digital images as a function of position. Truth
data was generated by certified radiologists marking truth boxes
over image regions associated with cancer. In addition to the
mammogram images, the radiologists also had access to patient
histories and pathology reports.
[0115] The radiologists identified 57 regions of interest,
containing biopsy-confirmed cancers associated with clustered
microcalcifications, by means of truth boxes. All 978 images were
then processed by the microcalcification detector of the invention
to produce a plurality of feature vectors, a subset of which were
associated with the 57 truth boxes. Half of the subset feature
vectors were randomly chosen, along with about three times as many
feature vectors not associated with clustered microcalcifications,
to comprise the training set of feature vectors. The MLP NN, having
a predetermined number of hidden nodes, was then trained using the
training set. The remaining feature vectors were used as a test
database to evaluate the performance of the MLP NN after training.
Training of the MLP NN was carried out by means of the
Levenberg-Marquardt back propagation algorithm.
[0116] Alternatively, the MLP NN can be trained with other learning
algorithms and may have nonlinearities other than the hyperbolic
tangent in either or both layers. In an alternative embodiment with
sigmoidal output nodes, the Bayes optimal solution of the problem
of classifying clustered microcalcification detections as either
suspicious or non-suspicious may be obtained.
[0117] In one run of the preferred embodiment during testing,
before application of the MLP NN classifier to eliminate
false-positive clustered microcalcifications, the detection
procedure found about 93% of the true-positive clustered
microcalcifications in both the training and test databases while
indicating about 10 false-positive clustered microcalcifications
per image. It was found that after an MLP NN classifier having 25
first layer nodes was used with the respective optimum weights
found during training, 93% of the true-positive detections were
retained while 57% of the false-positive detections were
successfully removed. Referring to FIG. 24, there may be seen a
histogram of the results of testing on the testing database after
classification by the MLP NN. Of course, the MLP NN of the
invention may be operated with more or fewer first layer nodes as
desired.
DISPLAYING DETECTIONS
[0118] After the locations of clustered microcalcifications have
been determined, they are indicated on the original digitized
mammogram image, or a copy of the original image, by drawing
rectangular boxes around microcalcifications. Other means for
indicating the locations of microcalcifications may be used, such
as, for example, placing arrows in the image pointing at detections
or drawing ellipses around the detections.
[0119] The locations of clustered microcalcifications are passed to
the display detections procedure as a list of row and column
coordinates of the upper left and lower right pixels bounding each
of the clusters. The minimum row and column coordinates and maximum
row and column coordinates are computed for each cluster. Bounding
boxes defined by the minimum and maximum row and column coordinates
are added to the original digitized image, by means well known in
the art. The resulting image is then stored as a computer-readable
file, displayed on a monitor, or printed as a hard-copy image, as
desired.
[0120] In one embodiment of the system, the resulting image is
saved to a hard disk on a general-purpose computer having dual
Pentium II.RTM. processors and running a Windows NT.RTM. operating
system. The resulting image may be viewed on a VGA or SVGA monitor,
such as a ViewSonic PT813.RTM. monitor, or printed as a hard-copy
gray-scale image using a laser printer, such as a Lexmark Optra S
1625.RTM.. Of course, other hardware elements may be used by those
with ordinary skill in the art.
OPTIMIZING THE PARAMETERS
[0121] Genetic algorithms (GAs) have been successfully applied to
many diverse and difficult optimization problems. A preferred
embodiment of this invention uses an implementation of a GA
developed by Houck, et al. ("A Genetic Algorithm for Function
Optimization," Tech. Rep., NCSU-IE TR 95-09, 1995), the entire
disclosure of which is incorporated by reference herein, to find
promising parameter settings. The parameter optimization process of
the invention is shown in FIG. 25. This is a novel application of
optimization techniques as compared to current computer-aided
diagnosis systems that require hand tuning by experiment.
[0122] GAs search the solution space to maximize a fitness
(objective) function by use of simulated evolutionary operators
such as mutation and sexual recombination. In this embodiment, the
fitness function to be maximized reflects the goals of maximizing
the number of true-positive detections while minimizing the number
of false-positive detections. GA use requires determination of
several issues: objective function design, parameter set
representation, population initialization, choice of selection
function, choice of genetic operators (reproduction mechanisms) for
simulated evolution, and identification of termination
criteria.
[0123] The design of the objective function is a key factor in the
performance of any optimization algorithm. The function
optimization problem for detecting clustered microcalcifications
may be described as follows: given some finite domain, D, a
particular set of cluster detection parameters, x={t, f, k.sub.lo,
k.sub.hi, N, .mu.Cs.sub.min, d.sub.NN} where x.epsilon.D, and an
objective function f.sub.obj: D-, where denotes the set of real
numbers, find the x in D that maximizes or minimizes f.sub.obj.
When sloping local thresholding is used in the cluster detector,
the parameters N, A, B, and C are optimized. Radiologic imaging
systems may be optimized to maximize the TP rate subject to the
constraint of minimizing the FP rate. This objective may be recast
into the functional form shown in the following equation: 4 f obj (
x ) = { - FP ( x ) , TP ( x ) TP min FP penalty , otherwise ( 10
)
[0124] where maximization is the goal. For a particular set of
cluster detection parameters, if the minimum acceptable TP rate,
TP.sub.min, is exceeded, the objective function returns the
negative of the FP rate. Otherwise, if the TP rate falls below
TP.sub.min, the objective function returns a constant value,
FP.sub.penalty=-10. Other objective functions may also be used.
[0125] Since a real-valued GA is an order of magnitude more
efficient in CPU time than the binary GA, and provides higher
precision with more consistent results across replications, this
embodiment of the invention uses a floating-point representation of
the GA.
[0126] This embodiment also seeds the initial population with some
members known beforehand to be in an interesting part of the search
space so as to iteratively improve existing solutions. Also, the
number of members is limited to twenty so as to reduce the
computational cost of evaluating objective functions.
[0127] In one embodiment of the invention, normalized geometric
ranking is used, as discussed in greater detail in Houck, et al.,
supra, for the probabilistic selection process used to identify
candidates for reproduction. Ranking is less prone to premature
convergence caused by individuals that are far above average. The
basic idea of ranking is to select solutions for the mating pool
based on the relative fitness between solutions. This embodiment
also uses the default genetic operation schemes of arithmetic
crossover and nonuniform mutation included in Houck, et al.'s
GA.
[0128] This embodiment continues to search for solutions until the
objective function converges. Alternatively, the search could be
terminated after a predetermined number of generations. Although
termination due to loss of population diversity and/or lack of
improvement is efficient when crossover is the primary source of
variation in a population, homogeneous populations can be succeeded
with better (higher) fitness when using mutation. Crossover refers
to generating new members of a population by combining elements
from several of the most fit members. This corresponds to keeping
solutions in the best part of the search space. Mutation refers to
randomly altering elements from the most fit members. This allows
the algorithm to exit an area of the search space that may be just
a local maximum. Since restarting populations that may have
converged proves useful, several iterations of the GA are run until
a consistent lack of increase in average fitness is recognized.
[0129] Once potentially optimum solutions are found by using the
GA, the most fit GA solution may be further optimized by local
searches. An alternative embodiment of the invention uses the
simplex method to further refine the optimized GA solution.
[0130] The autocropping system may also benefit from optimization
of its parameters including contrast value, number of erodes, and
number of dilates. The method for optimizing the autocropper
includes the steps of generating breast masks by hand for some
training data, selecting an initial population, and producing
breast masks for training data. The method further includes the
steps of measuring the percent of overlap of the hand-generated and
automatically-generated masks as well as the fraction of
autocropped breast tissue outside the hand-generated masks. The
method further comprises selecting winning members, generating new
members, and iterating in a like manner as described above until a
predetermined objective function converges.
[0131] In FIGS. 26 and 27, there may be seen therein free response
receiver operating characteristic curves for the system of the
invention for the outputs of the optimized microcalcification
detector and the classifier, respectively. FIG. 26 represents the
performance of the optimized detector before classifying
detections, while FIG. 27 represents the performance of the system
after classifying detections.
[0132] Although the GA has been described above in connection with
the parameter optimization portion of the preferred embodiment,
other optimization techniques are suitable such as, for example,
response surface methodology. Of course, processing systems other
than those described herein may be optimized by the methods
disclosed herein, including the GA.
INCORPORATING CAD SYSTEM OUTPUTS FOR OPTIMAL SENSITIVITY
[0133] Performance metrics for detection of suspicious regions
associated with cancer are often reported in terms of sensitivity
and specificity. Sensitivity measures how well a system finds
suspicious regions and is defined as the percentage of suspicious
regions detected from the total number of suspicious regions in the
cases reviewed. Sensitivity is defined as: 5 Sensitivity = TP TP +
FN ( 11 )
[0134] where TP is the number of regions reported as suspicious by
a CAD system that are associated with cancers, and FN is the number
of regions that are known to be cancerous that are not reported as
suspicious. Specificity measures how well the system reports normal
regions as normal. Specificity is defined as: 6 Specificity = TN FP
+ TN ( 12 )
[0135] where TN represents regions correctly identified as not
suspicious and FP represents regions reported as suspicious that
are not cancerous.
[0136] Current CAD systems increase specificity by reducing FP.
However, FP and TP are coupled quantities. That is, a reduction of
FP leads to a reduction of TP. This implies that some of the
suspicious regions that could have been detected are missed when
the objective is to maintain high specificity.
[0137] FIGS. 28 and 29 illustrate relationships between the
quantities TP, FP, TN, and FN. A measurement from a screening
mammography image is represented by test statistic, x. The
probability density function of x is represented by p(x) and the
decision threshold is represented by .theta.. If x is greater than
.theta., a suspicious region is reported. Areas under the
probability density functions represent probabilities of events.
From FIG. 28 observe that increasing the threshold reduces the
probability of FP decisions. However, observe from FIG. 29 that
increasing the threshold simultaneously reduces the probability of
TP decisions.
[0138] Another metric that exists for CAD systems is positive
predictive value (PPV), which is defined as the probability that
cancer actually exists when a region of interest is labeled as
suspicious. PPV can be calculated from the following equation: 7
PPV = TP TP + FP ( 13 )
[0139] Note that increasing TP or reducing FP increases PPV.
[0140] Radiologists and computers find different suspicious
regions. FIG. 30 is a Venn diagram depicting a possible
distribution of suspicious regions for man and machine detections.
Some suspicious regions are found solely by a human interpreter or
radiologist, some solely by a CAD system, some are found by both,
and some are not found by either.
[0141] Referring to FIG. 31, there may be seen a preferred method
for incorporating the outputs of a CAD system, and more
particularly for the CAD system of the invention, with the
observations of a human interpreter of a screening mammography
image 10 for optimal sensitivity, wherein a radiologist examines
the screening mammography image 10 in a step 20 and reports a set
of suspicious regions 30 designated as S1. The CAD system then
operates on the image 10 in a step 40 and reports a set of
suspicious regions 50 designated as S2. The radiologist then
examines set S2 and accepts or rejects members of set S2 as
suspicious in a step 60, thereby forming a third set of suspicious
regions 70 denoted as S3, which is a subset of S2. The radiologist
then creates in a step 80 a set of workup regions 90 denoted as S4
which is the union of sets S1 and S3. The workup regions 90 are
then recommended for further examination such as taking additional
mammograms with greater resolution, examining the areas of the
breast tissue corresponding to the workup regions by means of
ultrasound, or performing biopsies of the breast tissue.
[0142] While the invention has been described in connection with
detecting clustered microcalcifications in mammograms, it should be
understood that the methods and systems described herein may also
be applicable to other medical images such as chest x-rays.
[0143] While the methods herein described, and the forms of
apparatus for carrying these methods into effect, constitute
preferred embodiments of this invention, it is to be understood
that the invention is not limited to these precise methods and
forms of apparatus, and that changes may be made in either without
departing from the scope of the invention which is defined in the
appended claims.
* * * * *