U.S. patent application number 11/556226 was filed with the patent office on 2007-03-15 for method for automatic construction of 2d statistical shape model for the lung regions.
Invention is credited to David H. Foos, Hui Luo.
Application Number | 20070058850 11/556226 |
Document ID | / |
Family ID | 32468817 |
Filed Date | 2007-03-15 |
United States Patent
Application |
20070058850 |
Kind Code |
A1 |
Luo; Hui ; et al. |
March 15, 2007 |
METHOD FOR AUTOMATIC CONSTRUCTION OF 2D STATISTICAL SHAPE MODEL FOR
THE LUNG REGIONS
Abstract
A method for automatic construction of 2D statistical shape
models for the lung regions in chest radiographic images. The
method includes: extracting the anatomical structure contours from
a chest radiograph image; generating a polygonal shape
approximation for each contour; aligning a plurality of polygonal
shape approximations by minimizing their distance; assigning a
plurality of landmarks on each shape contour; and determining a
statistical shape model using aligned shape contours.
Inventors: |
Luo; Hui; (Rochester,
NY) ; Foos; David H.; (Rochester, NY) |
Correspondence
Address: |
Pamela R. Crocker;Patent Legal Staff
Eastman Kodak Company
343 State Street
Rochester
NY
14650-2201
US
|
Family ID: |
32468817 |
Appl. No.: |
11/556226 |
Filed: |
November 3, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10315855 |
Dec 10, 2002 |
|
|
|
11556226 |
Nov 3, 2006 |
|
|
|
Current U.S.
Class: |
382/132 ;
382/203 |
Current CPC
Class: |
G06T 2207/20081
20130101; G06T 2207/10116 20130101; G06T 7/149 20170101; G06T
2207/30012 20130101; G06T 2207/20104 20130101; G06T 7/11 20170101;
G06T 2207/20124 20130101; G06T 2207/30061 20130101 |
Class at
Publication: |
382/132 ;
382/203 |
International
Class: |
G06K 9/00 20060101
G06K009/00 |
Claims
1. A method for automatic construction of 2D statistical shape
models for the lung regions in chest radiographs, comprising:
extracting the anatomical structure contours from a chest
radiograph image; generating a polygonal shape approximation for
each contour; aligning a plurality of polygonal shape
approximations by minimizing their distance; assigning a plurality
of landmarks on each shape contour; and determining a statistical
shape model using aligned shape contours.
2. The method of claim 1, wherein aligning a plurality of polygonal
shape approximations is accomplished by: creating a template shape;
computing a minimal distance and associated transformation
parameters between the polygon approximations of the template shape
and a shape instance; and transforming the shape instance based on
the transformation parameters.
3. The method of claim 2, wherein computing the minimal distance is
accomplished by: computing turning functions of polygon
approximations of a shape instance and the template shape; and
transforming the turning function of the shape instance to search
the minimal distance between the turning function of the shape
instance and the turning function of the template shape; and
determining the transformation parameters using the minimal
distance.
4. The method of claim 2, wherein the computing the minimal
distance is accomplished by: choosing predetermined polygon
vertices with salient shape features as landmarks; assigning a
plurality of landmarks between the located landmarks using the
shape contour; computing the minimal distance between the shape
instance and the template shape using the landmarks; and
determining the transformation parameters using the minimal
distance.
5. The method of claim 4, wherein assigning a plurality of
landmarks between the located landmarks comprises locating a fixed
number of equidistant points between the located landmarks.
6. The method of claim 4, wherein computing the minimal distance is
accomplished by: computing the minimal distance and associated
transformation parameters between the polygon approximations of the
template shape and a shape instance; transforming the shape
instance based on the transformation parameters; and minimizing the
weighted sum of distances between the corresponding landmarks of
the template and the shape instance.
7. The method of claim 4, wherein the minimal distance is computed
by minimizing the weighted sum of distances between the
corresponding landmarks.
8. The method of claim 7, wherein the weights are chosen to give
more significance to the landmarks associated with salient shape
features.
9. The method of claim 1, wherein assigning a plurality of
landmarks on shape contours comprises: choosing a certain polygon
vertices with salient shape features as landmarks; and assigning a
plurality of landmarks between the located landmarks using the
aligned shape contour;
10. The method of claim 9, wherein assigning a plurality of
landmarks between the located landmarks comprises locating a fixed
number of equidistant points between the located landmarks.
11. The method of claim 1, wherein the statistical model is
determined according to a principle component analysis.
12. A method for automatic construction of 2D statistical shape
models for an anatomical structure, comprising: extracting the
anatomical structure contours from a radiograph image; generating a
polygonal shape approximation for each contour; aligning a
plurality of polygonal shape approximations by minimizing their
distance; assigning a plurality of landmarks on each shape contour;
and determining a statistical shape model using aligned shape
contours.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This is a continuation of U.S. Ser. No. 10/315,855 entitled
"METHOD FOR AUTOMATIC CONSTRUCTION OF 2D STATISTICAL SHAPE MODEL
FOR THE LUNG REGIONS", and filed on Dec. 10, 2002 in the names of
Luo et al., and which is assigned to the assignee of this
application.
FIELD OF THE INVENTION
[0002] This invention relates in general to lung shape modeling,
and in particular to a method for automatically constructing
two-dimension (2D) statistical shape model of the lung regions from
sets of chest radiographic images.
BACKGROUND OF THE INVENTION
[0003] The use of shape as an anatomical object property is a
rapidly increasing portion of research in the field of medical
image analysis. Shape representations and shape models have been
used in connection with segmentation of medical images, diagnosis,
and motion analysis. Among different types of shape models, Active
Shape Models (ASMs) have been frequently applied and proven a
powerful tool for characterizing objects and segmenting medical
images. In order to construct such models, sets of labeled training
images are required. The labels in the training sets consist of
landmark points defining the correspondences between similar
structures in each image across the set. Manual definition of
landmarks on 2D shapes has proven to be both time-consuming and
error prone. To reduce the burden, semi-automatic systems have been
developed. In these systems, a model is built from the current set
of examples, and used to search the next image. The user can edit
the result where necessary, then add the example to the training
set. Though this can considerably reduce the time and effort
required, labeling large sets of examples is still labor
intensive.
[0004] Because of the importance of landmark labeling, a few
attempts have been made to automate the shape alignment/average
process. For example, Lorenz and Krahnstover automatically locate
candidates for landmarks via a metric for points of high curvature,
Lorenz C., Krahnstove N. Generation of point-based 3D statistical
shape models for anatomical objects. CVIU, vol 77, no. 2, February
2000, pp. 175-191. Davatzikos et al. used curvature registration on
contours produced by an active contour approach, (C. Davatzikos, M.
Vaillant, S. M. Resnich, J. L. Prince, S. Letovsky, and R. N.
Bryan, A Computerized Approach for Morphological Analysis of the
Corpus Callosum, J. Computer Assisted Tomography, vol. 20, 1996,
pp. 88-97). Duncan et al. (J. Duncan, R. L. Owen, L. H. Staib, and
F. Anandan, Measurement of non-rigid motion using contour shape
descriptors, in IEEE Conference on Computer Vision and Pattern
Recognition, 1991, pp. 318-324). And Kambhamettu et al, (C.
Kambhamettu and D. B. Goldgof, Point correspondence recovery in
non-rigid motion, IEEE Conference on Computer Vision and Pattern
Recognition, 1992, pp. 545-561), propose methods of correspondence
based on the minimization of a cost function that involves the
difference in the curvature of two boundaries. However, as pointed
out by several studies, curvature is a rigid invariant of shape and
its applicability is limited in case of nonlinear shape
distortions. In addition, it is hard to find sufficient high
curvature points on lung contours.
[0005] Hill et al. employed a sparse polygonal approximation to one
of two boundaries which is transformed onto the other boundary via
an optimization scheme, (A. Hill, C. J. Taylor, and A. D. Brett, A
Framework for Automatic Landmark Identification Using a New Method
of Nonrigid Correspondence, IEEE Trans. Pattern Analysis and
Machine Intelligence, vol. 22, no. 3, 2000, pp. 241-251). The
polygonal matching is based on an assumption that arc path-lengths
between consecutive points are equal. This assumption may be
violated in case of severe shape difference and is especially
difficult to satisfy in polygonal approximation of lung shape
contours.
[0006] As a result, the prior art does not fit the lung shape
modeling very well, therefore there exists a need for a method for
automatically constructing 2D statistical shape model of lung
regions in chest radiographs.
SUMMARY OF THE INVENTION
[0007] According to the present invention, a method is provided for
automatic construction of 2-D statistical shape models for the lung
regions in chest radiographic images. The method makes use of a set
of shape instances of lung regions from chest images, and
automatically aligns them to a pre-defined template shape using the
L.sub.2 distance and Procrustes distance analysis. Once the
training shapes are appropriately aligned, a set of landmarks is
automatically generated from each shape. Finally, a 2D statistical
model is constructed by Principle Component Analysis. The
statistical shape model consists of a mean shape vector to
represent the general shape and variation modes in the form of the
eigenvectors of the covariance matrix to model the differences
between individuals.
ADVANTAGEOUS EFFECT OF THE INVENTION
[0008] The invention has the following advantages.
[0009] 1. The entire alignment and labeling process is
automatic.
[0010] 2. The time and effort required to label sets of data is
dramatically reduced.
[0011] 3. User bias introduced by manual labeling is avoided.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Preferred embodiments of the present invention will be
described below in more detail, with reference to the accompanying
drawings:
[0013] FIG. 1 is a flowchart illustrating the overall scheme for
the automated method for constructing 2D statistical shape models
of lung regions.
[0014] FIG. 2 is a block diagram illustration of the shape
alignment algorithm.
[0015] FIG. 3(a) is a diagrammatic view illustrating the polygonal
shape approximations T.sub.p computed from the template shape.
[0016] FIG. 3(b) is a diagrammatic view illustrating the polygonal
shape approximations Sp computed from a shape instances.
[0017] FIG. (4a) is a diagrammatic view of turning angle vs.
arc-length showing the turning function .theta..sub.Tp(s) of the
template shape.
[0018] FIG. 4(b) is a diagrammatic view of turning angle vs.
arc-length showing the turning function .theta..sub.Sp(s) of the
shape instance.
[0019] FIG. 5(a) is a diagrammatic view showing the result of the
coarse shape alignment.
[0020] FIG. 5(b) is a graphical view of turning angle vs.
arc-length illustrating the relationships of the turning functions
in the coarse shape alignment.
[0021] FIG. 6 illustrates the determination of landmarks on the
left and right lung shape contours.
[0022] FIGS. 7(a) and 7(b) are diagrammatic views respectively
showing the corresponding landmark points on the template shape and
the shape instance.
[0023] FIG. 8 is a diagrammatic view displaying the final alignment
result.
[0024] FIG. 9 is a diagrammatic view which the Procrustes average
shape. The clouds are landmarks from the aligned set of shape
instances.
[0025] FIGS. 10(a) and 10(l) show some training shapes of the lung
region selected from a database.
[0026] FIG. 11(a) is a diagrammatic view which shows the effects of
varying the first parameter of the left lung shape model by two
standard deviations.
[0027] FIG. 11(b) is a diagrammatic view which shows the effects of
varying the second parameter of the left lung shape model by two
standard deviations.
[0028] FIG. 11(c) is a diagrammatic view which shows the effects of
varying the first parameter of the right lung shape model by two
standard deviations.
[0029] FIG. 11(d) is a diagrammatic view which shows the effects of
varying the second parameter of the right lung shape model by two
standard deviations.
[0030] FIG. 12 is a block diagram of a radiographic imaging system
incorporating the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0031] The present invention relates in general to the processing
of chest radiographic images. FIG. 12 is a block diagram of a
radiographic system incorporating the present invention. As shown a
radiographic image, such as a chest radiographic image is acquired
by an image acquisition system 1600. Image acquisition system 1600
can include one of the following. (1) A conventional radiographic
film/screen system in which a body part (chest) of a patient is
exposed to x-radiation from an x-ray source and a radiographic
image is formed in the radiographic image is formed in the
radiographic film. The film is developed and digitized to produce a
digital radiographic image. (2) A computed radiography system in
which the radiographic image of the patient's body part is formed
in a storage phosphor plate. The storage phosphor plate is scanned
to produce a digital radiographic image. The storage phosphor plate
is erased and reused. (3) A direct digital radiography system in
which the radiographic image of the patient's body part is formed
directly in a direct digital device which directly produces a
digital radiographic image.
[0032] The digital radiographic image is processed according to the
present invention by image processing system 1602. System 1602 is
preferably a digital computer or digital microprocessor by can
include hardware and firmware to carry out the various image
processing operations.
[0033] The processed digital radiographic image is provided to
image output 1604, such as a high resolution electronic display or
a printer which produces a hard copy (film) of the processed
radiographic image. The original as well as the processed image can
be transmitted to a remote location, can be stored in a
radiographic image storage system (PACS), etc.
[0034] The present invention discloses a method for automatically
constructing 2D statistical shape models for lung regions, which is
based on the combination of three processing steps as shown in FIG.
1. First, a digital radiographic image of chest regions of a
patient is provided (box 9) for digital image processing. Then a
manual contour tracing is first performed to extract the lung
region contours from the chest radiographs (box 10). Later a shape
alignment algorithm is used to align all shape instances as closely
as possible to a pre-defined template shape (box 11). Finally, a
statistical shape model is generated by principle component
analysis using the aligned shape instances (box 12).
[0035] The most difficult issue in the alignment is the one-to-one
correspondence between different shape instances. The present
invention provides an efficient method to achieve this goal by
first searching a set of landmarks related to the shape features
along the shape contour, and then filling the segments between them
with a fixed number of equidistant landmarks. The method includes
two stages, as shown in FIG. 2. In the first stage (the coarse
shape alignment), a template shape is selected (box 21), and for
each shape instance, a scale, rotation and translation are computed
based on the L.sub.2 distance between the turning functions of the
two polygons, which are used to approximate the template shape and
the shape instance. In the second stage of the process (detailed
shape alignment), sets of corresponding points are defined and a
least-squares type (Procrustes) distance is computed for a more
detailed shape alignment.
[0036] In the present invention, a polygonal shape approximation is
computed to simplify the representation of a shape (box 22) and a
turning function .theta.(s) is defined to measure the angle of the
counter clockwise tangent from a reference point O on the shape
approximation (box 23). The reference point orientation .theta.(O)
is associated with the image coordinates (such as the x-axis).
.theta.(s), as a function of the arc-length s, keeps track of the
turning that takes place, increasing with left-hand turns and
decreasing with right-hand turns, as shown in FIGS. 4(a) and 4(b).
To ensure generality, the perimeter length of each polygon is
normalized. Thus for a simple closed contour, .theta.(s) starts at
.theta.(O) (assuming that the reference point O is placed at
differential point along the contour) and increases to
.theta.(1)=.theta.(O)+2.pi.. The function .theta.(s) has several
properties that make it especially suitable for shape alignment. It
is piecewise constant for polygons, making computations
particularly easy and fast. According to the definition, the
function .theta.(s) is invariant under translation and scaling to
the polygon. Rotation of the polygon corresponds to a simple shift
of .theta.(s) in the .theta. direction (the vertical direction),
while changing the location of the reference point O by an amount
t.di-elect cons.[0,1] along the perimeter of polygon corresponds to
a horizontal shift of the function .theta.(s).
[0037] In the implementation of coarse shape alignment, the method
chooses one shape instance as the template shape T, whose size is
close to the mean size of all shape instances. Then two polygonal
shape approximations T.sub.p and S.sub.p are computed from the
template shape 31 and a shape instances 32, respectively, as shown
in FIGS. 3(a) and 3(b). The degree to which T.sub.p and S.sub.p are
similar can be measured by taking the minimal L.sub.2 distance
between the turning functions .theta..sub.Tp(s) and
.theta..sub.Sp(s), as defined by D 2 T p , S p .function. ( t ,
.theta. ) = ( min .theta. .di-elect cons. , t .di-elect cons. [ 0 ,
1 ] .times. .intg. 0 1 .times. .theta. T p .function. ( s + t ) -
.theta. S p .function. ( s ) + .theta. 2 .times. .times. d s ) 1 2
( 1 ) ##EQU1##
[0038] where t represents the position of the reference point along
the polygon, and .theta. corresponds to the rotation of polygon.
Based on the proofs given by Arkin et al. (E. M. Arkin, L. P. Chew,
D. P. Huttenlocher, K. Kedem, and J. S. Mitcheel, An efficiently
computable Metric for Comparing Polygonal shapes. IEEE Trans. On
Pattern Analysis and Machine Intelligence. vol. 13, no. 3, 1991,
pp. 209-215), this problem can be solved by D 2 T p , S p
.function. ( t , .theta. ) = { min t .di-elect cons. [ 0 , 1 ]
.times. [ .intg. 0 1 .times. ( .theta. T p .function. ( s + t ) -
.theta. S p .function. ( s ) ) 2 .times. d s - ( .theta. *
.function. ( t ) ) 2 ] } 1 2 ( 2 ) ##EQU2##
[0039] Where .theta.* is the optimal orientation for any fixed t
which is given by
.theta.*=.intg..sub.0.sup.1.theta..sub.S.sub.p(s)ds-.intg..sub.-
0.sup.1.theta..sub.T.sub.p(s)ds-2.pi.t (3)
[0040] By solving the above equations, two matrices are obtained
(box 24). One is D.sub.2 matrix from Eq. (2) and the other is
.theta..sub.c matrix from Eq. (3). The correct orientation of the
shape instance can be found by searching the minimal L.sub.2
distance in D.sub.2 matrix and the corresponding element in
.theta..sub.c matrix (box 25). As for the other two parameters, the
scale is simply determined from the perimeters of two shapes.
s.sub.c=P.sub.Sp/P.sub.Tp (4)
[0041] The translation can be calculated from the gravity centers
of two shapes t.sub.c=t.sub.Tp-t.sub.Sp (5)
[0042] Once the coarse shape alignment is done (box 26), the shape
instance 52 is well aligned with the template shape 51, as shown in
FIG. 5. However, this result still leaves a space for a more
accurate alignment. Thus, in the second stage, the aim is to
improve the alignment by minimizing the Procrustes distance between
the template shape contour and the shape instance contour.
[0043] To compute the Procrustes distance, a crucial requirement is
to correctly define point correspondence between the template shape
and shape instance, which can be easily achieved after the coarse
shape alignment. The idea is that, first of all, some landmarks
related to the shape features are located along the shape contour.
For example, in FIG. 6, the landmark O on the left lung contour 61
and the landmarks O1 and O2 on the right lung contour 62. Then, a
fixed number of equidistance points are filled in each segment
between the located landmarks (box 27). Finally a list of vertices
is constructed where each vertex represents a landmark point and
the index order is correspond to the counter-clockwise direction
along the contour. This last step is very important since it
ensures that all elements with the same index represent
corresponding shape information.
[0044] Given the template shape vector X.sub.T (63) and the shape
instance vector X.sub.S, (64) as shown respectively in FIGS. 7(a)
and 7(b), an appropriate rotation .theta..sub.d, a scale s.sub.d
and a translation t.sub.d=(t.sub.dx,t.sub.dy) are chosen (box 28)
and mapped onto M(X.sub.S)+t.sub.d to minimize the weighted sum
(box 29). E = ( X T - M .function. ( s d , .theta. d ) .times. X s
- t d ) T .times. W .function. ( X T - M .times. ( s d , .theta. d
) .times. X s - t d ) .times. .times. Where ( 6 ) M .function. ( s
d , .theta. d ) = [ s dx 0 0 s dy ] .function. [ cos .times.
.times. .theta. - sin .times. .times. .theta. sin .times. .times.
.theta. cos .times. .times. .theta. ] ( 7 ) ##EQU3##
[0045] W is a diagonal matrix of weights for each landmark. In the
present invention, the weights are chosen to give more significance
to those landmarks related to anatomical structures. FIG. 8 shows
the final alignment result of the template shape 81 and the shape
instance 82 after the detailed alignment.
[0046] After the shape alignment, there is a one-to-one
correspondence between the vector elements of a given index, which
ensures the vector element represents corresponding shape
information. By taking the average of the coordinates of the
corresponding vertices, a mean shape can be generated for left lung
91 and right lung 92, as shown in FIG. 9, and the shape model
variation can be also analyzed by applying a principal component
analysis on the training data. Each computed principal component
gives a `model of variation`, a way in which the landmark points
tend to move together as the shape varies.
[0047] For the 2D lung shape models in the present invention, there
are N landmarks on the shape contour. So a 2N*2N covariance matrix
S is calculated by using S = 1 M .times. i = 1 M .times. ( x i - x
_ ) .times. ( x i - x _ ) T ( 8 ) ##EQU4##
[0048] Where x.sub.i is a shape instance, {overscore (x)} is the
mean shape. M is the total number of the shape instances.
[0049] One particularly useful property of this matrix is that it
can demonstrate variation in some directions more than in others.
These variations' directions and importance may be derived from an
eigen-decomposition of S by solving the equation
S.sub.p.sub.i=.lamda..sub.ip.sub.i (9)
[0050] Solutions to Eq. (9) provide the eigenvector p.sub.i and
their eigenvalues .lamda..sub.i of S. Conventionally, these
eigenvalues are sorted in the decreasing order. It can be shown
that the eigenvectors associated with larger eigenvalues correspond
to the directions of larger variation in the underlying training
data.
[0051] Note that any shape in the training set can be approximated
using the mean shape and a weighted sum of these deviations
obtained from the first t modes. x.about.{overscore
(x)}+P.sub.tb.sub.t (10)
[0052] Where b=(b.sub.1,b.sub.2, b.sub.3, . . . b.sub.t) is the
vector of weights, which indicates how much variation is exhibited
with respect to each of the eigenvectors.
[0053] The present invention has been used to generate 2D
statistical shape models of lung regions from 65 training contours.
FIGS. 10(a)-10(l) show some training shapes of the lung region
selected from a database. Each shape contour is created by a user
delineating the lung region boarders.
[0054] FIGS. 11(a) and 10(b) show the shape variation by varying
the first two model parameters. In particular, FIG. 11(a) shows the
effects of varying the first parameter of the left lung shape
model. FIG. 11(b) shows the effects of varying the second parameter
of the left lung shape model. FIG. 11(c) shows the effects of
varying the first parameter of the right lung shape model. FIG.
11(d) shows the effects of varying the second parameter of the
right lung shape model.
[0055] In summary, a method for automatically constructing a 2D
statistical shape model for lung region in chest radiograph is
provided. Given a set of lung region shape instances, the method
generated the mean shape of lung region by automatically aligning
the training shape instances, selecting landmarks, and finally
deriving a statistical model by principle component analysis. This
method has been successfully applied to a set of 65 lung region
shape data sets. As expected, a large portion of total shape
variability is captured with the first few eigenvectors. The
present method can also be used to construct the shape models of
other anatomical structures, such as bones and organs.
[0056] The invention has been described in detail with particular
reference to certain preferred embodiments thereof, but it will be
understood that variations and modifications can be effected within
the spirit and scope of the invention.
PARTS LIST
[0057] 9 radiographic image [0058] 10 contour extraction [0059] 11
contour shape alignment [0060] 12 principle component analysis
[0061] 21 template shape [0062] 22 polygon approximation [0063] 23
shape approximation [0064] 24 multiple matrices [0065] 25
corresponding element [0066] 26 coarse shape alignment [0067] 27
located landmarks [0068] 28 translation chosen [0069] 29 minimize
the weighted sum [0070] 31 template shape [0071] 32 shape instances
[0072] 51 template shape [0073] 52 shape instance [0074] 61 left
lung contour [0075] 62 right lung contour [0076] 63 shape vector
[0077] 64 shape instance vector [0078] 81 template shape [0079] 82
shape instance [0080] 91 left lung [0081] 91 right lung [0082] 1600
image acquisition system [0083] 1602 image processing system [0084]
1604 image output
* * * * *