U.S. patent application number 11/344757 was filed with the patent office on 2006-07-06 for devices and methods to image objects by time delay integration.
Invention is credited to Greve Jan, Federik Schreuder, Leon W.M.M. Terstappen, Arjan G.J. Tibbe.
Application Number | 20060147901 11/344757 |
Document ID | / |
Family ID | 33552457 |
Filed Date | 2006-07-06 |
United States Patent
Application |
20060147901 |
Kind Code |
A1 |
Jan; Greve ; et al. |
July 6, 2006 |
Devices and methods to image objects by time delay integration
Abstract
Devices and methods for automated collection and image analysis
are disclosed that enable identification or classification of
microscopic objects aligned or deposited on surfaces. Such objects,
e.g. detectably labeled rare target cells, are magnetically or
non-magnetically immobilized and subjected to Time Delay
Integration imaging (TDI). Incorporation of TDI technology into
image cytometry analysis, like CellTracks.RTM., makes it possible
to image moving objects with very high sensitivity and
signal-to-noise ratios. Implementation of TDI camera technology
with dual excitation and multispectral imaging of enriched rare
cells provides a rapid system for detection, enumeration,
differentiation and characterization of imaged rare cells on the
basis of size, morphology and immunophenotype.
Inventors: |
Jan; Greve; (Enschede,
NL) ; Schreuder; Federik; (Enschede, NL) ;
Terstappen; Leon W.M.M.; (Huntingdon Valley, PA) ;
Tibbe; Arjan G.J.; (Deventer, NL) |
Correspondence
Address: |
IMMUNICON CORPORATION
3401 MASONS MILL ROAD
SUITE 100
HUNTINGDON VALLEY
PA
19006
US
|
Family ID: |
33552457 |
Appl. No.: |
11/344757 |
Filed: |
February 1, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10612144 |
Jul 2, 2003 |
|
|
|
11344757 |
Feb 1, 2006 |
|
|
|
Current U.S.
Class: |
435/4 ;
382/128 |
Current CPC
Class: |
G01N 33/54326
20130101 |
Class at
Publication: |
435/004 ;
382/128 |
International
Class: |
C12Q 1/00 20060101
C12Q001/00; G06K 9/00 20060101 G06K009/00 |
Claims
1. A method for analytical imaging of target entities, which method
comprises: a. obtaining a sample suspected of containing said
target entities, b. magnetically labeling said target entities with
magnetic particles that are specific for said target entities, c.
magnetically manipulating said target entities towards a collection
surface in a sample chamber, d. moving said sample chamber on a
scanning stage while illuminating said collected target entities,
e. detecting emitted light from said collected target entities, and
f. assessing characteristics of collected target entities from a
group consisting of intensity, color, morphology, and combinations
thereof.
2. The method of claim 1, further comprising: a. detecting emitted
light whereby sequential sub-images of said collected target
entities are collected, and b. re-combining said sub-images to
construct a complete image of said collected target entities.
3. The method of claim 1, in which detecting emitted light is by
TDI.
4. The method of claim 1, in which said illumination step further
comprises the use of multiple wavelength light sources
5. The method of claim 1, in which said illuminating is by
excitation with a laser.
6. The method of claim 4, in which said excitation is with a blue
and green laser.
7. The method of claim 1, in which said detecting is by fluorescent
emission.
8. The method of claim 1 wherein said emitted light is from a group
consisiting of green fluorescent emitted light, blue fluorescent
emitted light, brightfield light, and combinations thereof.
9. The method of claim 1, in which said target entities are from a
group consisting of epithelial cells, endothelial cells, tumor
cells, cell debris, and combinations thereof.
10. The method of claim 1, in which said magnetic labels are
colloidal magnetic particles specific for an antigenic site on the
target entity.
11. The method of claim 10, in which said colloidal magnetic
particles are specific for the Epithelial Cell Adhesion Molecule
(EpCAM).
12. The method of claim 1, in which said collection surface
comprises parallel Nickel lines on a glass substrate.
13. An improved apparatus for analytical imaging of target entities
having a sample chamber with a collection surface, an arragnement
of magnets cabable of manipulating magnetically labeled target
entities towards said collection surface, at least one light
source, and a computer, whereby said improvement comprises: a. a
scanning stage wit said sample chamber affixed for controlled speed
and positioning, and b. a TDI camera.
14. The apparatus of claim 13, in which said collection surface
comprises Nickel lines on a glass substrate.
15. The apparatus of claim 13, in which said light source is a
laser.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. application Ser.
No. 10/612,144, and U.S. Provisional Application No. 60/494,101,
filed 11 Aug. 2003, under 35 U.S.C. section 119 (e). The entire
disclosure of which is incorporated by reference herein.
TECHNICAL FIELD
[0002] This invention relates generally to devices and methods to
obtain to scan and obtain images of objects, and more particularly
to images reconstructed from partial sub-images of object such as
cells obtained from biological fluids that are distributed in a
two-dimensional plane. The scanning and imaging technique provided
by the invention is especially advantageous for the imaging of
cells that are aligned by magnetic means and examined by digitized
optoelectronic means through Time-Delay-Integration.
BACKGROUND OF THE INVENTION
[0003] The identification and enumeration of circulating carcinoma
cells of epithelial origin in the blood of cancer patients that may
be present at frequencies of less than one carcinoma cell per ml of
blood has been detected using magnetic enrichment and image
cytometry as in, for example, CellTrack.RTM. Systems (Immunicon
Corporation, Huntingdon Valley, Pa.). Using a combination of
epithelial cell enrichment by magnetic means in combination with
analysis by multi-parametric flow cytometry, significant
differences in the number of "circulating tumor cells" were found
between healthy individuals and patients with breast cancer (Racila
et al., Proc. Nat. Acad. Sci. 95, 45894594, 1998). In several
studies, such "circulating tumor cells" (CTC) were defined as
events expressing the following characteristics: positive for the
epithelial cell marker cytokeratin, negative for the leukocyte
marker CD45, positive staining with a nucleic acid dye, and light
scattering properties that are compatible with cells. However,
morphometric confirmations of the detected events as cells and
further molecular evidence is lacking in flow cytometric methods,
but is clearly needed to assure that the detected rare events are
indeed tumor cells derived from a primary tumor. Automated image
analysis systems have been introduced to reduce subjective errors
in cell classification between different operators in manual
methods, but such prior art systems without preliminary cell
enrichment steps still inherently lack sensitivity. Several
automated cell imaging systems have been described or are
commercially available for cell analysis. The system developed by
Chromavision, ACIS.TM. or Automated Cellular Imaging System
(Douglass et al., U.S. Pat. No. 6,151,405) uses calorimetric
pattern recognition by microscopic examination of prepared cells by
size, shape, hue and staining intensity as observed by an automated
computer controlled microscope and/or by visual examination by a
health care professional. The system uses examination of cells on
microscope slides and was designed for tissue sections. The
SlideScan.TM. or MDS.TM. systems of Applied Imaging Corp. (Saunders
et al., U.S. Pat. No. 5,432,054) is described as an automated,
intelligent microscope and imaging system that detects cells or
"objects" by color, intensity, size, pattern and shape followed by
visual identification and classification. In contrast to the ACIS
system this system has the ability to detect fluorescent labels
which provides more capability. However, these and other currently
available methodologies are not sufficiently sensitive for accurate
classification and typing of rare events such as circulating tumor
cells in blood.
[0004] Thus, an imaging system (U.S. Ser. No. 10/612,144) was
developed to detect fluorescent signals at a relatively high speed,
performing similar to standard flowcytometry. The system is based
on the separation of cells by first labeling with immunomagnetic
particles. These particles are colloidal paramagnetic paricles
linked to a monoclonal antibody specific for a characteristic cell
surface antigen found on the target cell. When sufficiently bound
to the target population of cells, a magnetic field forces the
immunomagnetic particle-cell complex to rise to the top of the
chamber where the cells align between nickel lines. When targeting
epithelial-derived cells with a monoclonal antibody to the
Epithelial Cell Adhesion molecule (EpCAM), erythrocytes and other
non-labeled cells separate from the labeled cells which rise up to
the upper glass surface of the chamber. Since this is in a known
volume, enumeration of cells can easily be determined. A red diode
laser (635 nm) is focused as an elliptical beam on the chamber,
inconjunction with the nickel lines. The reflection from the nickel
lines is used as a signal for the tracking and focusing of the
laser beam on the aligned cells. The sample chamber is moved with
the X-Y stage in the Y direction and the laser is scanning one line
at a time. The aligned cells are illuminated one by one and the
emitted fluorescence is measured with photomultipliers. The
advantages of this design are that only the target cells align on
the inner surface of the viewing portion of the chamber and the
aligned cells are easily relocated after the first analysis has
been completed. Both benefits ensure that extremely small
populations of target cells can be measured with the imaging system
described in U.S. Ser. No. 10/612,144 after magnetic enrichment.
Scatter plots from fluorescent data, generated from multi-labeling
experiments, can be used to select rare events. With the cells
aligned along the inner surface of the viewing portion of the
chamber, fluid underneath the cells can be replaced, and the cells
can be further analyzed.
[0005] Prior technology has been based upon a standard full frame
charge coupled device (CCD) camera with, for example, a frame rate
of 25 frames per second. The cell is scanned with a moving stage or
a scanning mirror. The images acquired with the camera are added
together, forming a complete image of the cell. The disadvantage is
that when scanning, the laser is only illuminating a small area of
the cell. Therefore it is necessary to scan the object (or cell) to
image the entire cell. To acquire enough photons and to prevent
bluring due to the movement of the cell it is necessaayr to scan
slowly. Another disadvantage of this type of imaging is that a full
frame camera is not collecting photons during readout of the CCD.
Consequently, all fluorescent photons emitted at that moment are
useless. The camera is closed for readout for about 16% of the
total time for a pixel area of 512.times.512 pixels at 25 fps and a
fast readout rate of 40 Mhz.
[0006] Initially used in military airborne reconnaissance and
satellite imaging, TDI technoogy allows for the capture of high
quality images of moving objects. TDI imagings devices have also
been applied in document scanning and industrial product
inspection.
[0007] A TDI imager is a standard full frame CCD imager with a
length (columns) which is much larger than its height (rows or TDI
stages). This line scanner has 96 TDI rows which results in an
enhanced sensitivity that is 96 times better than a standard line
scanner with a single row of pixels ant the same readout speed.
FIG. 1 illustrates how a moving object is interpreted with TDI
technology. S.sub.i represents the position of a moving object at
time t.sub.i. At time t.sub.i+3, the first row of TDI CCD collects
the light emitted by the moving object. The TDI shifts the
collected charge from the first row to the second row with the same
speed as the object that is moving from S.sub.i+3 to S.sub.i+4.
Thus after each shift, the accumulated charges are added to
photo-generated charges at the new site, integrating the charges
along the TDI column. This process is repeated until the end of the
sensor is reached. For example, the Dalsa Eclipse line scanner has
the number of TDI stages at 96 which means the process is repeated
96 times.
[0008] TDI imaging has the advantage of enhanced photosensitivity
and good signal-to-noise woithout affecting the output data rate.
Consequently, high scan speeds and the ability to image at low
light levels are obtained. The uniformity of pixel sensitivity is
improved with the averaging over all the TDI stages, resulting in a
more linear output. The dark current uniformity of the pixels is
improved as a consequence of the averaging over all the TDI stages.
Exposure is easily controlled through the scan speed of the object.
Further, large objects are able to be imaged with a small CCD
camera as they are moving.
[0009] Accordingly, the present invention seeks to improve upon
traditional CCD technology as related to image cytometric analysis,
such as CellSpotter.TM. Systems, and to permit detection,
enumeration and accurate classification of rare target species,
such as CTC in blood or other fluids.
SUMMARY OF THE INVENTION
[0010] This invention improves upon the devices and methods that
permit the application of novel imaging capabilities to such
systems as the CellTracks.RTM. analysis system as described by
Tibbe et al. (Nature Biotech. 17, 1210-13, 1999). The devices and
methods described by Tibbe and in the disclosure of this invention
can also be applied to other target objects. However, the primary
application is rapid immunomagnetic selection of rare cells from
blood followed by automated alignment of the isolated cells and
automated image analysis. Briefly, in a preferred embodiment of the
invention, after magnetic collection and enrichment from blood, the
magnetically labeled cells are aligned along ferromagnetic lines of
nickel (Ni) and are scanned by a laser focused by means of a
conventional objective lens such as from a compact disk player.
Since the cells have been selectively stained with one or more
fluorescent labels, the measured fluorescence emissions and the
intensities can be used to identify or classify the cell type.
[0011] No liquid flow system is required by the system of the
present invention. The magnetic fields induced by the angular
magnets in proximity of the nickel lines keep the magnetically
labeled cells in fixed positions. This allows revisiting the
detected events after measuring the fluorescence emissions and
intensities for a more extensive analysis to further identify the
detected events. One can microscopically view the images of such
events and apply independent morphometric criteria to identify the
events as actual cells.
[0012] The present invention provides a novel scanning and imaging
diagnostic system for detection, classification and enumeration of
cells, which comprises an efficient automated means for collecting
and aligning immunomagnetically labeled target cells from body
fluids, and in which such collected cells also bear at least one
immuno-specific fluorescent label that differentiates target from
non-target cells labeled with different fluorescent label(s).
[0013] In accordance with the present invention, the new TDI
scanning and imaging techniques can be integrated into a system
such as the imaging system in U.S. Ser. No. 10/612,144 to obtain
high quality fluorescence images. The discoveries described and
claimed herein have greatly improved the detection, enumeration and
classification of rare cells over systems and methods in prior art.
Efficient detection of cells at very low frequencies, so called
rare events, requires minimal sample handling to avoid losses of
cells. Furthermore, the volume from which the rare cells are
separated and enriched should be as large as possible to increase
the sensitivity of detection. With the development and application
of the disclosed novel techniques, fluorescent images of specific
events can now be obtained resulting in a highly accurate
identification, thus making the inventive system a powerful tool
for the detection of rare events in body fluids.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] FIG. 1
[0015] Object interpretation by TDI imaging. The object represented
by the circle is moving along the vertical axis. When in the sensor
area, the charge in the pixel well is coninuously increaseing with
increasing exposure and is mvoing along with the passing cell. At
time Ti+11 the end of the sensor is reached and the readout
register reads the amount of chaarge integrated over all TDI
stages.
[0016] FIG. 2
[0017] a) Scatter plot of CD45 antibody-APC/Cy7 versus CAM5.2
antibody-APC fluorescence of SKBR3 cells spiked into whole blood,
captured and aligned by EpCAM antibody-labeled magnetic
nanoparticles. Some representative sub-images of the measured
events of region 1, the SKBR3 cell region, and of the broad band
containing the debris are shown. Region 2 is the region where the
leukocytes would appear, if present, and aligned along the Ni
lines.
[0018] b) Full image of an SKBR3 cell with its corresponding
measured fluorescence signals.
[0019] FIG. 3
[0020] Schematic representatio of a typical imaging system,
disclosed in U.S. Ser. No. 10/612,144.
[0021] FIG. 4
[0022] A schematic representation of the image reconstruction
method is shown. The detected events or cells are scanned with the
laser by moving the stage that is equipped with an encoder. The CCD
camera captures the individual sub-images and stores them in
computer memory along with the corresponding encoder for
positioning the x-y coordinates of the stage. After scanning is
complete, the sub-images are combined to form a full reconstructed
image of the object by using the encoder values, which are
calculated back to the number of pixels that the subsequent
sub-images should be shifted with respect to each other. Summation
of the shifted sub-images gives the complete reconstructed
fluorescent image of the cell.
[0023] FIG. 5
[0024] a) Fluorescent signals captured when a homogeneous layer of
dye is scanned.
[0025] b) Two graphs showing the sums of the fluorescent
intensities in the x- and y-directions for the same scanned
dye.
[0026] FIG. 6
[0027] a) A graphical representation of the solid angle captured by
the objective as a function of the position in the chamber. The
numerical aperture (NA) of the compact disk (CD) objective in air
is 0.45, resulting in an effective NA of 0.34 inside the chamber on
the stage. The NA corresponds to a solid angle of 0.37 sr. The
circle represents an aligned object with a diameter of 7 microns.
The effective collecting angle of two points is indicated.
[0028] b) The graph shows the relative sum of the calculated solid
angles of FIG. 4a in the z-direction.
[0029] FIG. 7
[0030] SNR measurements for several laser powers when scanning
cells labelled with rhodamine 6G.
[0031] FIG. 8
[0032] SNR measurements at different magnfications when scanning
cells labelled with Rhodamine 6G and illuminated with a laser power
of 10 mW.
[0033] FIG. 9
[0034] Illustration of the CCD signal that is collecting charge
from the dark current, readout noise and signal photons froma
constant illumination and by a fluorescent dye. The vertical line
is indicating the time when the SNR is optimal for the emission
from a dye.
[0035] FIG. 10
[0036] Schematic representation of the Cell Tracks system with TDI.
Exemplary Dual lasers (red/green) enable several types of dyes to
be used.
[0037] FIG. 11
[0038] Laser spot illuminating a sample. The nickel lines reflect
light on the left and right sides.
[0039] FIG. 12
[0040] Numbe of pixels that overlap due to misalignment.
[0041] FIG. 13
[0042] Cone of light for illumination and collection.
[0043] FIG. 14
[0044] Scatter plot of Oxazine 750 fluorescence versus CD4-APC
fluorescence of white blood cells in whole blood captured and
aligned by CD45-labeled magnetic nanoparticles. Some representative
images of the monocyte and granulocyte regions are shown.
[0045] FIG. 15
[0046] Time resolved imaging of Oxazine 750 stained CD45 ferrofluid
captured leukocytes in whole blood utilizing the Cell Tracks
system: time span is 0 to 120 sec at 20 sec intervals.
DETAILED DESCRIPTION OF THE INVENTION
[0047] Racila et al. (Proc. Nat. Acad. Sci. 95, 4589-94, 1998)
described a method for separating breast carcinoma cells from blood
using a sequence of steps including immunomagnetic labeling
followed by immunophenotyping analysis in a flow cytometer. As
stated in U.S. Ser. No. 10/612,144, this imaging system is capable
of immunophenotyping individual cells and confirming their idenity
by providing a fluorescence image. The procedure was tested by the
detection of cultured cells of the breast cancer cell line, SKBR3,
spiked into whole blood. The spiked sample was prepared as
described in the Examples. FIG. 2a shows the scatter plot of the
APC/Cy7 channel versus the APC channel in flow cytometry. The SKBR3
cells are located in Region 1. Debris appears as the broad band and
leukocytes, if present, are located in Region 2. After measuring
the scatter plot, some of the events were imaged with the novel
imaging technique disclosed herein. After selecting an object or
cell in the scatter plot, the imaging system automatically goes
back to the measured position of the cell and the imaging routine
is started. The set of images taken from events in Region 1 show
cells with fluorescently stained cytoplasm. The nucleus of these
cells is visible as a darker region. The images are different for
debris, which is not located in Region 1. FIG. 2b shows an image of
an SKBR3 cell with its corresponding measured fluorescence signals.
The fluorescent image and measured PMT signals correlate well.
[0048] As shown in FIG. 3, the imaging system, disclosed in U.S.
Ser. No. 10/612,144, consists of a standard monochrome surveillance
charge-coupled device (CCD) camera at a frame rate 25 Hz, having
manual gain adjustment (U.S. Ser. No. 10/612,144). Using this
arrangement with the insertion of a removable mirror into the light
beam, the fluorescent light captured by the compact disk (CD)
objective is focused by the spherical achromatic lens (f=150 mm)
onto the CCD instead of a simple pinhole. The CD objective consists
of a single aspherical lens, optimized to obtain a
diffraction-limited spot size at a wavelength of 780 nm, similar to
those used in current CD players. With a preferred numerical
aperture (NA) of the lens is 0.45 and a lens diameter of 4 mm, the
elliptical shape of the image is created by the two cylindrical
lenses that are placed at a distance slightly larger than twice
their individual focal lengths. The short axis of the elliptical
focus is set at 4 .mu.m (FWHM) which is smaller than the diameter
of a cell, limiting the focus to one cell at any one time. The
longer axis is larger than the Ni line spacing. The light focused
on the Ni lines is reflected and used for feedback control. The Ni
lines are present on a 0.5 mm thick glass substrate. Focusing the
laser light (635 nm) onto the Ni lines through the glass substrate
with the CD-objective results in a non-homogeneous laser focus.
[0049] The imaging system is based on magnetic collection methods
that does not stress or distort the cells as is commonly observed
in cytospin systems using centrifugal deposition on slides. Hence,
the magnetically aligned cells maintain their native
three-dimensional shape and volume. The device and method described
herein can also be used to obtain a confocal image of cells and,
therefore, to enhance the image quality by allowing a more accurate
determination of the 3D-distribution of the fluorescent dye inside
the cells.
[0050] A full image of a measured event, wherein a full image is
defined as a reconstructed image consisting of combined multiple
digitized sub-images, is obtained by the acquistion of positional
information. Before image acqisiton of a previously measured event,
the event must be relocated on the sample chamber. Consequently,
spatial information, such as x-y sample coordinates for each
sub-image, is needed. To obtain positional information in the
y-direction, the stage which moves both the magnets and the sample
chamber has been equipped with an encoder that has a resolution of
0.2 microns. The line number for a sub-image of a specific event is
measured to give positonal information in the x-direction. The
encoder signals, together with the line number, are stored in
memory and are coupled to the signals measured from the
photomultiplier tube (PMT) for each sub-image. In this manner, all
the measured sub-image events are associated with and indexed to an
x-y position for the sample.
[0051] To return to the position where a specific event or
sub-image was measured, the laser focus shifts by the number of
lines equal to the current line number minus the line number on
which the specific event was measured. Concurrently, the stage
moves in the y-direction to the specfic encoder position. An
alternative method for obtaining encoder signals is to add profiles
to the Ni lines to record positon. This approach further permits
the use of a significantly simpler and cheaper stage to move the
sample. It should be further noted that while Ni lines are used to
align the cells and to re-locate the cells using the line number on
which the cell is measured with its corresponding encoder value the
imaging invention described herein is not dependent on the presence
of Ni lines or magnetic lines and can be used on any surface on
which cells or other fluorescent objects of interest are present or
can be deposited. Only the encoder data in the scan direction would
be needed for reconstructing the image from the stored and grabbed
sub-images.
[0052] The intensity profile at a particular position of the laser
focus is non-homogeneous and, since the diameter of a cell is
typically between 5 and 20 microns, it is also smaller than the
cell diameter. Therefore, uniform illumination with a laser focus
that is smaller than the cell diameter and has a non-homogeneous
intensity profile is obtained by scanning the laser across the cell
surface with the movement of an optical component in the beam, as
is done in a laser scanning microscope (Corle, TR, Confocal
Scanning Microscopy and Related Imaging Systems, Academic Press,
NY, 1995). In the imaging system of U.S. Ser. No. 10/612,144,
uniform illumination by this method would result in a loss of
feedback, which in turn would result in a loss of positional
information in the x-direction. The intensity profile is obtained
after adding the individual pixel intensities of the focal spot
image in the y-direction. With a Ni line spacing of 10 microns, the
summed intensity profile has an intensity variation of .+-.6%
across the line spacing. Moving a cell in the y-direction through
this focus has the result that every part of the cell has received
an almost equal illumination after it has passed through the laser
focus. This method is used to obtain a full high quality
fluorescent image of an aligned cell, based on summation of
individual sub-images.
[0053] The magnets and chamber are positioned on a stage that moves
the cells through the focus in the y-direction. To obtain a full
image of a specific cell, the laser focus is shifted to the line
where the specific sub-image event was measured and the stage is
moved in the y-direction to the corresponding encoder position with
a speed of 10 mm/sec. The stage is slowed down to a speed of 5
um/sec when the distance to the cell position is 25 microns. While
moving the stage at this low constant speed in the y-direction, the
cell is scanned by the laser focus, and the fluorescence signal for
each sub-image is captured on the CCD (FIG. 4).
[0054] A frame grabber card captures the CCD sub-images at 25 Hz
and these are stored in memory. In each subsequent sub-image, a
different part of the cell is illuminated since the laser focus in
the scan direction is smaller than the cell diameter. Together with
the sub-image capture, the encoder position is read and both are
stored in computer memory. A total of 40 microns are scanned,
corresponding to 200 captured sub-images each with 150.times.250
pixels. There is an average of a 0.2 micron shift with respect to
each other. This corresponds to a shift of 2.63 pixels on the CCD
surface. Using the encoder values, the captured sub-images are
shifted over a number of pixels corresponding to the difference in
their associated encoder values x 2.63, which are then summed or
combined. This results in a reconstructed full cell image as is
illustrated in FIG. 3. The sub-image resolution in the y-direction
is determined by the encoder resolution (0.2 microns). The
resolution in the x-direction is determined by the number of pixels
in the image recorded in the x-direction (0.07 microns/pixel). Both
resolutions are smaller than the diffraction limit. Hence, it will
be appreciated be those skilled in the art that the ultimate image
resolution will not be determined by the encoder or the camera but
by the imaging optics.
[0055] An integral component of the imaging optics of the present
invention is the homogenity of illumination throughout the image.
Thus, a profile of this homogeneity was assessed with a thin layer
of a concentrated fluorescent dye solution, placed in the same
plane of the magnetically labeled cells such that they would be
aligned between the Ni lines which are spaced 10 microns apart. The
dye layer was scanned and imaged at a speed of 5 .mu.m/sec as
previously described. The sequentially captured sub-images are
presented in FIG. 5a. For a uniform layer of fluorescent dye, one
would expect a homogeneous fluorescent image, if the illumination
were uniform. The obtained image is shown in FIG. 5b. The observed
signals, as measured in intensity units along the center trace in
the y-direction of the image, were found to vary by .+-.7%. One
explanation for this variation may be non-homogeneity of the dye
layer, which would affect the emitted and captured fluorescent
light. A second explanation is that the stage did not move with a
constant speed. The position of the stage is not synchronized with
the frame rate of the camera and the frame grabber card, but images
were grabbed at 25 Hz regardless of the speed and position of the
stage in the y-direction. If the stage moves faster than 5
microns/sec in a certain region, fewer images would be captured of
this region resulting in a lower total intensity. However, the
variation in the speed of the stage was measured and turned out to
be much smaller than the observed variation in the measured image
intensity. The apparent non-uniformity along the center trace must
therefore be due the non-homogeneity of the dye layer. The
intensity profile in the x-direction or perpendicular to the Ni
lines and scan direction is also presented in FIG. 5b. The
intensity of the sub-images in this direction express a maximum
centered between the Ni lines which falls off near the edges of the
Ni lines. The Ni lines obstruct the emitted fluorescent light
resulting in a smaller collecting angle, which in turn results in a
smaller effective NA of the objective. In FIG. 6a, the effective
solid angle detected by the objective is calculated as a function
of the position in the chamber. The simulation was performed using
a line spacing of 10 microns and a depth or layer thickness of 14
.mu.m. The graph in FIG. 6b shows the sum of the calculated solid
angle values of FIG. 5a in the z-direction, thus providing a
measure of the collected and measured intensities as a function of
the x-position. The calculated intensity profile is in agreement
with the measured intensities for the uniform dye layer. The
effective NA of objects close to the Ni lines is largely reduced
resulting in a non-uniform captured intensity in the x-direction
even though uniform illumination is used. Objects with a diameter
smaller than 7 microns, as is indicated by the circle in FIG. 6a,
are imaged without loss in intensity due to the shielding effect of
the Ni lines, since the effective NA in this region is reduced by
the Ni lines.
[0056] The preferred camera used in the present invention is a
Dalsa eclipse high sensitivity TDI linescan camera. It is connected
to a computer with a framegrabber board. The camera has 0 to 255
grey levels (8 bit) with a total of 512 horizontal pixels and 96
TDI stages in the vertical direction. The pixel size is 13 microns
and the maximum line rate is 64 kHz.
[0057] Responsivity of a CCD is dependent upon the wavelength,
expressed in Digital Numbers (DN). The Dalsa Eclipse line scanner
has a high responsivity due to the high quantium efficiency in the
visible region. This responsivity is typically 1950
DN/(nJ/cm.sup.2) for a 0 dB gain in each column for the sensor.
Responsivity also depends upon the pixel area, the Fill Factor of
each pixel, the quantum efficiency of the CCD, QE, in
electrons/photon, the number of TDI stages (96) and the electronic
gain (dB). The fill factor of the camera is 100% which means that
the entire area of the CCD sensor is photo-sensitive. The amount of
photons that are actually converted to signal electrons depends on
the quantum efficiency (QE) of the sensor. The QE efficiency is
normally in the range of 0.4 to 0.7. While the photo-response for
each pixel varies a bit, for the TDI CCD camera the photo-response
uniformity is very good compared to a standard photodiode linear
array. The photo response improves with the averaging over 96 TDI
stages and the non-uniformity reduced by a factor of 1/96. With the
same effect of averaging seen in the dark-current distribution, a
good uniformity of the pixels results in the improved linear output
of the camera.
[0058] Several noise sources are present in a CCD camera. The major
sources are the readout noise, the photon noise, and dark current
noise. These noises arise from the imaging array, the readout
register, and the output structure. Photon noise is defined as
photons incident on the CCD which are converted to photoelectrons
within the silicon layer. These photoelectrons comprise the signal
but also carry a statistical variation of fluctuations in the
photon arrival rate at a given point. Readout noise is electronic
noise which is inherent to the CCD. Readout noise refers to the
uncertainity introduced during the process of quantifying the
electronic signal on the CCD. Dark current noise is noise that
arises from the statistical variation of thermally generated
electrons. These electrons come from the CCD material through
thermal vibration. All these noise sources contribute to the
quality of the overall signal which is expressed as the signal to
noise ration (SNR).
[0059] SNR describes the quality of a measurement. Specifically, it
is the ratio of the measured signal to the overall measured noise
at a pixel. High SNR is particularly important in applications
requrieing low light measurement. Taken together, the SNR for each
pixel for a CCD camera is dependent upon the following relation:
SNR = N s .eta. q ( N s + N b ) .eta. q + I d T int + N r 2 ( 1 )
##EQU1## where T.sub.int is the total integration time (seconds),
N.sub.s is the number of signal photons hitting the CCD in a time
T.sub.int (photons), N.sub.b is the number of background photons
hitting the CCD in a time T.sub.int (photons), I.sub.d is the dark
current (electrons/pixel/second), N.sub.r is the readout noise, and
n.sub.q is the quantum efficiency.
[0060] In the imnging system without TDI, SNR is improved by
reducing the imaging scan speed, resulting in larger number of
captured sub-images of a specific event and a better signal to
noise ratio. However, no improvement in image quality for the APC
labeled SKBR3 cells is observed when the image scanning speed was
reduced. The limit of the imaging scan velocity is determined by
the photo-bleaching rate of the dye molecules. No fluorescence is
detected with the CCD camera if an SKBR3 cell is scanned for the
second time, indicating that most of the APC molecules have already
been photo-destroyed after the first scan. Reducing the scan speed
would, therefore, make no difference in the detected fluorescence
signal and would only result in increased fluorescent background.
The optimal scan speed is, therefore, dependent on the individual
dye characteristics and is different for each fluorescent dye.
Therefore, to the improve signal to noise ratio it may be better to
scan faster. On the other hand scanning faster than 5 microns/sec
when using a camera with a frame rate of 25 Hz would result in a
loss of resolution since the captured images would be spaced more
than 0.2 microns apart. A CCD camera with higher frame rate would
be needed to scan faster without losing resolution. Replacing the
standard surveillance CCD camera with a more sensitive camera would
also allow imaging of dimly stained cells.
[0061] The SNR in an imaging system with a TDI camera as claimed in
the present invention depends upon the camera, the total
integration time and the amount of signal photons hitting the CCD
during integration. These signal photons come from fluorescent dyes
that are attached to the cells. The total amount of fluorescence
signal photons is not constant for these dyes, and is compounded by
a reduced total time due to bleaching of the dyes. The result is
that at slow scanning speeds (long integration times) most of the
fluorescence gathered results in a high SNR. However, the scanning
speed can be too slow where the number of photons emitted per
second will be so small that the dark current dominates the charge
distribution which results in a lower SNR.
[0062] Consequently, the scanning speed has to be adjusted to the
particular dye used to obtain the maximum SNR. A typical situation
would be to determine the optimum scanning speed for the sample
stage in the Y-direction when a fluorescence indicator is used in
order to obtain the higest SNR with the TDI imager of the present
invention. The dependence of the SNR will be on the bleaching rate,
the laser illumination and magnification. FIG. 7 shows the
calculated SNR for rhodomine 6G with several laser powers.
Measurements are based upon parameters in Table 1. TABLE-US-00001
TABLE 1 Simulation parameters Noise Dark current (N.sub.d) 8000
e-/pixel/sec Readout noise (N.sub.r) 60 e-/pixel Back ground
photons (N.sub.b) 0 Illumination Collection efficiency 0.1 Aspect
ratio 0.3 Magnification 65 Cell/label Dye Rhodamine 6G #
fluophores/cell 10000 # Cell diameter 10 .mu.m
At a low scanning velocity, the SNR improves with increasing
velocity. The SNR also improves with higher illumination power.
Maximum SNR is obtained when the total amount of photons is
increased to more than the amount of dark current photons.
Accordingly when scanning too slowly, the SNR will decrease while
the rate of photone emitted by the fluophores will not be high
enough. When scanning to fast, smaller amounts of photons are
collected, resulting in a decreased SNR. FIG. 8 shows the
calculated SNR for several magnifications for a laser power of 10
mW. The SNR improves when the magnification is lower with the
density of photons on the CCD higher for a smaller object. FIG. 7
shows an optimum speed of approximately 1.8 mm/sec for a 65.times.
magnification and a 10 mW laser power. As the magnification is
elevated (FIG. 8), the optimum speed decreases.
[0063] As mentioned above, SNR is also a function of the
integration time. FIG. 9 depicts the amont of photons that reach
the CCD and the associated relation with SNR as a funtion of
integration time. The optimum SNR for a constant illumination is
constantly increasing and is limited by the amount of charge the
pixel can hold. For the best performance, a high illumination rate
will shorten the total collecting time and reduce the total amount
of dart current.
[0064] SNR for the imaging system with TDI would preferably scan
the object with a velocity ranging from about 1 to 2.5 mm/sec.
While not limiting the scope of the present invention, this SNR is
a calculation based upon Rhodamine 6G, and depends upon the
particular dye.
[0065] The original imaging system (U.S. Ser. No. 10/612,144) used
a red diode laser as an excitaiton source for several fluorescent
labels. An encoder was used to give positional information of the
cells and to relocate the aligned cells.
[0066] CellTracks with the TDI imager incorporates all red
excitable dyes and the emissions of these dyes, along with a second
green laser (532 nm) to extend the amount of dyes that can be used
and to have an emission spectrum that is further apart (See FIG.
10). This green laser beam is expanded 4 times with a beam
expander. With a cylindrical lens, an elliptical spot is created
which results in two focal planes after focussing with the CD
objective (FIG. 11). A dichronic mirror that reflects the green
laser light and transmits the red laser light is used to combine
the two laser lines. The green laser spot overlaps the spot of the
red laser. An achromat replaces the CD lens to obtain a focal plane
that is equal for the red and the green laser light in combination
within the beam. This allows the beams to run in parallel. A
mirror/dichroic combination is used to project the red fluorescence
at a different position on the CCD which is followed by the green
fluorescence. The fluorescent signal is then measured with the PMT
or TDI camera as in the present invention. The green fluorescence
is projected on the lafte area of the CCD and the fluorescence form
the red laser excitation is projected on the right side of the
CCD.
[0067] The magnification necessary for the image using a TDI camera
depends on the resolution of the positional encode and the camera.
The preferred encoder is a Heidenhein MT 2571 with a resolution of
0.2 microns. This encoder will give a TTL pulse every 0.2 microns,
used to trigger the TDI camera and pulse results in one TDI row
shift.
[0068] To achieve a square pixel image, the pixel size in the
y-direction equals the size of a pixel in the x-direction. After
each pulse the TDI row is moved with a coresponding distance lp=13
microns on the CCD. This results in a magnification of 13/0.2=65
for this configuration. To use a different magnification, a counter
is used. This counter gives 1 TTL pulse to the camera after
1,2,4,6, . . . pulses of the encoder. With this counter, the
following magnifications can be used: M = 13 0.2 k , k = 1 , 2 , 4
, 6 .times. .times. ( 2 ) ##EQU2##
[0069] When the encoder is used, the set-up is constrained for a
cerain magnification and it is essential that the magnification is
correct to ensure that the movement of the object and the shifting
of the TDI rows is in synchrony. A mismatch of the magnificiation
of the stage and the camera will result in a bluring of the image.
When the magnification is too large or too small the image will be
stretched in the forward or reverse direction, respectively. When
the magnification is in synchrony, the speed of the x-y stage will
vary, but will be limited by the maximum and minimum line rate of
the camera. The minimum line rate of the camera is approximately
100 Hz, corresponding to a line time of 0.01 seconds and a total
exposure time of 0.96 seconds for all TDI stages. Also if no
encoder is used, the line rate and speed of the x-y stage can have
a fixed value for a certain magnification.
[0070] The encoder sends pulses to trigger the camera for control
of the line rate. Also connected, the framegraber sends a pulse in
LVDS format to the camera for each falling edge of the encoder
signal. To verify the resolution of the encoder, the amount of
steps is counted for a period of 5 msec. Counter values are
measured with a sampling frequence of 20 KHz.
[0071] Image resolution, particularly laterally, depends on the
wavelength, focal distance, and entrance diameter, all limited by
the diffraction limit of the objective. The one objective
considered in the present invention is a Sony CD pick-up system
(KSS-210ARP). The accurator of this CD system is used with a focal
distance of f=3.8 mm, a numerical aperature of N.A.=0.45, and an
opening entrance of D=4 mm. The CD lens is designed to operate at a
wavelength of 720 nm and, therefore, is expected to produce
spherical and chromatic aberrations for other wavelengths which
will reduce the resolution.
[0072] The resolution for the encoder is 0.2 microns for M=65 and,
with the shifting CCD rows, the illumination is always spread on
two pixels, resulting in a resoluiton of 0.4 microns for the image.
This resoluton is higher than the optical limit and is limited by
the optical resolution. In the X-direction, a mismatich between the
scanning direction and the TDI column results in a decrease of
resolution. The movement of the cell should be parallel with the
TDI columns. The number of overlaping pixels increases with a
higher deviation angle (FIG. 12). In the Y-direction, a mismatch of
the magnification will also cause a reduced resolution with a
blurring effect due to overlapping pixels.
[0073] The collection and illumination efficiency is determined by
the light gathering power (N.A.) of the lens. In the present
invention, the CD-lens is replaced with the achromat lens in the
pickup accurator. The achromat lens has a focal distance of 7.5 mm
and an entrance opening of 6.5 mm. The N.A. is approximately 0.27.
The solid angle (FIG. 13) is 0.21 with the achromat lens, replacing
the solid angle of 0.72 for the CD lens. The achromat is also used
for the green laser inorder to have the same folcal distance.
However, the solid angle of the achromat is much smaller than the
solid angle in the CD lens. The solid angle is proportional to the
intensity of the collected light, making the CD lens more efficient
than the achromat. Although the achromat lens is limiting the
sensitivity (by approximately a factor of 15 for both illumination
and collection of photons), it is more convenient to implement with
a second laser, making tracking and focusing of the system
possible. An achromat with a higher N.A. would be preferable, with
a numerical aperature (N.A.>0.5). This results in a higher SNR
and a higher resolution. The present invention is also considered
with the use of a microscope objective, although this type of
objective would mandate changing the tracking and focusing
mechanism.
[0074] For imaging and spectral separation of the fluorescent
emission, a prism or combination of filters can be used. Instead of
using a dichroic mirror combination, a prism is inserted in front
of the CCD. With a direct vision prism, the deflection angle
depends on the wavelength and the deflection angle is almost
parallel with the incomign ray of light. To obtain an image where
the fluorescence is separated with no spectral overlap, the
spectral separation must be large enough to separate more than the
maximum cell size. For example, with a maximum cell size of 20
microns the emission of APC and PE is about 100 nm apart and the
bandwidth of emission is approximately 10 nm, resulting in a pixel
resolution of 2 microns in the X-direction. Thus, dyes with smaller
emission spectrums are required for optical resolutions of 0.7 to
around 1 micron. A dichroic/mirror combination can also be used
whereby a high pass filter and an almost parallel mirror can be
used to separate the fluorescent emission into two disticnt areas.
For example, the emission of PE excited with the green laser is
reflected on the filter surface while the emission of APC is
transmitted by the filter and reflected on the mirror.
[0075] Implementation of TDI imaging into the imaging systems like
CellTracks provides a method with good image resolution and an
improved signal to noise compared to the analysis with the scanning
mirror and the full frame CCD. The scan speed is also increased
with the time to image one line under TDI imaging around 10 sec (at
a speed of 1.5 mm/sec). Previous methods would only measure 1 cell
within this time period. With the dual excitation laser, it is
possible to image the fluorescent emission with more than one dye.
Dyes that have the same absorption spectrums but different emission
spectrums have an emission that is close together, demanding a
narow bandwidth of the filters to image the emission of these
dyes.
[0076] It will be apparent to those skilled in the art that the
improved scanning and imaging system of the invention is not to be
limited by the foregoing descriptions of preferred embodiments, and
that the preferred embodiments of the invention which incorporate
these improvements, as previously described, have also been found
to enable the invention to be employed in many fields and
applications to diagnosis of cells and to particulate target
species in general. The following Examples illustrate specific of
the invention, but are not thereby limited in scope.
EXAMPLE 1
[0077] For these experiments, 10 .mu.l of fixed SKBR3 cells (50,000
cells/ml) were mixed with 290 .mu.l of EDTA blood. Also added at
the same time were 100 .mu.l of magnetic ferrofluid coated with
anti-EpCAM (magnetic particles of about 200 nm size coated with
proteins, streptavidin and biotinylated EpCAM antibody), an
antibody specific for epithelial cells and known to be present on
SKBR3 cells (cultured at Immunicon Corp., Huntingdon Valley, Pa.),
10 .mu.l of allophycocyanin (APC) conjugated to monoclonal
antibodies recognizing anti-cytokeratin species or cytoskeletal
proteins present in epithelial cells (e.g. SKBR3 cells that are
epithelium derived) and 10 .mu.l CD45-APC/Cy7 (Caltag, Burlingame,
Calif.) to identify leukocytes and identify leukocytes that may
nonspecifically bind to cytokeratin antibody. After 15 minutes
incubation, 50 .mu.l of this blood reaction mixture was injected
into the chamber. The chamber was placed in the Cell Tracks magnet
assembly and after two minutes collection time, the feedback system
was switched on and the measurements were started. In a single
measurement, 40 lines with aligned cells, each 15 mm in length and
with a line period of 30 .mu.m were scanned. At a chamber height of
0.5 mm, the scanned volume represents 9 .mu.l. The results of
scanning the collected labeled SKBR3 cells with the corresponding
measured immuno-fluorescent signals are shown in FIGS. 2a and 2b.
FIG. 2a shows a scatter plot of CD45-APC/Cy7 dye versus CAM5.2
antibody-APC fluorescence of SKBR3 cells in whole blood, captured
and aligned by EpCAM-labeled magnetic nanoparticles. Some
representative images of the measured events of Region 1, the SKBR3
cell region, and of the broad band containing the debris are shown.
Region 2 is the region where the leukocytes would appear, if
present and aligned along the Ni lines. FIG. 2b shows an image of
one SKBR3 cell with its corresponding measured fluorescence
signals.
EXAMPLE 2
[0078] In this experiment 100 .mu.l of EDTA anti-coagulated blood,
50 .mu.l of ferrofluid containing 5 .mu.g of CD45-labeled
ferromagnetic nanoparticles, 1.5 .mu.l CD4-APC and 25 .mu.l of
10.sup.-5 M Oxazine 750 perchlorate (Exciton Inc., Dayton, Ohio)
were added. The optimum concentration of the reagents was obtained
by serial titration of each of the reagents. After incubation for
15 minutes, 300 .mu.l PBS was added and 50 .mu.l of the blood
mixture was placed into the capillary that was already placed
between the magnets. The capillary has a glass bottom shaped in a
way that it fits between the 70.degree. tilted faces of the
magnets. Two strips of double-sided tape with a thickness of 0.5 mm
(3M Co., St. Paul, Minn.) were placed on the glass with spacing of
3 mm to form the sidewalls of the capillary. Ni lines, about 30
.mu.m wide and about 0.2 .mu.m thick, were produced by standard
photolithographic techniques on a 7740 Pyrex.RTM. glass wafer
(Corning International, Germany). Wafers were cut in pieces of 4
mm.times.25 mm and these were placed, with the Ni lines facing the
bottom, on the double sided tape to form the top of the capillary.
The inner dimensions of the capillary are height=0.5 mm, length=25
mm, width=3 mm. In the measurement presented here the scan speed in
the y-direction was 4 mm/sec, the chamber was scanned over 15 mm
and 40 lines were scanned, resulting in a measuring time of two and
a half minutes. Since the period of the lines is 30 .mu.m, the
surface scanned is 18 mm.sup.2. As the height of the chamber was
0.5 mm, the scanned volume is 9 .mu.l. For the differential white
blood cell count, the addition of reagents resulted in a dilution
factor of 4.77. To shorten the time that the cells need to align
between lines and to assure that even the weakly magnetic labeled
cells would be attracted to the upper surface, the capillary
together with the magnets was placed upside down after the blood
was placed into the capillary. After two minutes the capillary with
the magnets was inverted again and, after approximately one minute,
the feedback system was switched on and the measurement was
started. To separate the emission spectra, a 660df32 band-pass
filter for the APC fluorescence and a 730df100 band-pass filter
(both filters from Omega Optical Co., Brattleboro, Vt.) for the
Oxazine 750 were used. As the fluorescence intensity of Oxazine 750
stained cells is significantly greater than that of
immuno-fluorescent CD4-APC labeled cells, compensation of the
spectral overlap is required. A typical example of the scatter plot
obtained after compensation is shown in FIG. 14. Four populations
are clearly visible and were identified as CD4+ lymphocytes, CD4+
monocytes, CD4- lymphocytes and neutrophilic granulocytes. The gate
settings illustrated in the figure were used to determine the
number of events in each gate. Total number of leukocytes measured
was 12,350 and the measuring time was 2.5 minutes. To examine the
distribution of the fluorescence from the detected objects,
software was written to allow the user to point at the object of
interest in the scatter plot. The system then moved to the location
of this event and an image was taken. Surprisingly, the images
clearly demonstrated that the fluorescence obtained from the
Oxazine 750 staining was not derived from the nucleus but from the
granules (Shapiro HM, Stephens S: Flow cytometry of DNA Content
Using Oxazine 750 or Related Laser Dyes With 633 nm Excitation.
Cytometry 1986; 7: 107-110). Six images obtained from the events in
the granulocyte gate and two images from events in the monocyte
gate are shown.
EXAMPLE 3
[0079] The experiment described in example 2 was repeated but time
resolved images were taken with the CellTracks imaging system from
Oxazine 750 stained and CD45 ferrofluid captured leukocytes in
whole blood. FIG. 15 shows four examples of images taken at 20
seconds intervals. The distribution of the fluorescence within the
cells is clearly changing between the time intervals and different
cells behave differently as is obvious from the cells followed in
frame 3 and 4. In both frames images from two cells in close
proximity are taken and the differences in uptake and cellular
distribution of the Oxazine 750 are apparent. From these examples
it is obvious that this imaging system has a unique capability to
perform functional analysis of cells as, for example, one can study
the responses of cells in blood to drugs or other components in
real time.
EXAMPLE 4
[0080] TDI incorporation in the CellTracks imaging system was
tested with fluorescent beads and HELA tumor cells. A non-cooled
Dalsa Eclipse camera with a 96 CCD surface was used. The stage,
moving with the sample chamber, was equipped with an encoder that
has a resolution of 0.2 microns. The TDI rows shifted on every
encoder pulse which guaranteed a one to one relationship between a
moving cell and its image onto the CCD. HELA tumor cells were
labeled with anti-cytokeratine-PE and the nucleus stained with
DAPI. A brightfield image was made with 880 nm IR-LED. TDI imaging
resulted in rapid imaging of cells with good SNR at an optimum
scanning speed.
[0081] Although the present invention has been described with
reference to specific embodiments, workers skilled in the art will
recognize that many variations may be made therefrom, for example
in the particular experimental conditions herein described, and it
is to be understood and appreciated that the disclosures in
accordance with the invention show only some preferred embodiments
and objects and advantages of the invention without departing from
the broader scope and spirit of the invention. It is to be
understood and appreciated that these discoveries in accordance
with this invention are only those which are illustrative of the
many additional potential applications of the apparatus and methods
that may be envisioned by one of ordinary skill in the art, and
thus are not in any way intended to be limiting of the invention.
Accordingly, other objects and advantages of the invention will be
apparent to those skilled in the art from the following detailed
description, together with the appended claims.
* * * * *