U.S. patent application number 10/911996 was filed with the patent office on 2005-03-24 for methods of monitoring effects of chemical agents on a sample.
This patent application is currently assigned to MediSpectra, Inc.. Invention is credited to Flewelling, Ross, Hed, Ze'ev, Kaufman, Howard, Schmid, Philippe, Zelenchuk, Alex.
Application Number | 20050064602 10/911996 |
Document ID | / |
Family ID | 26866607 |
Filed Date | 2005-03-24 |
United States Patent
Application |
20050064602 |
Kind Code |
A1 |
Kaufman, Howard ; et
al. |
March 24, 2005 |
Methods of monitoring effects of chemical agents on a sample
Abstract
The invention provides methods and systems for monitoring
effects of chemical agents on optical signals produced by samples
in response to the chemical agents. Preferred methods comprise
application of multiple chemical agents that interact to alter an
optical signal from the sample. Methods and systems of the
invention also comprise monitoring an optical signal from an
endogenous chromophore upon application of a chemical agent to a
sample. Methods and systems of the invention also comprise the use
of triggers, atomizers and image alignment to enhance the results
of methods described herein.
Inventors: |
Kaufman, Howard; (Newton,
MA) ; Zelenchuk, Alex; (Stoughton, MA) ;
Flewelling, Ross; (Chelmsford, MA) ; Schmid,
Philippe; (Lausanne, CH) ; Hed, Ze'ev;
(Nashua, NH) |
Correspondence
Address: |
TESTA, HURWITZ & THIBEAULT, LLP
HIGH STREET TOWER
125 HIGH STREET
BOSTON
MA
02110
US
|
Assignee: |
MediSpectra, Inc.
Lexington
MA
|
Family ID: |
26866607 |
Appl. No.: |
10/911996 |
Filed: |
August 5, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10911996 |
Aug 5, 2004 |
|
|
|
09738614 |
Dec 15, 2000 |
|
|
|
60170972 |
Dec 15, 1999 |
|
|
|
Current U.S.
Class: |
436/164 |
Current CPC
Class: |
A61B 5/0071 20130101;
A61B 5/0084 20130101; A61B 1/00142 20130101; A61B 5/0075 20130101;
A61B 1/0008 20130101; A61B 1/00009 20130101; A61M 11/00
20130101 |
Class at
Publication: |
436/164 |
International
Class: |
G01N 021/00 |
Claims
What is claimed is:
1-32. (canceled).
33 A method for observing effects of a chemical agent on a tissue
sample, the method comprising the steps of: dispensing a chemical
agent on a tissue sample; initiating a measurement period relative
to the dispensing step; compensating for motion of the sample
during the measurement period; and measuring at least one optical
signal from the sample within the measurement period.
34 The method of claim 33, wherein the compensating step comprises
capturing a plurality of sequential images of the tissue sample
during the measurement period and aligning a subset of the
plurality of images.
35 The method of claim 33, wherein the initiating step comprises
providing an automated triggering signal.
36 The method of claim 33, wherein the at least one optical signal
comprises at least one of fluorescent light and backscatter
light.
37 The method of claim 33, wherein the at least one optical signal
comprises both fluorescent light and backscatter light.
38 The method of claim 33, wherein the measuring step comprises
determining at least one of a fluorescence spectrum and a
reflectance spectrum.
39 The method of claim 33, wherein the chemical agent comprises
acetic acid.
40 The method of claim 33, wherein the chemical agent comprises at
least one member of the group consisting of acetic acid, formic
acid, propionic acid, butyric acid, Lugol's iodine, Shiller's
iodine, methylene blue, toluidine blue, osmotic agents, ionic
agents, and indigo carmine.
41 The method of claim 33, wherein the tissue sample comprises
cervical tissue.
42 The method of claim 33, wherein the tissue sample comprises at
least one member selected from the group consisting of cervical
tissue, colorectal tissue, skin, uterine tissue, and epithelial
tissue.
43 The method of claim 33, wherein the tissue sample is in
vivo.
44 The method of claim 33, further comprising the step of:
identifying a classification of the tissue based at least in part
on the at least one optical signal.
45 The method of claim 44, wherein the classification comprises a
member of the group consisting of Normal, CIN I, Normal/CIN I, CIN
II, CIN m, and CIN II/III.
46 The method of claim 44, further comprising the step of:
representing the classification on a display.
47 The method of claim 46, wherein the representing step comprises
superimposing a color on an image of the tissue sample.
48 The method of claim 46, further comprising the step of:
determining an area of the tissue sample for biopsy.
49 An apparatus for observing effects of a chemical agent on a
tissue sample, the apparatus comprising: a timer adapted to track a
measurement period relative to a dispensing of a chemical agent on
a tissue sample; and a probe comprising probe optics adapted to
illuminate the tissue sample and to collect light from the tissue
sample within at least part of the measurement period.
50 The apparatus of claim 49, further comprising a camera adapted
to obtain a plurality of images for compensating for motion of the
tissue sample during the measurement period.
51 The apparatus of claim 49, wherein the probe optics comprise at
least one light source.
52 The apparatus of claim 51, wherein the at least one light source
comprises at least one of a UV light source and a white light
source.
53 The apparatus of claim 52, wherein the at least one light source
comprises a UV light source and a white light source.
54 The apparatus of claim 51, wherein the at least one light source
comprises at least one member of the group consisting of a nitrogen
laser, a frequency-tripled Nd:YAG laser, and a xenon flash
lamp.
55 The apparatus of claim 49, further comprising a dispenser
adapted to dispense the chemical agent on the tissue sample.
56 The apparatus of claim 55, wherein the dispenser comprises an
atomizer.
57 The apparatus of claim 49, further comprising a disposable
sheath configured to cover at least a portion of the probe.
58 The apparatus of claim 49, further comprising a detector from
which a spectral result is obtained.
59 The apparatus of claim 49, wherein the probe optics are adapted
to collect at least one of fluorescent light and backscatter light
from the tissue sample.
60 The apparatus of claim 49, wherein the probe optics are adapted
to collect fluorescent light and backscatter light from the tissue
sample.
61 The apparatus of claim 49, further comprising a spectrometer
adapted to obtain a spectral result from the collected light.
62 The apparatus of claim 49, further comprising a colposcope
adapted to provide an image of the tissue sample.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to and the benefit of U.S.
provisional patent application Ser. No. 60/170,972, filed Dec. 15,
1999, the disclosure of which application is hereby incorporated by
reference.
FIELD OF THE INVENTION
[0002] This invention relates generally to spectral analysis. More
particularly, in one embodiment, the invention relates to
determining chemically-induced changes of optical spectra.
BACKGROUND OF THE INVENTION
[0003] Direct visual observation alone is often inadequate for
identification of abnormalities in a specimen being examined,
whether the specimen is a biological specimen or otherwise.
Observation of many medical conditions in biological specimens of
all kinds is well known. It is common in medical examination to
perform visual examinations in disease diagnosis. For example,
visual examination of the cervix can discern areas where there is a
"suspicion" of pathology. In some instances, filters can be used to
improve visual differentiation of normal and abnormal tissues. In
other situations, when tissues of the cervix are examined in vivo,
chemical agents such as acetic acid can be applied to enhance the
differences in appearance between normal and pathological areas.
These techniques form an integral part of a colposcopic examination
of the cervix. Colposcopists may amplify the difference between
normal and cancerous tissue with the application of various
"activation" agents, the most common being acetic acid, at
approximately 3% to 5% concentration, or an iodine solution, such
as Lugol's iodine or Shiller's iodine. Even when the cervical
tissues are viewed through a colposcope by an experienced
practitioner with the application of acetic acid, correct diagnosis
can be affected by subjective analysis. A variety of methods using
optical techniques have been directed towards the diagnosis of
cancer and other pathologies, particularly involving the cervix.
Certain of these systems and methods have limitations that render
them unsuitable for use as screening procedures.
[0004] While there have been extensive developments in the field of
cancer diagnosis, none of these are well adapted for screening
large populations. Currently, disease diagnoses are made
predominately from pathological examinations of biopsied tissue.
Techniques such as biopsies, while being the definitive
determination of the presence of disease, are labor-intensive and
operator-dependent, thus unsuitable for screening large
populations. As another example, medical imaging techniques,
depending on their cost, resource requirements and patient
accessibility, may be unsuitable for population screening.
[0005] To be well accepted in the medical community, a screening
method should be sufficiently sensitive and specific to identify
abnormalities accurately. Furthermore, a screening method ideally
is easy to perform so that it can be carried out rapidly on an
otherwise healthy patient. In. addition, to be cost effective the
screening method should not require the use of expensive resources,
including a significant time commitment from costly, highly trained
medical personnel. Generally, screening settings advantageously
employ less skilled operators and more operator-independent
technology.
SUMMARY OF THE INVENTION
[0006] The invention provides systems and methods for quickly and
efficiently screening samples, especially biological samples.
According to the invention, changes in the spectral properties of
tissues upon exposure to chemical agents are characteristic of the
physiological state of the tissue. In particular, the invention
relates to changes in spectral properties of a sample in response
to chemical treatment. The sample can be a sample of tissue, and
the response can be indicative of a state of health of the tissue
or the patient from whom the sample is obtained. Upon exposure to
chemical agents, the light emission properties of a sample change.
In the case of a sample of tissue, the temporal evolution of these
changes is characteristic of the state of health of the tissue
generally. When exposed to light, tissues emit light having
spectral properties that are characteristic of the physiological
and biochemical make-up of the tissue. When exposed to a chemical
agent, such as a contrast agent, the spectral properties of the
tissue are changed by the interaction of the agent with endogenous
molecules in the tissue. As the chemical agent diffuses out of the
area of application, or otherwise becomes less abundant in the
tissue, the emission spectrum of the tissue returns to pre-exposure
levels. According to the invention, changes in tissue produced by
endogenous chemical agents provide insight into the sample, such as
the clinical health of the tissue as described in detail below. The
invention also involves systems and methods of performing the
application of one or more chemical agents, including the amount of
material dispensed, dispensing patterns, and triggering a
measurement relative to the time of dispensing.
[0007] Accordingly, the invention provides methods and systems for
monitoring effects of chemical agents on a sample by exposing a
sample to one or more chemical agents, and measuring a change in an
optical signal from the sample. A preferred method of the invention
comprises dispensing a plurality of chemical agents on a sample,
wherein the agents interact to alter an optical signal from the
sample and measuring the chemical agents are selected from the
group consisting of acetic acid, formic acid, propionic acid,
butyric acid, Lugol's iodine, Shiller's iodine, methylene blue,
toluidine blue, osmotic agents, ionic agents, and indigo carmine.
The chemical agents may be applied substantially simultaneously, or
by dispensing at least two of the plurality of chemical agents
sequentially.
[0008] The invention is applicable to any sample type. Preferred
methods of the invention comprise using a biological sample. In a
preferred embodiment, the sample is selected from epithelial
tissue, cervical tissue, colorectal tissue, skin, and uterine
tissue.
[0009] In another aspect, a preferred embodiment of the invention
relates to a method of monitoring effects of a chemical agent on a
sample comprising dispensing a chemical agent on a sample,
providing an automated triggering signal to initiate a measurement
period relative to the dispensing, and measuring an optical signal
from the sample. The automated triggering signal can be provided
prior to, substantially simultaneously with, or after dispensing
the chemical agent. In preferred embodiments, the measurement is
initiated at a predetermined time relative to the automatic
triggering signal. In yet another aspect, methods of the invention
comprise of diagnosing the state of health of a applying the
chemical agent or agents as a mist onto the sample.
[0010] In a preferred embodiment, the predefined pattern is
substantially circular. In another preferred embodiment, the
predefined pattern is substantially annular.
[0011] In preferred embodiments, the chemical agent is dispensed at
a controlled rate, or a controlled volume of the chemical agent is
dispensed, or both.
[0012] In a still further aspect, the invention comprises
dispensing a chemical agent on a sample, capturing a plurality of
sequential images of the sample during a measurement period,
automatically aligning a subset of the plurality of images to
spatially correlate the subset of images, measuring an optical
signal from the subset of the spatially correlated images, and
providing a diagnosis of a state of health of the sample based at
least in part on the optical signal.
[0013] In a preferred embodiment, aligning further comprises
aligning the subset to compensate for relative motion between the
sample and a spectral observation device. In another preferred
embodiment, aligning further comprises aligning the subset to
compensate for relative motion between a first portion of the
sample and a second portion of the sample.
[0014] In a still further aspect, the invention provides methods
for determining a tissue response in which a chemical agent is
applied to a tissue and an optical property of an endogenous
molecule in the tissue is measured. In a preferred embodiment. the
endogenous molecule is a chromophore, for example a fluorophore.
Method of the invention comprise applying the chemical agent and
monitoring an optical signal from the endogenous molecule. The
presence, absence, or change in the signal may be indicative of
disease when compared to known standards. Such standards may be
empirically derived or may be obtained from the art. The endogenous
chromophore is preferably hemoglobin, a porphoryin, NADH, a flavin,
elastin, or collagen.
[0015] In preferred methods, the optical signal is a light signal,
such as a fluorescent or white light spectrum. The optical signal
may also be a spectrum produced, at least in part by
light-scattering properties of the tissue.
[0016] Also in preferred methods, the optical signal may be a decay
function. The optical signal is compared to a standard response
associated with healthy or diseased tissue, including tissue at
various stages of disease. Such standards may be determined
empirically or known in the art. Alteration of an optical signa
alone may be indicative of the health of the patient from whom a
sample was obtained.
[0017] The foregoing and other objects, aspects, features, and
advantages of the invention will become more apparent from the
following description and from the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The objects and features of the invention can be better
understood with reference to the drawings described below, and the
claims. The drawings are not necessarily to scale, emphasis instead
generally being placed upon illustrating the principles of the
invention. In the drawings, like numerals are used to indicate like
parts throughout the various views.
[0019] FIG. 1 shows an exemplary spectroscopic system that employs
a plurality of spectral types according to an illustrative
embodiment of the invention;
[0020] FIG. 2 shows an exemplary operational block diagram of the
spectroscopic system of FIG. 1;
[0021] FIG. 3 is a detailed schematic flow diagram showing
exemplary steps of combining a fluorescence spectrum analysis with
a reflectance spectrum analysis according to an illustrative
embodiment of the invention;
[0022] FIG. 4 is a schematic diagram of another illustrative system
useful for monitoring the effects of a chemical agent on a
specimen, and which embodies principles of the invention;
[0023] FIG. 5 is a graph that shows trend lines of data observed
according to principles of the invention;
[0024] FIGS. 6A-6C are diagrams that show raw data observed from
various specimens, according to principles of the invention;.
[0025] FIG. 7 is a graph showing curves representing averages of
data processed according to principles of the invention;
[0026] FIG. 8 is a graph plotting computed ratios that show a basis
for differentiating CIN II/III lesionf from CIN I and normal tissue
for individual specimens, according to principles of the
invention;
[0027] FIG. 9 is a graph showing responses, normalized at 480 nm,
from tissues as function of wavelength, according to principles of
the invention;
[0028] FIG. 10 is a functional block diagram of an embodiment of a
another illustrative system useful for monitoring the effects of a
chemical agent on a specimen according to the invention;
[0029] FIG. 11 is a functional block diagram of an embodiment of an
illustrative hand-held system useful for monitoring the effects of
a chemical agent on a specimen according to the invention;
[0030] FIGS. 12A-12C depict schematic arrangements for illustrative
filter wheels useful in the system of FIG. 11;
[0031] FIG. 13 is a functional block diagram of an embodiment of a
system useful for monitoring the effects of a chemical agent on a
specimen according to the invention;
[0032] FIG. 14 is a schematic diagram of a filter wheel useful in
the system of FIG. 13;
[0033] FIG. 15 is a functional block diagram of an embodiment of a
system useful for monitoring the effects of a chemical agent on a
specimen according to the invention;
[0034] FIG. 16 shows a schematic diagram of a CCD device for use in
the system of FIG. 15;
[0035] FIG. 17 is a graph showing the time dependence of
backscattered responses at 600 nm for various tissue classes (NED,
CIN II and CIN III), recorded using systems and methods of the
invention;
[0036] FIGS. 18A-18C are diagrams depicting various aspects of a
mucosal atomizer device used to spray a chemical agent uniformly
onto the surface of a specimen, and to provide a trigger mechanism
useful for initiating an optical observation, according to
principles of the invention;
[0037] FIG. 19 is a graph of absorption spectra recorded for NADH
using a chemical agent according to the invention;
[0038] FIG. 20 is a graph of fluorescence spectra as a function of
time for specimens treated with a chemical agent according to
principles of the invention;
[0039] FIG. 21 is a graph of flourescence spectra recorded before
and after treatment of a specimen with a chemical agent, according
to principles of the invention;
[0040] FIG. 22 is a one-dimensional diagram of watershed
segmentation;
[0041] FIG. 23 is a graph of a signal and its first derivative;
and
[0042] FIG. 24 shows a sigmoidal scaling function used to enhance
the contrast between light and dark regions of an image.
DETAILED DESCRIPTION
[0043] Acetowhitening of cervical tissue has long been known to be
a qualitative aid to locating lesions during colposcopic
examination. However, accurate quantitative measurements of
acetowhitening of cervical epithelial tissue, as a function of time
and wavelength, have not been reported. Quantitative analysis of
the acetowhitening process can significantly increase the
sensitivity and specificity of traditional colposcopy.
[0044] The invention will be described in terms of multiple
embodiments that relate to the observation of chemically-induced
changes in optical spectra, particularly in the area of medical
diagnostics, and especially as it relates to the analysis of
spectra obtained from human cervical tissue in the detection of
cervical cancer. However, the invention has applicability generally
in the area of chemically-induced changes in optical spectra.
[0045] FIG. 1 depicts an exemplary spectroscopic system 100
employing a plurality of spectral data types in methods and systems
according to an illustrative embodiment of the invention. The
spectroscopic system includes a console 102 connected to a probe
104 by a cable 106. The cable 106 carries electrical and optical
signals between the console 102 and the probe 104. The probe 104
accommodates a disposable component 108 which is used only once,
and discarded after use. The console 102 and the probe 104 are
mechanically connected by an articulating arm 110, which can also
support the cable 106. The console 102 contains much of the
hardware and the software of the system, and the probe 104 contains
the necessary hardware for making suitable spectroscopic
observations. The details of the system are further explained in
conjunction with FIG. 2.
[0046] FIG. 2 shows an exemplary operational block diagram 200 of a
spectroscopic system of the type depicted in FIG. 1. According to
an illustrative embodiment, the spectroscopic system of FIGS. 1 and
2 is subtantially the same as single-beam spectrometer devices, but
is adapted to include the features of the invention. The console
102 includes a computer 202 which executes software that controls
the operation of the spectroscopic system 100. The software
includes one or more modules recorded on machine-readable media,
which can be any medium such as magnetic disks, magnetic tape,
CD-ROM, semiconductor memory, or the like. Preferably, the
machine-readable medium is resident within the computer 202. In
alternative embodiments, the machine-readable medium can be
connected to the computer 202 by a communication link. In
alternative embodiments, one can substitute computer insructions in
the form of hardwired logic for software, or one can substitute
firmware (i.e., computer instructions recorded on devices such as
PROMs, EPROMS oe EEPROMs, or the like) for software. The term
machine-readable instructions as used herein is intended to
encompass software, hardwired logic, firmware and the like.
[0047] The computer 202 is a general purpose computer. The computer
202 can be an embedded computer, or a personal computer such as a
laptop or desktop computer, that is capable of running the
software, issuing suitable control commands, and recording
information in real time. The computer 202 has a display 204 for
reporting information to an operator of the spectroscopic system
100, a keyboard 206 for enabling the operator to enter information
and commands, and a printer 208 for providing a print-out, or
permanent record, of measurements made by the spectroscopic system
100 and for printing diagnostic results, for example, for inclusion
in the chart of a patient. As described below in more detail, in an
illustrative embodiment of the invention, some commands entered at
the keyboard, enable a user to select a particular spectrum for
analysis or to reject a spectrum, and to select particular segments
of a spectrum for normalization. Other commands enable a user to
select the wavelength range for each particular segment and to
specify both wavelength contiguous and non-contiguous segments.
[0048] The console 102 also includes an ultraviolet (UV) source 210
such as a nitrogen laser or a frequency-tripled Nd:YAG laser, a
white light source 212 such as one or more Xenon flash lamps, and
control electronics 214 for controlling the light sources both as
to intensity and as to the time of onset of operation and the
duration of operation. One or more power supplies 216 are included
in the console 102, to provide regulated power for the operation of
all of the components. The console 102 also includes at least one
spectrometer and at least one detector (spectrometer and detector
218) suitable for use with each of the light sources. In some
embodiments, a single spectrometer can operate with both the UV
light source and the white light source. In some embodiments, the
same detector can record UV and white light signals, and in some
embodiments different detectors are used for each light source.
[0049] The console 102 also includes coupling optics 220 to couple
the UV illumination from the UW light source 210 to one or more
optical fibers in the cable 106 for transmission to the probe 104,
and for coupling the white light illumination from the white light
source 212 to one or more optical fibers in the cable 106 for
transmission to the probe 104. The console 102 also includes
coupling optics 222 to couple the spectral response of a specimen
to UV illumination from the UV light source 210 observed by the
probe 104 and carried by one or more optical fibers in the cable
106 for transmission to the spectrometer and detector 218, and for
coupling the spectral response of a specimen to the white light
illumination from the white light source 212 observed by the probe
104 and carried by one or more optical fibers in the cable 106 for
transmission to the spectrometer and detector 218. The console 102
includes a footswitch 224 to enable an operator of the
spectroscopic system 100 to signal when it is appropriate to
commence a spectral observation by stepping on the switch. In this
manner, the operator has his or her hands free to perform other
tasks, for example, aligning the probe 104.
[0050] The console 102 includes a calibration port 226 for
calibrating the optical components of the spectrometer system. The
operator places the probe 104 in registry with the calibration port
226 and issues a command that starts the calibration operation. In
the calibration operation, a calibrated light source provides
illumination of known intensity as a function of wavelength as a
calibration signal. The probe 104 detects the calibration signal.
The probe 104 transmits the detected signal through the optical
fiber in the cable 106, through the coupling optics 222 to the
spectrometer and detector 218. A test spectral result is obtained.
A calibration of the spectral system is computed as the ratio of
the amplitude of the known illumination at a particular wavelength
divided by the test spectral result at the same wavelength.
[0051] The probe 104 includes probe optics 230 for illuminating a
specimen to be analyzed with UV and white light from the UV source
210 and the white light source 212, and for collecting the
fluorescent and backscatter (or reflectance) illumination from the
specimen that is being analyzed. The probe includes a scanner
assembly 232 that provides illumination from the UV source 210 in a
raster pattern over a target area of the specimen of cervical
tissue to be analyzed. The probe includes a video camera 234 for
observing and recording visual images of the specimen under
analysis. The probe 104 includes a targeting souce 236, which can
be used to determine where on the surface of the specimen to be
analyzed the probe 104 is pointing. The probe 104 also includes a
white light illuminator 238 to assist the operator in visualizing
the specimen to be analyzed. Once the operator aligns the
spectroscopic system and depresses the footswitch 224, the computer
202 controls the actions of the light sources 210, 212, the
coupling optics 220, the transmission of light signals and
electrical signals through the cable 106, the operation of the
probe optics 230 and the scanner assembly 232, the retreival of
observed spectra via the cable 106, the coupling of the observed
spectra via the coupling optics 222 into the spectrometer and
detector 218, the operation of the spectrometer and detector 218,
and the subsequent signal procesing and analysis of the recorded
spectra.
[0052] FIG. 3 is a detailed schematic flow diagram 300 showing
exemplary steps of combining fluorescence spectrum analysis with
reflectance spectrum analysis to perform tissue characterization
according to an illustrative embodiment of the invention. Step 310
indicates that fluorescence spectra from a test specimen of unknown
condition or unknown state of health are available. At step 320,
the computer 202 determines whether the test specimen can be
classified as "normal," or "metaplasia," or can not be classified
by fluorescence spectroscopy alone. As indicated in step 325, a
decision is taken as to whether the test specimen has a definitive
state of health, for example that the specimen is "normal." If the
test specimen can be classified, for example as normal, the method
ends at step 330.
[0053] In the event that a definitive condition or state of health
cannot be ascribed to a test specimen, the computer 202 further
analyses information available from a reflectance spectrum or from
a plurality of reflectance spectra taken from the test specimen. At
step 335, the computer 202 provides processed reflectance
spectra.
[0054] If the specimen cannot be classified, a mean normalization
step is performed by computer 202, as indicated at step 340. The
mean normalization is carried out using a plurality of reflectance
spectra taken from specimens that are known to represent normal
squamous tissue. In one embodiment, a single test specimen is
examined at multiple locations, each location measuring
approximately one millimeter in diameter. If one or more locations
of the test specimen provide fluorescence spectra that indicate
that those locations can be classified as representing normal
squamous tissue, the reflectance spectra recorded from those
locations are used to mean normalize the reflectance spectra
obtained from locations that are not capable of being classified as
"normal" or "metaplasia" solely on the basis of fluorescence
spectra.
[0055] As indicated in step 350, the computer 202 can carry out an
analysis using a metric, for example using the Mahalanobis distance
as a metric in N-dimensional space. In one embodiment, the test
reflectance spectra are truncated to the wavelength regions 391 nm
to 484 nm, and 532 nm to 625 nm. In one embodiment, the
classifications CIN I and CIN II/II are the classifications that
are possible for a test spectrum that is neither classified as
"normal" nor "metaplasia" by fluorescence spectral analysis. As
indicated at step 350, the computer 202 classifies the test
specimen as having a condition or state of health selected from CIN
I and CIN II/III based on the value of the metric computed by the
computer 202, provided that the value of the metric does not exceed
a pre-determined maximum value.
[0056] At step 360, the computer 202 presents the results of the
classification of the test specimen, as a condition or state of
health corresponding to one of normal, metaplasia, CIN I and CIN
II/III.
[0057] FIG. 4 shows a schematic diagram of an illustrative system
600 embodying principles of the invention. A standard colposcope
610 (Zeiss, Model 1-FC ZMS-506-II) is modified by adding video
image capture capability with permanent and electronic storage of
data to allow capturing of time-sequenced images during a routine
colposcopic examination. The colposcope 610 has magnification
capabilities of 4.times., 6.times., 10.times., 16.times., and
25.times., and is illuminated by a fiber optic-coupled 12 volt/100
watt halogen lamp 620 with 20.times. eye binoculars. A
three-channel charge-coupled device color video camera (DAGE-MTI,
Model DC-330) 630 is mounted to the colposcope 610. The computer
650 includes an integrated video frame-grab board and video display
card at 24-bit resolution for capturing images. Images can be
captured at a rate of at least about one image per second. The
computer 650 also includes image control software (TeleComputing
Solutions, ColpoShot.TM.) that interfaces with the video frame grab
board for archiving images, for example into patients' medical
records. The ColpoShot.TM. software is modified to allow for
intensity measurements at specific sites as a function of time and
wavelength (as resolved by four discrete filters in a filter wheel
640, described in more detail below). The computer 650 also
includes control software for controlling the change of filters in
the high-speed filter wheel 640. This software is synchronized with
the data collection (image capture) software so each image is
associated with a spectral region corresponding to a particular
filter. Time-stamping of each image is performed so each image can
be placed in proper time sequence.
[0058] In the illustrative system 600, the filter wheel 640 is from
a Ludl Electronics Ltd., with an RS 232 and GPIB 488 computer
interface for resolving optical signals with respect to wavelength.
Images are measured and recorded at three separate wavelength bands
in the visible spectral region. The first wavelength band is near
400 nm, with a bandwidth of about 20 nm to about 30 nm. The second
wavelength band is near 525 nm with a bandwidth of about 30 nm. The
third wavelength band is near 680 nm with a nominal bandwidth of
about 30 nm. In addition to the images taken through the filter
wheel 640, a fourth image using unfiltered illumination is taken as
part of the data set. The unfiltered images allow data analysis of
red (R), green (G), and blue (B) components for comparison with
filtered image data. As described before, crossed polarizers
mounted in the optical path, one associated with the light coming
from the illumination source 620, and one associated with the light
from the image to be observed and recorded, are used to reduce
unwanted glare from the surface of the cervix.
[0059] The illustrative system 600 is controlled by the computer
650, having capabilities similar to the computer 140 described
earlier. The computer 650 has associated with it software to
operate the computer 650, to provide input and output interactions
with an instrument user, to control and synchronize the various
components of the illustrative system 600, and to record, analyze,
and report data obtained from the illustrative system 600.
[0060] The illustrative system 600 is configured to capture
time-separated images of the specimen during routine colposcopic
examinations. Digital images are recorded at a 4.times.
magnification giving a panoramic view of the entire cervical field
at maximal acetic whitening. In the illustrative embodiment, images
are taken about every second for about 5 minutes after the
application of acetic acid. The computer 650 rotates the filter
wheel 640 to allow for imaging at different wavelengths.
[0061] In operation, an illustrative embodiment of the process of
obtaining images is as follows. The first image following the
application of the acetic acid is an unfiltered image. Next, the
filter wheel 640 is rotated to bring the short-wavelength
(.about.400 nm) filter into place and the next image is recorded.
Then, the .about.525 nm filter is positioned, and the next image is
recorded Next, the long-wavelength (.about.680 nm) filter is
positioned and the last image of the sequence is recorded. This
process takes four seconds to complete. After this first cycle
through the filter wheel 640, the process repeats with another
unfiltered image, followed by the sequence of filtered images. The
process of observing and recording images continues without
stopping for a duration of 300 seconds. The resulting data are
seventy-five unfiltered images of the evolution of an optical
signal from a specimen treated with a chemical agent, such as
cervical acetowhitening, and a total of seventy-five images in each
of the three filtered spectral regions. As will be appreciated by
those of skill in the spectroscopic arts, the precise sequence of
observing and recording images in the various wavelength bands
depends on the sequence of placement of filters within the filter
wheel 640 and the sense of rotation of the wheel 640. Alternative
sequences of observation can be employed with substantially
equivalent results. The duration of operation can be shortened or
extended from the illustrative 300 seconds just described depending
on the situation, which can be influenced by the kind of specimen
and how it is to be examined (e.g., specimen characteristics, such
as cervix, larynx, skin, and the like, specimen in vivo or in
vitro, use of different chemical agents, the disease conditions to
be investigated, and the like).
[0062] Illustratively, time-stamped images are saved to disk at 20
second intervals. In one embodiment, treatment of a specimen with a
chemical agent is accomplished as follows. A solution of 5% acetic
acid is applied with solution-soaked cotton balls placed in contact
with the surface of the cervix for about 15 seconds. An alternative
method of application of a chemical agent is discussed below. In
one embodiment, the time sequence image capturing software is run
immediately before the application of acetic acid, to obtain
baseline measurements.
[0063] In one embodiment, the parameters that are extracted from
the observations include the rate of acetowhitening, the maximum
intensity of the whitening, and the final rate of decay of the
whitening. Once the data is collected, the images are analyzed by
the computer 650 with software that calculates four parameters
(mean Luminance, and mean red (R ), green (G), and blue (B)
intensities) within user-defined Regions of Interest (ROI's). The
software enables the user to mark, with a mouse controlled
cross-hair cursor, 5 pixel by 5 pixel ROI's on a location in an
image. A biopsy can subsequently be taken by the colposcopist, to
permit a comparison of the results obtained from the methods of the
invention with the results of the biopsy. Once ROI's have been
manually marked on all images in the timed-sequence, mean Luminance
and mean R, G, B intensities within the 5 pixel by 5 pixel ROI's
are calculated and output in tabular form. Also included in the
output recorded in the table are the following data elements; image
number, ROI location, elapsed time in seconds, and the standard
deviation and median of the Luminance and R, G, B values. In one
embodiment, the ratio of the mean green intensity to the mean red
intensity is found to yield accurate results.
[0064] In this embodiment, to calibrate the utility of the system
and method, five (5) biopsy-confirmed CIN II/III lesions are
measured, five (5) biopsy-confirmed CIN I lesions are measured,
five (5) colposcopy-confirmed normal mature squamous tissue regions
are measured and one (1) biopsy-confirmed normal mature squamous
tissue region is measured. Data are analyzed by graphing the Green
intensity divided by the Red intensity and normalizing by the
maximum intensity within each patient.
[0065] FIG. 5 is a diagram 700 that shows the trend lines of ROIs
correlated to CIN II/III lesions (curve 706), CIN I lesions (curve
708), and normal mature squamous tissue (curve 710). The trend
lines are plotted using the ratio of mean green intensity to mean
red intensity, normalized to maximum intensity, as the vertical
axis 702 (expressed in arbitrary units), and using the time after
application of acetic acid to the tissue, expressed in seconds, as
the horizontal axis 704.
[0066] FIGS. 6A-6C are diagrams, generally 800, that show graphs of
raw data plotted using the ratio of mean green intensity to mean
red intensity, normalized to maximum intensity, as the vertical
axis 802 (expressed in arbitrary units), and using the time after
appllication of acetic acid to the tissue, expressed in seconds, as
the horizontal axis 804. FIG. 6A is a diagram that shows the raw
data of ROIs correlated to CIN II/III lesions, as curves 810, 812,
814, 816, 818 representing observations taken from five
individuals. FIG. 6B is a diagram that shows the raw data of ROIs
correlated to CIN I lesions, as curves 820, 822, 824, 826, 828
representing observations taken from five individuals. FIG. 6C is a
diagram that shows the raw data of ROIs correlated to normal mature
squamous tissue, as curves 830, 832, 834, 836, 838 representing
observations taken from five individuals.
[0067] An operator of the illustrative system and method defines a
region of interest on an image. The intensity readings of the
pixels in this region are averaged to provide a quantitative value
of brightness as recorded through the particular filter (or
unfiltered). By plotting these values as functions of time, a
picture of the evolution of the acetowhitening at the selected
location in the image is created.
[0068] A clinically useful tool based on the acetowhitening kinetic
characteristics analyzes the data to differentiate CIN II/III
lesions from CTN I lesions and normal mature squamous tissue.
According to one illustrative embodiment, the technique uses mean
values from 100 second segments of individual patient kinetic
curves. The curves are processed by calculating the mean of
segments along the curve, i.e. the mean value of the data in the
temporal range from about 100-200 seconds after application of the
chemical agent, the mean value of the data in the temporal range
from about 200-300 seconds after application of the chemical agent,
and so forth. FIG. 7 is a graph 900 showing curves of the averages
of data processed in this manner, in which the average values are
plotted along the vertical axis 902 (expressed in normalized
units), and using the time after application of acetic acid to the
tissue, expressed in seconds, as the horizontal axis 904. The curve
906 represents data relating to CIN II/III lesions. The curve 908
represents data relating to CIN I lesions. The curve 910 represents
data relating to normal mature squamous tissue.
[0069] FIG. 8 is a scatter plot 1000 generated by taking the ratio
of mean values from two time intervals (100-200 seconds) and
(200-300 seconds) for data from individual specimens. In FIG. 8,
the average values for the time interval 200 seconds to 300 seconds
(expressed in normalized units) are plotted along the vertical axis
1002, and the time interval 100 seconds to 200 seconds (expressed
in normalized units), is plotted as the horizontal axis 1004. The
points 1006 represent data relating to CIN II/III lesions. The
points 1008 represent data relating to CIN I lesions. The points
1010 represent data relating to normal mature squamous tissue. FIG.
8 shows a basis for differentiating CIN II/III for individual
specimens. An illustrative line 1020 is a line of demarcation
between the CIN II/III data and the remaining data. A second
technique using the first derivative of the curves shown in FIG. 7
is also operative. This technique yields similar results to those
shown in FIG. 8.
[0070] According to another illustrative embodiment of the
invention, an indication of the presence or lack of cancerous or
precancerous tissue is obtained by recording the optical response
in two parts of the visible spectrum. In this embodiment, the
inventors have observed that at short wavelengths, such as 380 nm,
absorption by hemoglobin can reduce signal intensities. Optical
responses are recorded in that part of the spectrum where optical
response variation can be detected due to morphological changes in
tissue which are associated with cancerous and precancerous tissue,
such morphological variations having a strong impact on light
scattering. At longer wavelengths, beyond 590 nm and to about 750
nm, scattering of light from cancerous tissue was substantially
greater than from normal tissue, and thus the reflected responses
from cancerous tissue in that spectral range were greater than from
normal tissue.
[0071] It is desirable to standardize the responses from the tissue
using a signal at a wavelength where both of these influences are
relatively weak. In one embodiment, the system of the invention
standardizes responses at 480 nm for this purpose. In one
embodiment, the response, e.g., the observed reflectance, is
recorded at three wavelengths, and the responses obtained at the
short wavelength (between 360 and 440 nm) and at the long
wavelength (between 590 and 750 nm) are divided by the response at
480 nm. According to one illustrative methodology of the invention,
normalized reflections at longer wavelengths indicate cancerous and
precancerous tissue, while lower intensity normalized refelections
indicate healthy tissue. According to a further illustrative
methodology of the invention, reflections in the short wavelength
part of the spectrum indicate cancerous and precancerous tissue,
while higher intensity reflections indicate healthy tissue.
[0072] An algorithm using the rate of change of white light
reflection at some specific wavelength, for instance, at 600 nm,
can provide accurate differentiation between pathologic and healthy
tissue within the first 60 seconds after the application of a
pathology differentiating agent like acetic acid. Other algorithms,
using both the aforementioned rate of change, or the time lapsed to
reach maximum back scattering after application of a
differentiating agent, or the time required to attain specific back
scattered (normalized) threshold values, permit the diagnosis of
the presence or absence of cancer in the screened cervix.
[0073] As an aspect of the invention, methods are provided that
employ specific algorithms to analyze the back-scattered responses
obtained at the preselected wavelength or wavelengths either with
or without a chemical agent. Algorithms further provide for
classifying examined tissues as normal or pathological. In certain
embodiments, these systems are characterized by ease of operation,
simplicity and ruggedness.
[0074] FIG. 9 presents a graph 1100 showing two data curves 1102,
1104 obtained from healthy (no evidence of disease, or NED) and
cancerous (CIN) tissue respectively. Normalized intensity is
plotted along the vertical axis 1106 and wavelength (in units of
nm) is plotted along the horizontal axis 1108. All received
responses (I.sub.80 ) are normalized by dividing the intensities
received by the intensity obtained at an arbitrary wavelength.
Reflections measured at 480 nm are used for this purpose, since it
is in a part of the spectrum where the responses' intensities are
relatively independent of the tissue state (healthy or
pathological). FIG. 9 shows that the received intensities at longer
wavelength (between 550 to 750 nm) are consistently higher for
cancerous tissue (curve 1104) than for healthy tissue (curve 1102).
The data indicate that the use of three wavelengths from the
reflected spectrum of tissue provides correlation between the
presence or absence of cancer in the target tissue.
[0075] In one embodiment, an algorithm utilizes the reflected
reading from the tissue at the three selected wavelengths to
produce an indicator of the presence or absence of a pathology in
the target tissue, or to create an artificial pathology image of
the tissue observed. In the first step of the algorithm, the
responses are collected at three wavelengths for each point
observed. In one embodiment, the following three wavelengths can be
used:
[0076] .lambda..sub.1=380 nm
[0077] .lambda..sub.2=480 nm
[0078] .lambda..sub.3=650 nm
[0079] It is understood that one can select wavelength ranges
rather than specific narrow bands as illustrated here. Normalized
reflected intensities may then be defined:
[0080] R.sub.380=I(.lambda..sub.1)/I(.lambda..sub.2)
[0081] R.sub.650=I(.lambda..sub.3)/I(.lambda..sub.2)
[0082] where I(.lambda..sub.1), I(.lambda..sub.2) and
I(.lambda..sub.3) are the measured reflected intensities at
.lambda..sub.1, .lambda..sub.2 and .lambda..sub.3 respectively.
These normalized intensities R.sub.380 and R.sub.650 (which are
dimensionless), can vary from about 0.2 to about 6. In one
embodiment, the intensity of the reflected light at 380 and 650 nm
are normalized, where the normalization parameter is the reflected
intensity at 480 nm. It should be evident to those of ordinary
skill in the art that while in one embodiment R.sub.380 is defined
at .lambda.=380 nm and R.sub.650 at .lambda.=650 nm, one can define
R(low .lambda.) and R(high .lambda.) around neighboring wavelengths
in the respective ranges as well, using data such as presented in
FIG. 9 from a number of subjects and tissue with varying
pathologies in those subjects as a "training set" to calibrate the
apparatus being employed. The selection of the "bandwidth" around
the center wavelength is related to the kind of instrumentation
selected for the actual device, as described below in more
detail.
[0083] As long as the bandwidths selected during the calibration or
training of the device and its subsequent use in the field for
screening purposes are the same, good correlation is found between
high values of R.sub.650 coupled with low values of R.sub.380 and
the presence of cancerous and precancerous, or CIN, tissue.
Similarly, good correlation is found between low values of
R.sub.650 and high values of R.sub.380 and the presence of healthy,
or NED, tissue. Specifically, for cervical tissue that when
R.sub.650<3.1 and R.sub.380>1.1, the tissue is healthy (NED)
and when R.sub.650>3.3 and R.sub.380<0.9 the tissue is
cancerous or precancerous (CIN of all grades).
[0084] In one embodiment, a grading algorithm is incorporated in a
data processing unit employed by these systems and methods. The
grading algorithm utilizes the pair (R.sub.650, R.sub.380) and
classifies the reflections from each site observed into three
groups. In the case of cervical tissue, the algorithm classifies
reflections for which the pair obeys R.sub.650<2.9,
R.sub.380>0.1.1 as "healthy tissue" or NED. Similarly, a second
group of sites, for which the pair obeys R.sub.650>3.5,
R.sub.380<0.9 is classified as cancerous or precancerous tissue
or CIN. Finally, a third group of tissue, including those tissues
for which the reflections pairs obey the relationships
2.9<R.sub.650<3.5, 0.9<R.sub.380<0.1.1, is classified
as tissue for which a determination cannot be made. An algorithm
according to these systems and methods classifies each point in the
observed tissue as healthy or unhealthy. If this classification can
not be performed for a particular tissue area, that area is
segregated into a third, "unclassifiable" class.
[0085] An algorithm according to these systems and methods maps
tissue for the presence or absence of a pathology. In one
embodiment, an algorithm utilizes an independently determined set
of threshold values for R.sub.380 and R.sub.650. These threshold
values are determined in clinical studies from a large number of
patients from which both readings of R.sub.380 and R.sub.650 are
compared with biopsies taken from the tissues from which these
values are determined. The threshold values as well as the actual
wavelengths where the reflections are taken (and the normalizing
wavelength utilized to determine from I(.lambda.) the normalized
reflection R.sub.80 ) can vary from the values presented herein, as
long as the short wavelengths reflections correlate well with
absorption by hemoglobin and the long wavelengths reflections with
variations of scattering between healthy and pathological
tissues.
[0086] The wavelengths presented in the example above and shown in
FIG. 9 are useful in the diagnosis of cervical tissue
abnormalities. It is understood, however, that other wavelengths
may be useful, particularly when other tissue areas are studied.
Furthermore, the critical threshold values of the short and long
wavelengths standardized reflections, R, are subject to
determination for each type of tissue targeted.
[0087] In another embodiment of the invention, a tissue integral
algorithm is used, where the cervix as a whole is examined to
determine if a pathology exists without actually obtaining an image
of the location of such pathology within the tissue. This algorithm
is used as follows. The computer 650 collects the normalized
reflection R.sub.650 for all measured sites on the tissue and
determine the minimum R.sub.650(min) of the set {R.sub.650}. The
computer 650 determines the maximum value R.sub.650(max) of the set
{R.sub.650}. In one embodiment, if the condition
R.sub.650(max)<1.2R.sub.650(min) of the set {R.sub.650} is true
(e.g., if all observed values of R.sub.650 are smaller than 120% of
the smallest value of R.sub.650 R.sub.650(min)), then the tissue is
free of pathology. If this condition is not met, pathology of some
type is indicated, and the subject should be referred for
additional diagnostic tests to identify the type and location of
the suspected cervical pathology.
[0088] A similar algorithm involving R.sub.380 can be used, whereby
the computer 650 determines the minimum R.sub.380(min) of the set
{R.sub.380}, for the normalized reflection R.sub.380 observed for
all tissue locations. The computer 650 determines the maximum value
R.sub.380(max) of the set {R.sub.380). In one embodiment, if the
condition R.sub.380g(max)<1.20R.sub.380(min) of the set
{R.sub.380} is true, (e.g., if all observed values of R.sub.380 are
smaller than 120% of the smallest value of R.sub.380,
R.sub.380(min)), tissue is free of pathology. If this condition is
not met, pathology of some type is indicated, and the subject
should be referred for additional diagnostic tests to identify the
type and location of the suspected cervical pathology.
[0089] It is understood that an algorithm in which both of the
above conditions are met also results in a valid classification of
the subject population into healthy and possibly pathological
tissue. It should further be clear that an algorithm based on
simultaneously satisfying both conditions can be a useful grading
system of tissue for the presence or lack of pathology. Such an
algorithm can be expected to result in a greater number of
"undetermined" cases. However, the confidence level of correctly
grading healthy and pathologic tissue is higher that when using
either one of the tissue integral algorithms described above
individually.
[0090] It should furthermore be evident to those of ordinary skill
in these arts that other algorithms can be constructed without
departing from the scope of the systems and methods described above
but that nonetheless rely upon the fact that scattering from
non-pathological tissue at wavelengths between about 600 nm and
about 750 nm is consistently greater for pathological tissue than
for healthy tissue, or that rely upon the fact that absorption of
light in the range of about 370 nm to about 430 nm is greater for
pathological tissue than for healthy tissue. Such algorithms,
consistent with these systems and methods, are useful in
classifying a subject's cervix for the presence or lack thereof of
pathological tissue (e.g., a state of health of a subject's
cervix). In other embodiments, algorithms can employ data collected
at other wavelengths in order to diagnose pathologies of the cervix
or pathologies of other body tissues.
[0091] FIG. 10 shows an illustrative embodiment of a device for
determining the presence or absence of pathology in a tissue of the
cervix according to the invention. In this figure, a screening
device (shown generally at 1200) is an integral part of a
colposcope 1202, and is used to make determinations of tissue
pathology point by point. A colposcope 1202 is provided with a high
intensity light source and optics to view cervical tissue, all
included within the colposcope 1202. The image viewed by an
observer 1203 is recorded with a video camera 1210 and recorded for
future reference on magnetic media through a video tape recorder
1211. The instrument depicted in FIG. 10 is used for determining
the presence or absence of pathology point by point. Since it is
paired with a colposcope 1202, this embodiment is suitable for use
by highly trained professionals, such as gynecologists. A beam
splitter 1212 is used to select a site in the target tissue 1213
which is illuminated with white light 1214 from the light source
provided in the colposcope 1202. The position control of the beam
splitter (and thus selection of the point examined in the target
tissue) is accomplished with a "joystick" 1215. The optical head
1216 includes a small laser diode (wavelength at about 635 nm)
1217, having a beam coaxial with the optical head's detection
optics. In operation, the red beam is pointed toward the tissue
1213. Since the optical axis of the laser diode 1217 and the
collection optics in the optical head 1216 are the same, the
optical head 1216 measures the light reflected from the point
illuminated by the red laser diode beam. In order to maintain the
calibration of the optical head's sensor 1218, a white reflector
1219 is provided within the illumination path of the colposcope, to
which the operator directs the seeking beam from the laser diode
1217. The reflectance from the white reflector 1219 is used as a
standard for calibrating the sensor 1218. Such a white reflector
can be made from Spectrolon.TM., from Labsphere Corporation.
Alternatively, high purity BaSO.sub.4 reflecting paint from the
Kodak Corporation can be applied to a flat surface and used.
[0092] In some embodiments of the invention, a polarizer is
interposed in the back scattered beams which considerably reduces
the specular reflection from the target tissues. The specular
reflection is understood to comprise the light reflected from the
thin film of moisture overlaying the target tissue that has not
interacted with the underlying tissue.
[0093] In operation, the physician directs the beam 1214 to a
specific site on the suspected tissue 1213. The reflected light
from this site is collected by the optical head 1216. A
spectrometer 1220 (which can be either a refractive or dispersive
spectrometer) disperses the light so that the intensity of the
reflected light at preselected wave lengths can be measured in the
detector 1218. In one illustrative embodiment, three preselected
wavelengths are chosen. In certain embodiments, the sensor 1218
comprises a plurality of sensors corresponding in number to the
number of preselected wavelengths, so that one sensor is dedicated
to each wavelength. The sensor 1218 can be an ICCD, a standard CCD,
or any other detector system known in the art or envisioned by
those of ordinary skill in these arts.
[0094] Data from the sensor 1218 is analyzed in a computer
processor 1221 by applying an algorithm system as described above,
and a score is obtained from the data processing that relates to
the presence or absence of pathology at the tissue area being
illuminated by the laser diode 1217. This score is graphically
represented on a display 1222. The digital information
corresponding to the score is made available electronically for
further processing or representation. In certain embodiments,
points for which pathological scores are obtained can be
represented on a display 1222 as superimposed upon an image
provided by a video camera 1210. In one embodiment, abnormal points
are identified graphically with an artificial color not commonly
found in cervical tissue, such as shades of green. It will be seen
below that other embodiments provide for creation of artificial
images or representations of pathologies. The embodiment
illustrated in FIG. 10 is suitable for operation by a gynecologist
in conjunction with colposcopy. In this setting, the device is well
adapted for use as an assisting device for determining which areas
of the cervix may require biopsies.
[0095] In one embodiment, the systems and methods of the invention
provide a hand held device adapted for illuminating a target tissue
with white light and further adapted for detecting reflections or
backscattered responses at three specific wavelengths. FIG. 11
shows an illustrative embodiment suitable for screening
applications. This embodiment provides features of a visualization
colposcope and features of a screening device according to the
present invention. In this embodiment, a superimposition of
pathological findings on a cervical image may be produced.
[0096] FIG. 11 shows a colposcreener 1330 consisting of two
orthogonal optical paths 1331 and 1332. The first optical path 1331
includes a plurality of lenses (for example lenses 1333, 1334 and
1335) to image the tissue 1336 so that it can be viewed by an
observer 1303. The second optical path 1332 includes a distal
portion 1331 a of the first optical path 1331 (for example lenses
1334 and 1335), a beam splitter, 1338 and additional lenses
(forexample, lenses 1337 and 1339). The beam splitter 1338 couples
the two optical paths 1331 and 1332 to the distal portion 1331a of
the first optical path 1331, directing half of the light reflected
back from the tissue 1336 to be viewed by the observer 1303 through
the ocular 1333 and directing half to a sensor 1340. The sensor
1340 is coupled to the optics via a mirror 1341, as shown in FIG.
11, or the sensor 1340 is positioned in the image plane of the
second optical path 1332. In some embodiments, the sensor 1340
comprises a plurality of sensors corresponding in number to the
number of preselected wavelengths, so that one sensor is dedicated
to each wavelength. The sensor 1340 can be, for example an ICCD, a
standard CCD, or any other detector system known in the art or
envisioned by those of ordinary skill in these arts. A filter wheel
1342 is placed in the optical path of the detected beam 1332, to
allow at any given time only one wavelength to reach the detector
1340. The filter wheel 1342 is mounted on an appropriate driving
mechanism, for instance, a stepper motor 1343, which sequentially
indexes the wheel to the appropriate filter.
[0097] Arrangements of filter wheels are shown in more detail in
FIGS. 12A, 12B and 12C. In one embodiment, as shown in FIG. 12A,
the filter wheel 1442 has three filters, 1444, 1445 and 1446, each
capable of blocking most of the spectrum of the reflected beam
except around the three selected wavelengths, 380 nm, 480 nm and
650 nm respectively (for the filters 1444, 1445 and 1446). It will
be understood by those of ordinary skill in the art that a number
of duplications of these filters can be employed for drive
simplicity, in particular when the cross section of the reflected
beam is narrowed (at the common focal point of the lens on both
sides of the filter wheel 1442), so as to allow more than three
wavelengths to be determined per rotation of the filter wheel 1442.
In such an arrangement in the illustrated embodiment, the number of
filter slots would be a multiple of three. In another embodiment,
as shown in FIG. 12B, a different filter wheel, 1447, is used in
place of the previously illustrated filter wheel 1442. The filter
wheel 1447 has four positions (or multiples of four). The first
three, positions 1448, 1449 and 1450, are filters transmitting at
380 nm, 480 nm and 650 nm respectively, as cited above, and a
fourth slot, 1451, being spectrally neutral, namely it is either a
simple open slot in the filter wheel 1447, or a neutral filter that
reduces the transmission of all wavelengths by a constant factor.
The latter case simplifies the task of maintaining the signals
received by the sensor 1440, (for instance a CCD) under a given
threshold and thus preventing sensor's saturation.
[0098] In one embodiment, the shape of the colposcreener 1330 is
similar to the device depicted in FIG. 11. FIG. 11 shows a
colposcreener 1330 shaped like a gun, with a trigger 1352 used to
initiate the processes of obtaining optical reflection data and
viewing the tissue 1336. In operation of this embodiment, pressing
the trigger 1352 switches on a light source in the control console
1353. This light source is concentrated into an optical fiber
bundle (not shown) included in the control cable 1354 which
connects the control console 1353 and the colposcreener 1330. The
optical fiber bundle 1354 delivers light to the distal end 1355 of
the colposcreener 1330. In one embodiment, a cone of white light
1357 illuminates the target tissue 1336 homogeneously. It is
understood that other shapes and configurations of the
colposcreener 1330 may be envisioned by those of ordinary skill in
these arts without departing from the scope of the systems and
methods disclosed herein. Furthermore, while the colposcreener 1330
is adapted for examination of the cervix, other shapes and
embodiments consistent with these systems and methods may be
devised that are structurally adapted for other anatomic areas.
[0099] FIG. 11 further shows that light reflected from the tissue
is split by the beam splitter 1338 into a viewing beam carried
along the optical path 1331 and a detection beam carried along the
optical path 1332. In that manner, the tissue screened is viewed
directly through the ocular 1333 while the detection beam is being
sequentially scanned for the three wave length discussed above. The
tissue is imaged onto the sensor 1340. In one embodiment the sensor
1340 comprises a CCD array, whereby the light intensity reflected
for each point in the tissue is measured. The data from the sensor
1340 is transmitted through a data cable 1356 to a data processing
unit 1358 for further analysis. Synchronization signals generated
in the control console 1353 provide correct indexing of the streams
of data for each one of the three filters in the filter wheel 1342.
This may be achieved by using the signal sent to the stepper motor
1343 to coordinate with the data stream from the sensor 1340.
[0100] In one illustrative embodiment, the synchronization task is
simplified by using the geometry of the filters in the filter wheel
1342. In this embodiment, the motor 1343 is used in a continuous
rather then a stepping manner, thus the filter wheel 1342 rotates
continuously. An embodiment using a filter wheel in this way is
shown in FIG. 12C, where the filter wheel 1459 is depicted as
having three unequal filters 1460, 1461 and 1462, separated by
unequal spaces 1463, 1464 and 1465. In this embodiment, a CCD or a
CCD array is advantageously employed as the sensor 1340, as
previously described. Since the CCD has its highest sensitivity in
the red part of the spectrum, and the light source is typically
richer in the red part of the spectrum as well, the 650 nm red
filter 1460 in FIG. 12C is much shorter, with shorter collection
time. The short collection time is used to indicate to the data
analysis unit 1358 that the red filter 1460 (transmitting
selectively at 650 nm) is being used. To better equilibrate the
intensities received, the green filter 1461 (transmitting
selectively at 480 nm) is larger than the red filter 1460. The blue
filter 1462 (transmitting selectively at 380 nm where the CCD is
least sensitive) occupies the longest segment of the circumference.
The signals received at various wavelengths are more homogeneous
and easier to analyze. Integration times can be adjusted
accordingly. The adjustment of the integration time can be keyed on
the "No signal" periods between the filters, represented by the
unequal spacings 1463, 1464 and 1465 between the filters.
[0101] In this illustrative approach, the actual normalized
intensities, R.sub.380, R.sub.480 and R.sub.650 as discussed above
are modified to account for the time variability of data
acquisition between the three different filters. Since these
factors depend on the specific integration time selected, the
normalized reflections R.sub.80 provided above are used,
understanding that algorithms based on these findings are devised
once a calibration for a specific design is available.
[0102] The data received for each one of the three filters is
analyzed for each pixel and is displayed on the display monitor
1322 in a dual fashion. The first display generates a
Red/Green/Blue image of the tissue by taking the raw data
(normalized for spectral differences in the CCD sensitivity as well
as variations of integration times when using the filter wheel 1559
shown in FIG. 12C) from each CCD's pixel and presenting it as a
normal full color picture. This is achieved with well known "frame
grabbing" electronics readily available commercially.
[0103] Each pixel, P.sub.ij, has associated with it three values
(residing in the grabbed frame), I.sub.ij,380, I.sub.ij,480 and
I.sub.ij,650, from which are derived normalized intensities
R.sub.ij,380 and R.sub.ij,650. A strongly discriminating algorithm
selects all pixels P.sub.ij for which both of the conditions
R.sub.ij,650>3.3 and R.sub.ij,380<0.9, namely those pixels
for which a pathology is identified. These pixels form a group Qij
of pathological tissue. A "weaker" discrimination defines as
"pathological" only those P.sub.ij for which R.sub.ij,650>3.3
and the so defined Q.sub.ij are then painted on the total image as
a pathology.
[0104] The display superimposes an image of all the pixels Q.sub.ij
having a "pathological" signature on the natural picture of the
tissue. This is achieved by selecting a color uncommon to the
tissue (such as green, or blue) and painting said all Q.sub.ij
(pathological) pixels all in the same color, thus obtaining an
artificial-looking image of the extent of the pathology in the
tissue. The filter depicted in FIG. 12B wherein one of the
positions is a neutral filter uses the image generated by the
neutral filter (which will appear on the display having shades of
gray) as the background image of the tissue on which the pathology
is superimposed in any desired color. To maintain the system of
FIG. 11 in calibration, a standard white reflector is used, as
described above for FIG. 10.
[0105] FIG. 13 depicts an embodiment in which the apparatus is
configured as a screening device 1570 without providing for direct
visualization of the tissue being screened by an observer. In the
illustrative embodiment, the screening optical head 1571 contains
an optical train 1572, an illuminator 1573, a CCD array 1574 and a
filter wheel 1575. The filter wheel 1575 is rotated as previously
described, either continuously or in a stepping fashion with a
stepper motor, 1576. The light source is within the data
processing/control console 1577, and the data is displayed on a
display 1578. Light from the light source is collected into a
bundle of optical fibers 1573 which is an integral part of the
cable 1579, between the console 1577 and the screening device 1571.
While FIG. 13 shows the optical fibers 1573 as nested within the
screening device 1571 at one location, it is understood that the
fibers within the bundle can be arranged circumferentially or in
any other geometric arrangement in order to provide homogeneous
illumination of the target tissue 1580.
[0106] FIG. 14 shows a filter wheel 1600 suitable for use with the
device depicted in FIG. 13. The illustrated filter wheel 1600 is
configured to select light at wavelengths of 380 nm, 480 nm and 650
nm, with an open area to facilitate synchronization. It should also
be evident to practitioners of ordinary skill that any arrangement
of filter wheels understood in the art, including those illustrated
in FIGS. 12A-12C above, could be used in the depicted system, with
the operation of the apparatus being adjusted accordingly.
[0107] In operation the screening device 1571 may be pointed to the
target tissue 1580. The tissue may be illuminated through the
optical fiber bundle 1573 and reflections from the tissue may be
recorded by the CCD array 1574 at about 380 nm, about 480 nm and
about 650 nm. The data processing unit 1577 analyzes the recorded
data using any one of the algorithms described above. Tissues with
color enhanced pathologies are represented on the display 1578. In
one embodiment of the invention, visual display is not provided and
only a reading or printout of the status of the subject (having or
nor having a pathology in the target tissue) is presented. In this
embodiment, the instrument advantageously uses the above-mentioned
tissue integral algorithm. To use this algorithm, the data
processing unit 1577, after obtaining the values R.sub.ij,650 and
R.sub.ij,380 for each pixel P.sub.ij, determines the maximum and
minimum values obtained for R.sub.650 and R.sub.380. If the
conditions R.sub.650(max)<1.2R.sub.650(min) and
R.sub.380(max)<1.2.sub.380(min) are met, the subject is clas
free of pathologies. Otherwise, the subject is referred for
additional diagnostic evaluation to determine the nature and the
extent of the suspected pathology.
[0108] In another illustrative embodiment, depicted in FIG. 15, the
filter wheel is eliminated. Further, in lieu of a standard CCD
array a color CCD array may be used. In the illustrated embodiment,
a screening device 1700 includes three modules 1701, 1704 and 1703.
The first module, the screening probe 1701, is operably connected
to the second module 1704 through a cable 1703. The first module
1701 contains a circumferentially arranged optical fiber bundle
1706 for transmitting light to a target 1710, and an optical path
1702 comprising optical elements for receiving light emitted from
the target 1710. The second module 1704 contains a light source
coupled to an optical fiber bundle 1706. The fibers in the optical
fiber bundle 1706 are distributed circumferentially at the distal
end of the probe. Furthermore, the second module 1704 contains a
data processing unit, including an electronic frame grabbing
submodule to process data received from the color CCD array 1900 in
the probe module. Results are displayed on the display module 1705,
which is connected to the second module 1704 by way of cable
1707.
[0109] The color CCD array 1900, as used in the illustrated
embodiment, may be typically divided into pixels each having four
elements. FIG. 16 shows a segment of the surface of such an array
1800. For illustrative purposes, an array of 10.times.10 elements
organized as an array of 5.times.5 pixels is shown. However, it is
understood that such an array can comprise in excess of
500.times.500 elements and thus more that 250.times.250 pixels.
Each one of the pixels 1801 has two green filters 1802 and 1803,
overlaying two of the elements of the four elements in pixel 1801.
The other two elements, 1804 and 1805, have respectively a red and
a blue filter overlaid thereupon. While the specific filters
employed in standard color CCD devices can vary from the three
wavelengths selected above, and can vary from manufacturer to
manufacturer, standard color CCDs can be used in the invention.
[0110] The operation of the device 1700 depicted in FIG. 15 is
similar to the operation of the system depicted in FIG. 13, except
that no filter wheel is employed. In contrast to the embodiment
depicted in FIG. 13, in the embodiment of FIG. 16 the whole image
in three chroma is taken at once, and the frame grabbing module
transfers the intensities received for each one of the three colors
to a data processing device which undertakes the normalization of
the long and short wavelength reflection with the middle of the
spectrum responses and proceeds to apply to the two artificial
intensities so derived any of the previously described
algorithms.
[0111] In the illustrative embodiment, the system optics 1702
images the target tissue 1710 onto the color CCD 1800, and the
signal from each pixel is captured in a frame grabbing device in
module 1704. The intensities registered for the two green filtered
elements are averaged and used as the normalization value for the
intensities registered for the red filtered element and for the
blue filtered element. In this fashion, normalized values
R.sub.ij(B) and R.sub.ij(R) are obtained for each pixel having a
row i and a column j. These normalized values respectively
represent the normalized reflected intensities in the blue and red
part of the spectrum.
[0112] While the filters used in commercial color CCD do not
correspond exactly to the wavelength 380 nm and 650 nm mentioned
above, and furthermore the bandwidth of those filters are
relatively wide, satisfactory calibration and discrimination
between pathological and healthy tissue can be achieved. The
threshold values can be changed for R(B) and R(R) relative to those
shown above for R.sub.380 and R.sub.650. These values vary somewhat
depending on the source of the color CCD. To alleviate the problem
of variability, an array of filters with the appropriate fixed
wavelengths of about 380 nm, about 480 nm and about 650 nm can be
overlaid over a standard CCD array to obtain a screening device
that has no moving parts (such as the filter wheel) in some of the
embodiments mentioned above. The general algorithm
R.sub..lambda.(Max)<.alpha.R.sub..lambda. (min) where
.alpha.>1 and is a function of the specific .lambda. selected is
advantageously employed without undue experimentation by ordinary
skilled practitioners in these arts to discriminate between healthy
and pathological tissue.
[0113] In another illustrative embodiment, these systems and
methods are used in conjunction with an acetic acid delivery
system, as shown in FIG. 17. FIG. 17 shows a graph 1900 of measured
reflectance I, as normalized by the initial reflectance immediately
after the application of the acetic acid, as function of time,
beginning with the application of an acetic solution to the cervix,
at a wavelength of about 600 nm. The graph 1900 has normalized
intensity plotted along the vertical axis 1902 and time expressed
in seconds plotted along the horizontal axis 1904. The data
collection is achieved by using a single narrow bandpass filter,
transmitting around 600 nm, overlaying a CCD. FIG. 17 represents
measurements from a number of tissue samples (in vivo, followed by
determination of the pathology from biopsies). The time dependence
of the reflected responses from tissues fall into three well
differentiated zones. Healthy (NED) tissues have a response 1910
which is independent of time. CIN II tissue shows an increasing
response 1920 for about 30 to 60 seconds, and then the reflectance
slowly fades out and returns to normal within about three minutes
or a little longer. CIN III tissue shows a response 1930 that
includes a strong change with time for the first 20 to 30 seconds,
and then, the response 1930 stays strong for longer than about
three minutes. FIG. 17 shows that the optical behavior of the
various tissue classes following the application of the amplifying
agent differentiates between healthy and pathologic tissue from
measurements taken during the first 10 to 20 seconds after the
application of the amplifying agent (or chemical agent), in this
case acetic acid.
[0114] A useful algorithm employs the rate of change of the
normalized intensity I with time, dI/dt at between 10 to 20 seconds
after the application of the amplifying agent. According to this
algorithm, if dI/dt <0.055 sec.sup.-1, the tissue is classified
as healthy (NED). If 0.075 sec.sup.-1<dI/dt<0.11 sec.sup.-1,
the tissue is classified as CIN II. If di/dt>0.11 sec.sup.-1,
the tissue is classified as CIN III. In one embodiment, the higher
dI/dt during the first 10 to 20 seconds after the application of
the acetic acid solution, the more severe the pathology is.
[0115] In another embodiment, an algorithm involves the measurement
of the normalized reflectance after either 10 or 20 seconds from
the application of the acetic acid solution. If I is greater than
1.25 after 10 seconds (or about 1.5 after 20 seconds), the tissue
is classified as pathologic, and the patient is directed to have a
more detailed analysis of the condition, sometimes, including a
biopsy. This embodiment is applied, as an example, when the probe
is used in true screening situations rather than in more
traditional colposcopic examinations.
[0116] In another embodiment, an algorithm is based on the time
required to reach a maximum in the back reflected response of the
tissue. According to this embodiment, the longer it takes to reach
this maximum the more severe the condition, providing, however,
that the maximum is more than about 3.0 times the minimum back
scattered response for the same tissue. The disadvantage of this
approach is that longer exposure may be required, particularly in
the case of CIN III, where back scattered responses continue to
increase even after more than 200 seconds.
[0117] To shorten that time interval, another algorithm compares
the maximum normalized response at 600 nm during any interval of
time greater than 10 seconds from the application of the acetic
acid solution, to the initial response, and if that response is
more than 30% larger than the initial response, the tissue is
classified as CIN in general. This algorithm is used when fast
classification of cervixes in a screening environment is
desired.
[0118] In yet another embodiment of the invention, a screening
algorithm takes an ititial reading of responses for each point
probed prior to the application of the acetic acid, stores the
values as a standard set, and then takes a number of images
sequentially. The screening algorithm performs a point by point
subtraction of the value of the stored initial responses from the
responses obtained after the application of the acetic acid. The
time dependence for various classes of tissue results in
distributions similar to those shown in FIG. 17, except that the
scale of the back scattered intensities is now changed. The
algorithm utilizes the differential responses of various classes in
a manner as described above.
[0119] Apparatus and methods for controlled delivery amount and
delivery pattern of a chemical agent are disclosed below. Apparatus
and method for accurately and synchronously triggering the optical
measurements with regard to the time of delivery of the chemical
agent are disclosed below. Image capture software to record
time-stamped images and user-defined regions of interest to be
defined on a master image is disclosed below. This analysis
software automates the calculation and display of acetowhitening
characteristics from a motion corrected time-sequence of patient
images. This improves the ability to correlate instrument
measurements to the pathological evaluation of biopsied tissue.
[0120] In another embodiment of the invention, when using an
amplifying agent such as acetic acid, an automated system delivers
the amplifying agent to the target tissue. A triggering mechanism
applies the chemical reproducibly and eliminates variability of
time delays between the application and the start of obtaining
optical responses from the target tissue.
[0121] FIG. 18A shows an illustrative embodiment of a system with a
screening probe 2000 similar in construction to the probe shown in
FIG. 11. This apparatus and the associated techniques are used to
improve the demarcation of a start time and to guarantee the
application of a constant volume of acetic acid. Another embodiment
is a mucosal atomizer device comprising a 3 cc syringe and 6"
stylet tubing extension nozzle is used to spray 2 cc of 5% acetic
acid uniformly onto the surface of the cervix. The distal part 2013
of the probe 2000 is covered with a composite disposable sheath
2011 having attached at its own distal periphery a hollow toroidal
structure 2014. The hollow toroidal structure 2014 contains the
chemical agent or amplifying agent, for example, acetic acid at a
3% to 5% concentration. A retractor 2012 compresses the toroidal
structure 2014 when the operator is ready to apply the amplifying
agent to the target tissue. When the retractor 2012 is retracted
using a trigger like mechanism 2015 the toroidal structure 2014 is
compressed and its content is sprayed onto the tissue.
Simultaneously with the retraction action, the probe is signaled to
start taking measurements, by the actuation of a switch 2016 in the
handle of the probe. The chemical or amplifying agent is sprayed
through a plurality of perforations 2017 as shown in FIG. 18B, a
top view of the distal end of the assembled sheath/retractor
assembly. To prevent accidental expulsion of the solution, a
covering such as an adhesive tape may be attached to the distal end
2013, covering the toroidal structure 2014. In one embodiment, the
covering may be removed after mounting the sheath on the probe and
just prior to the insertion of the probe into a target area such as
the vagina. While FIG. 18A depicts a cylindrical structure, it
should be apparent to those of ordinary skill in the art that a
conical structure can be utilized to improve packaging and nesting
of multiple disposable sheaths, and furthermore, that any other
structure may be constructed which is adapted for the functions
depicted in FIG. 18A and which is further adapted for the anatomic
region in which it will be used.
[0122] In the embodiment shown in FIGS. 18B and 18C, the retractor
2012 is designed to leave an optical window 2018 for the reception
of responses from the illuminated tissue. Illumination is achieved
through circumferentially distributed optical fibers, as previously
described in FIG. 11, and the light is transmitted through a
transparent part of the peripheral distal end of the retractor
2012. The toroidal container 2014 for the amplifying agent is
affixed to the sheath 2011, as shown in FIG. 18C, which shows a
cross sectional view of the distal end of the sheath/retractor
assembly. The toroidal container 2014 is directly attached to the
sheath (as shown at 2019) or it is affixed in any other suitable
way.
[0123] While in FIG. 18A we show a toroidal container 2014 which
discharges its content upon compression, it should be clear that
other shapes may be useful in the practice of the invention. For
instance, the cross section of the toroidal container 2014 can be
oval or rectangular. In certain embodiments, the amplifying agent
container is constructed as a side mounted syringe having a plunger
that causes discharge of the amplifying agent in a spray form,
while providing simultaneously a signal to the probe that
amplifying agent is applied, and thus providing a starting point
for the temporal measurements of reflections from the target
tissue. While this figure shows one embodiment of a system for
automating the application of the amplifying agent and for
standardizing the time lapse between the application of the
amplifying agent and the measurement of back scattered responses
from the target tissue, it should be evident to a person trained in
the art that other mechanisms achieving the same goal can be
devised without deviating from the spirit of the invention. Such
other applicators could include, but are not limited to, surgical
applicators like sponges, cloths or swabs that are made to be
retracted after the application of the amplifying solution so leave
open the optical path between the tissue and the probe's distal end
2013.
[0124] When the algorithms use normalized responses, as normalized
against time zero, the trigger actuates a timer within the probe
controller that sets up a predetermined time interval for the first
measurement (typically within I second of amplifying agent
application). When the algorithm used normalizes responses relative
to the response obtained prior to the application of the amplifying
agent, an image of the cervix is taken prior to the application and
recorded with the frame grabber in the data processing unit 1704.
After the trigger 2016 signals the probe to start taking responses,
the responses are taken and normalized (pixel by pixel) and one of
the algorithrns described above analyzes the data. The data are
presented as either a "positive" or "negative" finding for the
whole cervix, or alternatively, an artificial image of the
pathology is presented for those pixels where the algorithms
returned positive findings. This image is superimposed on a visual
image of the cervix and recorded to allow post screening accurate
location of tissue requiring subsequent biopsy.
[0125] In some embodiments of the invention, spatial data are
averaged over groups of neighboring pixels (between 2.times.2 to
6.times.6), and these averages (both for the standardizing
measurement, or normalizing measurement) are used as normalized
intensities. Other methods for averaging or normalizing spatial
data can be used. Different methods of normalizing can be related
to the resolution of the CCD used in that specific interest.
[0126] In another embodiment, a plurality of chemical agents are
applied to a specimen, either simultaneously or sequentially. The
use of multiple chemical agents causes any of a number of different
effects. One chemical agent is used to control or change pH (e.g.,
hydrogen ion concentration), change the concentration of one or
more other ionic species, or change an osmotic pressure, while
another chemical agent is used to induce another sort of change,
for example, staining a material, activating or passivating a
material, or otherwise changing a physical property of a
material.
[0127] Application of an exogenous contrast agent when combined
with the activation of an endogenous contrast agent gives rise to a
combined contrast that provides more valuable information than
either agent alone. For example, application of acetic acid to
epithelial tissue results in time-dependent effects in the
fluorescence emission spectrum resulting from activation of
endogenous native fluorophores in tissue, such as NADH, collagen,
elastin, favins (e.g., FAD) or porphyrins.
[0128] This effect arises from at least two different sources. One
source is the penetration of the acetic acid into the tissue
followed by the resulting pH change on the spectral properties of
the endogenous fluorophore. The effect of pH is shown for NADH in
FIG. 19. FIG. 19 shows four absorption spectra recorded over the
wavelength range of about 320 nm to aboout 600 nm. The absorbance
is plotted along the vertical axis 2102 and the wavelength in nm is
plotted along the horizontal axis 2104. A baseline spectrum 2110 is
taken for a 5% acetic acid solution containing no NADH, and shows
little absorption. The spectrum 2120 is taken at pH 4.0 and shows
two strong absorption peaks at approximately 350 nm and at
approximately 430 nm. The spectrum 2130 is taken at pH 5.0 and
shows a strong absorption peak at approximately 350 nm and a much
weaker absorption peak at approximately 430 nm as compared to the
pH 4.0 spectrum. The spectrum 2140 is taken at pH 7.0 and shows a
strong absorption peak at approximately 350 nm and virtually no
absorption peak at approximately 430 nm as compared to the pH 4.0
spectrum. The spectral properties of NADH absorption are
significantly affected by pH. Since absorption is the first step in
fluorescence, it is reasonable to expect that pH will affect the
emitted fluorescence as well.
[0129] Acetic acid penetrates into different types of tissues and
cells at different rates depending on the type of tissue present.
In addition, the amount of NADH in cells typically differs
according to the type of cell and its metabolic state. Consequently
the kinetics of this pH response can be indicative of the tissue or
cell type and its metabolic condition.
[0130] Acetic acid causes acetowhitening when applied to certain
tissues, such as epithelial surfaces. The acetowhitening effect is
produced by light scattering changes. These changes have further
secondary effects on spectral measurements, such as induced
fluorescence. Changes in the induced fluorescence result from
either of two sources. One source is the direct effect of
acetowhitening on the penetration of the UV excitation light. A
second effect results from the light scattering on the observed
spectral shape of the emitted fluorescence. Since the
acetowbitening is time dependent, these secondary effects are time
dependent as well.
[0131] Temporal changes observed in fluorescence emission following
the application of acetic acid to a cell suspension is shown in
FIG. 20. FIG. 20 shows a graph 2200 of spectra measured as a
function of time after application of acetic acid. The spectral
intensity is plotted along the vertical axis 2202 and the
wavelength in nm is plotted along the horizontal axis 2204. Curve
2210 is recorded at the time of application of the acetic acid
solution. Curve 2220 is measured 0.5 minutes after acetic acid
application. Curves 2230, 2240, 2250 and 2260 are recorded 1, 2, 3,
and 7 minutes after acetic acid application, respectively. The
fluorescence spectrum originally observed at the time of acetic
acid application quickly decreases in intensity, and recovers
slowly thereafter.
[0132] Fluorescence spectra in cervical tissue have similar changes
over time following application of acetic acid. FIG. 21 is a graph
2300 that shows the changes in fluorescence spectra before (curve
2310) and after (curve 2320) application of acetic acid to cervical
tissue. The spectral intensity is plotted along the vertical axis
2302 and the wavelength in nm is plotted along the horizontal axis
2304. Note that the fluorescent intensity at some wavelengths below
about 450 nm changes more substantially than the intensity at
wavelengths above about 450 nm. Since the time-course of those
changes is related to the type of tissue being probed (as described
above) these spectral differences can differentiate the type of
tissue under study.
[0133] Relative motion between the patient and the colposcope can
cause problems with registration of the different images for that
patient during analysis. A robust motion detection and correction
technique is disclosed below. This technique uses the
cross-correlation of two successive images to determine global
motion. The cross-correlation is computed in the Fourier domain
using a fast Fourier transform. In one embodiment, the image
registration technique is used after the specimen data is
collected. In an alternative embodiment, systems according to the
invention incorporate the technique on-line, as it does not require
excessive processing overhead.
[0134] Image processing is used to extract relevant features from
the data observed and recorded using the systems and methods of the
invention. Image processing techniques that can be applied include,
but are not limited to, color space transformations, filtering,
artifact detection and removal, image enhancement, extraction of
three-dimensional shape information, manipulations using
mathematical morphology, and segmentation.
[0135] A color space transfromation is intended to transform the
three primary colors, red (R), blue (B), and green (G), into a new
set of colors or values using a kinear or non-linear
transformation. A number of well-known transformations interconvert
R,G, B and luminance/chrominance components, for example, as used
in converting light recorded in a camera into broadcast signals,
and rendering broadcast signals on a television display. Filtering
is useful in image processing, and is used to suppress noise and
unwanted interfereing signals. Many filters and filtering processes
are known. Filters can include both hardware filters such as
optical filters and electronic filters, as well as filters applied
in software, such as digital filters. For example, the median
filter replaces every pixel of an image with the median value
computed in a given neighborhood of the pixel.
[0136] Artifact detection and removal is used to eliminate spurious
information from a set of data to be analyzed. Some artifacts, such
as portions of an optical field of view that are extraneous, may be
eliminated by changing the height and or width of the field of
view, or by masking portions of the field of view, for example when
a physician observes that the field of view includes material that
is not of interest.
[0137] Image enhancement can include processing to improve the
visual contrast between adjacent portions of an image. A number of
known techniques are available, including applying a weighting
function to a range of intensities or gray scale values.
[0138] Extraction of three-dimensional shape information is useful
in representing a surface that is non-planar in two-dimensions. An
example is computing the three-dimensional features of the cervix
to account for the nonuniformities of illuminating a
three-dimensional surface.
[0139] Manipulations using mathematical morphology are well-known.
Image processing using the principles of mathematical morphology
provides a representation of an image in a form that simplifies the
computational burden in image processing.
[0140] Morphological operators are based on the mathematics of set
theory. A set in mathematical morphology represents the shape of an
object in an image. In the case of two-dimensional (binary) images,
the sets are members of Z.sup.2 and each element represents the
(x,y) coordinates of a black (or white, depending on the
convention) pixel in the image. Gray-scale, color, time-varying
components, or any vector-valued information can be included by
extending the Euclidean space size.
[0141] The basic morphological operators are described in terms of
gray-scale images below. Let the input image be described by a
function f: Z.sup.2.fwdarw.R. Gray-scale dilation is defined
as:
(f.sym.b)(v,w)=max{f(v-x,w-y)+b(x,y).vertline.(v-x,w-y).epsilon.
D.sub.f;(x,y).epsilon. D.sub.h}
[0142] where b: Z.sup.2.fwdarw.R is a function called a structuring
element, D.sub.f is the domain offand D.sub.6 is the domain of b.
The structuring element has a key role in this operator: it is
added morphologically to the image at each pixel location.
[0143] The opposite of dilation is erosion. The erosion operator is
defined as:
(f.sym.b)(v,w)=min{f(v+x,w+y)-b(x,y).vertline.(v+x,w+y).epsilon.
D.sub.f;(x,y).epsilon. D.sub.h}
[0144] In this case the structuring element is subtracted
morphologically from the image at each pixel location.
[0145] Two important morphological operators are defined using
erosion and dilation: opening and closing. They are respectively
defined as:
f.multidot.b=(f.THETA.b).sym.b
f.multidot.b=(f.sym.b).THETA.b.
[0146] The effect of opening is to preserve holes and remove peaks,
while closing preserves peaks and closes holes according to the
structuring element's shape. The structuring element b is fitted
from inside (below an image) in the opening case and fitted from
outside (above an image) in the closing case.
[0147] A morphological filter can be defined as any combination of
morphological operators. For example (fo b).multidot.b, opening
followed by closing, or (f.multidot.b)o b, closing followed by
opening. These operators are neither commutative, nor associative
or distributive and the filtering operators cited above are not
equal. One of the following two filters is used: 1 f_b = 1 2 [ ( f
b ) + ( f b ) ] , and f_b = 1 2 [ ( f b ) b + ( f b ) b ]
[0148] where the _ symbol means f filtered by b.
[0149] A more elegant way to achieve a morphological filtering with
better geometrical characteristics is to use geodesic
reconstruction after a morphological opening. The reconstruction
process uses geodesic dilation which for gray-scale images is
defined by:
(f.sym.b).sup.(1)(v,w)=min{(f.sym.b)(v,w),
f.sub.0(v,w)},(v,w).epsilon. D.sub.f,
[0150] where f.sub.0 is the reference image, usually the original
image, and g is a small structuring element, usually a four pixel
(cross) or eight pixel (square) connected element. Geodesic
reconstruction is obtained by repeating the geodesic dilation n
times ((f.sym.b).sup.(n)) until idempotency is reached. The
geodesic reconstruction is then written:
Rg=(f.sym.b).sup.(i), with
(f.sym.b).sup.(i+1)=(f.sym.b).sup.(i)
[0151] An equivalent operator can be defined for reconstruction
after morphological closing which uses geodesic erosion. For
gray-scale images it is defined as:
(f.sub.--b).sup.(1)(v,w)=max{(f.sub.--b)(v,w),
f.sub.0(v,w)},(v,w).epsilon- . D.sub.f
[0152] An example of geodesic reconstruction after morphological
opening suppresses the square shape deformation introduced by the
opening process. These geodesic reconstruction operators
significantly improve any filtering process for a modest additional
computation time.
[0153] The most natural example of diffusion process is heat
transfer inside matter. This physical phenomenon is mathematically
expressed by the following partial differential equations: 2 q = -
k T , c p T t = - q + f ,
[0154] leading to the following second order elliptic equation: 3 c
p T t = - ( k T ) + f ,
[0155] Heat transfer involves a thermal flux q. The whole system
must obey the law of energy conservation. The symbol .gradient. is
the differential operator, which is defined as
.gradient.=(.differential./.differential..t- imes..sub.1, . . . ,
.differential./.differential..times..sub.d). The parameterp is the
density of the medium, k is the thermal conductivity, c is the
specific heat capacity, and f the capacity of internal heat
sources. An analogy exists between temperature variation and value
variation in images. The basic formulation is obtained when the
medium is assumed to be homogeneous, without sources and with
constant conductivity.
[0156] In image processing applications the ideal objective is to
obtain an image where only strong edges are preserved while noise
and small structures are smoothed out. Diffusion is used as an edge
preserving filtering method. The thermal conductivity is replaced
by a conductivity function which adapts the diffusion to the local
gradient: decreasing diffusion for increasing gradient. The above
diffusion equation becomes: 4 v t = ( D u ) ,
[0157] where v(x,t) is the signal value at time t and position x,
and D is a conductivity matrix. The latter defines the type of
diffusion:
[0158] if D reduces to a constant value k then the diffusion is
isotropic,
[0159] if D reduces to a nonlinear function g(.multidot.) then the
diffusion is nonlinear isotropic,
[0160] if D is a tensor whose elements are functions gij
(.multidot.) then the diffusion is anisotropic. The analysis uses
the case where D=g(.multidot.), and the following conductivity
function: 5 g ( ) = { 1 if k o . k o / if > k o . , f ( ) = { 1
if k o . 0 if > k o . ,
[0161] Histogram equalization re-assigns pixel values in order to
obtain a uniform distribution. Let v(x) be the pixel value at
location x and P(v) be the probability density function associated
to v. The following transformation is used: 6 e q ( ) = 0 P ( s )
s
[0162] where 0.ltoreq.v.ltoreq.1. In the discrete case, the uniform
distribution is only approximated and the following equation is
used: 7 u k , e q = j = 0 k n j n ,
[0163] where n is the total number of pixels and nj the number of
pixels with value equal to j.
[0164] The fitting technique used is called linear least squares.
The idea is to fit a linear combination of arbitrary functions
(linear or nonlinear) given by: 8 l ( x ) = k = 0 M - 1 k f k ( x )
,
[0165] where x is an N-dimensional coordinate vector (N=2 in the
case of images), to a set of data l.sub.i(x.sub.i) with i=0, . . .
, n-1. In one embodiment, the following series of functions are
used:
1, x, x.sup.2, x.sup.3, . . . ,
[0166] in the 1-D case and:
1, x, yx.sup.2, xy, y.sup.2, x.sup.3, x.sup.2y, xy.sup.2, y.sup.3,
. . . ,
[0167] in the 2-D case.
[0168] The fitting criteria is the minimization of the following
least-square error: 9 x 2 = i - 0 n - 1 [ l i - K = 0 M - 1 k f k (
x i ) i ] 2
[0169] where .sigma..sub.i is the measurement variance at location
x.sub.i. In one embodiment, set .sigma..sub.i-1, .A-inverted.i=1, .
. . ,n-1.
[0170] By defining the n.times.M matrix A whose elements A.sub.ij
are given by:
A.sub.ij=f.sub.j(x.sub.i),
[0171] and the vector b of length n whose elements b.sub.i are
given by b.sub.i=l.sub.i, then the following system must be
solved:
a=(A.sup.TA).sup.-1A.sup.T.b,
[0172] where a=[.sigma..sub.0. . . .sigma..sub.m-1]. Since the
A.sup.TA product is positive definite, Cholesky decomposition can
be used to compute the inverse.
[0173] Segmentation is a morphological technique that splits an
image into different regions according to a pre-defined criterion.
In the analysis of the state of health of a biological specimen, it
is meaningful to compare the proprties of different areas of the
specimen. Segmentation is a method that directly provides
information on how many regions are present in the image of a
specimen, and the location of each region.
[0174] In one embodiment, colposcopic images are segmented to track
regions in a time series of images. Relevant features are extracted
from the labeled regions and their evolution is analyzed as a
function of time, to measure and localize acetowhitening effects.
Colposcopic images are segmented using a watershed based algorithm.
An efficient pre-processing scheme is used, as are two region
merging techniques. The use of markers to track the segmentation in
time-series of images is used, and the problem of global motion and
local deformations related to the precise tracking of these markers
is discussed.
[0175] A segmentation scheme for colposcopic images separates the
image of the cervix into a number of regions according to an
intensity criterion. Segmentation techniques are well known in the
mathematical morphology arts. In one embodiment, the object (e.g.,
the cervix) is known and multiple regions with different intensity
content within the cervix are to be identified.
[0176] A technique based on the structural and spatial information
rather than on the spectral information is suited to analyze
colposcopic images. One approach uses the watershed technique. The
watershed technique uses structural information. The watershed
technique provides a fine to coarse segmentation of an image in
combination with region merging techniques. The flooding technique
views a gray-level image as a 3-dimensional surface and
progressively floods this surface from below. Each local minimum in
the surface is thought of as a hole. A rising water level floods a
region as soon as a hypothetical water level reaches the associated
minimum. FIG. 22 illustrates this concept on a 1-D signal. The
arrows 2402a-2402d show the flooding origins and directions and the
solid lines 2406a-2406d are the watersheds. The flooded minima are
called catchment basins and the borders between neighboring basins
are called watersheds. Only the catchment basins are of interest.
They constitute the segmented image. Fast implementations use
first-in-first-out (FIFO) queues and sorted data.
[0177] FIG. 23 is a graph 2500 of a signal 2510 and its first
derivative 2520, both plotted with amplitude as a vertical axis
2502 and position as a horizontal axis 2504. FIG. 23 shows a signal
2510 used to represent watersheds with three distinct regions (hole
2512, plateau 2514, and peak 2516) and its derivative function
2520, or gradient. Image segmentation with the watershed transform
is performed on the image gradient 2520. The signal shows three
distinct regions, and the direct watershed transform would produce
a one lowest region. The gradient 2520 separates the signal into
its three regions. An analogous principle holds for two-dimensional
signals, i.e. images.
[0178] FIG. 24 shows a graph 2600 of a sigmoidal scaling function
2610 used to enhance the contrast between light and dark regions of
an image. The sigmoidal scaling function 2610 is plotted as output
along a vertical axis 2602 as a function of an input that is
indiated along a horizontal axis 2604. The use of the watershed
transform often leads to a severe over-segmentation. Pre-processing
is performed to reduce the number of regions in a segmented image.
In one embodiment, a pre-processing scheme reduces the number of
regions from several thousand to several hundreds:
[0179] An algorithm that treats images to provide a segmented image
includes the following steps:
[0180] computing luminance component L*;
[0181] performing 3-D shape compensation;
[0182] performing sigmoidal scaling;
[0183] morphological closing/opening with a cylindrical structural
element;
[0184] performing geodesic reconstruction;
[0185] computing image gradient;
[0186] computing threshold gradient;
[0187] performing closing; and
[0188] performing geodesic reconstruction of the gradient
image.
[0189] The uniform luminance component L* is well adapted to the
segmentation process, and is computed for an image of interest.
[0190] 3-D shape compensation removes an artifact (e.g., a
stair-case effect) in the segmented image. The illumination is
non-uniform when falling on a curved surface (e.g., the cervix),
which in turn influences the gradient values used for the
segmentation.
[0191] A sigmoidal scaling function is used to improve the contrast
between light and dark regions. FIG. 24 is a graph of a scaled
sigmoid, used in this study. The sigmoid increases contrast in the
median intensity range by providing a larger number of
quantification levels. Applying a sigmoidal scaling function causes
dark and very light areas to exhibit diminished contract, while the
limit between light and dark regions is enhanced.
[0192] Closing and opening morphological operators, respectively,
are used to suppress small regions corresponding to holes or peaks
in the images. The geodesic reconstruction keeps the geometrical
aspect of the image as close as possible to that of the original
image. A morphological closing with geodesic reconstruction is
performed on the gradient of the filtered image to remove plateaus,
which are visualized as separate regions in the watershed
transform. The diameter of the cylinder used as structuring element
defines the minimum size of the regions in the segmented image.
[0193] Finite-element approximations are used to compute
derivatives. Alternative approaches involve using a Sobel operator,
which is a 3.times.3filter. Another alternative is the use of a
local cubic polynomial approximation, which is a 5.times.5 filter.
Before processing the gradient for the watershed extraction,
application of a threshold removes small values.
[0194] In one embodiment, the gradient is computed using cubic
mean-square approximation. In another embodiment, a morphological
closing/opening filter with geodesic reconstruction is first
applied and then the gradient is computed. Spurious regions are
smoothed out and contrast enhanced in large regions by both
methods.
[0195] In one embodiment, the watershed algorithm is modified as
follows. The data is represented in floating point, and the
interval steps between successive flooded levels is not uniform.
Also, each watershed pixel is merged into a neighboring region
according to a nearest neighbor criterion.
[0196] The region merging step following the watershed transform
step reduces over-segmentation. In one embodiment, neighboring
regions are merged if an intensity change along their border is
greater than a given threshold. Alternatively, neighboring regions
are merged if a difference in mean intensity value is greater than
a given threshold.
[0197] In both embodiments, a map of all border pixels is used. For
each segment, a computer computes the difference between the mean
value of pixels of each of the two regions under consideration. If
the absolute value is below a given value, the segment is removed
from the mask, and the regions are merged. The final mask is used
to map out the gradient image. The watersheds are recomputed.
[0198] An alternative merging algorithm uses the same routines. The
alternative algorithm uses the mean value image as input in place
of the original image. Since all pixels in a region have the same
(mean) value, the algorithm works differently, in that border
segments are suppressed if the difference in mean value between
neighboring regions is smaller than a given threshold. A
morphological distance is an approximation of the distance, in
pixels, from a pixel to the nearest segment border. A method to use
markers to track the segmentation in time series of images is now
presented. The extraction of markers is necessary in order to
initialize the flooding process in the watershed transform computed
in successive images (e.g., in time-series).
[0199] The approach used comprises the steps of finding pixels
having minimum value for each region, and selecting the minimum
with the largest morphological distance for each region.
[0200] The first step selects the minimum value as an initial
marker, since the flooding used in the watersheds start at local
minima. The pixel with minimum value and largest morphological
distance is used to avoid a small deformation of a region pushing a
marker outside of the region.
[0201] A homotopy modification of the gradient image obtained with
the markers is used to suppress catchment basins corresponding to
minima that have not been marked, in order to speed up the
computation. The homotopy modification of v is the geodesic
reconstruction of v (mask) from {acute over (v)} (marker). 10 ~ ( ,
l ) = ( ( , ) if ( , ) M max y ( , ) otherwise .
[0202] In one embodiment, more than one marker per region is
considered. Pixels having a morphological distance greater than a
given value (typically 2-3) or being at least equal to the largest
morphological distance within a given region are considered. These
markers are used to zero out gradient values in the following
image, in order to reduce influence of local maxima on the homotopy
modification. It is assumed that the borders between neighboring
regions are located somewhere between the marked regions.
[0203] Further, the markers are used to initialize the watershed
algorithm with the gradient image of the next image. Tracking
schemes are employed to take into account global and local
motion.
[0204] One illustrative tracking scheme for the detection of
patient motion during an acquisition cycle uses the
cross-correlation of two sub-images of two successive images to
determine the global motion. An alternative algorithm is used to
track motion for segmentation tasks.
[0205] Typically, images of specimens exhibit large homogeneous
regions which are difficult to track. Structural information is
used to improve motion tracking. Derivatives are used instead of
the original image. Using the gradient improves the system's
sensitivity to glare, while the use of the sum of the gradients in
both the x and y directions highlights low-contrast structures.
Using the Laplacian operator (the sum of second derivatives)
provides similar results. Applying a low-pass filter before
computing the derivatives yields results similar to the Laplacian
of Gaussian used for edge detection. The low pass filter suppresses
noise and smoothes out glare.
[0206] The embodiment further comprises three modifications. The
true cross-correlation of successive images is computed in a
512.times.512 pixels window. The two windows used for each image
are different and are of sizes 302.times.302 and 210.times.210,
respectively. A Hamming window is used to extract these two
signals. The two windows have different sizes to make sure that the
second signal is completely contained in the first one, and the use
of a Hamming window avoids small oscillations, especially at the
transition of the selected signals and the zero-padded areas. The
equations used for the motion detection are given below.
[0207] Optical flow algorithms are used to measure local motions.
In order to save computation time, the optical flow is computed
only for the markers. Optical flow is defined as the distribution
of apparent velocities of movement of brightness patterns in an
image, and is used to reconstruct three-dimensional surfaces in
medical imaging applications.
[0208] Additional embodiments of motion detection algorithms
include the following steps: implementation of a local deformation
tracking system for improving the precision of marker tracking;
extraction of feature signals from the series of segmentation
results; analysis of feature signals and classification into groups
of interest; and use of group information to correlate the
evolution with the histology.
[0209] For motion detection, only a single frame is used. The three
RGB color components are transformed into a single intensity
component using the following relationship:
I=0.299.multidot.R+0.587.multidot.G+0.114.multidot.B
[0210] In order to suppress high-frequencies due to noise and to
the interlaced video signals, we apply a Gaussian low-pass filter
to the intensity component: 11 g ( x ) = 1 2 2 exp ( ( x - ) T ( x
- ) 2 2 )
[0211] where {overscore (.mu.)} is the center (mean value) of the
Gaussian and .sigma. is its standard deviation. Finally, we use
only derivative information to compute the translation parameters,
either the sum of derivatives 12 x + y
[0212] or the Laplacian 13 2 x 2 + 2 y 2 .
[0213] Motion can be detected using a cross-correlation operator
applied to two successive images in a sequence.
[0214] The cross-correlation is computed in the Fourier domain
using a fast Fourier transform. The following relationship is
used:
.PHI.=X.multidot.Y*
[0215] where .PHI., X, and Y are the Fourier transform of the
cross-correlation function, the first, and the second signal,
respectively. The * symbol represents the complex conjugate. Note
that the cross-correlation of two signals of length N.sub.1 and
N.sub.2 provides N.sub.1+N.sub.2-1 values and therefore, in order
to avoid aliasing problems due to under-sampling, the two signals
must be padded with zeros up to N.sub.1+N.sub.2-1 samples.
[0216] For discrete signals (i.e. sampled and quantized signals),
the discrete Fourier transform (DFT) and the inverse discrete
Fourier transform (IDFT) are given respectively by: 14 v ( k , l )
= m = 0 N - 1 n = 0 M - 1 u ( m , n ) exp ( - j 2 m k N ) exp ( - j
2 nl M ) u ( m , n ) = 1 NM k = 0 N - 1 l = 0 M - 1 v ( m , n ) exp
( j 2 m k N ) exp ( j 2 n l M )
[0217] This transform expands the signal onto an orthonormal basis
of exponential functions. Once the inverse discrete transform is
computed, the location of the maximum value corresponds to the
translation necessary to align both images.
[0218] Different types of windows are used for spectral analysis,
when only part of a signal is analyzed. The goal is to avoid
oscillation around discontinuities (Gibbs phenomenon). A Hamming
window can be used, which is given by the following
relationship:
.omega..sub.h(k)=1/2[1+cos(2.pi.k/N)]
[0219] where N is the number of samples, k is the sample index, and
-N/2.rarw.k .rarw.-N/2. In the frequency domain, the Fourier
transform of the signal is convolved with the Fourier transform of
the Hamming window. For the two-dimensional case the Hamming window
is constructed as a separable function, i.e.
.omega..sub.h(k,l)=.omega..sub.h(k).multidot..om- ega..sub.h(k),
where (k,l) are the pixel coordinates.
[0220] As mentioned above, in some embodiments, the
cross-correlation of the sum-of-derivatives images is the basis of
a motion detection algorithm.
[0221] The cross-correlation of two images in a sequence provides
information about the translation necessary to obtain the best
match in the inner-product sense. However, this does not
necessarily mean that the two images are perfectly aligned. A
validation method is necessary to measure the "quality" of the
matching.
[0222] The Sobel operator is given by: 15 fs = 1 8 ( - 1 0 1 - 2 0
2 - 1 0 1 )
[0223] This filter is obtained by convolving the finite element
approximation to derivatives with a weight matrix: 16 fs = 1 2 ( 0
0 0 - 1 0 1 0 0 0 ) ** 1 4 ( 0 1 0 0 2 0 0 1 0 )
[0224] where ** is the two-dimensional convolution. The second
filter in Equation 2 is a low-pass filter in the direction
perpendicular to the derivative operator, which renders the filter
less sensitive to noise. The derivative along the y-axis is
obtained by using the transposed version of the Sobel operator.
[0225] Another way to compute derivatives is to use a local
polynomial approximation by minimizing the mean-square error (MSE)
with the underlying image pixels. The approximation is given by: 17
( , ) i = 0 8 b i i ( , )
[0226] where (k,l) are the coordinates in the local 5.times.5
domain centered on the current pixel. The b.sub.i coefficients are
the optimal weights in the MSE sense and the .omega..sub.i are
orthogonal polynomials (1, k, l, k.sup.2-2/3, l.sup.2-2/3, kl,
(k.sup.2-2/3)l, (l.sup.2-2/3)k, (k.sup.2-2/3)(l.sup.2-2/3) the
following filter for the first derivatives: 18 f = 1 50 ( - 2 - 1 0
1 2 - 2 - 1 0 1 2 - 2 - 1 0 1 2 - 2 - 1 0 1 2 - 2 - 1 0 1 2 ) - 17
300 ( - 1 2 0 - 2 1 - 1 2 0 - 2 1 - 1 2 0 - 2 1 - 1 2 0 - 2 1 - 1 2
0 - 2 1 ) + 1 144 ( 4 2 0 - 2 - 4 - 2 - 1 0 1 2 - 4 - 2 0 2 4 - 2 -
1 0 1 2 4 2 0 - 2 - 4 )
[0227] The center of each image is divided into an 8.times.8 array
of blocks of size 32.times.32. The array is chosen to avoid the
image borders. The borders can contain extraneous material, that
is, not part of the cervix. In normal use, the physician attempts
to keep the cervix in the middle of the image.
[0228] For each of the blocks the normalized inner product with the
corresponding block in the adjacent motion compensated image is
computed: 19 P i , j = X B i , j I 2 ( x ) I 2 ( x ) B i , j I 1 2
( x ) x B i , j I 2 2 ( x )
[0229] where B.sub.ijN.sup.2 is the domain of definition of block
(ij), and I.sub.1,2 are the two processed images. The absolute
value of P.sub.ij is used as a quality measure.
[0230] The method is used for series of 28 images and the motion is
estimated for each of them, except for the first one. Once the
motion parameters are determined, the frame is shifted to its
computed "correct" location and the above block-based correlation
is computed with the previous shifted image. The result obtained
when using the intensity values for each image is plotted with the
x-axis corresponding to the different blocks while the y-axis
corresponds to the different images. For the example presented
above, the shifted images match perfectly and the output is zero
intensity everywhere.
[0231] In an alternative embodiment, in which edge information is
the only information used in the matching correlation, the
intensity images are replaced with sum-of-derivatives images. To
date, this embodiment has provided less favorable motion
compensation than the other embodiment. However, the sum-of
derivatives approach appears to provide better identification of
sudden or gross motion.
[0232] While the invention has been particularly shown and
described with reference to specific preferred embodiments, it
should be understood by those skilled in the art that various
changes in form and detail may be made therein without departing
from the spirit and scope of the invention as defined by the
appended claims.
* * * * *