U.S. patent application number 15/275760 was filed with the patent office on 2017-01-12 for in-vivo retinal imaging system.
The applicant listed for this patent is CITY UNIVERSITY. Invention is credited to Stephen GRUPPETTA.
Application Number | 20170007121 15/275760 |
Document ID | / |
Family ID | 43401718 |
Filed Date | 2017-01-12 |
United States Patent
Application |
20170007121 |
Kind Code |
A1 |
GRUPPETTA; Stephen |
January 12, 2017 |
IN-VIVO RETINAL IMAGING SYSTEM
Abstract
An optical imaging system (1) for in-vivo retinal imaging, the
system (1) comprising: an optical source (3) for generating
incoherent light in a plurality of wavelength bands; an optical
imaging sub-system (6) configured to split light from said optical
source (3) into a plurality of beams, to introduce a path
difference between said beams of light, and recombine those beams
to form interference fringes that are imaged on a subject (21); and
an image capture device (29) configured to capture light from the
subject (21) being imaged, and to form an image of said subject
(21).
Inventors: |
GRUPPETTA; Stephen; (London,
GB) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
CITY UNIVERSITY |
London |
|
GB |
|
|
Family ID: |
43401718 |
Appl. No.: |
15/275760 |
Filed: |
September 26, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14694153 |
Apr 23, 2015 |
|
|
|
15275760 |
|
|
|
|
13882596 |
Aug 5, 2013 |
9089289 |
|
|
PCT/EP2011/069375 |
Nov 3, 2011 |
|
|
|
14694153 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A61B 3/102 20130101;
G01J 3/4535 20130101; A61B 3/113 20130101; A61B 3/13 20130101; A61B
3/0008 20130101; A61B 3/14 20130101; A61B 3/12 20130101 |
International
Class: |
A61B 3/12 20060101
A61B003/12; A61B 3/00 20060101 A61B003/00; A61B 3/14 20060101
A61B003/14 |
Foreign Application Data
Date |
Code |
Application Number |
Nov 3, 2010 |
GB |
1018560.1 |
Claims
1. An optical imaging system for in-vivo retinal imaging, the
system comprising: an optical source for generating incoherent
light in a plurality of wavelength bands; an optical imaging
sub-system configured to split light from said optical source into
a plurality of beams, to introduce a path difference between said
beams of light, and recombine those beams to form interference
fringes that are imaged on a subject; and an image capture device
configured to capture light from the subject being imaged, and to
form an image of said subject.
2.-35. (canceled)
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. patent
application Ser. No. 13/882,596, filed Aug. 5, 2013, which is a
national stage application under 35 U.S.C. 371 of PCT Application
No. PCT/EP2011/069375 having an international filing date of 3 Nov.
2011, which designated the United States, which PCT application
claimed the benefit of Great Britain Application No. 1018560.1
filed 3 Nov. 2010, each of which are incorporated herein by
reference in their entirety.
FIELD OF THE INVENTION
[0002] This invention relates to in-vivo retinal imaging
systems.
[0003] To facilitate a proper understanding of the teachings of the
present invention, particular embodiments will be described below
with particular reference to in-vivo imaging of the retina of a
subject.
BACKGROUND TO THE INVENTION
[0004] A variety of different devices have previously been proposed
for in-vivo imaging of a subject's eye. Such devices are typically
used by a clinician or physician to look for abnormalities
(congenital or acquired) and symptoms of disease.
[0005] One such previously proposed device is the so-called
"ophthalmoscope". Commonly available opthalmoscopes range from
relatively simple pocket-sized opthalmoscopes such as the Welch
Allyn PocketScope.TM. Ophthalmoscope, to more complex devices such
as the Welch Allyn Panoptic.TM. Ophthalmoscope (each of which are
available from Welch Allyn Inc., Corporate Headquarters, 4341 State
Street Road, Skaneateles Falls, N.Y. 13153-0220, USA and viewable
at: www.welchallyn.com). In general terms, such devices allow an
operator to shine a light into a subject's eye to illuminate the
retina whilst the operator looks for abnormalities and symptoms of
disease.
[0006] A more sophisticated device for imaging the retina of a
subject is the so-called Scanning Laser Ophthalmoscope (often
referred to as an "SLO"). The SLO is more accurate than a
traditional ophthalmoscope and provides a greater field of view of
the retina. The SLO comprises a laser source and a series of
vertical and horizontal mirrors that can be operated to scan the
laser over a subject's retina to generate a raster image that can
be displayed on a conventional television monitor.
[0007] Whilst the SLO is able to image the retina in real time,
reflections from the eye, astigmatism and higher order aberrations
introduced by the cornea, tear film, lens and eye movements, can
cause problems with the data produced by the device, resulting in
poorer images. To address these issues, a device known as an
Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) has more
recently been proposed. This device uses adaptive optics to remove
optical aberrations from images obtained with an SLO. In
particular, in an AOSLO, a laser is collimated and then reflected
off of a beam-splitting mirror. As with a conventional SLO, light
is passed through both a horizontal and a vertical scanning mirror
before and after the eye is scanned to align the moving beam for
eventual retinal raster images of the retina. Additionally, the
light is reflected off of a deformable mirror before and after
exposure to the eye to compensate for optical aberrations. The
laser enters the eye through the pupil to illuminate the region it
has been focused onto and light reflected back passes to a beam
splitter where it is directed simultaneously toward a
photomultiplier tube (PMT) and toward a Shark-Hartmann wavefront
sensor array. The light going toward the photomultiplier is focused
through a confocal pinhole to remove light not reflecting off of
the plane of interest and is then recorded in the PMT. Light
directed to the wavefront sensor array is split up by the lenslets
in the array and then recorded onto a Charge-coupled device (CCD)
camera for detection of optical aberrations. These aberrations are
optically compensated for by using the deformable mirror to
increase lateral and axial resolution.
[0008] Another commonly used technique is called Optical Coherence
Tomography (OCT). OCT provides a powerful clinical tool for
monitoring retinal physiology in patients, and utilises low
coherence interferometry to differentiate tissues within the eye
and create a cross section of a living patients' retina
non-invasively. OCT provides better axial resolution than AOSLO,
however AOSLO represents tends to provide better translational
resolution than OCT and can thus be used to track minor lateral
physical changes, such as the effects of eye movements on the
retina. A combination of AOSLO and OCT has also recently been
proposed, which combination should provide, at high speed, three
dimensional images of individual cone cells and an illustration of
the overall cone mosaic near the fovea of a subject's eye.
[0009] Whilst these devices are all of use in imaging the eye of a
subject, a principal problem with SLO, AOSLO and OCT devices is
that they are relatively expensive, typically in the order of tens
of thousands of pounds. The effect of this is that such devices
tend not to be available to individual or small groups of
practitioners, and instead tend to be limited to larger
organisations, such as hospitals.
[0010] One technique that might be employed to address this issue
is broadly similar to a technique that is commonly referred to as
"structured illumination microscopy". Structured illumination
microscopy enables optical sectioning of a three-dimensional (3D)
object resulting in images similar to those obtained using a
confocal scanning microscope. It has also been used for enhanced
lateral resolution, allowing superresolution beyond the diffraction
limit. In the depth-resolving case, the basic principle between
structured illumination microscopy and confocal microscopy is
similar, namely that only planes that are in focus are imaged
efficiently and out-of-focus planes contribute significantly less
to the image.
[0011] However they are fundamentally different optical systems.
Structured illumination microscopy has the advantage of being an
optically simple technique that does not require laser illumination
nor scanning of the beam or sample. The non-scanning configuration
of the structured illumination microscope and the absence of a
laser source enable a simple optical set up to be used that has
minimal moving parts, and thus has potential for cost-effectiveness
and robustness.
[0012] Another drawback of the confocal microscope is that the
detector pinhole rejects light in order to achieve axial
sectioning, and in practice, especially in the ophthalmic imaging
case of the confocal Scanning Laser Ophthalmoscope (SLO), trade
offs have to be made between pinhole size and confocality, thus
limiting the axial sectioning capabilities of the device.
Structured illumination microscopy does not reject any light and
can therefore image the sample more efficiently.
[0013] In structured illumination microscopy, the sample is
illuminated with a sinusoidal pattern along one of its lateral
dimensions. For weak objects, it has been shown that it is only the
zero-order spatial frequency that does not attenuate with defocus.
As it is possible with sinusoidal illumination to recover an image
in which the zero-order term is absent; all remaining spatial
frequencies tend to attenuate with defocus thereby providing that
the in-focus plane is the one that contributes most significantly
to the image obtained. A drawback of this technique is that it is
necessary to acquire three successive images with the sinusoidal
pattern displaced by phases of 2/3 .pi. and -2/3 .pi. with respect
to the first image. From these three images, an optically sectioned
image of the sample can be obtained.
[0014] Whilst this approach appears useful, in the context of
imaging non-stationary objects the requirement for multiple images
proves problematic. This is particularly the case in the context of
in-vivo retinal imaging where involuntary and voluntary tremors and
saccades of the eye result in a typically continuously moving
sample.
[0015] Another problem to be addressed is the manner in which the
retina, in this particular example is to be illuminated. In
particular in one technique known as "grid projection", the
frequency of the sinusoid needs to be carefully controlled so that
the resultant frequency of the light illuminating the retina is in
the region of 500 cycles per degree. However, a problem with this
approach is that the optics of the eye does not transmit a sinusoid
with a frequency above about 60 cycles per degree.
[0016] One way to address this last problem is to illuminate the
retina with coherent sources that are allowed to interfere and
thereby generate fringes on the retina. However, if coherent
sources are used, all layers of the eye may an equal contribution
and hence the ability to image the retina in three-dimensions is
lost.
[0017] One previously proposed attempt to resolve these issues is
disclosed in US2009/0046164. In this patent application the system
disclosed is primarily intended to provide lateral superresolution
(i.e. lateral resolution beyond the Rayleigh limit), and hence
axial resolution (or in other words, three-dimensionality) is not
of concern. This patent discloses the use of grid projection
techniques using incoherent light, and fringe projection techniques
using coherent laser light. However, incoherent grid projection
techniques cannot function as a means to image the retina in-vivo
due to the aforementioned transmission limit of around 60 cycles
per degree (it would, of course, function adequately when used to
image a sample in-vitro), and fringe projection techniques using
coherent laser sources cannot provide axial resolution. In
addition, in this patent application the aforementioned problem
with a subject moving between successive images is countered by two
different techniques, in one technique where the movement concerned
is anticipated, phase shifts are estimated a priori. In another
technique where movements cannot be anticipated, an algorithm is
employed to estimate phase shifts a posteriori from peaks in the
Fourier transform. As will be appreciated, in either case the
accuracy of the resulting image is only as good as the phase shift
estimations.
[0018] The present invention has been devised with the foregoing
problems in mind.
SUMMARY OF THE INVENTION
[0019] In accordance with a presently preferred embodiment of the
present invention, there is provided an optical imaging system for
in-vivo retinal imaging, the system comprising: an optical source
for generating incoherent light in a plurality of wavelength bands;
an optical imaging sub-system configured to split light from said
optical source into a plurality of beams, to introduce a path
difference between said beams of light, and recombine those beams
to form interference fringes that are imaged on a subject; and an
image capture device configured to capture light from the subject
being imaged, and to form an image of said subject.
[0020] In another aspect, the invention provides an imaging method
for in-vivo retinal imaging, the method comprising: operating an
optical source to generate incoherent light in a plurality of
wavelength bands; splitting light from said optical source into a
plurality of beams, introducing a path difference between said
beams of light, and recombining those beams to form interference
fringes that are imaged on a subject; capturing light from the
subject being imaged, and forming an image of said subject.
[0021] Other aspects and features of the invention are set out in
the dependent claims, and further features and advantages of
aspects of the invention are set out hereafter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Various aspects of the teachings of the present invention,
and arrangements embodying those teachings, will hereafter be
described by way of illustrative example with reference to the
accompanying drawings, in which:
[0023] FIG. 1 is a schematic representation of an optical imaging
system with all branches of an interferometer superimposed for
clarity;
[0024] FIG. 2 is a diagrammatic representation of the geometry for
an illustrative optical imaging system;
[0025] FIG. 3 is a diagrammatic representation of an optical
imaging system according to an embodiment of the present
invention;
[0026] FIG. 4 is a schematic representation of an illustrative
optical source; and
[0027] FIG. 5 is a diagrammatic representation of an optical
imaging system according to an embodiment of the present
invention.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0028] Before embarking on a detailed description of preferred
embodiments, it is appropriate at this juncture to provide a
general explanation of the scientific theory on which aspects and
teachings of the present invention rely. Specific reference will be
made below to in-vivo retinal imaging, but it should be remembered
(as mentioned previously) that this particular application of the
teachings of the present invention is merely illustrative and not
intended to be a limitation of the scope of the present
invention.
[0029] In very general terms, the teachings of the present
invention concern a novel implementation of structured illumination
in which three images with displaced sinusoidal illumination are
acquired simultaneously instead of sequentially, thus avoiding the
aforementioned difficulties associated with imaging by
non-stationary subjects, such as in-vivo imaging of a subject's
retina.
[0030] In one envisaged implementation this is achieved by
illuminating a subject to be imaged, such as the retina, with three
different wavelengths each projecting a sinusoidal pattern with the
required phase. In a particularly preferred implementation the
wavelengths are chosen to match the peak responsivity of the three
detector types in a standard colour CCD camera so that, after
appropriate filtering to reduce cross-talk, the three required
images can be extracted from the three colour channels of a single
image from the CCD camera from which the final optically sectioned
image can be retrieved.
[0031] The resulting multispectral image will be a result of three
sub-images that are not exactly identical owing to the difference
in reflectivity of the imaged layer for each of the three
wavelengths. However, the spatial frequency of the grid will be
high with respect to the structures being imaged (as discussed
below) and therefore any artefacts will be small. Furthermore, in
practical situations it is likely that a number of successive
optically sectioned images will be aligned and averaged to improve
contrast and reduce noise, and in this scenario the
non-stationarity of the object is now advantageous as the averaging
process is likely to cause any inhomogeneities due to wavelength
differences in a given single image also to be averaged. The final
image will therefore be a composite of three wavelengths, likely to
yield more information than a monochromatic image.
[0032] As mentioned briefly above, when imaging the human retina,
standard structured illumination microscopy encounters a further
problem that makes its application to retinal imaging problematic.
Conventionally, the structured pattern illuminating the sample is
achieved in one of two ways, namely grid projection or fringe
projection.
[0033] In the former, a sinusoidal grid is illuminated incoherently
and imaged onto the sample, in the latter a laser beam is used to
generate a coherent fringe pattern on the sample. However, it has
been shown that axial sectioning can only be obtained under
incoherent imaging and therefore the fringe projection method can
only be used for fluorescence microscopy as the incoherence is
obtained through the mutual independence of the fluorofores. Since
non-invasive retinal imaging is not a fluorescence technique,
fringe projection cannot be used.
[0034] In addition, grid projection is limited by the Modulation
Transfer Function (MTF) of the condenser lens as the spatial
frequency of the grid affects the axial resolution. Because of the
relatively poor optics of the human eye (which acts as both
condenser and objective in retinal imaging) the highest spatial
frequency that can be obtained by the grid projection technique
(typically about 60 cycles per degree (cpd), this being the
frequency cut-off of a typical eye) is too low for achieving the
desired axial resolution.
[0035] The technique embodied by the teachings of the present
application addresses these issues by adopting a novel fringe
projection technique. The technique, described in detail below, can
project fringes whose spatial frequency is not limited by the
optics of the eye (or more generally by the collector lens) and can
illuminate the sample incoherently as is preferred for
non-fluorescent axial sectioning using structured illumination.
[0036] In one envisaged implementation of the teachings of the
invention, the aforementioned novel projection technique employs a
Michelson interferometer in which an incoherent extended light
source is used to project straight Fizeau fringes directly onto the
retina (note however that this application is merely illustrative
of the teachings of the invention, and that the teachings of the
invention may more widely be applied--for example to structured
illumination microscopy in general).
[0037] FIG. 1 is a schematic representation of a Michelson
interferometer in which all branches are superimposed onto the same
optical axis. The plane represents the extended incoherent light
source, M.sub.1 and M.sub.2 represent the mirrors (one of which is
actually the image of the mirror on a beamsplitter, not shown). The
axial optical path length of the mirrors from the source is equal,
though one mirror is tilted with respect to the other by a small
angle .alpha.. If we consider a point source S on the extended
source .SIGMA., then each mirror creates an image of this source,
S.sub.1' and S.sub.2' respectively. The fringes are localised at
the plane of the air wedge produced by the two mirrors, and
therefore the eye will focus at the plane containing M.sub.1 and
M.sub.2.
[0038] It is thus appropriate to consider that the light reaching
the eye is coming from the two point sources S.sub.1' and S.sub.2'
both of which are emitting diverging spherical waves, so that at
the pupil plane P of the eye we can define wavefronts W.sub.1 and
W.sub.2 coming from the respective sources. These wavefronts have
the same curvature since the two sources S.sub.1' and S.sub.2' are
equidistant from the eye for small angle .alpha., and their
respective tilt is .alpha.. As the eye is focused at the plane
containing the M.sub.1 and M.sub.2, these wavefronts will focus
before the retina R (as shown by the dotted red lines in FIG. 1)
and we can therefore define two diverging wavefronts W.sub.1' and
W.sub.2' at the retina that are again substantially identical
except for tilt. The interference produced by these wavefronts
forms an interference pattern on the retina that consists of a
plurality of parallel straight fringes. It should be noted that
because .alpha. is small, the angular subtense of the sources
S.sub.1' and S.sub.2' at the eye can be assumed to be well within
the isoplanatic patch of the eye, and as such any aberrations
introduced at the pupil plane P of the eye will be common to both
W.sub.1 and W.sub.2, and hence W.sub.1' and W.sub.2'. Thus the
fringe pattern produced is not affected by the optics of the eye
and the spatial frequency of the sinusoidal pattern is only a
function of the angle .alpha. and the wavelength .lamda.. This
holds for all point sources S on the extended incoherent source
plane .SIGMA., thereby providing an incoherently illuminated
sinusoidal grid pattern whose spatial frequency can be tuned by
rotating one of the mirrors to vary the angle .alpha.. The phase of
the illuminating sinusoidal pattern can be altered by moving one of
the mirrors axially to alter the relative optical path difference
between the two branches.
[0039] Since the Fizeau fringes produces are localised at the plane
R' of the optical air wedge formed by the two mirrors, and hence at
the retinal plane R conjugate to this plane, it is also necessary
to consider the nature of the illumination for out-of-focus planes.
At the localisation plane on the retina R, the interference pattern
produced by the virtual point sources S.sub.1' and S.sub.2' due to
the actual point source S of the extended source .SIGMA. is only a
function of the tilt .alpha.. In other words, the interference at
any point on the localisation plane depends only on the thickness
of the air wedge at that point, as expected for Fizeau fringes
(fringes of equal thickness). For out-of-focus planes this is no
longer the case; the interference pattern produced by each point
source S on .SIGMA. is now also dependant on the location of the
point source so that each point source will produce a pattern that
has a slight phase shift with respect to its neighbour. As a result
the pattern will now be dependant on the effective size of the
extended source and the distance of the source from the mirrors.
The aperture of any detection system (such as the eye) will also
impact on the effective size of the source. Therefore in general,
the modulation of the fringes will decrease with increasing defocus
but the rate of change of the modulation with defocus depends on
the system set up.
[0040] In an envisaged implementation where non-stationary objects
are to be imaged, for example as occurs in in-vivo retinal imaging,
an implementation of the teachings of the invention provides a
Structured Illumination Ophthalmoscope, depicted schematically in
FIG. 3 and described later in detail, that employs the simultaneous
projection of fringes at three different wavelengths. In the
particular implementation depicted in FIG. 3, each wavelength has
an associated mirror and the spatial frequency and phase of each
fringe pattern can be adjusted separately to give sinusoidal
patterns with the same spatial frequency and relative phases of
.phi..sub.0, .phi..sub.0+2.pi./3 and .phi..sub.0-2.pi./3. As will
be appreciated, the angle .alpha. affects the spatial frequency of
the fringes at the plane of the mirrors M.sub.1 and M.sub.2 (FIG.
1). In practical cases, there will be a magnification which is less
than unity between this plane and the retinal plane R. This
magnification is a parameter that can be used to optimise the rate
of change of spatial frequency of the illumination pattern with
changing angle .alpha..
[0041] The image formation theory of the structured illumination
microscope is well documented for grid projection and fringe
projection techniques, and a similar approach can be adopted to
formalise the theory for the novel fringe projection technique
described herein.
[0042] If we let (x.sub.o,y.sub.0) represent the lateral
coordinates at the object plane (i.e. in this specific example, the
retina), we can then define the normalised coordinates (t.sub.0,
w.sub.0)=k(x.sub.0, y.sub.0)n sin .alpha. where k=2.pi./.lamda. and
n sin .alpha. is the numerical aperture NA (FIG. 2 shows, for
illustration, a simplified schematic of the imaging system). If the
amplitude reflectance of the retina is r(t.sub.0,w.sub.0) and it is
illuminated by a structured incoherent intensity pattern given
by:
I.sub.illumination(t.sub.0, w.sub.0)=1+.mu. cos(vt.sub.0+.phi.),
(1)
where .mu. and v are the modulation and frequency respectively of
the sinusoidal pattern, and .phi. is the phase, then the object
intensity becomes
I.sub.object(t.sub.0, w.sub.0)=[1+.mu.
cos(vt.sub.0+.phi.)].rho.(t.sub.0, w.sub.0), (2)
where .rho.=|r|.sup.2 is the intensity reflectance of the retina.
The intensity image of this object formed incoherently at the image
plane (t.sub.i,w.sub.i) is therefore
I(t.sub.i, w.sub.i)=.intg..intg.[1+.mu.
cos(vt.sub.0+.phi.)].rho.(t.sub.0, w.sub.0)|h(t.sub.i+t.sub.0,
w.sub.i+w.sub.0)|.sup.2dt.sub.0dw.sub.0, (3)
where h is the amplitude point spread function of the objective
(i.e. the optics of the eye). We assume unit magnification between
object and image plane throughout these derivations, and
integration is over all space. We can now expand this last equation
using the expansion of cosine in terms of Euler's formula, and for
compactness we pre-define the following functions:
I.sub.0(t.sub.i, w.sub.i)=.intg..intg..rho.(t.sub.0,
w.sub.0)|h(t.sub.i+t.sub.0,
w.sub.i+w.sub.0)|.sup.2dt.sub.0dw.sub.0, (4)
I.sub.v(t.sub.i, w.sub.i)=.intg..intg.e.sup.ivt.sup.0.rho.(t.sub.0,
w.sub.0)|h(t.sub.i+t.sub.0,
w.sub.i+w.sub.0)|.sup.2dt.sub.0dw.sub.0, (5)
I.sub.-v(t.sub.i,
w.sub.i)=.intg..intg.e.sup.-ivt.sup.0.rho.(t.sub.0,
w.sub.0)|h(t.sub.i+t.sub.0,
w.sub.i+w.sub.0)|.sup.2dt.sub.0dw.sub.0, (6)
which yields
I ( t i , w i ) = I 0 ( t i , w i ) + .mu. 2 .phi. I v ( t i , w i
) + .mu. 2 .phi. I - v ( t i , w i ) . ( 7 ) ##EQU00001##
I.sub.0(t.sub.i,w.sub.i) is simply the conventional incoherent
image in a standard microscope with homogenous illumination
(.mu.=0), and we also note that I.sub.-v=I.sub.v* where * denotes
the complex conjugate. The intensity image obtained using the
structured illumination therefore can be considered as having three
components, one of which is equivalent to the standard microscope.
The relative weighting of these three components depends on the
modulation of the illuminating pattern, .mu.. Before proceeding to
show that I.sub.v and I.sub.-v possess axial sectioning properties,
we note that in order to extract these components we require more
than one intensity image so that I.sub.0 can be eliminated. Thus,
three intensity images I.sub.1, I.sub.2 and I.sub.3 are obtained
with phases .phi..sub.1=.phi..sub.0,
.PHI. 2 = .PHI. 0 + 2 .pi. 3 and .PHI. 3 = .PHI. 0 - 2 .pi. 3
##EQU00002##
respectively. We can therefore show that the desired component can
be obtained through either of the following two expressions:
I .+-. v = I 1 + I 2 .-+. 2 .pi. / 3 + I 3 .+-. 2 .pi. / 3 , ( 8 )
I .+-. v = ( ( I 1 - I 2 ) 2 + ( I 1 - I 3 ) 2 + ( I 2 - I 2 ) 2 2
) 1 2 ( 9 ) ##EQU00003##
The conventional incoherent image can also be easily recovered from
the three acquired images through;
I 0 = 1 3 ( I 1 + I 2 + I 3 ) ( 10 ) ##EQU00004##
[0043] We can now define the object intensity spectrum
(m,n)={.rho.(t.sub.0,w.sub.0)} where represents the Fourier
transform operator and (m,n) are spatial frequencies corresponding
to (t.sub.0,w.sub.0), we can therefore substitute for .rho. in Eq.
5 to give
I.sub.v(t.sub.i,
w.sub.i)=.intg..intg..intg..intg.e.sup.ivt.sup.0(m,n)e.sup.-i(mt.sup.0.su-
p.+mw.sup.0.sup.)|h(t.sub.i+t.sub.0,
w.sub.i+w.sub.0)|.sup.2dt.sub.0dw.sub.0dmdn (11)
Since, P(m,n)=.sup.-1{h(t.sub.0, w.sub.0)} where P is the
generalised pupil function then through use of the shift theorem
and autocorrelation theorem for Fourier transforms we have
( mt i + nw i ) P ( m , n ) P * ( m , n ) = - 1 { h ( t o + t i , w
0 + w i ) 2 } ( 12 ) = .intg. .intg. h ( t o + t i , w 0 + w i ) 2
( 13 ) - ( mt o + nw o ) t o w o . ##EQU00005##
In anticipation of our final result we define the transfer
function, and following a further application of the shift theorem
to take into account the exponential term in Eq. 11 we get:
I.sub.v(t.sub.i,w.sub.i)=e.sup.ivt.sup.i.intg..intg.(m,n)C(m+v,n)e.sup.i-
(mt.sup.i.sup.+mw.sup.i.sup.)dmdn (14)
[0044] In order to investigate the effect of defocus on the
structured illumination microscope, we need to consider the effect
of defocus on the illumination pattern. This is fairly
straightforward for the grid projection and fringe projection
techniques. In the former, as the illumination pattern is an image
of a sinusoidal grid formed on the sample, the axial behaviour of
the structured pattern is determined by the three-dimensional point
spread function of the collector lens which is responsible for the
illumination. The modulation of the sinusoidal pattern therefore
decreases with defocus. In fringe illumination the sinusoidal
pattern is formed through the interference of two laser beams and
is independent of axial position; the modulation therefore does not
decrease with defocus. For the novel fringe projection technique
described herein, the defocus considerations are more involved as
described above.
[0045] One practical scenario is the case when the extended source
is small or distant so that we can assume all rays are nearly
parallel to the optical axis. This is a valid assumption for an
ophthalmic imaging system owing to the restrictions imposed by the
pupil of the eye. In this case we can assume that the modulation
.mu. of the sinusoidal pattern is not a function of defocus.
Equation 3 can therefore be rewritten as
I(t.sub.i,w.sub.i;u)=.intg..intg.[1+.mu.
cos(vt.sub.0+.phi.)].rho.(t.sub.0,w.sub.0;u)|h(t.sub.i+t.sub.0,w.sub.i+w.-
sub.0;u)|.sup.2dt.sub.0dw.sub.0, (15)
where u is the normalised axial coordinate representing defocus,
related to the actual axial coordinate z through u=4knz sin
sin.sup.2 .alpha./2. Similarly, all subsequent equations derived
from Eq. 3 become functions of u.
[0046] Therefore we note that the transfer function C(m+v,n;u) of
the structured illumination brightfield microscope with the
(incoherent) fringe projection technique described herein is
identical to the transfer function of a structured illumination
fluorescence microscope illuminated with the (coherent) fringe
projection technique, and will therefore express the same axial
sectioning characteristics which are comparable to those of the
confocal microscope. Similarly, the imaging system will exhibit an
increase in lateral resolution owing to the higher cut-off
frequency of C(m+v,n;u) with respect to that of the standard
incoherent microscope and the confocal microscope, for all non-zero
values of v. It should be noted that the assumption leading to
constant modulation of the structured pattern with defocus
represents a worst case scenario, since any attenuation would lead
to better axial sectioning. The choice of size and distance of the
extended source in designing the system can therefore to some
extent enhance the optical sectioning properties of the imaging
system. On the other hand, the use of different wavelengths to
obtain the three required intensity images, while making the
technique feasible for ophthalmic use, will affect the axial
sectioning capabilities as the I.sub.0 term will not be fully
eliminated in Eqs. 8 and 9.
[0047] In the preceding paragraphs we have described and
theoretically assessed a novel structured illumination technique
that is well suited for the imaging of non-stationary objects,
including in-vivo retinal imaging. The technique described includes
a new technique for providing the sinusoidal illumination pattern
that is required for structured illumination, namely a novel
(incoherent) fringe projection technique, and a multiple wavelength
illumination system that enables the three images required with
phase-shifted structured patterns to be acquired simultaneously,
rather than sequentially, thereby enabling moving objects to be
imaged.
[0048] We have also shown that for practical implementations in the
ophthalmic case, the theoretical axial sectioning is on a par with
that obtained through fluorescence imaging through structured
illumination with the coherent fringe projection system, and in the
general case the geometry of the extended source in relation to the
objective can be altered to further improve the achievable axial
sectioning.
[0049] Referring now to FIG. 3 of the accompanying drawings, there
is depicted an optical system, hereafter referred to as a
structured illumination ophthalmoscope (SIO) that embodies the
teachings of the present invention.
[0050] The SIO 1 comprises an optical source 3 that is operable to
generate incoherent light in three discrete wavelength bands. In
one envisaged implementation the optical source comprises three
light sources that are operable to generate incoherent light in
respective discrete wavelength bands. In a preferred implementation
one said light source comprises a source of red light, one a source
of green light, and the last a source of blue light. In theory any
incoherent source of light may be used, but in a particularly
preferred arrangement each said light source comprises one or more
light emitting diodes. In one envisaged arrangement depicted
schematically in FIG. 4, the light source 3 comprises a plurality
of red, green and blue light sources 2 (for example LEDs) arranged
on a support 4, such as a PCB, so that when viewed from a distance
the support appears to emit white light. In a particularly
preferred arrangement the light sources 3 of each wavelength may be
dispersed throughout the support 4 so that light of a given
wavelength range is emitted from disparate point sources on the
support.
[0051] In another envisaged implementation, which may be provided
in combination with the arrangement shown in FIG. 4 or in
combination with another arrangement of light sources, a rotating
diffuser 28 may be provided so that the sources provide diffuse
illumination. In another envisaged arrangement, a spatially
incoherent source of illumination may be provided in place of a
light source and diffuser. In either case, a polariser 27 may also
be provided.
[0052] The light source 3 is configured to illuminate an optical
sub-system 6 that comprises a Michelson interferometer. The
interferometer comprises a first beamsplitter 5 which passes, in a
manner known in the art, some of the incident light to a common
reference mirror 7 in one branch of the interferometer, and
reflects the remainder to a prism 9 in the other branch of the
interferometer.
[0053] The prism 9 functions to split incident light from the beam
splitter into discrete red, blue and green light beams that are
each directed to an associated (preferably wavelength specific)
mirror 13(i), (ii) and (iii). Optionally, to avoid cross-talk
between different light sources, appropriate bandpass filters (not
shown) may be inserted between the prism 9 and one or more of the
mirrors 13.
[0054] Light reflected by the mirrors 13 travels back through the
prism 9, and then through a second beamsplitter 15 before being
focused by the optics 17 (including the pupil, lens and cornea) of
the subject's eye 19 onto the retina 21. A half-wave plate 23 is
provided between the first beamsplitter 5 and the common reference
mirror 7 to control fringe modulation, and light returning through
the half-wave plate is reflected by the first beamsplitter 5
through the second beamsplitter 15 and into the subject's eye 19.
In a preferred implementation the second beamsplitter 15 is a
polarising beamsplitter, and this polarising beamsplitter operates
in conjunction with a quarter-wave plate 25 between the subject's
eye and the second beamsplitter 15 to counteract reductions in
returning light due to birefringence caused by the eye.
[0055] The common reference mirror 7 is movable back and forth
towards and away from the first beamsplitter 5, and the mirrors
13(i), (ii) and (iii) are movable in two dimensions (towards/away
and angularly) with respect to the prism 9. Movement of the mirrors
7, 13 adjusts the relative path lengths in each branch of the
interferometer as well as adjusting the focus of the beams on the
retina of the subject.
[0056] As will be appreciated by persons skilled in the art, the
aforementioned Michelson interferometer may more generally be
described as a means for splitting incident light into a number of
beams, and introducing a path difference between those beams of
light (that in this instance are each comprised of three colours,
namely red, green and blue), and then recombining those beams to
form interference fringes that are imaged on--in this instance--the
retina of the subject.
[0057] Light reflected from the retina of the subject returns
through the optics of the eye 17 and quarter-wave plate 25 before
being reflected by the second beamsplitter 15 towards an image
capture device 29, which may comprise an RGB CCD camera. In an
envisaged implementation the RGB light sources are chosen to emit
light in wavelength ranges that complement those that can be
detected by the image capture device, and in a particularly
preferred implementation the image capture device is configured to
be capable of outputting R, G or B channels separately, or
outputting a pseudo-colour image by combining respective channels.
An imaging lens 31 (the like of which is known in the art) may be
provided, if desired, to focus incident light onto the image
capture device 29.
[0058] One advantage of this arrangement is that as certain
structures within the eye are more responsive to light of
particular wavelengths (for example, green light is particularly
good for imaging blood vessels within the eye as haemoglobin in the
blood absorbs red light and hence blood vessels tend to appear as
dark regions on a brighter background) these structures can be
imaged particularly well. Another advantage is that a pseudo-colour
image inherently provides more information to the operator than
would be discernible from a black/white or greyscale image.
[0059] In an alternative implementation, the single RGB image
capture device could be replaced with discrete red, green and blue
image capture devices and a prism or a series of frequency-specific
beamsplitters/mirrors that divert respective colours to the
associated imaging device. However such an arrangement would
needlessly complicate the device without providing any discernible
advantages, and hence is less preferred.
[0060] In the arrangement described above with reference to FIG. 3
of the drawings, three images of a subject's retina are
simultaneously acquired and subsequently used to generate a
three-dimensional image of the retina. By acquiring the images
simultaneously, problems associated with movement of the subject's
eye between images can be avoided.
[0061] In another application of the principles of the present
invention, a SIO system is provided that captures successive images
of the eye of a subject, and then generates a three-dimensional
representation of the subject's retina from at least some of those
images. In such a system, movement of the eye between successive
images is quantified by illuminating the eye with a reference beam
as each image is acquired, and then calculating the degree of eye
movement that has occurred between successively captured images.
Since movement of the eye causes a phase shift in the illuminating
pattern from which the images are acquired, a determination of the
degree of movement enables the phase shift in the illuminating
pattern between successive images to be calculated, and by
selecting three images attributable to light having relative phases
of .phi..sub.0, .phi..sub.0+2.pi./3 and .phi..sub.0-2.pi./3, or
more than three images with other combinations of relative phase
shifts, the images can be combined to provide a three-dimensional
image of the subject's retina.
[0062] FIG. 5 is a schematic representation of such a system 40 in
which features common to the system depicted in FIG. 3 are labelled
with the same reference numeral.
[0063] The system 40 includes a first optical source 42 that is
configured to generate incoherent light at a reference wavelength
(typically at a wavelength, such as infra-red (for example, 800 to
900 nm), that is not visible), and a second optical source 44 that
is configured to generate incoherent light at a second wavelength
different from that of the light from the first source 42.
[0064] The light source 44 is configured to illuminate an optical
sub-system 6 that comprises a Michelson interferometer. The
interferometer comprises a first beamsplitter 5 which passes, in a
manner known in the art, some of the incident light to a common
reference mirror 7 in one branch of the interferometer, and
reflects the remainder to a mirror 46 in the other branch of the
interferometer.
[0065] Light reflected by the mirror 46 travels back through the
beamsplitter 5, and then through a second, a third and a fourth
beamsplitter 47, 48 and 50 before being focused by the optics 17
(including the pupil, lens and cornea) of the subject's eye 19 onto
the retina 21. A half-wave plate 23 is provided between the first
beamsplitter 5 and the common reference mirror 7 to control fringe
modulation, and light returning through the half-wave plate 23 is
reflected by the first beamsplitter 5 through the second, third and
fourth beamsplitters 47, 48 and 50 and into the subject's eye 19.
In a preferred implementation at least one of the third and fourth
beamsplitters 48, 50 is a polarising beamsplitter that operates in
conjunction with a quarter-wave plate 25 between the subject's eye
and the fourth beamsplitter 50 to counteract reductions in
returning light due to birefringence caused by the eye.
[0066] The common reference mirror 7 is movable back and forth
towards and away from the first beamsplitter 5, and the mirror 46
is movable in one dimension (towards/away) with respect to the
beamsplitter 5. Movement of the mirrors 7, 46 adjusts the relative
path lengths in each branch of the interferometer as well as
adjusting the focus of the beams on the retina of the subject.
[0067] As will be appreciated by persons skilled in the art, the
aforementioned Michelson interferometer may more generally be
described as a means for splitting incident light into a number of
beams, and introducing a path difference between those beams of
light, and then recombining those beams to form interference
fringes that are imaged on--in this instance--the retina of the
subject.
[0068] Light from the light source 42 illuminates the
aforementioned second beamsplitter 47 and is reflected through the
third and fourth beamsplitters 48, 50 and quarter-wave plate 25
before being focused by the optics 17 of the subject's eye 19 onto
the retina 21.
[0069] The third and fourth beamsplitters 48, 50 are each
configured to reflect light of a particular range of wavelengths.
In one envisaged implementation, the fourth beamsplitter 50 is
configured to reflect light from the first optical source 42, and
the third beamsplitter 48 is configured to reflect light from the
second optical source 44. By virtue of this arrangement, light
reflected from the retina of the subject returns through the optics
of the eye 17 and quarter-wave plate 25 before being reflected by
one of the third and fourth beamsplitters 48, 50 towards an
associated image capture device 52, 54, such as a video camera.
Imaging lenses 31 (the like of which is known in the art) may be
provided, if desired, to focus incident light onto the respective
image capture devices 52, 54.
[0070] In a particularly preferred implementation, the camera 52
associated with the fourth beamsplitter 50 is tuned for detecting
light of the wavelength emitted by the first source 42. The camera
54 associated with the third beamsplitter 48 may be tuned to detect
light of the wavelength emitted by the second source 44, or in
another envisaged arrangement the camera 54 associated with the
third beamsplitter 48 may be configured to be sensitive to a
broader range of wavelengths.
[0071] This latter arrangement is particularly useful, as it would
allow the SIO depicted in FIG. 5 to be used for hyper- or
multispectral imaging of a subject's retina. In such an
implementation, by sweeping the wavelength at which light is
emitted by the second source 44 over a wide range of wavelengths
(and capturing images at each said wavelength) it would be possible
to generate a hyper- or multispectral image of the retina, which
image would allow certain structures within the eye (such as blood
vessels) to be imaged particularly well. In a similar way, the
arrangement shown in FIG. 3 may be used for hyper- or multispectral
imaging by varying the wavelengths at which the light sources 3
emit light. Such wavelength variations may be introduced in a
number of ways, for example by means of suitable filters.
[0072] Referring again to FIG. 5, the system 40 further comprises a
processor 56, such as a computer, that is configured to process
images from the respective cameras 52, 54. In particular, the
processor is configured to use conventional digital image
processing techniques (such as those described in Digital Image
Processing (2.sup.nd Edition), by Gonzalez and Woods, published by
Prentice Hall) to identify landmarks in an image and then track
those landmarks as they move between successive frames of the
signal from the camera 52. By tracking these landmarks the
processor is able to determine a measure of the extent to which the
eye being imaged has moved between successive images. The processor
derives, from this measurement, an indication of the phase shift
induced by the respective eye movements and selects three
corresponding frames from the signal output by camera 54 that
exhibit the required relative phase difference of .phi..sub.0,
.phi..sub.0+2.pi./3 and .phi..sub.0-2.pi./3, or selects more than
three corresponding frames with other combinations of phase
differences. The processor may then combine these images to
generate a three-dimensional representation of the subject's
retina.
[0073] In the foregoing it has been explained how the novel
technique described herein can provide structured illumination
imaging, particularly (but not exclusively) in the field of retinal
imaging. The achievement of good lateral and axial resolution when
imaging the living human retina is important for early detection
and diagnosis of retinal disease, when treatment tends to be both
more effective and more cost-efficient. Imaging devices that can
resolve small retinal structures both laterally and in depth also
aid clinicians who study these diseases, treatment and
management.
[0074] The SLO was the first device to offer optical sectioning of
the retina, and Optical Coherence Tomography (OCT) has the
capability of achieving good axial resolution, but despite this the
Structured Illumination Ophthalmoscope (SIO) described herein
provides a number of advantages over existing systems.
[0075] For example, unlike the SLO and OCT techniques, the SIO does
not employ lateral scanning as the illumination is wide-field. This
makes the SIO a relatively simple device, at least optically, that
does not rely on mechanical scanning devices and hence incur the
optical design trade-offs associated with scanning systems. In
addition to the potential for reduced design, engineering and
production costs as compared with existing devices, the SIO avoids
distortion due to intra-frame eye movements and other potential
artefacts introduced by the scanning processes characteristic of
SLO and OCT devices.
[0076] Another advantage is that the use of incoherent light
sources in the SIO reduces speckle effects that can introduce
artefacts particularly when imaging at higher resolutions. Another
advantage is associated with the fact that as less of the
illumination light is discarded at the sample, as compared with the
confocal pinhole in the SLO or the interference conditions required
in OCT, the SIO is light efficient. In the SLO in particular, the
trade-off between having a small confocal pinhole size to provide
higher axial resolution, and having sufficient signal-to-noise
ratio at the detector is a major drawback that is mitigated by the
SIO. This latter point is especially important in retinal imaging
since the incident light on the sample is limited by ocular safety
considerations. Yet another advantage is that the multispectral
imaging characteristics of the SIO enables efficient imaging of
more retinal layers and structures within a single image than
previous techniques.
[0077] It is apparent, therefore, that in retinal imaging
applications, the SIO provides axially sectioned images that are at
least comparable to those of the SLO. It also offers a number of
advantages over the SLO which include better light efficiency,
improved lateral resolution, multi-spectral imaging and a marked
reduction in optical and opto-mechanical complexity as no lateral
scanning mechanisms are required. The latter point has implications
in image quality but also in potential future development and
manufacture costs, and maintenance and reliability of commercial
devices. The potential for an inexpensive retinal imaging system
with high quality 3D imaging capabilities is one of importance
clinically in the drive to detect retinal disease early through
screening.
[0078] It will be appreciated that whilst various aspects and
embodiments of the present invention have heretofore been
described, the scope of the present invention is not limited to the
particular arrangements set out herein and instead extends to
encompass all arrangements, and modifications and alterations
thereto, which fall within the scope of the appended claims. For
example, it will be apparent to persons skilled in the art that
adaptive optical techniques of the type previously described may
readily be applied to the SIO technique described herein, with
similar benefits (the application of such techniques being
straightforward for persons of ordinary skill in the art).
[0079] It will further be appreciated by persons skilled in the art
that in the aforementioned embodiment where an object is
simultaneously illuminated by three discrete beams of light from
separate light sources, an equivalent arrangement would be to
provide a single light source that is subsequently split (for
example by a plurality of suitable filters) into the aforementioned
three discrete beams of light. Such a modification should be
considered as being within the scope of the present invention. An
advantage of such an arrangement, as compared to a system that
employs discrete RGB light sources, is that the wavelengths of
light used to illuminate the object can more easily be varied--for
example by using different sets of filters.
[0080] In addition, whilst in the arrangement depicted in FIG. 5 it
is preferred for the reference light source 42 to generate
incoherent light, it is anticipated that one could instead arrange
for the light source to generate coherent light. However, in such
circumstances a coherent light source would tend to generate
artefacts in the images, and it would likely be difficult (i.e.
computationally intensive) to distinguish these artefacts from
landmarks in those images. In this light of this, it is preferred
(but not essential) for the light source 42 to output incoherent
light.
[0081] It should also be noted that whilst the accompanying claims
set out particular combinations of features described herein, the
scope of the present invention is not limited to the particular
combinations hereafter claimed, but instead extends to encompass
any combination of features herein disclosed.
* * * * *
References