U.S. patent application number 17/513391 was filed with the patent office on 2022-02-17 for structured illuminating microscopy apparatus.
This patent application is currently assigned to NIKON CORPORATION. The applicant listed for this patent is NIKON CORPORATION. Invention is credited to Tomoya NODA, Hiroshi OHKI, Yosuke OKUDAIRA.
Application Number | 20220050283 17/513391 |
Document ID | / |
Family ID | |
Filed Date | 2022-02-17 |
United States Patent
Application |
20220050283 |
Kind Code |
A1 |
OHKI; Hiroshi ; et
al. |
February 17, 2022 |
STRUCTURED ILLUMINATING MICROSCOPY APPARATUS
Abstract
An acquiring unit of a structured illuminating microscopy
apparatus acquires at least two modulated images having the same
wave number vector and the different phases; and a calculating unit
of the structured illuminating microscopy apparatus, in a spatial
frequency spectrum of each of at least the two modulated images
acquired by the acquiring unit, separates a 0th-order modulating
component and .+-.first-order modulating components of
observational light fluxes superimposed on arbitrary two
observation points based on at least four observation values
regarding the two observation points which are mutually displaced
by an amount of the wave number vector.
Inventors: |
OHKI; Hiroshi;
(Yokohama-Shi, JP) ; NODA; Tomoya; (Saitama-shi,
JP) ; OKUDAIRA; Yosuke; (Konosu-shi, JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
NIKON CORPORATION |
Tokyo |
|
JP |
|
|
Assignee: |
NIKON CORPORATION
Tokyo
JP
|
Appl. No.: |
17/513391 |
Filed: |
October 28, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16290149 |
Mar 1, 2019 |
11187883 |
|
|
17513391 |
|
|
|
|
14597495 |
Jan 15, 2015 |
10261304 |
|
|
16290149 |
|
|
|
|
PCT/JP2013/004338 |
Jul 16, 2013 |
|
|
|
14597495 |
|
|
|
|
International
Class: |
G02B 21/36 20060101
G02B021/36; G02B 21/06 20060101 G02B021/06; G02B 27/58 20060101
G02B027/58 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 19, 2012 |
JP |
2012-160805 |
Claims
1. A structured illuminating microscopy apparatus, comprising: an
illuminating optical system performing a spatial modulation on a
sample by fringes; an image-forming optical system forming an image
of an observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
acquiring a modulated image of the sample; and a calculating unit
generating an image of the sample based on the modulated image
acquired by the acquiring unit, wherein: the acquiring unit
acquires a first modulated image having a first wave number vector
of the fringes and a first phase of the fringes, a second modulated
image having the first wave number vector of the fringes and a
second phase of the fringes, a third modulated image having a
second wave number vector of the fringes, and a fourth modulated
image having a third wave number vector of the fringes being a
linear combination of the first wave number vector of the fringes
and the second wave number vector of the fringes; and the
calculating unit restores the modulated image based on observation
points mutually displaced by an amount of the first wave number
vector in a spatial frequency spectrum of the first modulated image
having the first wave number vector and the first phase of the
fringes and observation points mutually displaced by the amount of
the first wave number vector in a spatial frequency spectrum of the
second modulated image having the first wave number vector and the
second phase of the fringes, and generates an image of a specimen
based on the first modulated image, the second modulated image, the
third modulated image, and the fourth modulated image.
2. The structured illuminating microscopy apparatus according to
claim 1, wherein the calculating unit generates the image of the
specimen based on observation points in a first spatial frequency
spectrum of the first modulated image, in a second spatial
frequency spectrum of the second modulated image, in a third
spatial frequency spectrum of the third modulated image, and in a
fourth spatial frequency spectrum of the fourth modulated
image.
3. The structured illuminating microscopy apparatus according to
claim 1, wherein the calculating unit generates the image of the
specimen based on three observation points in a first spatial
frequency spectrum of the first modulated image, three observation
points in a second spatial frequency spectrum of the second
modulated image, three observation points in a third spatial
frequency spectrum of the third modulated image, and three
observation points in a fourth spatial frequency spectrum of the
fourth modulated image.
4. The structured illuminating microscopy apparatus according to
claim 2, wherein two observation points among the three observation
points in the first spatial frequency spectrum are mutually
displaced the amount of the first wave number vector.
5. The structured illuminating microscopy apparatus according to
claim 3, wherein two observation points among the three observation
points in the first spatial frequency spectrum are mutually
displaced the amount of the second wave number vector.
6. The structured illuminating microscopy apparatus according to
claim 3, wherein the three wave number vectors are the vectors
having a same magnitude and whose directions are different from one
another by 120.degree..
Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application is a Division of U.S. application Ser. No.
16/290,149 filed Mar. 1, 2019, which is a Division of U.S.
application Ser. No. 14/597,495 filed Jan. 15, 2015, which is a
Continuation of International Application No. PCT/JP2013/004338
filed Jul. 16, 2013, designating the U.S., and claims the benefit
of priority from Japanese Patent Application No. 2012-160805 filed
Jul. 19, 2012, the entire contents of which are incorporated herein
by reference.
BACKGROUND
1. Field
[0002] The present application relates to a structured illuminating
microscopy apparatus.
2. Description of the Related Art
[0003] As a method of performing a super-resolved observation on an
observational object such as an organism sample, there is one in
which a spatial frequency of a structure of the observational
object is modulated by illuminating lights (refer to the
specification of U.S. Pat. No. RE 38,307).
[0004] In this method, the observational object is illuminated by
the spatially-modulated illuminating lights, and information
regarding a high spatial frequency that exceeds a resolution limit
included in the structure of the observational object is made to be
contributed to an image formation of a microscopy optical system.
Further, by performing a calculation on a plurality of pieces of
data after being subjected to modulating image formation obtained
by switching phases of spatial illumination and under mutually
different phases (referred to as "modulated images", hereinafter),
data after being subjected to demodulating image formation
(referred to as "demodulated image" or "super-resolved image",
hereinafter) is acquired.
[0005] However, in order to observe one piece of super-resolved
image, there is a need to acquire a plurality of pieces of
modulated images, and to generate spectra of the respective
modulated images, so that it is difficult to perform the
observation at high speed.
SUMMARY
[0006] One example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires at least two
modulated images each having the wave number vector being the same
and the phase being different; and the calculating unit, in a
spatial frequency spectrum of each of at least the two modulated
images acquired by the acquiring unit, separates a 0th-order
modulating component and .+-.first-order modulating components of
the observational light fluxes superimposed on arbitrary two
observation points based on at least four observation values
regarding the two observation points which are mutually displaced
by an amount of the wave number vector.
[0007] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires, among three
wave number vectors having a mutually closed relationship, one
modulated image under each of two wave number vectors, and acquires
at least two modulated images each having the phase being different
under the other one wave number vector; and the calculating unit,
in a spatial frequency spectrum of each of at least the four
modulated images acquired by the acquiring unit, separates a
0th-order modulating component and .+-.first-order modulating
components of the observational light fluxes superimposed on
arbitrary three observation points based on twelve observation
values regarding the three observation points which are mutually
displaced by an amount of the three wave number vectors.
[0008] Note that it is also possible that the acquiring unit sets a
phase contrast between the at least two modulated images each
having the phase being different to .pi..
[0009] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires one modulated
image under each of three wave number vectors having a mutually
closed relationship, and acquires one non-modulated image; and the
calculating unit, in a spatial frequency spectrum of each of the
three modulated images and the one non-modulated image acquired by
the acquiring unit, separates a 0th-order modulating component and
.+-.first-order modulating components of the observational light
fluxes superimposed on arbitrary three observation points based on
twelve observation values regarding the three observation points
which are mutually displaced by an amount of the three wave number
vectors.
[0010] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires four modulated
images each having the phase being different by using the fringes
simultaneously having three wave number vectors having a mutually
closed relationship; and the calculating unit, in a spatial
frequency spectrum of each of the four modulated images acquired by
the acquiring unit, separates a 0th-order modulating component and
.+-.first-order modulating components of the observational light
fluxes superimposed on arbitrary three observation points based on
twelve observation values regarding the three observation points
which are mutually displaced by an amount of the three wave number
vectors.
[0011] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires three
modulated images each having the wave number vector being the same
and the phase being different; and the calculating unit, in a
spatial frequency spectrum of each of the three modulated images
acquired by the acquiring unit, separates a 0th-order modulating
component, first-order modulating components and .+-.second-order
modulating components of the observational light fluxes
superimposed on arbitrary two observation points based on six
observation values regarding the two observation points which are
mutually displaced by an amount of the wave number vector.
[0012] Note that it is also possible that the acquiring unit sets a
phase contrast among the three modulated images to 2.pi./3.
[0013] Further, it is also possible that the acquiring unit
acquires the three modulated images each having the phase being
different under each of three wave number vectors having different
directions; and the calculating unit performs the separation on
each of the three wave number vectors.
[0014] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires two modulated
images each having the phase being different under each of three
wave number vectors having a mutually closed relationship, and
acquires one non-modulated image; and the calculating unit, in a
spatial frequency spectrum of each of the six modulated images and
the one non-modulated image acquired by the acquiring unit,
separates .+-.first-order modulating components of the
observational light flux superimposed on arbitrary three
observation points based on twenty-one observation values regarding
the three observation points which are mutually displaced by an
amount of the three wave number vectors, and separates a 0th-order
modulating component and .+-.second-order modulating components of
the observational light fluxes superimposed on arbitrary three
observation points based on twenty-one observation values regarding
the three observation points which are mutually displaced by a
doubled amount of the three wave number vectors.
[0015] Note that it is also possible that the acquiring unit sets a
phase contrast between the two modulated images acquired under at
least one of the wave number vector to .pi..
[0016] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires four modulated
images each having the phase being different under each of three
wave number vectors having a mutually closed relationship; and the
calculating unit, in a spatial frequency spectrum of each of the
twelve modulated images acquired by the acquiring unit, separates
.+-.first-order modulating components of the observational light
flux superimposed on arbitrary three observation points based on
thirty-six observation values regarding the three observation
points which are mutually displaced by an amount of the three wave
number vectors, and separates a 0th-order modulating component and
.+-.second-order modulating components of the observational light
fluxes superimposed on arbitrary three observation points based on
thirty-six observation values regarding the three observation
points which are mutually displaced by a doubled amount of the
three wave number vectors.
[0017] Further, one example of a structured illuminating microscopy
apparatus of the present embodiment includes an illuminating
optical system performing a spatial modulation on a sample by
fringes; an image-forming optical system performing a modulating
image formation of the sample by forming an image of an
observational light flux from the sample being performed the
spatial modulation; an acquiring unit controlling at least one of a
wave number vector of the fringes and a phase of the fringes, and
capturing a result of the modulating image formation to acquire a
modulated image of the sample; and a calculating unit generating an
image of the sample based on the modulated image acquired by the
acquiring unit, in which the acquiring unit acquires, among three
wave number vectors having a mutually closed relationship, two
modulated images each having the phase being different under each
of two wave number vectors, and acquires four modulated images each
having the phase being different under the other one wave number
vector; and the calculating unit, in a spatial frequency spectrum
of each of the eight modulated images acquired by the acquiring
unit, separates .+-.first-order modulating components of the
observational light flux superimposed on arbitrary three
observation points based on twenty-four observation values
regarding the three observation points which are mutually displaced
by an amount of the three wave number vectors, and separates a
0th-order modulating component and .+-.second-order modulating
components of the observational light fluxes superimposed on
arbitrary three observation points based on twenty-four observation
values regarding the three observation points which are mutually
displaced by a doubled amount of the three wave number vectors.
[0018] Note that it is also possible that the acquiring unit sets a
phase contrast among the four modulated images each having the
phase being different to .pi./2.
[0019] Further, it is also possible that the three wave number
vectors are the vectors having a same magnitude and whose
directions are different from one another by 120.degree..
BRIEF DESCRIPTION OF THE DRAWINGS
[0020] FIG. 1 is a configuration diagram of a structured
illuminating microscopy apparatus 1 of a first embodiment.
[0021] FIG. 2A and FIG. 2B are diagrams each explaining a beam
branching section 14.
[0022] FIG. 3A and FIG. 3B are diagrams each explaining a function
of a 1/2 wavelength plate 19 of a beam selecting section 18.
[0023] FIG. 4A, FIG. 4B, and FIG. 4C are diagrams each explaining a
function of a beam selecting element 20 of the beam selecting
section 18.
[0024] FIG. 5 is a diagram explaining a function of the beam
selecting section 18.
[0025] FIG. 6 is a diagram explaining a rotating mechanism 18A of
the beam selecting section 18.
[0026] FIG. 7A and FIG. 7B are diagrams each explaining an
operation of a translatory shifting mechanism 15 of the beam
branching section 14.
[0027] FIG. 8 is a diagram explaining a beam selecting element 20'
for a 3D-SIM mode.
[0028] FIG. 9 is a diagram explaining a demodulating calculation in
a conventional 2D-SIM mode.
[0029] FIG. 10 illustrates a distribution of a reciprocal of
condition number of a matrix M used in the conventional 2D-SIM.
[0030] FIG. 11 is a diagram explaining a demodulating calculation
in Section 1.3.
[0031] FIG. 12A illustrates a range capable of being restored under
a condition of .DELTA..PHI..noteq..pi., and FIG. 12B illustrates a
range capable of being restored under a condition of
.DELTA..PHI.=.pi..
[0032] FIG. 13 illustrates a range capable of being restored when a
number of directions of interference fringes is set to three in
Section 1.3.
[0033] FIG. 14A illustrates an area capable of being restored in a
first step in Section 1.4, and FIG. 14B illustrates an area capable
of being restored in a second step.
[0034] FIG. 15A illustrates an area capable of being restored in a
third step in Section 1.4, and FIG. 15B illustrates an area capable
of being restored in a fourth step.
[0035] FIG. 16A is a graphical representation of an expression 1.27
in Section 1.4, and FIG. 16B is a graphical representation of a
modified version of the expression 1.27.
[0036] FIG. 17 illustrates a restored area according to a first
example in Section 1.5.
[0037] FIG. 18 illustrates a restored area according to a second
example in Section 1.5.
[0038] FIG. 19A and FIG. 19B are graphical representations of an
expression 1.33 in Section 1.6.
[0039] FIG. 20 illustrates an area restored in Section 1.6.
[0040] FIGS. 21A to 21D are diagrams explaining, in detail, the
expression 1.33 in Section 1.6.
[0041] FIG. 22A and FIG. 22B are graphical representations of an
expression 1.63 in Section 1.9.
[0042] FIG. 23 is a diagram illustrating a relationship of a
grating structure of three-direction interference fringes and
fundamental vectors a.sub.1 and a.sub.2 of the grating.
[0043] FIG. 24 is a diagram illustrating a relationship of
interference fringe intensity distribution among four modulated
images.
[0044] FIG. 25A and FIG. 25B are diagrams explaining a projection
method of the three-direction interference fringes.
[0045] FIG. 26 is a diagram explaining another projection method of
the three-direction interference fringes.
[0046] FIG. 27A is a diagram illustrating a frequency range (xy
cross section) of a demodulated image of a conventional 3D-SIM, and
FIG. 27B is a diagram illustrating a frequency range (zx cross
section) of the demodulated image of the conventional 3D-SIM.
[0047] FIG. 28A is a diagram illustrating a frequency range (xy
cross section) of a demodulated image in Section 2.4, and FIG. 28B
is a diagram illustrating a frequency range (zx cross section) of
the demodulated image in Section 2.4.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0048] Hereinafter, a structured illuminating microscopy apparatus
will be described as an embodiment of the present invention.
[0049] [Explanation of Apparatus]
[0050] First, a configuration of a structured illuminating
microscopy apparatus will be described.
[0051] FIG. 1 is a configuration diagram of a structured
illuminating microscopy apparatus 1. As illustrated in FIG. 1,
there are provided, in the structured illuminating microscopy
apparatus 1, a laser unit 100, an optical fiber 11, an illuminating
optical system 10, an image-forming optical system 30, an imaging
sensor 42, a controlling device 43, an image storing-calculating
device 44, and an image displaying device 45. Note that the
illuminating optical system 10 is one of epi-illumination type, and
illuminates a sample 2 by utilizing an objective lens 31 and a
dichroic mirror 33 of the image-forming optical system 30.
[0052] In the laser unit 100, there are provided a first laser
light source 101, a second laser light source 102, shutters 103 and
104, a mirror 105, a dichroic mirror 106, and a lens 107. Each of
the first laser light source 101 and the second laser light source
102 is a coherent light source, and exit wavelengths of the laser
light sources are mutually different. Here, it is assumed that a
wavelength .lamda.1 of the first laser light source 101 is longer
than a wavelength .lamda.2 of the second laser light source 102
(.lamda.1>.lamda.2). The first laser light source 101, the
second laser light source 102, and the shutters 103 and 104 are
respectively driven by the controlling device 43.
[0053] The optical fiber 11 is formed of, for example, a
single-mode fiber of polarization-maintaining type to guide a laser
light exited from the laser unit 100. A position in an optical axis
direction of an exit end of the optical fiber 11 can be adjusted by
a position adjusting mechanism 11A. The position adjusting
mechanism 11A is driven by the controlling device 43.
[0054] In the illuminating optical system 10, there are disposed a
collector lens 12, a polarizing plate 13, a beam branching section
14, a collecting lens 17, a beam selecting section 18, a lens 21, a
field stop 22, a field lens 23, an excitation filter 24, the
dichroic mirror 33, and the objective lens 31, in this order from
an exit end side of the optical fiber 11.
[0055] The beam branching section 14 is provided with a translatory
shifting mechanism 15 and a diffractive optical element
(diffraction grating) 16, and the beam selecting section 18 is
provided with a 1/2 wavelength plate 19, a beam selecting element
20, and a rotating mechanism 18A. Each of the beam branching
section 14 and the beam selecting section 18 is driven by the
controlling device 43.
[0056] In the image-forming optical system 30, there are disposed
the objective lens 31, the dichroic mirror 33, an absorption filter
34, and a secondary objective lens 35, in this order from a side of
the sample 2.
[0057] The sample 2 is, for example, a culture fluid dropped on a
surface of parallel-plate glass, and a cell having a fluorescence
exists in the vicinity of a glass interface in the culture fluid.
In the cell, both of a first fluorescent area which is excited by a
light with the wavelength .lamda.1, and a second fluorescent area
which is excited by a light with the wavelength .lamda.2, are
exhibited.
[0058] The imaging sensor 42 is a two-dimensional imaging sensor
formed of a CCD, a CMOS or the like. When the imaging sensor 42 is
driven by the controlling device 43, it captures an image formed on
its imaging plane 41, and generates an image. The image is taken
into the image storing-calculating device 44 via the controlling
device 43.
[0059] The controlling device 43 drives and controls the laser unit
100, the position adjusting mechanism 11A, the beam branching
section 14, the beam selecting section 18, and the imaging sensor
42.
[0060] The image storing-calculating device 44 performs calculation
with respect to the image given via the controlling device 43,
stores an image after being subjected to the calculation in a
not-illustrated internal memory, and at the same time, it sends the
image to the image displaying device 45.
[0061] Next, a behavior of laser light in the structured
illuminating microscopy apparatus 1 will be described.
[0062] A laser light with the wavelength .lamda.1 exited from the
first laser light source 101 (first laser light) is incident on the
mirror 105 via the shutter 103, and reflected by the mirror 105 to
be incident on the dichroic mirror 106. Meanwhile, a laser light
with the wavelength .lamda.2 exited from the second laser light
source 102 (second laser light) is incident on a beam splitter 106
via the shutter 104, and combined with the first laser light. The
first laser light and the second laser light exited from the
dichroic mirror 106 are incident on an incident end of the optical
fiber 11 via the lens 107. Note that when the controlling device 43
controls the laser unit 100, the wavelength (=use wavelength
.lamda.) of laser light which is incident on the incident end of
the optical fiber 11, is switched between the long wavelength
.lamda.1 and the short wavelength .lamda.2.
[0063] The laser light incident on the incident end of the optical
fiber 11 propagates in the optical fiber 11, and generates a point
light source at the exit end of the optical fiber 11. The laser
light exited from the point light source is converted into a
collimated light flux by the collector lens 12 to be incident on
the diffraction grating 16 of the beam branching section 14 via the
polarizing plate 13, and then branched into diffractive light
fluxes of respective orders. The diffractive light fluxes of
respective orders are collected by the collecting lens 17 at
mutually different positions on a pupil conjugate plane 25.
[0064] Here, the pupil conjugate plane 25 indicates a focal
position of the lens 17 (rear focal position), and a position
conjugated with a pupil 32 of the objective lens 31 (a position at
which .+-.first-order diffractive lights are collected) via the
lens 23 and the lens 21 (note that a position determined by a
person skilled in the art by taking the design requirements such as
aberration, vignetting and the like of the collecting lens 17, and
the lenses 21 and 23 into consideration, also falls into the
concept of "conjugate position").
[0065] Note that since the laser light exited from the optical
fiber 11 is basically linearly polarized, the polarizing plate 13
can be omitted, but, the polarizing plate 13 is effective to
securely cut an excess polarization component. Further, in order to
increase a utilization efficiency of the laser light, an axis of
the polarizing plate 13 desirably coincides with a polarization
direction of the laser light exited from the optical fiber 11.
[0066] The diffractive light fluxes of respective orders directed
to the pupil conjugate plane 25 are incident on the beam selecting
section 18 which is disposed in the vicinity of the pupil conjugate
plane 25.
[0067] The beam selecting section 18 makes only a pair of
diffractive light fluxes (only .+-.first-order diffractive light
fluxes, in this case), out of the incident diffractive light fluxes
of respective orders, to be selectively passed therethrough.
[0068] The .+-.first-order diffractive light fluxes passed through
the beam selecting section 18 form, with the use of the lens 21, a
plane conjugated with the diffraction grating 16 in the vicinity of
the field stop 22, and are then converted into a collimated light
by the field lens 23. Further, the collimated light passes through
the excitation filter 24, and is reflected by the dichroic mirror
33 to be collected at mutually different positions on the pupil
plane 32 of the objective lens 31.
[0069] The respective .+-.first-order diffractive light fluxes
collected on the pupil plane 32 are turned into collimated light
fluxes when being exited from a tip of the objective lens 31, and
interfere with each other on a surface of the sample 2, to thereby
form interference fringes. The interference fringes are used as
structured illuminating lights.
[0070] When the sample 2 is illuminated by such structured
illuminating lights, a moire fringe corresponding to a difference
between a pitch structure of the structured illuminating lights and
a pitch structure of the sample 2 (of the fluorescent area)
appears, in which on the moire fringe, a structure of high
frequency of the sample 2 is shifted to a side of frequency that is
lower than the original frequency, so that a light (fluorescence)
that exhibits this structure is directed to the objective lens 31
at an angle smaller than the original angle. Therefore, when the
sample 2 is illuminated by the structured illuminating lights, even
structural information of the high frequency of the sample 2 (of
the fluorescent area) is transmitted by the objective lens 31.
[0071] The fluorescence generated in the sample 2 is incident on
the objective lens 31, and converted into a collimated light by the
objective lens 31, and after that, the collimated light transmits
through the dichroic mirror 33 and a barrier filter 34, and a
modulating image formation of the sample 2 is performed on the
imaging plane 41 of the imaging sensor 42 via the secondary
objective lens 35.
[0072] A result of the modulating image formation is subjected to
imaging by the imaging sensor 42, and a result thereof is taken
into the image storing-calculating device 44 via the controlling
device 43. Further, the image storing-calculating device 44
performs a demodulating calculation (details will be described
later) on the modulated image taken therein, thereby generating a
demodulated image (super-resolved image). Further, the
super-resolved image is stored in the internal memory (not
illustrated) of the image storing-calculating device 44, and at the
same time, it is sent to the image displaying device 45.
[0073] Next, the beam branching section 14 will be described in
detail.
[0074] FIG. 2 are diagrams each explaining the beam branching
section 14, in which FIG. 2A is a diagram in which the diffraction
grating 16 of the beam branching section 14 is seen from the
optical axis direction, and FIG. 2B is a diagram illustrating a
positional relationship of collecting points formed on a pupil
conjugate plane by .+-.first-order diffractive light fluxes. Note
that FIG. 2A is a schematic diagram, so that a structural pitch of
the diffraction grating 16 illustrated in FIG. 2A is not always the
same as an actual structural pitch.
[0075] As illustrated in FIG. 2A, the diffraction grating 16 is a
two-dimensional diffraction grating having pitch structures along
mutually different plural directions perpendicular to the optical
axis of the illuminating optical system 10. Here, the diffraction
grating 16 has a pitch structure along each of a first direction
V1, a second direction V2, and a third direction V3, the directions
being different from one another by 120.degree., and it is assumed
that pitches of the pitch structures are common.
[0076] Note that the pitch structure of the diffraction grating 16
may be either a pitch structure of density type formed by utilizing
a density (transmittance), or a pitch structure of phase type
formed by utilizing a level difference (phase contrast), but, the
pitch structure of phase contrast type is more preferable in a
point that a diffraction efficiency of +first-order diffractive
light is high.
[0077] A collimated light flux incident on such a diffraction
grating 16 is converted into a first diffractive light flux group
branched along the first direction V1, a second diffractive light
flux group branched along the second direction V2, and a third
diffractive light flux group branched along the third direction
V3.
[0078] The first diffractive light flux group includes a 0th-order
diffractive light flux and .+-.first-order diffractive light
fluxes, and the .+-.first-order diffractive light fluxes each
having a common order, out of the above, travel in directions
symmetric with respect to the optical axis.
[0079] In like manner, the second diffractive light flux group
includes a 0th-order diffractive light flux and .+-.first-order
diffractive light fluxes, and the .+-.first-order diffractive light
fluxes each having a common order, out of the above, travel in
directions symmetric with respect to the optical axis.
[0080] In like manner, the third diffractive light flux group
includes a 0th-order diffractive light flux and .+-.first-order
diffractive light fluxes, and the .+-.first-order diffractive light
fluxes each having a common order, out of the above, travel in
directions symmetric with respect to the optical axis.
[0081] The .+-.first-order diffractive light fluxes of the first
diffractive light flux group, the .+-.first-order diffractive light
fluxes of the second diffractive light flux group, and the
.+-.first-order diffractive light fluxes of the third diffractive
light flux group, are collected, by the aforementioned collecting
lens 17, at mutually different positions within the pupil conjugate
plane.
[0082] Further, as illustrated in FIG. 2B, collecting points 25d
and 25g of the .+-.first-order diffractive light fluxes of the
first diffractive light flux group are symmetric with respect to
the optical axis, and an arranging direction of the collecting
points 25d and 25g corresponds to the first direction V1.
[0083] Further, collecting points 25c and 25f of the
.+-.first-order diffractive light fluxes of the second diffractive
light flux group are symmetric with respect to the optical axis,
and an arranging direction of the collecting points 25c and 25f
corresponds to the second direction V2. Note that a displace amount
between the collecting points 25c and 25f of the second diffractive
light flux group is the same as a displace amount between the
collecting points 25d and 25g of the first diffractive light flux
group.
[0084] Further, collecting points 25b and 25e of the
.+-.first-order diffractive light fluxes of the third diffractive
light flux group are symmetric with respect to the optical axis,
and an arranging direction of the collecting points 25b and 25e
corresponds to the third direction V3. Note that a displace amount
between the collecting points 25b and 25e of the third diffractive
light flux group is the same as a displace amount between the
collecting points 25d and 25g of the first diffractive light flux
group.
[0085] In the above-described beam branching section 14, the
translatory shifting mechanism 15 is formed of a piezoelectric
motor or the like. The translatory shifting mechanism 15 makes the
diffraction grating 16 to be translatory shifted in a direction
perpendicular to the optical axis of the illuminating optical
system 10, and a direction which is not perpendicular to each of
the above-described first direction V1, second direction V2, and
third direction V3. When the diffraction grating 16 is translatory
shifted in this direction, a phase of fringes of the structured
illuminating lights is shifted (details will be described
later).
[0086] Next, the beam selecting section 18 will be described in
detail.
[0087] FIG. 3 and FIG. 4 are diagrams explaining the beam selecting
section 18. The 1/2 wavelength plate 19 of the beam selecting
section 18 sets a polarization direction of incident diffractive
light fluxes of respective orders, as illustrated in FIG. 3, and
the beam selecting element 20 of the beam selecting section 18 is a
mask which makes only the .+-.first-order diffractive light fluxes
of any one of the first to third diffractive light flux groups to
be selectively passed therethrough, as illustrated in FIG. 4.
[0088] Further, a not-illustrated rotating mechanism of the beam
selecting section 18 makes the beam selecting element 20 rotate
around the optical axis to switch the .+-.first-order diffractive
light fluxes to be selected among the first to third diffractive
light flux groups, and maintains a polarization direction when the
selected .+-.first-order diffractive light fluxes are incident on
the sample 2 to S polarization, by rotating the 1/2 wavelength
plate 19 around the optical axis in conjunction with the beam
selecting element 20.
[0089] Specifically, the beam selecting section 18 switches the
direction of fringes of the structured illuminating lights while
keeping a state of the fringes of the structured illuminating
lights. Hereinafter, conditions for keeping the state of fringes
will be concretely described.
[0090] First, a direction of a fast axis of the 1/2 wavelength
plate 19 is required to be set so that a polarization direction of
the .+-.first-order diffractive light fluxes becomes perpendicular
to the branching direction of the .+-.first-order diffractive light
fluxes to be selected (any one of the first direction V1 to the
third direction V3). Note that the fast axis of the 1/2 wavelength
plate 19 described here indicates a direction in which a phase
delay amount when a light polarized in a direction of the axis
passes through the 1/2 wavelength plate 19 is minimized.
[0091] Further, an opening pattern of the beam selecting element 20
is formed of a first opening portion 20A and a second opening
portion 20B through which one and the other of the .+-.first-order
diffractive light fluxes belonging to the same diffractive light
flux group are individually passed, and a length of each of the
first opening portion 20A and the second opening portion 20B around
the optical axis is set to a length which enables the diffractive
light flux linearly polarized in the above-described direction to
pass through each of the opening portions. Therefore, a shape of
each of the first opening portion 20A and the second opening
portion 20B is a shape close to a partial ring-belt shape.
[0092] Here, as illustrated in FIG. 3A, a rotating position of the
1/2 wavelength plate 19 when the direction of the fast axis of the
1/2 wavelength plate 19 becomes parallel to a direction of axis of
the polarizing plate 13, is set to a reference of the rotating
position of the 1/2 wavelength plate 19 (referred to as "first
reference position", hereinafter).
[0093] Further, a rotating position of the beam selecting element
20 when a beam selecting direction (=a branching direction of the
.+-.first-order diffractive light fluxes to be selected) of the
beam selecting element 20 becomes perpendicular to the direction of
axis of the polarizing plate 13, is set to a reference of the
rotating position of the beam selecting element 20 (referred to as
"second reference position", hereinafter).
[0094] At this time, a rotation amount of the 1/2 wavelength plate
19 from the first reference position should be controlled to be
half of a rotation amount of the beam selecting element 20 from the
second reference position, as illustrated in FIG. 3B.
[0095] Specifically, when the rotation amount of the 1/2 wavelength
plate 19 from the first reference position is .theta./2, the
rotation amount of the beam selecting element 20 from the second
reference position is set to .theta..
[0096] Accordingly, when the rotating mechanism 18A of the beam
selecting section 18 rotates the beam selecting direction of the
beam selecting element 20 in the right direction from the second
reference position by a rotation angle .theta.1, as illustrated in
FIG. 4A, to select the .+-.first-order diffractive light fluxes of
the first diffractive light flux group (the branching direction is
the first direction V1), the rotating mechanism 18A rotates the
direction of the fast axis of the 1/2 wavelength plate 19 in the
right direction from the first reference position by a rotation
angle .theta.1/2.
[0097] At this time, although the polarization directions of the
diffractive light fluxes of respective orders before the
diffractive light fluxes pass through the 1/2 wavelength plate 19
are parallel to the direction of axis of the polarizing plate 13,
as indicated by dashed bidirectional arrows in FIG. 4A, the
polarization directions of the diffractive light fluxes of
respective orders after the diffractive light fluxes pass through
the 1/2 wavelength plate 19 are rotated in the right direction by
the rotation angle .theta.1, so that the polarization directions of
the selected .+-.first-order diffractive light fluxes become
perpendicular to the branching direction of the .+-.first-order
diffractive light fluxes (the first direction V1), as indicated by
solid bidirectional arrows in FIG. 4A.
[0098] Further, when the rotating mechanism 18A of the beam
selecting section 18 rotates the beam selecting direction of the
beam selecting element 20 in the right direction from the second
reference position by a rotation angle .theta.2, as illustrated in
FIG. 4B, to select the .+-.first-order diffractive light fluxes of
the second diffractive light flux group (the branching direction is
the second direction V2), the rotating mechanism 18A rotates the
direction of the fast axis of the 1/2 wavelength plate 19 in the
right direction from the first reference position by a rotation
angle .theta.2/2.
[0099] At this time, although the polarization directions of the
diffractive light fluxes of respective orders before the
diffractive light fluxes pass through the 1/2 wavelength plate 19
are parallel to the direction of axis of the polarizing plate 13,
as indicated by dashed bidirectional arrows in FIG. 4B, the
polarization directions of the diffractive light fluxes of
respective orders after the diffractive light fluxes pass through
the 1/2 wavelength plate are rotated in the right direction by the
rotation angle .theta.2, so that the polarization directions of the
selected .+-.first-order diffractive light fluxes become
perpendicular to the branching direction of the .+-.first-order
diffractive light fluxes (the second direction V2), as indicated by
solid bidirectional arrows in FIG. 4B.
[0100] Further, when the rotating mechanism 18A of the beam
selecting section 18 rotates the beam selecting direction of the
beam selecting element 20 in the left direction (when seen from the
sample side, which is similarly applied to the following
description) from the second reference position by a rotation angle
.theta.3, as illustrated in FIG. 4C, to select the .+-.first-order
diffractive light fluxes of the third diffractive light flux group
(the branching direction is the third direction V3), the rotating
mechanism 18A rotates the direction of the fast axis of the 1/2
wavelength plate 19 in the left direction from the first reference
position by a rotation angle .theta.3/2.
[0101] At this time, although the polarization directions of the
diffractive light fluxes of respective orders before the
diffractive light fluxes pass through the 1/2 wavelength plate 19
are parallel to the direction of axis of the polarizing plate 13,
as indicated by dashed bidirectional arrows in FIG. 4C, the
polarization directions of the diffractive light fluxes of
respective orders after the diffractive light fluxes pass through
the 1/2 wavelength plate 19 are rotated in the left direction by
the rotation angle .theta.3, so that the polarization directions of
the selected .+-.first-order diffractive light fluxes become
perpendicular to the branching direction of the .+-.first-order
diffractive light fluxes (the third direction V3), as indicated by
solid bidirectional arrows in FIG. 4C.
[0102] Therefore, the rotating mechanism 18A of the beam selecting
section 18 is only required to move the 1/2 wavelength plate 19 and
the beam selecting element 20, in a conjunctive manner, at a gear
ratio of 2:1.
[0103] FIG. 5 is a diagram explaining the function of the beam
selecting section 18 described above. Note that in FIG. 5,
bidirectional arrows surrounded by a circular frame indicate a
polarization direction of a light flux, and bidirectional arrows
surrounded by a quadrangular frame indicate an axial direction of
the optical element.
[0104] Further, as illustrated in FIG. 6, a plurality of (six, in
an example illustrated in FIG. 6) cutouts 20C are formed on an
outer peripheral portion of the beam selecting element 20, and the
rotating mechanism 18A is provided with a timing sensor 20D for
detecting the cutouts 20C. Accordingly, the rotating mechanism 18A
can detect not only the rotating position of the beam selecting
section 18 but also the rotating position of the 1/2 wavelength
plate 19.
[0105] Next, the translatory shifting mechanism 15 of the beam
branching section 14 will be described in detail.
[0106] FIG. 7 are diagrams explaining an operation of the
translatory shifting mechanism 15 of the beam branching section
14.
[0107] First, in order to enable to conduct the demodulating
calculation (details thereof will be described later), at least two
modulated images in which directions of interference fringes are
common and phases are different, being modulated images related to
the same sample 2, are required. This is because in a modulated
image generated by the structured illuminating microscopy apparatus
1, a 0th-order modulating component, a +first-order modulating
component, and a -first-order modulating component being
information of structure in which a spatial frequency is modulated
by the structured illuminating lights, out of a structure of the
sample 2 are superimposed, and there is a need to make the mutually
superimposed three unknown parameters to be known in the
demodulating calculation (details thereof will be described
later).
[0108] Accordingly, in order to shift the phase of the interference
fringes, the translatory shifting mechanism 15 of the beam
branching section 14 shifts the diffraction grating 16 along a
direction which is perpendicular to the optical axis of the
illuminating optical system 10, and which is not perpendicular to
all of the aforementioned first direction V1, second direction V2,
and third direction V3 (x direction), as illustrated in FIG.
7A.
[0109] Note that a shift amount L of the diffraction grating 16
required to shift the phase of the interference fringes by a
desired shift amount .PHI. is not always the same when the beam
selecting direction selected by the beam selecting section 18 is
the first direction V1, when the direction is the second direction
V2, and when the direction is the third direction V3.
[0110] As illustrated in FIG. 7B, if a structural pitch (pitch) in
each of the first direction V1, the second direction V2, and the
third direction V3 of the diffraction grating 16 is set to P, an
angle made by a shift direction of the diffraction grating 16 (x
direction) and the first direction V1 is set to .theta.1, an angle
made by the shift direction of the diffraction grating 16 (x
direction) and the second direction V2 is set to .theta.2, and an
angle made by the shift direction of the diffraction grating 16 (x
direction) and the third direction V3 is set to .theta.3, a shift
amount L1 in the x direction of the diffraction grating 16 required
when the beam selecting direction is the first direction V1 is
represented by L1=.PHI..times.P/(4.pi..times.|cos .theta.1|), a
shift amount L2 in the x direction of the diffraction grating 16
required when the beam selecting direction is the second direction
V2 is represented by L2=.PHI..times.P/(4.pi..times.|cos .theta.2|),
and a shift amount L3 in the x direction of the diffraction grating
16 required when the beam selecting direction is the third
direction V3 is represented by L3=.PHI..times.P/(4.pi..times.|cos
.theta.3|).
[0111] Specifically, the shift amount L in the x direction of the
diffraction grating 16 required to set the phase shift amount of
the interference fringes to have the desired value .PHI. is
represented, as in an expression (1), by the angle .theta. made by
the wavelength selecting direction (any one of the first direction
V1, the second direction V2, and the third direction V3) and the x
direction.
L=.PHI..times.P/(4.pi..times.|cos .theta.|) (1)
[0112] Incidentally, a shift amount L in the x direction of the
diffraction grating 16 required to set the phase shift amount .PHI.
of the interference fringes to 2.pi. is represented by
P/(2.times.|cos .theta.|). This is an amount corresponding to a
half pitch of the diffraction grating 16. Specifically, only by
shifting the diffraction grating 16 by a half pitch, the phase of
the structured illuminating light can be shifted by one pitch (this
is because a fringe pitch of the interference fringes formed of the
.+-.first-order diffractive lights corresponds to twice the
structural pitch of the diffraction grating 16).
[0113] [Basic Operation of Image Storing-Calculating Device 44]
[0114] The above-described image storing-calculating device 44 is
formed of a computer which performs calculation by executing a
program for calculation, an operation circuit which performs
calculation processing, or a combination of both of the computer
and the operation circuit. Further, the computer may be a
general-purpose computer on which the program for calculation is
installed via a storage medium or a communication network.
[0115] Hereinafter, a basic procedure of the demodulating
calculation performed by the image storing-calculating device 44
will be described. The basic procedure includes the following four
steps.
[0116] First step: Each of a plurality of modulated images is
subjected to Fourier transform, to thereby generate a plurality of
spatial frequency spectra.
[0117] Second step: A 0th-order modulating component of
fluorescence, a +first-order modulating component of fluorescence,
and a -first-order modulating component of fluorescence
superimposed on each of the spatial frequency spectra are separated
from one another on a Fourier space.
[0118] Third step: The mutually separated 0th-order modulating
component of fluorescence, +first-order modulating component of
fluorescence, and -first-order modulating component of fluorescence
are arranged again on the Fourier space, to thereby generate a
spatial frequency spectrum of a demodulated image.
[0119] Fourth step: The spatial frequency spectrum of the
demodulated image is subjected to inverse Fourier transform, to
thereby acquire the demodulated image (=super-resolved image).
[0120] Note that at least two of these steps may be collectively
executed by using one arithmetic expression.
[0121] [2D/3D Switching]
[0122] Hereinafter, explanation will be made on a 2D/3D switching
of the structured illuminating microscopy apparatus described
above.
[0123] In the above explanation, the interference fringes projected
onto the sample 2 are set to two-beam interference fringes
(specifically, the example of using the structured illuminating
microscopy apparatus 1 in the 2D-SIM mode is explained), but, it is
also possible to set the interference fringes projected onto the
sample 2 to three-beam interference fringes (specifically, to use
the structured illuminating microscopy apparatus 1 in the 3D-SIM
mode).
[0124] In the 3D-SIM mode, a beam selecting element 20' as
illustrated in FIG. 8 is used, instead of using the beam selecting
element 20 illustrated in FIG. 6. The beam selecting element 20'
corresponds to the beam selecting element 20 illustrated in FIG. 6
in which an opening portion 20E through which the 0th-order
diffractive light flux passes is provided. Note that a place of
formation of the opening portion 20E is in the vicinity of the
optical axis, and a shape of the opening portion 20E is a circular
shape, for example. With the use of such a beam selecting element
20', it is possible to make not only the .+-.first-order
diffractive light fluxes but also the 0th-order diffractive light
flux contribute to the interference fringes.
[0125] As described above, the interference fringes generated by
the interference of three diffractive light fluxes (three-beam
interference) are spatially-modulated not only in the surface
direction of the sample 2 but also in the depth direction of the
sample 2. Therefore, with the use of the interference fringes, it
is possible to obtain the super-resolution effect also in the depth
direction of the sample 2.
[0126] Note that between the 2D-SIM mode and the 3D-SIM mode, the
contents of the demodulating calculation to be executed by the
image storing-calculating device 44 are different. This is because,
although the three components being the 0th-order modulating
component of fluorescence, the +first-order modulating component of
fluorescence, and the -first-order modulating component of
fluorescence are superimposed on the modulated image generated in
the 2D-SIM mode, five components being the 0th-order modulating
component of fluorescence, the +first-order modulating component of
fluorescence, the -first-order modulating component of
fluorescence, a +second-order modulating component of fluorescence,
and a -second-order modulating component of fluorescence are
superimposed on the modulated image generated in the 3D-SIM
mode.
[0127] Further, since the number of modulating components
superimposed on the modulated image are different between the
2D-SIM mode and the 3D-SIM mode, the number of frames of the
modulated images to be acquired by the controlling device 43 and
the like are also different. Detailed explanation will be given
hereinafter.
[0128] [Section 1.1 (Prerequisite of 2D-SIM)]
[0129] In this Section, a prerequisite of the 2D-SIM will be
described.
[0130] Here, an interference fringe intensity distribution in the
2D-SIM mode is defined as follows.
[0131] A fluorescent material density of a sample is set to
I.sub.0(x), and an interference fringe intensity distribution on a
sample plane is set to K (x). Further, it is assumed that a
fluorescence generated in the sample is in proportion to an
illumination intensity. In this case, a fluorescence intensity
distribution I.sub.fl (x) is represented as follows.
I.sub.fl(x)=I.sub.0(x)K(x) (1.1)
[0132] Further, a fluorescence generated at each point of the
sample is incoherent, so that an image as a result of capturing the
fluorescence intensity distribution I.sub.fl (x) using an objective
lens, namely, a modulated image I (x) is represented as follows by
an expression of incoherent image formation.
I(x)=.intg..intg.dx'dy'PSF(x-x')I.sub.fl(x') (1.2)
[0133] Hereinafter, the Fourier transform of each function is
represented as follows.
(.xi.)=[I(x)](.xi.)
[0134] In this case, a result of representing the modulated image
on a Fourier space (namely, a spatial frequency spectrum of the
modulated image) is represented as follows.
(.xi.)=OTF(.xi.) .sub.fl(.xi.) (1.3)
[0135] Since OTF becomes zero under a condition of |.xi.|>2NA,
the spatial frequency spectrum of the modulated image also becomes
zero under the condition of |.xi.|>2NA. Note that the following
relationship is used in this case.
OTF(.xi.)=[PSF(x)]
[0136] Further, the fluorescence intensity distribution on the
Fourier space is represented as follows.
.sub.fl(.xi.)=.intg..intg.d.xi.'d.eta.' .sub.0(.xi.-.xi.'){tilde
over (K)}(.xi.') (1.4)
[0137] Hereinafter, a coefficient which is not necessary in the
explanation of the demodulating calculation will be ignored.
[0138] [Section 1.2 (Conventional 2D-SIM)]
[0139] In this Section, a demodulating calculation of the
conventional 2D-SIM will be described for comparison.
[0140] First, an interference fringe intensity distribution of the
2D-SIM is represented as follows (fringes have a sinusoidal
intensity distribution).
K .function. ( x ) = 1 + cos .function. ( 2 .times. .pi. .lamda.
.times. .xi. 0 x - .PHI. ) ( 1.5 ) ##EQU00001##
[0141] Note that .xi..sub.0 indicates a spatial frequency
(modulation frequency) of the interference fringes.
[0142] Therefore, the interference fringe intensity on the Fourier
space is represented as follows.
K ~ .function. ( .xi. ) = .delta. .function. ( .xi. ) + .delta.
.function. ( .xi. - .xi. 0 ) .times. e - i .times. .PHI. + .delta.
.function. ( .xi. + .xi. 0 ) .times. e i .times. .PHI. 2 ( 1.6 )
##EQU00002##
[0143] Note that indicates a coordinate on the Fourier space.
[0144] According to this expression 1.6, the expression 1.3, and
the expression 1.4, it can be understood that the modulated image
on the Fourier space is represented as follows.
(.xi.)=OTF(.xi.)(1/2e.sup.-i.PHI. .sub.0(.xi.-.xi..sub.0)+
.sub.0(.xi.)+1/2e.sup.i.PHI. .sub.0(.xi.+.xi..sub.0)) (1.7)
[0145] Hereinafter, the spatial frequency spectrum on the Fourier
space is simply referred to as "spectrum". Further, to a modulated
image acquired when a phase of the interference fringes is
.PHI..sub.i, a subscript ".PHI..sub.i" corresponding to the image
is attached.
[0146] Here, as described above, on the observation point .xi. in
the spectrum of the modulated image acquired in the 2D-SIM, the
three components being the -first-order modulating component of
fluorescence, the +first-order modulating component of
fluorescence, and the 0th-order modulating component of
fluorescence, are superimposed. Three terms in the right side of
the expression 1.7 correspond to the respective modulating
components. Specifically, since the sample (fluorescence) is
spatially modulated by the fringes having the sinusoidal intensity
distribution, the spectrum of the modulated image can be
represented by the three modulating components (the 0th-order
modulating component and the .+-.first-order modulating components)
of fluorescence. The +first-order modulating component superimposed
on the observation point is a value (restoration value) which
should be possessed by a restoration point (.xi.-.xi..sub.0) in a
spectrum of a demodulated image, the -first-order modulating
component superimposed on the observation point .xi. is a value
(restoration value) which should be possessed by a restoration
point (.xi.+.xi..sub.0) in the spectrum of the demodulated image,
and the 0th-order modulating component superimposed on the
observation point .xi. is a value (restoration value) which should
be possessed by a restoration point .xi. in the spectrum of the
demodulated image. This applies to each observation point in the
spectrum of the modulated image. One large black point in FIG. 9
corresponds to a certain observation point, and the large black
point and two small black points on both sides of the large black
point correspond to three restoration points restored from the
observation point.
[0147] Accordingly, in the demodulating calculation of the
conventional 2D-SIM, in order to make the three modulating
components superimposed on each observation point in the spectrum
of the modulated image to be separated from one another, three
modulated images with different phases of fringes have been
acquired to generate spectra of the respective modulated images,
and the spectra have been applied to three expressions (the
following expression 1.8, expression 1.9, and expression 1.10), to
thereby obtain three equations. Conventionally, by solving the
three equations, restoration values of a painted-out area (normal
resolution range and super-resolution range) in FIG. 9 have been
determined.
.sub..PHI.1(.xi.)=OTF(.xi.)(1/2e.sup.-i.PHI..sup.1
.sub.0(.xi.-.xi..sub.0)+ .sub.0(.xi.)+1/2e.sup.i.PHI..sup.1
.sub.0(.xi.+.xi..sub.0)) (1.8)
.sub..PHI.2(.xi.)=OTF(.xi.)(1/2e.sup.-i.PHI..sup.2
.sub.0(.xi.-.xi..sub.0)+ .sub.0(.xi.)+1/2e.sup.i.PHI..sup.2
.sub.0(.xi.+.xi..sub.0)) (1.9)
.sub..PHI.3(.xi.)=OTF(.xi.)(1/2e.sup.-i.PHI..sup.3
.sub.0(.xi.-.xi..sub.0)+ .sub.0(.xi.)+1/2e.sup.i.PHI..sup.3
.sub.0(.xi.+.xi..sub.0)) (1.10)
[0148] Incidentally, when it is described that .tau.=OTF (.xi.) for
simplification, the expression 1.8, the expression 1.9, and the
expression 1.10 can be rewritten as follows.
[ I ~ .PHI.1 .function. ( .xi. ) I ~ .PHI.2 .function. ( .xi. ) I ~
.PHI.3 .function. ( .xi. ) ] = .tau. 2 .function. [ e - i .times.
.PHI. 1 2 e i .times. .PHI. 1 e - i .times. .PHI. 2 2 e i .times.
.PHI. 2 e - i .times. .PHI. 3 2 e i .times. .PHI. 3 ] .function. [
I ~ 0 .function. ( .xi. - .xi. 0 ) I ~ 0 .function. ( .xi. ) I ~ 0
.function. ( .xi. + .xi. 0 ) ] ( 1.11 ) ##EQU00003##
[0149] Note that when a determinant of a matrix (which is set as M,
hereinafter) of this expression is not zero, it is possible to
determine, from three observation values (left side) of a certain
observation point in a spectrum of each of three modulated images,
restoration values (right side) of three restoration points
corresponding to the observation points.
[0150] Here, the spatial frequency (modulation frequency)
.xi..sub.0 of the fringes in the conventional 2D-SIM is set so that
|.xi..sub.0|<2NA is satisfied, and it is possible to determine a
restoration value of a restoration point which satisfies
|.xi..+-..xi..sub.0|>2NA by an observation value obtained from a
normal resolution area in which |.xi.|<2NA is satisfied.
Accordingly, in the conventional 2D-SIM, it is possible to restore
a restoration value of an area outside of the normal resolution
area (super-resolution area), namely, it is possible to obtain a
super-resolved image as a demodulated image.
[0151] Note that the above-described matrix M does not depend on
.xi.. Specifically, the above-described matrix M does not depend on
the coordinate (=spatial frequency) on the Fourier space.
Accordingly, when a condition number of the matrix M is plotted by
setting the phase .PHI..sub.i as a parameter, a result as
illustrated in FIG. 10 is obtained.
[0152] FIG. 10 illustrates a distribution of a reciprocal of the
condition number of the matrix M. Note that in this case, a phase
.PHI..sub.1 of a first modulated image is set to 0.degree., and
each of a phase .PHI..sub.2 of a second modulated image and a phase
.PHI..sub.3 of a third modulated image is set to a variable. A
horizontal axis in FIG. 10 indicates .PHI..sub.2, and a vertical
axis in FIG. 10 indicates .PHI..sub.3.
[0153] From FIG. 10, it can be understood that when .PHI..sub.2
equals to 120.degree. and .PHI..sub.3 equals to 240.degree., the
reciprocal of the condition number takes the maximum value of 0.5,
and thus the best condition is provided. For this reason, the
conventional 2D-SIM generally sets a phase contrast among three
frames to 120.degree..
[0154] [Section 1.3 (Two-Image-Two-Point Restoration in
2D-SIM)]
[0155] In this Section, "two-image-two-point restoration" will be
described as a demodulating calculation in the 2D-SIM of the
present embodiment. It is set that the acquirement of modulated
images in this Section is performed when the aforementioned
controlling device 43 controls the respective parts, and the
calculation in this Section is executed by the aforementioned image
storing-calculating device 44 (which similarly applies to the other
Sections).
[0156] This Section focuses attention on the fact that, in a
spectrum of one piece of modulated image acquired by the 2D-SIM,
modulating components with mutually common values are superimposed
on two observation points .xi. and (.xi.+.xi..sub.0) which are
separated by a modulation frequency .xi..sub.0 in a modulating
direction.
[0157] Concretely, each of a -first-order modulating component of
fluorescence superimposed on the observation point .xi. and a
0th-order modulating component of fluorescence superimposed on the
observation point (.xi.+.xi..sub.0) corresponds to a restoration
value of a restoration point (.xi.+.xi..sub.0), each of a
+first-order modulating component of fluorescence superimposed on
the observation point (.xi.+.xi..sub.0) and a 0th-order modulating
component of fluorescence superimposed on the observation point
.xi. corresponds to a restoration value of a restoration point.
Specifically, the two observation points .xi. and (.xi.+.xi..sub.0)
include the mutually common restoration values of the two
restoration points .xi. and (.xi.+.xi..sub.0). The
two-image-two-point restoration in this Section utilizes this
relationship. Concrete description will be made hereinafter.
[0158] First, if it is assumed that the interference fringe
intensity distribution is set in a similar manner to that of the
conventional 2D-SIM, a range of observation of the spectrum of the
modulated image is represented by |.xi.|<2NA, by using NA of the
objective lens.
[0159] The spatial frequency (modulation frequency) .xi..sub.0 of
the fringes in this Section is set so that |.xi..sub.0|<2NA is
satisfied. Note that the spatial frequency (modulation frequency)
.xi..sub.0 of the fringes is set by a grating pitch of the
diffraction grating 16 (fringe pitch formed on the sample).
[0160] In this case, it is possible to obtain observation values at
two observation points .xi. and (.xi.+.xi..sub.0) which are
separated by .xi..sub.0, from a spectrum of one piece of modulated
image. Note that a range capable of obtaining the observation value
at (.xi.+.xi..sub.0) is limited to a range in which satisfies a
condition of |.xi.+.xi..sub.0|<2NA.
[0161] Here, the observation value at the observation point .xi.
and the observation value at the observation point
(.xi.+.xi..sub.0) in the spectrum of the one piece of modulated
image are represented by the following expression.
.sub..PHI.(.xi.)=OTF(.xi.)(1/2e.sup.-i.PHI.
.sub.0(.xi.-.xi..sub.0)+ .sub.0(.xi.)+1/2e.sup.i.PHI.
.sub.0(.xi.+.xi..sub.0)) (1.12)
.sub..PHI.(.xi.-.xi..sub.0)=OTF(.xi.-.xi..sub.0)(1/2e.sup.-i.PHI.
.sub.0(.xi.)+ .sub.0(.xi.+.xi..sub.0)+1/2e.sup.i.PHI.
.sub.0(.xi.+2.xi..sub.0)) (1.13)
[0162] In the right side of the expression 1.12 and the expression
1.13, restoration values (unknowns) of four restoration points
appear. In order to make these four restoration values to be known,
two more expressions are necessary.
[0163] Accordingly, in this Section, a spectrum of each of two
modulated images whose phases .PHI. are mutually different is
generated, four observation values, in total, regarding two
observation points .xi. and (.xi.+.xi..sub.0) are referred to from
the respective two spectra, and the four observation values are
applied to the expression 1.12 and the expression 1.13, to thereby
obtain four expressions, in total, including four restoration
values (unknowns).
[0164] Here, when it is set that .tau..sub.1=OTF(.xi.),
.tau..sub.2=OTF(.xi.+.xi..sub.0), the phase .PHI. of the first
modulated image is .PHI..sub.1, and the phase .PHI. of the second
modulated image is .PHI..sub.2, for simplification, the four
expressions can be represented by the following matrix.
[ I ~ .PHI.1 .function. ( .xi. ) I ~ .PHI.1 .function. ( .xi. +
.xi. 0 ) I ~ .PHI.2 .function. ( .xi. ) I ~ .PHI.3 .function. (
.xi. + .xi. 0 ) ] = 1 2 .function. [ .tau. 1 .times. e - i .times.
.PHI. 1 2 .times. .tau. 1 .tau. 1 .times. e i .times. .PHI. 1 0 0
.tau. 2 .times. e - i .times. .PHI. 1 2 .times. .tau. 2 .tau. 2
.times. e i .times. .PHI. 1 .tau. 1 .times. e - i .times. .PHI. 2 2
.times. .tau. 1 .tau. 1 .times. e i .times. .PHI. 2 0 0 .tau. 2
.times. e - i .times. .PHI. 2 2 .times. .tau. 2 .tau. 2 .times. e i
.times. .PHI. 1 ] .function. [ I ~ 0 .function. ( .xi. - .xi. 0 ) I
~ 0 .function. ( .xi. ) I ~ 0 .function. ( .xi. + .xi. 0 ) I ~ 0
.function. ( .xi. + 2 .times. .xi. 0 ) ] ( 1.14 ) ##EQU00004##
[0165] Accordingly, in this Section, when a determinant of the
matrix (which is set as M, hereinafter) is not zero, it is possible
to determine the four restoration values (right side) from the four
observation values (left side) in the spectra of the two pieces of
modulated images.
[0166] Here, out of two circular frames in FIG. 11, the circular
frame on the inside indicates an outer edge of the normal
resolution range (|.xi. |=2NA). Further, the circular frame on the
outside indicates an outer edge of the super-resolution range
(|.xi. |=4NA).
[0167] Two large black points in FIG. 11 indicate two certain
observation points displaced by an amount of the spatial frequency
(modulation frequency) .xi..sub.0 of the fringes, and the two large
black points and two small black points in FIG. 11 indicate four
restoration points restored from the two observation points.
[0168] In this Section, the spectra of the two pieces of modulated
images in which the phases of the interference fringes are
different are obtained, so that from the two observation points in
each of the two spectra, the four observation values, in total, are
obtained. Further, by applying the four observation values to the
aforementioned expression 1.14, the restoration values of the
respective four restoration points are determined.
[0169] Further, in this Section, by repeatedly calculating four
restoration values while moving two observation points within the
normal resolution range, restoration values of the entire
painted-out area in FIG. 11 are determined.
[0170] Therefore, in this Section, regardless of the fact that the
number of pieces of acquired modulated images (the number of
generated spectra) is only two, it is possible to determine at
least a part of restoration values in the normal resolution range
and at least a part of restoration values in the super-resolution
range.
[0171] [Section 1.3.1 (Restoration Capable Condition)]
[0172] In this Section, a necessary condition for the demodulating
calculation in Section 1.3 will be described.
[0173] In order to make the above-described expression 1.14 have a
unique solution, it is only required that the determinant of the
matrix M takes a value other than zero. Here, the determinant of
the matrix M is represented as follows.
det .times. .times. M = 1 2 .function. [ .tau. 1 .times. e - i
.times. .PHI. 1 2 .times. .tau. 1 .tau. 1 .times. e i .times. .PHI.
1 0 0 .tau. 2 .times. e - i .times. .PHI. 1 2 .times. .tau. 2 .tau.
2 .times. e i .times. .PHI. 1 .tau. 1 .times. e - i .times. .PHI. 2
2 .times. .tau. 1 .tau. 1 .times. e i .times. .PHI. 2 0 0 .tau. 2
.times. e - i .times. .PHI. 2 2 .times. .tau. 2 .tau. 2 .times. e i
.times. .PHI. 2 ] ( 1.15 ) .times. = .tau. 1 2 .times. .tau. 2 2
.times. 1 2 .function. [ e - i .times. .PHI. 1 2 e i .times. .PHI.
1 0 0 e - i .times. .PHI. 1 2 e i .times. .PHI. 1 e - i .times.
.PHI. 2 2 e i .times. .PHI. 2 0 0 e - i .times. .PHI. 2 2 e i
.times. .PHI. 2 ] ( 1.16 ) .times. = 4 .times. .tau. 1 2 .times.
.tau. 2 2 .function. ( cos .function. ( .PHI. 1 - .PHI. 2 ) - 1 ) 2
( 1.17 ) ##EQU00005##
[0174] Therefore, as long as a phase contrast .DELTA..PHI. between
the phase .PHI..sub.1 of the first modulated image and the phase
.PHI..sub.2 of the second modulated image satisfies a condition of
.DELTA..PHI..noteq.0, a condition of detM.noteq.0 is satisfied, and
the expression 1.14 has a unique solution. As a result of the
above, it can be understood that the necessary condition for the
demodulating calculation in Section 1.3 is the condition of
.DELTA..PHI..noteq.0.
[0175] [Section 1.3.4 (Characteristic of Condition of
.DELTA..PHI.=.pi.)]
[0176] In this Section, a characteristic of a condition of
.DELTA..PHI.=.pi. will be described.
[0177] When the condition of .DELTA..PHI.=.pi. is satisfied, the
following expression is satisfied.
I ~ .PHI.1 .function. ( .xi. ) = .tau. 1 2 .times. ( I ~ 0
.function. ( .xi. - .xi. 0 ) + 2 .times. I ~ 0 .function. ( .xi. )
+ I ~ 0 .function. ( .xi. + .xi. 0 ) ) ( 1.21 ) I ~ .PHI.2
.function. ( .xi. ) = .tau. 1 2 .times. ( - I ~ 0 .function. ( .xi.
- .xi. 0 ) + 2 .times. I ~ 0 .function. ( .xi. ) - I ~ 0 .function.
( .xi. + .xi. 0 ) ) ( 1.22 ) ##EQU00006##
[0178] In this case, the phases multiplied with respect to the
following expression become equal.
.sub.0(.xi.-.xi..sub.0), .sub.0(.xi.+.xi..sub.0)
[0179] Accordingly, I.sub.0 (.xi.) can be easily solved, and the
following expression is satisfied.
I ~ 0 .function. ( .xi. ) = 1 2 .times. .tau. 1 .times. ( I ~
.PHI.1 .function. ( .xi. ) + I ~ .PHI.2 .function. ( .xi. ) ) (
1.23 ) ##EQU00007##
[0180] Therefore, if the condition of .DELTA..PHI.=.pi. is set to
be satisfied, it is possible to eliminate an area which cannot be
restored in the normal resolution area.
[0181] Although a painted-out area in FIG. 12A indicates a range
capable of being restored under a condition of
.DELTA..PHI..noteq..pi., a painted-out area in FIG. 12B indicates a
range capable of being restored under a condition of
.DELTA..PHI.=.pi. (when |.xi..sub.0|=2NA is satisfied, in both of
the above cases). Out of two circles in each of FIG. 12, the circle
on the inside indicates an outer edge of the normal resolution
range (|.xi. |=2NA), and the circle on the outside indicates an
outer edge of the super-resolution range (|.xi. |=4NA).
[0182] Note that here, although the number of direction of the
interference fringes is assumed to be one, if the number of
directions of the interference fringes is set to three, and a
demodulating calculation similar to that in Section 1.3 is applied
with respect to each direction, it is possible to restore a wide
area such as one illustrated in FIG. 13.
[0183] [Section 1.4 (Two-Pass Restoration in 2D-SIM)]
[0184] In this Section, "Two-pass restoration" will be described as
a demodulating calculation in the 2D-SIM of the present embodiment.
In the Two-pass restoration, the number of directions of the
interference fringes is set to two.
[0185] Hereinafter, in order to distinguish a plurality of
interference fringes in which directions and pitches are mutually
different, the individual interference fringes are represented by
wave number vectors. A magnitude of the wave number vector
indicates a magnitude of the spatial frequency of the interference
fringes, and a direction of the wave number vector indicates the
direction of the interference fringes.
[0186] In this Section, the following four steps are executed.
[0187] First step: Two pieces of modulated images in which wave
number vectors are .xi..sub.0, and phases are different are
acquired, and spectra of the respective two pieces of modulated
images are generated. The respective two pieces of modulated images
are represented as follows.
{I.sub..PHI..sub.1.sup.(0),I.sub..PHI..sub.2.sup.(0)}
[0188] Further, by performing a demodulating calculation similar to
that in Section 1.3 on the spectra of the respective two pieces of
modulated images, restoration values of an area illustrated in FIG.
14A are determined.
[0189] Second step: Two pieces of modulated images in which wave
number vectors are, and phases are different are acquired, and
spectra of the respective two pieces of modulated images are
generated. The respective two pieces of modulated images are
represented as follows.
{I.sub..PHI..sub.1.sup.(1),I.sub..PHI..sub.2.sup.(1)}
[0190] Further, by performing a demodulating calculation similar to
that in Section 1.3 on the spectra of the respective two pieces of
modulated images, restoration values of an area illustrated in FIG.
14B are determined.
[0191] Third step: Based on the restoration values determined in
the above-described steps and the following expression, restoration
values of an area illustrated in FIG. 15A are determined.
I ~ .PHI.1 ( 1 ) .function. ( .xi. ) = .tau. 1 2 .times. ( e - i
.times. .PHI. 1 .times. I ~ 0 .function. ( .xi. - .xi. 1 ) + 2
.times. I ~ 0 .function. ( .xi. ) + e i .times. .PHI. 1 .times. I ~
0 .function. ( .xi. + .xi. 1 ) ) ( 1.24 ) I ~ .PHI.2 ( 1 )
.function. ( .xi. ) = .tau. 1 2 .times. ( e - i .times. .PHI. 2
.times. I ~ 0 .function. ( .xi. - .xi. 1 ) + 2 .times. I ~ 0
.function. ( .xi. ) + e i .times. .PHI. 2 .times. I ~ 0 .function.
( .xi. + .xi. 1 ) ) ( 1.25 ) ##EQU00008##
[0192] Specifically, by applying the restoration values determined
in the above-described steps, namely, by applying the following
expression to an expression 1.26, the restoration values of the
area illustrated in FIG. 15A are determined.
.sub.0(.xi.)
[0193] Note that the expression 1.26 is an expression as a result
of solving an expression 1.24 and an expression 1.25 based on the
following expression.
.times. I ~ 0 .function. ( .xi. - .xi. 1 ) , I ~ 0 .function. (
.xi. + .xi. 1 ) .times. [ I ~ 0 .function. ( .xi. - .xi. 1 ) I ~ 0
.function. ( .xi. + .xi. 1 ) ] = 1 2 .times. i .times. .times. s
.times. .times. in .function. ( .PHI. 1 - .PHI. 2 ) .function. [ e
- i .times. .PHI. 2 - e - i .times. .PHI. 1 - e i .times. .PHI. 2 e
i .times. .PHI. 1 ] .function. [ 2 .tau. 1 .times. I ~ .PHI.1 ( 1 )
.function. ( .xi. ) - I ~ 0 .function. ( .xi. ) 2 .tau. 1 .times. I
~ .PHI.2 ( 1 ) .function. ( .xi. ) - I ~ 0 .function. ( .xi. ) ] (
1.26 ) ##EQU00009##
[0194] Note that in order to enable to conduct the present step, in
at least the second step, a condition of .DELTA..PHI..noteq..pi.n
(n is an integer) is set.
[0195] Fourth step: Based on the restoration values in the normal
resolution range determined in the second step (a part satisfying
the condition of |.xi.|<2NA, out of the painted-out area in FIG.
14B), restoration values of an area illustrated in FIG. 15B are
determined in a similar manner.
[0196] Note that in order to enable to conduct the present step, in
the first step, the condition of .DELTA..PHI..noteq..pi.n (n is an
integer) is set.
[0197] The first step and the third step can also be collectively
represented as in FIG. 16B. Specifically, four black points lined
in a horizontal direction on a horizontal line in FIG. 16B indicate
positions on the Fourier space (wave number space) of four
restoration values (unknowns) determined by simultaneous equations
equivalent to the expression 1.14 solved in the first step.
[0198] On each of two vertical lines in FIG. 16B, a large black
point at a center among three black points lined in a vertical
direction indicates a position, on the Fourier space (wave number
space), of one known number determined by the expression in the
first step, and small black points on both ends among the three
black points indicate positions, on the Fourier space (wave number
space), of two restoration values (unknowns) in the simultaneous
equations 1.26 solved in the third step.
[0199] The mutual positional relationship among the eight black
points is the same even if any example is chosen. A range in which
the black points can be positioned on the Fourier space (wave
number space) is limited by a range in which the two large black
points (at the center) can be positioned on the Fourier space (wave
number space). The range in which the two large black points can be
positioned is represented by |.xi.|<2NA, so that the range in
which the restoration values (unknowns) are positioned, which can
be determined through the calculation in the first step and the
third step, corresponds to a range of painted-out area in FIG.
16B.
[0200] Note that FIG. 16A is a diagram illustrating the second step
and the fourth step, in a similar manner to FIG. 16B.
[0201] Therefore, in this Section, it is possible to restore the
entire painted-out area illustrated in FIG. 17.
[0202] [Section 1.5 (Example of Super Resolution in 2D-SIM)]
[0203] In this Section, two examples of super resolution will be
described, based on the results obtained in Sections up to the
previous Section.
[0204] First, in the first example, the Two-pass restoration is
conducted by considering that a suppression of the number of pieces
of modulated images (number of spectra) is important. In order to
conduct the Two-pass restoration, in the first example, the number
of directions of the wave number vectors is set to two, and under
each of mutually different two wave number vectors .xi..sub.1 and
.xi..sub.2, two pieces of modulated images with different phases
are acquired (four pieces of modulated images, in total, are
acquired), and spectra of the respective four pieces of modulated
images are generated (four spectra, in total, are generated).
Further, in order to enable to conduct the Two-pass restoration, a
phase contrast .DELTA..PHI. between the two pieces of modulated
images acquired under the same wave number vector is set to
.DELTA..PHI..noteq..pi.. Further, a magnitude of the wave number
vector |.xi..sub.i| is set to |.xi..sub.i|=2NA (i=1, 2). In this
case, the painted-out area illustrated in FIG. 17 is restored.
[0205] Next, in the second example, a calculation accuracy is
considered as important, and accordingly, not the Two-pass
restoration, but the "two-image-two-point restoration" is
conducted. In order to conduct the two-image-two-point restoration,
in the second example, the number of directions of the wave number
vectors is set to three, and under each of mutually different three
wave number vectors .xi..sub.1, .xi..sub.2, and .xi..sub.3, two
pieces of modulated images with different phases are acquired (six
pieces of modulated images, in total, are acquired), and spectra of
the respective six pieces of modulated images are generated (six
spectra, in total, are generated). Further, a phase contrast
.DELTA..PHI. between the two pieces of modulated images acquired
under the same wave number vector is set to .DELTA..PHI.=.pi..
Further, in order to avoid a generation of gap in a restored area,
|.xi..sub.i| is intentionally set to be smaller than 2NA.
Concretely, the magnitude |.xi..sub.i| of the wave number vector is
set to |.xi..sub.i|=( 3).times.NA (i=1, 2) to maximize |.xi..sub.i|
within a range of generating no gap in the restored area. In this
case, an area illustrated in FIG. 18 is restored.
[0206] [Section 1.6 (Four-Image-Three-Point Restoration in
2D-SIM)]
[0207] In this Section, "four-image-three-point restoration" in
2D-SIM will be described as a demodulating calculation in the
2D-SIM of the present embodiment.
[0208] In this Section, the number of directions of the wave number
vectors is set to three (modulated images are acquired under each
of three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3, and spectra of the respective modulated images are
generated).
[0209] Further, the three wave number vectors .xi..sub.1,
.xi..sub.2, and .xi..sub.3 are set to have a closed relationship
(.xi..sub.3=.xi..sub.1-.xi..sub.2).
[0210] Further, in any one direction (wave number vector
.xi..sub.1) out of the three directions, the phase number is set to
two (two pieces of modulated images I.sup.(0) and I.sup.(1) in
which directions of fringes are the same and phases are different
are acquired), and in each of the other two directions (wave number
vectors .xi..sub.2 and .xi..sub.3), the phase number is suppressed
to one (two pieces of modulated images I.sup.(2) and I.sup.(3) in
which directions of fringes are different are acquired).
[0211] Further, a phase contrast .DELTA..PHI. between the two
pieces of modulated images in which the directions of fringes are
the same is set to .DELTA..PHI.=.pi..
[0212] Further, in this Section, a magnitude of each wave number
vector is set to |.xi..sub.i|=2NA (i=1, 2, 3).
[0213] At this time, an interference fringe intensity distribution
of each of the four pieces of modulated images I.sup.(0),
I.sup.(1), I.sup.(2), and I.sup.(3) is represented as follows.
K ( 0 ) = 1 + cos .function. ( 2 .times. .pi. .lamda. .times. .xi.
1 x + .PHI. 0 ) ( 1.29 ) K ( 1 ) = 1 + cos .function. ( 2 .times.
.pi. .lamda. .times. .xi. 1 x + .PHI. 1 ) ( 1.30 ) K ( 2 ) = 1 +
cos .function. ( 2 .times. .pi. .lamda. .times. .xi. 2 x + .PHI. 2
) ( 1.31 ) K ( 3 ) = 1 + cos .function. ( 2 .times. .pi. .lamda.
.times. ( .xi. 1 - .xi. 2 ) x + .PHI. 3 ) ( 1.32 ) ##EQU00010##
[0214] Here, in each of spectra of the four pieces of modulated
images I.sup.(0), I.sup.(1), I.sup.(2), and I.sup.(3), a triangle
drawn by the three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3 is imagined, and an attention is focused on three
observation points .xi., (.xi.+.xi..sub.1), and (.xi.+.xi..sub.2)
positioned at vertices of the triangle (large black points).
[0215] From the three observation points .xi., (.xi.+.xi..sub.1),
and (.xi.+.xi..sub.2) in the spectrum of each of the four pieces of
modulated images I.sup.(0), I.sup.(1), I.sup.(2), and I.sup.(3)
acquired in this Section, twelve observation values, in total, are
obtained, so that twelve expressions each corresponding to the
expression 1.7 corresponding to each of the twelve observation
values can be obtained. In this Section, the following condition is
required to solve simultaneous equations made of the twelve
expressions.
cos(.PHI..sub.0-.PHI..sub.1).noteq.1 (1.40)
cos(.PHI..sub.0-.PHI..sub.2-.PHI..sub.3).noteq.cos(.PHI..sub.1-.PHI..sub-
.2-.PHI..sub.3) (1.41)
[0216] FIG. 19 are graphical representations of calculation. Note
that it is set that |.xi..sub.1|=|.xi..sub.2|, and
.xi..sub.1.xi..sub.2=|.xi..sub.1.parallel..xi..sub.2|/2.
[0217] Three large black points in FIG. 19A indicate three
observation points positioned at vertices of a triangle drawn by
the three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3 in a spectrum of modulated image, and the three large
black points and nine small black points in FIG. 19A indicate
restoration points (twelve restoration points, in total) restored
from the three observation points.
[0218] As described above, since the number of pieces of modulated
images (number of spectra) is four in this Section, twelve
observation values, in total, are obtained from the three
observation points. By creating simultaneous equations of the
twelve expressions regarding the twelve observation values and
solving the equations, restoration values of twelve restoration
points are individually determined.
[0219] Further, in this Section, by repeatedly calculating twelve
restoration values while moving three observation points,
restoration values of the entire painted-out area illustrated in
FIG. 19A are determined.
[0220] Note that FIG. 19B is a graphical representation when a
similar restoration is performed by inverting the direction of the
triangle. The restoration in FIG. 19A and the restoration in FIG.
19B can be conducted in a parallel manner. In this Section, the two
ways of restorations are conducted, to thereby restore the entire
painted-out area illustrated in FIG. 20.
[0221] FIG. 20 illustrates a result of synthesizing the restored
area illustrated in FIG. 19A and the restored area illustrated in
FIG. 19B.
[0222] Hereinafter, an example of calculation of solving the
simultaneous equations made of the twelve expressions described in
this Section will be described in detail.
[0223] FIG. 21 are diagrams explaining the calculation in this
Section in divided three steps.
[0224] FIG. 21A illustrates four restoration points 1 to 4 restored
in a first step.
[0225] FIG. 21B illustrates four restoration points 5 to 8 restored
in a second step.
[0226] FIG. 21C illustrates four restoration points 9 to 12
restored in a third step.
[0227] FIG. 21D illustrates a correspondence between numbers 1 to
12 in the drawings and restoration values.
[0228] First step: Four observation values regarding two
observation points 1 and 2 lined in a direction of the wave number
vector .xi..sub.1 at an interval of |.xi..sub.1|, are applied to
the expression of the two-image-two-point restoration, to thereby
determine restoration values of respective restoration points 1, 2,
3, and 4.
[0229] Second step: The restoration values of the respective
restoration points 1 and 2 are used to determine restoration values
of respective four restoration points 5, 6, 7, and 8, which are
displaced from the restoration points 1 and 2 by an amount of each
of the wave number vectors .xi..sub.2 and .xi..sub.3. Expressions
used at this time are four expressions, in total, including two
expressions (corresponding to two phases) regarding the direction
of the wave number vector .xi..sub.1, one expression regarding a
direction of the wave number vector .xi..sub.2, and one expression
regarding a direction of the wave number vector .xi..sub.3.
[0230] Third step: The restoration values of the respective
restoration points 1, 2, and 5 are used to determine restoration
values of the remaining respective restoration points 9, 10, 11,
and 12, which are displaced from the restoration points 1, 2, and 5
by an amount of each of the wave number vectors .xi..sub.2 and
.xi..sub.3. Expressions used at this time are four expressions, in
total, including two expressions (corresponding to two observation
points) regarding the direction of the wave number vector
.xi..sub.2, and two expressions (corresponding to two observation
points) regarding the direction of the wave number vector
.xi..sub.3.
[0231] The solving method of the simultaneous equations made of the
twelve expressions described in this Section is not limited to
follow the above-described procedure, as a matter of course.
[0232] [Section 1.7 (Modified Example of Four-Image-Three-Point
Restoration in 2D-SIM)]
[0233] This Section describes a modified example of the
four-image-three-point restoration.
[0234] In this Section, the phase number in all of the three
directions is suppressed to one, and instead of that, one piece of
non-modulated image is acquired, and a spectrum of the
non-modulated image is generated.
[0235] The non-modulated image corresponds to an image acquired
under a condition of K.sup.(0)=1, and can be acquired in a state
where, for example, the diffraction grating 16 and the beam
selecting section 18 described above are removed from the optical
path. Further, the spectrum of the non-modulated image is obtained
by performing Fourier transform on the non-modulated image.
[0236] As described above, in this Section, the number of pieces of
the modulated images (number of spectra of the modulated images) is
three, and the number of piece of the non-modulated image (number
of spectrum of the non-modulated image) is one, so that twelve
observation values, in total, are obtained from three observation
points. By creating simultaneous equations of the twelve
expressions regarding the twelve observation values (nine
expressions 1.7 regarding the spectra of the modulated images and
three expressions 1.53 regarding the spectrum of the non-modulated
image) and solving the equations, restoration values of twelve
restoration points are individually determined.
[0237] Note that in this Section, the following condition is
required.
cos(.PHI..sub.1-.PHI..sub.2-.PHI..sub.3).noteq.0 (1.52)
I.sup.(0)=OTF(.xi.)I.sub.0(.xi.) (1.53)
[0238] [Section 1.9
(Simultaneous-Three-Direction-Four-Image-Three-Point Restoration in
2D-SIM)]
[0239] In this Section,
"simultaneous-three-direction-four-image-three-point restoration"
will be described as a modified example of the
four-image-three-point restoration.
[0240] First, in this Section, interference fringes projected onto
a sample are set to be formed by summing up three interference
fringes with different directions (three-direction interference
fringes), as will be described below. Note that a projection method
of the three-direction interference fringes will be described
later.
K .function. ( x ) = 1 + 2 .times. a .times. i = 1 3 .times. cos
.function. ( 2 .times. .pi. .lamda. .times. .xi. i x - .PHI. i ) (
1.59 ) ##EQU00011##
[0241] Specifically, in this Section, three-direction interference
fringes having the three wave number vectors .xi..sub.1,
.xi..sub.2, and .xi..sub.3 at the same time are employed, the
three-direction interference fringes are used to acquire four
pieces of modulated images with mutually different phases, and
spectra of the respective four pieces of modulated images are
generated.
[0242] Note that it is set that |.xi..sub.1|.ltoreq.2NA (i=1, 2,
3), and the three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3 are set to have a closed relationship
(.xi..sub.3=.xi..sub.1-.xi..sub.2).
[0243] Further, a value of a which defines an amplitude of the
three-direction interference fringes is set to be selected to
satisfy the following expression.
K(x).gtoreq.0,.A-inverted.x
[0244] First, on a certain observation point in a spectrum of a
certain piece of modulated image acquired in this Section,
restoration values of seven restoration points, in total, including
a restoration value which should be given to a restoration point
.xi. in a spectrum of a demodulated image, and restoration values
which should be given to restoration points (.xi..+-..xi..sub.i)
(i=1, 2, 3) in the spectrum of the demodulated image, are
superimposed.
[0245] In other words, on the certain observation point .xi. in the
spectrum of the modulated image, seven components, in total, being
a 0th-order modulating component of fluorescence, .+-.first-order
modulating components of fluorescence based on the wave number
vector .xi..sub.1, .+-.first-order modulating components of
fluorescence based on the wave number vector .xi..sub.2, and
.+-.first-order modulating components of fluorescence based on the
wave number vector .xi..sub.3, are superimposed.
[0246] Here, in the spectrum of the modulated image, a triangle
drawn by the three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3 is imagined, and an attention is focused on three
observation points .xi., (.xi.+.xi..sub.1), and (.xi.+.xi..sub.2)
positioned at vertices of the triangle.
[0247] In the entire three observation points .xi.,
(.xi.+.xi..sub.1), and (.xi.+.xi..sub.2), restoration values of
twelve restoration points are included.
[0248] FIG. 22 are graphical representations thereof. Note that it
is set that |.xi..sub.1|=|.xi..sub.2|, and
.xi..sub.1.xi..sub.2=|.xi..sub.1.parallel..xi..sub.2|/2.
[0249] Three large black points in FIG. 22A indicate three
observation points positioned at vertices of a triangle drawn by
the three wave number vectors .xi..sub.1, .xi..sub.2, and
.xi..sub.3 in the spectrum of the modulated image, and the three
large black points and nine small black points in FIG. 22A indicate
restoration points (twelve restoration points, in total) restored
from the three observation points.
[0250] As described above, since the number of pieces of modulated
images (number of spectra) is four in this Section, twelve
observation values, in total, are obtained from the three
observation points. By creating simultaneous equations of the
twelve expressions regarding the twelve observation values and
solving the equations, restoration values of twelve restoration
points are individually determined.
[0251] Further, in this Section, by repeatedly calculating twelve
restoration values while moving three observation points,
restoration values of the entire painted-out area illustrated in
FIG. 22A are determined.
[0252] Note that FIG. 22B is a graphical representation when a
similar restoration is performed by inverting the direction of the
triangle. The restoration in FIG. 22A and the restoration in FIG.
22B can be conducted in a parallel manner. In this Section, the two
ways of restorations are conducted, to thereby restore an area same
as the painted-out area illustrated in FIG. 20.
[0253] Here, phases (formed of three components) of the
three-direction interference fringes reflected on the respective
four pieces of modulated images I.sup.(1), I.sup.(2), I.sup.(3),
and I.sup.(4) are represented as follows, for example.
(.PHI..sub.1.sup.(1),.PHI..sub.2.sup.(1),.PHI..sub.3.sup.(1)=(0,0,0)
(1.65)
(.PHI..sub.1.sup.(2),.PHI..sub.2.sup.(2),.PHI..sub.3.sup.(2)=(.pi.,0,.pi-
.) (1.66)
(.PHI..sub.1.sup.(3),.PHI..sub.2.sup.(3),.PHI..sub.3.sup.(3)=(0,.pi.,-.p-
i.) (1.67)
(.PHI..sub.1.sup.(4),.PHI..sub.2.sup.(4),.PHI..sub.3.sup.(4)=(.pi.,.pi.,-
0) (1.68)
[0254] Specifically, an interference fringe intensity distribution
of each of the four pieces of modulated images I.sup.(1),
I.sup.(2), I.sup.(3), and I.sup.(4) is represented as follows.
K 1 .function. ( x ) = 1 + 2 .times. a .function. ( cos .function.
( 2 .times. .pi. .lamda. .times. .xi. 1 x ) + cos .function. ( 2
.times. .pi. .lamda. .times. .xi. 2 x ) + cos .function. ( 2
.times. .pi. .lamda. .times. .xi. 3 x ) ) ( 1.69 ) K 2 .function. (
x ) = 1 + 2 .times. a .function. ( - cos .function. ( 2 .times.
.pi. .lamda. .times. .xi. 1 x ) + cos .function. ( 2 .times. .pi.
.lamda. .times. .xi. 2 x ) - cos .function. ( 2 .times. .pi.
.lamda. .times. .xi. 3 x ) ) ( 1.70 ) K 3 .function. ( x ) = 1 + 2
.times. a .function. ( cos .function. ( 2 .times. .pi. .lamda.
.times. .xi. 1 x ) - cos .function. ( 2 .times. .pi. .lamda.
.times. .xi. 2 x ) - cos .function. ( 2 .times. .pi. .lamda.
.times. .xi. 3 x ) ) ( 1.71 ) K 4 .function. ( x ) = 1 + 2 .times.
a .function. ( - cos .function. ( 2 .times. .pi. .lamda. .times.
.xi. 1 x ) - cos .function. ( 2 .times. .pi. .lamda. .times. .xi. 2
x ) + cos .function. ( 2 .times. .pi. .lamda. .times. .xi. 3 x ) )
( 1.72 ) ##EQU00012##
[0255] Incidentally, a sum of the interference fringe intensity
distributions of the four pieces of modulated images is represented
as follows.
i = 1 4 .times. K i .function. ( x ) = 1 ( 1.73 ) ##EQU00013##
[0256] Specifically, if four pieces of modulated images are
acquired under combinations of the interference intensity
distributions and the phases described above, respective parts of a
sample are illuminated by a mutually equal light intensity. Among
the four pieces of modulated images, there is a relationship such
that patterns of the three-direction interference fringes are
common, and only a position of the pattern is shifted. Accordingly,
the following relationship is satisfied.
K.sub.2(x)=K.sub.1(x+1/2a.sub.1) (1.74)
K.sub.3(x)=K.sub.1(x+1/2a.sub.2) (1.75)
K.sub.4(x)=K.sub.1(x+1/2a.sub.1+1/2a.sub.2) (1.76)
[0257] Note that a.sub.1 and a.sub.2 indicate fundamental vectors
of grating when a pitch structure of the interference fringes is
regarded as a crystal lattice, and when a reciprocal lattice vector
(wave number vector) k.sub.i is set such that
k.sub.1=(2.pi./.lamda.).xi..sub.1,
k.sub.2=(2.pi./.lamda.).xi..sub.2, and k.sub.3=e.sub.z (e.sub.z:
unit vector in z direction), a.sub.1 and a.sub.2 can be given by
the following expression.
a 1 = 2 .times. .pi. .times. k 2 .times. k 3 k 1 ( k 2 .times. k 3
) ( 1.77 ) a 2 = 2 .times. .pi. .times. k 3 .times. k 1 k 2 ( k 3
.times. k 1 ) ( 1.78 ) ##EQU00014##
[0258] FIG. 23 is a diagram illustrating a relationship of a
grating structure of the three-direction interference fringes and
the fundamental vectors a.sub.1 and a.sub.2 of the grating.
[0259] FIG. 24 is a diagram illustrating a relationship of
interference fringe intensity distribution among four pieces of
modulated images.
[0260] As illustrated in FIG. 24, among the four pieces of
modulated images, grating patterns move in parallel so as not to
overlap with one another. Further, a unit of movement amount of the
pattern is half the fundamental vector of the grating.
[0261] [Section 1.9.2 (Projection Method of Three-Direction
Interference Fringes)]
[0262] Here, the projection method of the three-direction
interference fringes will be described.
[0263] In order to make the three-direction interference fringes to
be generated in the above-described structured illuminating
microscopy apparatus 1, it is possible to use the aforementioned
diffraction grating 16 (FIG. 2A), similar to the case where other
interference fringes (one-direction interference fringes) are made
to be generated.
[0264] Note that an opening pattern of the beam selecting element
20 is set to cut, out of diffractive lights of three groups
generated at the diffraction grating 16, 0th-order diffractive
lights of the respective groups, high-order diffractive lights of
second-order or higher of the respective groups, and +first-order
diffractive lights of the respective groups, and to make only
-first-order diffractive lights of the respective groups transmit
therethrough. Accordingly, collecting points formed on a pupil
plane are only collecting points formed of the three -first-order
diffractive lights. FIG. 25A illustrates an arrangement of
collecting points when excess diffractive lights are not cut by the
beam selecting element 20, and FIG. 25B illustrates an arrangement
of collecting points when the excess diffractive lights are cut by
the beam selecting element 20. In this case, three collecting
points are formed at positions displaced from one another by
120.degree.. Three diffractive lights (three -first-order
diffractive lights, in this case) exited from the three collecting
points are incident, from three directions, on an illumination area
of a sample, to thereby form three-direction interference fringes
on the sample. Note that here, the diffractive lights which
contribute to the interference fringes are set to the three
-first-order diffractive lights, but, it is needless to say that
the diffractive lights can also be set to the three +first-order
diffractive lights.
[0265] Note that in this case, not the overlapped three ways of
two-beam interference fringes, but the three-beam interference
fringes are generated as the three-direction interference fringes,
resulting in that the super-resolution effect is lowered. Further,
since one of the .+-.first-order diffractive lights is cut, a
utilization efficiency of laser light is lowered.
[0266] Accordingly, it is also possible to design as follows.
Specifically, independent three laser light sources A, B, and C are
prepared, and each of a laser light exited from the laser light
source A, a laser light exited from the laser light source B, and a
laser light exited from the laser light source C is branched by a
two-direction branching fiber, to thereby form six point light
sources a, a', b, b', c, and c'. Note that the point light sources
a and a' are coherent light sources generated from the laser light
source A, the point light sources b and b' are coherent light
sources generated from the laser light source B, and the point
light sources c and c' are coherent light sources generated from
the laser light source C. Further, by appropriately laying the
fiber, the six point light sources a, a', b, b', c, and c' are
disposed on a pupil conjugate plane in a positional relationship
such as one illustrated in FIG. 26. Specifically, an arranging
direction of the point light sources a and a', an arranging
direction of the point light sources b and b', and an arranging
direction of the point light sources c and c' are set to directions
which are different from one another by 120.degree.. Six laser
lights exited from the six point light sources are incident on the
illumination area of the sample from six directions to form
three-direction interference fringes on the sample.
[0267] Here, laser lights La and La' exited from the point light
sources a and a', laser lights Lb and Lb' exited from the point
light sources b and b', and laser lights Lc and Lc' exited from the
point light sources c and c' do not interfere with one another.
Accordingly, the interference fringes formed on the sample are made
by overlapping three ways of two-beam interference fringes.
Therefore, the super-resolution effect is never lowered, and the
utilization efficiency of laser light is high.
[0268] Note that when the two-direction branching fiber is used,
instead of the diffraction grating 16, as a branching unit of light
as above, it is only required to change each of a phase contrast
between the laser lights La and La', a phase contrast between the
laser lights Lb and Lb', and a phase contrast between the laser
lights Lc and Lc', instead of translatory shifting the diffraction
grating 16 for changing the phases (formed of three components) of
the three-direction interference fringes.
[0269] [Section 2.1 (Prerequisite of 3D-SIM)]
[0270] In this Section, a prerequisite of a demodulating
calculation in the 3D-SIM will be described.
[0271] Here, an interference fringe intensity distribution in the
3D-SIM is assumed as follows.
[0272] If a wavelength of three-beam interference is set to
.lamda., an interference fringe intensity distribution K (r) in the
3D-SIM is represented as follows.
.times. K .function. ( r ) = j .times. a j .times. e .times. ? ? 2
.times. .times. ? .times. indicates text missing or illegible when
filed ( 2.1 ) ##EQU00015##
[0273] Note that it is set that k.sub.0=2.pi./.lamda., and j=-1, 0,
+1, and a vector k.sub.j is defined as follows.
k.sub.0=k.sub.0e.sub.2 (2.2)
k.sub.+=k.sub.0( {square root over
(1-.xi..sub.0.sup.2)}e.sub.z+.xi..sub.0) (2.3)
k.sub.-=k.sub.0( {square root over
(1-.xi..sub.0.sup.2)}e.sub.z+.xi..sub.0) (2.4)
[0274] Here, it is set that .xi..sub.0e.sub.z=0.
[0275] If, for simplification, it is assumed that a.sub.0=1,
a.sub.+=a=|a| e.sup.i.PHI., and a.sub.-=a*=|a| e.sup.-i.PHI., the
following expression is satisfied.
K(r)=|ae.sup.ik.sup.+.sup. r+e.sup.ik.sup.0.sup.
r+a*e.sup.ik.sup.-.sup. r|.sup.2
[0276] Accordingly, the interference fringe intensity distribution
K is represented as follows. The fringes are formed by overlapping
fringes of a first pitch having a sinusoidal intensity distribution
(interference fringes formed of, out of three light fluxes made of
a center light and lights on the right and left of the center
light, the right and left lights), and fringes of a second pitch
(which is double the first pitch) having a sinusoidal intensity
distribution (interference fringes formed of, out of three light
fluxes made of a center light and lights on the right and left of
the center light, the center light and the right light (or the left
right)).
K .function. ( x , z ) = 1 + 2 .times. a 2 + a * .times. e .times.
? + ae - ? + ae .times. ? + a * .times. e - ? + a * 2 .times. e - ?
+ a 2 .times. e .times. ? .times. .times. ? .times. indicates text
missing or illegible when filed ( 2.6 ) ##EQU00016##
[0277] Here, if it is set that .zeta..sub.0=
[1-.xi..sub.0.sup.2]-1, the following expression can be given.
K .function. ( x , z ) = 1 + 2 .times. a 2 + a * .function. ( e ?
.times. ? + a - ? ) .times. e .times. - ? .times. ? + a .function.
( e .times. ? ? + e - ? ) .times. e .times. ? ( 2.7 ) .times. + a *
2 .times. e - ? ? + a 2 .times. e .times. ? .times. .times. ?
.times. indicates text missing or illegible when filed ( 2.8 )
##EQU00017##
[0278] This is divided into a component which depends on z, and a
component which depends on x, and represented as follows.
.times. K .function. ( x , z ) = 2 .times. ? ? .times. K m
.function. ( z ) .times. J m .function. ( x ) .times. .times. ?
.times. indicates text missing or illegible when filed ( 2.9 )
##EQU00018##
[0279] Note that the following expressions are given.
K.sub.0(z)=1 (2.10)
K.sub..+-.1(z)=e.sup.ik.sup.0.sup..zeta..sup.0.sup.z+e.sup.-ik.sup.0.sup-
..zeta.z (2.11)
K.sub..+-.2(z)-1 (2.12)
J.sub.0=1+1|a|.sup.2 (2.13)
J.sub.1=ae.sup.ik.sup.0.sup..xi..sup.0.sup. x (2.14)
J.sub.2=a.sup.2e.sup.2ik.sup.0.sup..xi..sup.0.sup. x (2.15)
[0280] Note that it is set that J.sub.-1=J.sub.1*, and
J.sub.-2=J.sub.2*.
[0281] Incidentally, if a fluorescent material density of a sample
is set to I.sub.0 (x) and the interference fringes with the
interference fringe intensity distribution K described above are
projected onto the sample, a fluorescence intensity distribution of
the sample is assumed to be represented by I.sub.0 (r) K (r), and
an approximation (Born approximation) in which a fluorescence
generated at each point of the sample does not excite a fluorescent
material at another point, is employed.
[0282] At this time, a modulated image I (x, z) acquired in the
3D-SIM mode is represented as follows.
I(x,z)=(I.sub.0(x,z)K(x,z)PSF(x,z) (2.16)
[0283] Specifically, the following expression is given.
I .function. ( x , z ) = m .times. .intg. PSF .function. ( x - x '
, z - z ' ) .times. K m .function. ( z ) .times. J m .function. ( x
, z ) .times. I o .function. ( x , z ) .times. d 3 .times. x ( 2.17
) ##EQU00019##
[0284] Here, if a starting point in the z direction (optical axis
direction) of the interference fringes is set so that a z
coordinate (z') of an observation point always becomes a center,
the following expression is given.
I .function. ( x , z ) = m .times. .intg. PSF .function. ( x - x '
, z - z ' ) .times. K m .function. ( z - z ' ) .times. J m
.function. ( x , z ' ) .times. I o .function. ( x , z ' ) .times. d
3 .times. x ' ( 2.18 ) ##EQU00020##
[0285] If a three-dimensional OTF is set as represented by
OTF.sub.m(.xi.,.zeta.)=.sup.-1[PSF(x,z)K.sub.m(z)] (2.19)
OTF.sub..+-.2(.xi.,.zeta.)=OTF.sub.0(.xi.,.zeta.) (2.20)
OTF.sub..+-.2(.xi.,.zeta.)=OTF.sub.0(.xi.,.zeta.-.zeta..sub.0)+OTF.sub.0-
(.xi.,.zeta.+.zeta..sub.0) (2.21)
is given.
[0286] Further, a result of representing the modulated image on a
Fourier space (namely, a spatial frequency spectrum of the
modulated image) is represented as follows.
I ~ .function. ( .xi. , .zeta. ) = m .times. OTF m .function. (
.xi. , .zeta. ) .function. [ J ~ m .function. ( .xi. , .zeta. ) I ~
o .function. ( .xi. , .zeta. ) ] ( 2.22 ) ##EQU00021##
[0287] When this is written down, the following expression is
given.
I ~ .function. ( .xi. , .zeta. ) = OTF 0 .function. ( .xi. , .zeta.
) .times. I ~ o .function. ( .xi. , .zeta. ) + b * .times. OTF 1
.function. ( .xi. , .zeta. ) .times. I ~ o .function. ( .xi. - .xi.
0 , .zeta. ) + bOTF 1 .function. ( .xi. , .zeta. ) .times. I ~ o
.function. ( .xi. + .xi. 0 , .zeta. ) + c * .times. OTF 0
.function. ( .xi. , .zeta. ) .times. I ~ o .function. ( .xi. - 2
.times. .xi. 0 , .zeta. ) + cOTF 0 .function. ( .xi. , .zeta. )
.times. I ~ o .function. ( .xi. + 2 .times. .xi. 0 , .zeta. ) (
2.23 ) ##EQU00022##
[0288] Note that the following expression is given so that a
coefficient of a first term becomes one.
b = a .times. e i .times. .times. .PHI. 1 + 2 .times. a 2 ( 2.24 )
c = a 2 .times. e 2 .times. i .times. .times. .PHI. 1 + 2 .times. a
2 ( 2.25 ) ##EQU00023##
[0289] Note that a, b, and c are values determined based on an
intensity balance of the three light fluxes (.+-.first-order
diffractive lights and 0th-order diffractive light) which
contribute to the interference fringes in the 3D-SIM.
[0290] Hereinafter, the spatial frequency spectrum on the Fourier
space is simply referred to as "spectrum". Further, .PHI. appeared
in the expression is referred to as "phase", hereinafter.
[0291] [Section 2.2 (Conventional 3D-SIM)]
[0292] In this Section, a demodulating calculation of the
conventional 3D-SIM will be described for comparison.
[0293] First, on an observation point in a spectrum of a modulated
image acquired in the 3D-SIM mode, five components being a
-first-order modulating component of fluorescence, a +first-order
modulating component of fluorescence, a -second-order modulating
component of fluorescence, a +second-order modulating component of
fluorescence, and a 0th-order modulating component of fluorescence
are superimposed. The .+-.first-order modulating components
superimposed on the observation point are values (restoration
values) which should be possessed by restoration points
(.xi..+-..xi..sub.0) in a spectrum of a demodulated image, the
.+-.second-order modulating components superimposed on the
observation point .xi. are values (restoration values) which should
be possessed by restoration points (.xi..+-.2.xi..sub.0) in the
spectrum of the demodulated image, and the 0th-order modulating
component superimposed on the observation point .xi. is a value
(restoration value) which should be possessed by a restoration
point .xi. in the spectrum of the demodulated image.
[0294] Specifically, the .+-.first-order modulating components
superimposed on the observation point .xi. are components modulated
by fringes of the second pitch (which is double the first pitch)
having the sinusoidal intensity distribution (interference fringes
formed of, out of three light fluxes made of a center light and
lights on the right and left of the center light, the center light
and the right light (or the left right)), and the .+-.second-order
modulating components superimposed on the observation point .xi.
are components modulated by fringes of the first pitch having the
sinusoidal intensity distribution (interference fringes formed of,
out of three light fluxes made of a center light and lights on the
right and left of the center light, the right and left lights).
[0295] This applies to each observation point in the spectrum of
the modulated image. A large black point in FIG. 27 corresponds to
a certain observation point, and the large black point and four
small black points on both sides of the large black point
correspond to five restoration points restored from the observation
point.
[0296] Accordingly, in the demodulating calculation of the
conventional 3D-SIM, in order to make the five modulating
components superimposed on each observation point in the spectrum
of the modulated image to be separated from one another, five
pieces of modulated images with different phases have been acquired
to generate spectra of the respective modulated images.
Conventionally, by creating simultaneous equations of five
equations satisfied by the spectra and solving the simultaneous
equations, restoration values of a painted-out area (normal
resolution range and super-resolution range) in FIG. 27 have been
determined.
[0297] [Section 2.4 (Nine-Image-Two-Point Restoration in
3D-SIM)]
[0298] In this Section, "nine-image-two-point restoration" will be
described as a demodulating calculation of the 3D-SIM in the
present embodiment. It is set that the acquirement of modulated
images in this Section is performed when the aforementioned
controlling device 43 controls the respective parts, and the
calculation in this Section is executed by the aforementioned image
storing-calculating device 44 (which similarly applies to the other
Sections).
[0299] This Section focuses attention on the fact that, in a
spectrum of one piece of modulated image acquired by the 3D-SIM,
modulating components with mutually common values are superimposed
on two observation points .xi. and (.xi.+.xi..sub.0) which are
separated by a modulation frequency .xi..sub.0 in a modulating
direction.
[0300] Concretely, each of a -first-order modulating component of
fluorescence superimposed on the observation point .xi. and a
0th-order modulating component of fluorescence superimposed on the
observation point (.xi.+.xi..sub.0) corresponds to a restoration
value of a restoration point (.xi.+.xi..sub.0), each of a
-second-order modulating component of fluorescence superimposed on
the observation point .xi. and a -first-order modulating component
of fluorescence superimposed on the observation point
(.xi.+.xi..sub.0) corresponds to a restoration value of a
restoration point (.xi.+2.xi..sub.0), each of a +first-order
modulating component of fluorescence superimposed on the
observation point (.xi.+.xi..sub.0) and a 0th-order modulating
component of fluorescence superimposed on the observation point
corresponds to a restoration value of a restoration point .xi., and
each of a +second-order modulating component of fluorescence
superimposed on the observation point (.xi.+.xi..sub.0) and a
+first-order modulating component of fluorescence superimposed on
the observation point corresponds to a restoration value at
(.xi.-.xi..sub.0). Specifically, the two observation points .xi.
and (.xi.+.xi..sub.0) include the mutually common restoration
values of the four restoration points (.xi.-.xi..sub.0), .xi.,
(.xi.+.xi..sub.0), and (.xi.+2.xi..sub.0).
[0301] The nine-image-two-point restoration in this Section
utilizes this relationship. Concrete description will be made
hereinafter.
[0302] In a spectrum of one piece of modulated image, an
observation value at an observation point .xi. and an observation
value at an observation point (.xi.+.xi..sub.0) are represented by
the following expression.
I ~ .function. ( .xi. , .zeta. ) = OTF 0 .function. ( .xi. , .zeta.
) .times. I ~ o .function. ( .xi. , .zeta. ) + b * .times. OTF 1
.function. ( .xi. , .zeta. ) .times. I ~ o .function. ( .xi. - .xi.
0 , .zeta. ) + bOTF 1 .function. ( .xi. , .zeta. ) .times. I ~ o
.function. ( .xi. + .xi. 0 , .zeta. ) + c * .times. OTF 0
.function. ( .xi. , .zeta. ) .times. I ~ o .function. ( .xi. - 2
.times. .xi. 0 , .zeta. ) + cOTF 0 .function. ( .xi. , .zeta. )
.times. I ~ o .function. ( .xi. + 2 .times. .xi. 0 , .zeta. ) (
2.31 ) I ~ .function. ( .xi. + .xi. 0 , .zeta. ) = OTF 0 .function.
( .xi. + .xi. 0 , .zeta. ) .times. I ~ o .function. ( .xi. + .xi. 0
, .zeta. ) + b * .times. OTF 1 .function. ( .xi. + .xi. 0 , .zeta.
) .times. I ~ o .function. ( .xi. , .zeta. ) + bOTF 1 .function. (
.xi. + .xi. 0 , .zeta. ) .times. I ~ o .function. ( .xi. + 2
.times. .xi. 0 , .zeta. ) + c * .times. OTF 0 .function. ( .xi. +
.xi. 0 , .zeta. ) .times. I ~ o .function. ( .xi. - .xi. 0 , .zeta.
) + cOTF 0 .function. ( .xi. + .xi. 0 , .zeta. ) .times. I ~ o
.function. ( .xi. + 3 .times. .xi. 0 , .zeta. ) ( 2.32 )
##EQU00024##
[0303] Accordingly, in this Section, three pieces of modulated
images having the same wave number vector .xi..sub.0 and mutually
different phases .PHI. are acquired to generate spectra of the
respective three pieces of modulated images, six observation
values, in total, regarding the two observation points .xi. and
(.xi.+.xi..sub.0) are referred to from the respective three
spectra, and the six observation values are applied to these
expressions, thereby obtaining six expressions, in total, including
six restoration values (unknowns).
[0304] FIG. 28 are diagrams each illustrating a frequency range of
a demodulated image in the 3D-SIM in this Section. FIG. 28A
illustrates a xy cross section, and FIG. 28B illustrates a zx cross
section.
[0305] Large black points in FIG. 28 indicate two certain
observation points .xi. and (.xi.+.xi..sub.0) which are displaced
by .xi..sub.0, and the two large black points and four small black
points in FIG. 28 indicate restoration points (six restoration
points, in total) restored from the observation points .xi. and
(.xi.+.xi..sub.0).
[0306] The above-described explanation corresponds to an
explanation of restoration regarding a certain wave number vector
(one direction).
[0307] Accordingly, in this Section, three pieces of modulated
images with different phases are acquired under each of three wave
number vectors whose directions are mutually different, spectra of
the respective modulated images are generated, and restoration
processing similar to the above-described restoration processing is
performed on the spectra, for each direction. This makes it
possible to acquire a demodulated image with wide frequency
range.
[0308] Note that in this Section, it is desirable that a phase
contrast .DELTA..PHI. among the three pieces of modulated images
acquired under the same wave number vector is set to 2.pi./3.
[0309] [Section 2.5 (Seven-Image-Three-Point Restoration in
3D-SIM)]
[0310] In this Section, a seven-image-three-point restoration in
the 3D-SIM will be described. This Section applies Section 1.7 in
which the spectrum of the non-modulated image is utilized, to the
3D-SIM.
[0311] First, in this Section, the number of directions (number of
wave number vectors) is set to three.
[0312] Specifically, in this Section, the three wave number vectors
.xi..sub.1, .xi..sub.2, and .xi..sub.3 are set to have a closed
relationship (.xi..sub.3=.xi..sub.1-.xi..sub.2), and further, in
return for acquiring one piece of non-modulated image, the phase
number is reduced by one (the phase number is set to two) in each
of the three directions.
[0313] Further, in this Section, a phase contrast .DELTA..PHI.
between the two pieces of modulated images acquired under the same
wave number vector is set to .DELTA..PHI.=7.
[0314] Further, in this case, the number of each of the three wave
number vectors is set to k (k=1, 2, 3), and two pieces of modulated
images with mutually different phases acquired under the k-th wave
number vector are represented as follows.
I.sub..PHI..sub.k.sup.(k),I.sub..PHI..sub.k.sub.+.DELTA..PHI..sub.k.sup.-
(k)
[0315] Further, the non-modulated image is represented as
I.sup.(0).
[0316] From an expression 2.23, the following expression is
satisfied.
.sub..PHI..sub.k.sup.(k)(.xi.,.zeta.)=.tau..sub.0
.sub.0(.xi.,.zeta.)+.tau..sub.0'
.sub.1,.PHI..sub.k.sup.(k)(.xi.,.zeta.)+.tau..sub.0
.sub.2,.PHI..sub.k.sup.(k)(.xi.,.zeta.) (2.34)
[0317] Note that the respective .+-.first-order modulating
components and .+-.second-order modulating components are
collectively represented as follows.
.sub.1,.PHI..sub.k.sup.(k)(.xi.,.zeta.)=|b.sub.k|e.sup.-1.PHI..sup.k
.sub.0(.xi.-.xi..sub.k,.zeta.)+|b.sub.k|e.sup.i.PHI..sup.k
.sub.0(.xi.+.xi..sub.k,.zeta.) (2.35)
.sub.2,.PHI..sub.k.sup.(k)(.xi.,.zeta.)=|c.sub.k|e.sup.-2i.PHI..sup.k
.sub.0(.xi.-2.xi..sub.k,.zeta.)+|c.sub.k|e.sup.2i.PHI..sup.k
.sub.0(.xi.+2.xi..sub.k,.zeta.) (2.36)
[0318] Further, it is set that .tau..sub.0=OTF.sub.0(.xi., .zeta.),
and .tau..sub.0'=OTF.sub.1 (.xi., .zeta.).
[0319] Here, the phase contrast .DELTA..PHI. between the two pieces
of modulated images acquired under the same wave number vector is
set to .DELTA..PHI.=.pi., so that from an expression 2.35, the
following expression is satisfied.
.sub.1,.PHI..sub.k.sub.+.pi..sup.(k)(.xi.,.zeta.)=-
.sub.1,.PHI..sub.k.sup.(k)(.xi.,.zeta.) (2.37)
.sub.2,.PHI..sub.k.sub.+.pi..sup.(k)(.xi.,.zeta.)=
.sub.2,.PHI..sub.k.sup.(k)(.xi.,.zeta.) (2.38)
[0320] Accordingly, the following expression is obtained from an
expression 2.34.
.sub..PHI..sub.k.sup.(k)(.xi.,.zeta.)=.tau..sub.0
.sub.0(.xi.,.zeta.)+.tau..sub.0'
.sub.1.sup.(k)(.xi.,.zeta.)+.tau..sub.0 .sub.2.sup.(k)(.xi.,.zeta.)
(2.39)
.sub..PHI..sub.k.sub.+.pi..sup.(k)(.xi.,.zeta.)=.tau..sub.0
.sub.0(.xi.,.zeta.)-.tau..sub.0'
.sub.1.sup.(k)(.xi.,.zeta.)+.tau..sub.0 .sub.2.sup.(k)(.xi.,.zeta.)
(2.40)
[0321] Note that for simplification, a subscript .PHI.k is omitted
in the right side.
[0322] Further, the 0th-order modulating component (normal
resolution component) can be represented as follows.
.sup.(0)(.xi.,.zeta.)=.tau..sub.0 .sub.0(.xi.,.zeta.) (2.41)
[0323] The expression 2.39, the expression 2.40, and the expression
2.41 described above are represented as follows when being
collectively written by a matrix.
.times. [ I ~ ( 0 ) .function. ( .xi. , .zeta. ) I ~ .PHI. k ( k )
.function. ( .xi. , .zeta. ) I ~ .PHI. k ( k ) .times. ? .times. (
.xi. , .zeta. ) ] = [ T 0 0 0 T 0 T 0 ' T 0 T 0 - T 0 ' T 0 ]
.function. [ I ~ 0 .function. ( .xi. , .zeta. ) I ~ 1 ( k )
.function. ( .xi. , .zeta. ) I ~ 2 ( k ) .function. ( .xi. , .zeta.
) ] .times. .times. ? .times. indicates text missing or illegible
when filed ( 2.42 ) ##EQU00025##
[0324] Accordingly, in this Section, seven observation values
regarding observation points .xi. in spectra of respective seven
pieces of modulated images are first applied to an expression 2.42,
thereby determining each of seven restoration values in the right
side of the expression 2.42, namely, each of the following
restoration values (.+-.first-order modulating components,
.+-.second-order modulating components, and 0th-order modulating
component).
.sub.1.sup.(k), .sub.2.sup.(k)(k=1,2,3), .sub.0
[0325] A calculation to be performed hereinbelow is similar to the
calculation in Section 1.7. Specifically, in the spectrum
represented by
.sub.1.sup.(k)(k=1,2,3), .sub.0,
it is possible to separate, based on twelve observation values
regarding arbitrary three observation points which are mutually
displaced by an amount of each of the three wave number vectors k
(k=1, 2, 3), the +first-order modulating component and the
-first-order modulating component of fluorescence superimposed on
the three observation points.
[0326] Further, in the spectrum represented by
.sub.2.sup.(k)(k=1,2,3), .sub.0,
it is possible to separate, based on twelve observation values
regarding arbitrary three observation points which are mutually
displaced by a doubled amount of each of the three wave number
vectors 2.zeta..sub.k (k=1, 2, 3), the +second-order modulating
component and the - second-order modulating component of
fluorescence superimposed on the three observation points.
[0327] [Section 2.6 (Twelve-Image-Three-Point Restoration in
3D-SIM)]
[0328] In this Section, for the purpose of maintaining the
calculation accuracy, rather than reducing the number of pieces of
images (number of spectra), the number of directions (number of
wave number vectors) is set to three, and the phase number in each
direction is set to four, to thereby acquire twelve pieces of
modulated images, in total (twelve spectra, in total, are
generated).
[0329] Further, in this Section, a phase contrast .DELTA..PHI.
among the four pieces of modulated images acquired under the same
wave number vector is set to .DELTA..PHI.=.pi./2.
[0330] Further, in this case, the direction number is set to k
(k=1, 2, 3), and the phase number is set to I (I=0, 1, 2, 3).
[0331] First, from the expression 2.35, the .+-.first-order
modulating components are represented by the following
expression.
.times. I ~ 1 , .PHI. k ( k ) .function. ( .xi. , .zeta. ) = b k
.times. e - i .times. .times. .PHI. k .times. I ~ o .function. (
.xi. - .xi. k , .zeta. ) + b k .times. e i .times. .times. .PHI. k
.times. I ~ o .function. ( .xi. + .xi. k , .zeta. ) ( 2.47 ) I ~ 1
, .PHI. k + ( k ) .function. ( .xi. , .zeta. ) = - i .times. b k
.times. e - i .times. .times. .PHI. k .times. I ~ o .function. (
.xi. - .xi. k , .zeta. ) + i .times. b k .times. e i .times.
.times. .PHI. k .times. I ~ o .function. ( .xi. + .xi. k , .zeta. )
( 2.48 ) I ~ 1 , .PHI. k + ( k ) .function. ( .xi. , .zeta. ) = - b
k .times. e - i .times. .times. .PHI. k .times. I ~ o .function. (
.xi. - .xi. k , .zeta. ) - b k .times. e i .times. .times. .PHI. k
.times. I ~ o .function. ( .xi. + .xi. k , .zeta. ) ( 2.49 ) I ~ 1
, .PHI. k - .pi. 2 ( k ) .function. ( .xi. , .zeta. ) = i .times. b
k .times. e - i .times. .times. .PHI. k .times. I ~ o .function. (
.xi. - .xi. k , .zeta. ) - i .times. b k .times. e i .times.
.times. .PHI. k .times. I ~ o .function. ( .xi. + .xi. k , .zeta. )
( 2.50 ) ##EQU00026##
[0332] Further, from the expression 2.36, the +second-order
modulating components are represented by the following
expression.
.times. I ~ 2 , .PHI. k ( k ) .function. ( .xi. , .zeta. ) = c k -
2 .times. i .times. .times. .PHI. k .times. I ~ o .function. ( .xi.
- 2 .times. .xi. k , .zeta. ) + c k .times. e 2 .times. i .times.
.times. .PHI. k .times. I ~ o .function. ( .xi. + 2 .times. .xi. k
, .zeta. ) ( 2.51 ) I ~ 2 , .PHI. k + ( k ) .function. ( .xi. ,
.zeta. ) = - c k .times. e - 2 .times. i .times. .times. .PHI. k
.times. I ~ o .function. ( .xi. - 2 .times. .xi. k , .zeta. ) - c k
.times. e 2 .times. i .times. .times. .PHI. k .times. I ~ o
.function. ( .xi. + 2 .times. .xi. k , .zeta. ) ( 2.52 ) I ~ 2 ,
.PHI. k + ( k ) .function. ( .xi. , .zeta. ) = c k .times. e - 2
.times. i .times. .times. .PHI. k .times. I ~ o .function. ( .xi. -
2 .times. .xi. k , .zeta. ) + c k .times. e 2 .times. i .times.
.times. .PHI. k .times. I ~ o .function. ( .xi. + 2 .times. .xi. k
, .zeta. ) ( 2.53 ) I ~ 2 , .PHI. k - .pi. 2 ( k ) .function. (
.xi. , .zeta. ) = - c k .times. e - 2 .times. i .times. .times.
.PHI. k .times. I ~ o .function. ( .xi. - 2 .times. .xi. k , .zeta.
) - c k .times. e 2 .times. i .times. .times. .PHI. k .times. I ~ o
.function. ( .xi. + 2 .times. .xi. k , .zeta. ) ( 2.54 )
##EQU00027##
[0333] Further, the following expression is satisfied.
I ~ 2 , .PHI. k ( k ) .function. ( .xi. , .zeta. ) = I ~ 2 , .PHI.
k + .pi. ( k ) .function. ( .xi. , .zeta. ) = - I ~ 2 , .PHI. k +
.pi. 2 ( k ) .function. ( .xi. , .zeta. ) = - I ~ 2 , .PHI. k -
.pi. 2 ( k ) .function. ( .xi. , .zeta. ) ( 2.55 ) ##EQU00028##
[0334] Accordingly, when the demodulating calculation in this
Section is written by a matrix, the following expression is
given.
[ I ~ .PHI. k ( k ) .function. ( .xi. , .zeta. ) I ~ .PHI. k + .pi.
2 ( k ) .function. ( .xi. , .zeta. ) I ~ .PHI. k + .pi. ( k )
.function. ( .xi. , .zeta. ) I ~ .PHI. k - .pi. 2 ( k ) .function.
( .xi. , .zeta. ) ] = [ T 0 T 0 ' .times. b k * T 0 ' .times. b k T
0 T 0 - iT 0 ' .times. b k * iT 0 ' .times. b k - T 0 T 0 - T 0 '
.times. b k * - T 0 ' .times. b k T 0 T 0 iT 0 ' .times. b k * - iT
0 ' .times. b k - T 0 ] .function. [ I ~ 0 .function. ( .xi. ,
.zeta. ) I ~ o .function. ( .xi. - .xi. k , .zeta. ) I ~ o
.function. ( .xi. + .xi. k , .zeta. ) I ~ 2 ( k ) .function. ( .xi.
, .zeta. ) ] ( 2.56 ) ##EQU00029##
[0335] Therefore, in this Section, by applying observation values
of the spectra of the twelve pieces of modulated images (twelve
spectra) to this expression, the 0th-order modulating component and
the .+-.first-order modulating components are separated.
[0336] Further, the remaining .+-.second-order modulating
components are separated through a procedure similar to that in
Section 1.7.
[0337] As described above, in this Section, it is possible to
separate the .+-.first-order modulating components without
conducting the procedure in Section 2.5, so that a high sectioning
effect can be expected.
[0338] [Section 2.7 (Eight-Image-Three-Point Restoration in
3D-SIM)]
[0339] Note that the above-described Section 1.6 may also be
applied to the 3D-SIM in the following manner.
[0340] In this Section, the number of directions (number of wave
number vectors) is set to three.
[0341] Further, the three wave number vectors .xi..sub.1,
.xi..sub.2, and .xi..sub.3 are set to have a closed relationship
(.xi..sub.3=.xi..sub.1-.xi..sub.2).
[0342] Further, in any one direction (wave number vector
.xi..sub.1) out of the three directions, the phase number is set to
four, and the phase number in each of the other two directions is
suppressed to two.
[0343] Specifically, in this Section, in return for setting the
three wave number vectors .xi..sub.1, .xi..sub.2, and .xi..sub.3 to
have the closed relationship (.xi..sub.3=.xi..sub.1-.xi..sub.2),
the number of modulated images in each of the two directions is
reduced by one. Accordingly, the total number of modulated images
(total number of spectra) becomes eight.
[0344] Note that in this Section, a phase contrast among the
modulated images in the first direction (k=1) is set to
.DELTA..PHI.=.pi./2, a phase contrast between the modulated images
in the second direction (k=2) is set to .DELTA..PHI.=.pi., and a
phase contrast between the modulated images in the third direction
(k=3) is set to .DELTA..PHI.=.pi..
[0345] With the use of the spectra of the eight pieces of modulated
images acquired as above, it is possible to separate the respective
modulating components. Specifically, it is only required to
determine the 0th-order modulating component using the spectra of
the four pieces of modulated images regarding the first direction
(k=1) in the first step, and then to perform the separation of the
.+-.first-order modulating components, and the +second-order
modulating components in the second step.
[0346] The many features and advantages of the embodiments are
apparent from the detailed specification and, thus, it is intended
by the appended claims to cover all such features and advantages of
the embodiments that fall within the true spirit and scope thereof.
Further, since numerous modifications and changes will readily
occur to those skilled in the art, it is not desired to limit the
inventive embodiments to the exact construction and operation
illustrated and described, and accordingly all suitable
modifications and equivalents may be restored to, falling within
the scope thereof.
* * * * *