U.S. patent application number 14/398029 was filed with the patent office on 2018-03-15 for image capturing apparatus and focusing method thereof.
This patent application is currently assigned to HAMAMATSU PHOTONICS K.K.. The applicant listed for this patent is HAMAMATSU PHOTONICS K.K.. Invention is credited to Hideshi Oishi.
Application Number | 20180074307 14/398029 |
Document ID | / |
Family ID | 51209204 |
Filed Date | 2018-03-15 |
United States Patent
Application |
20180074307 |
Kind Code |
A9 |
Oishi; Hideshi |
March 15, 2018 |
IMAGE CAPTURING APPARATUS AND FOCUSING METHOD THEREOF
Abstract
In the image capturing apparatus, the optical path difference
producing member is disposed on the second optical path. Thereby,
at the second imaging device, it is possible to suppress the amount
of light on image pickup of an optical image which is focused at
the front of an optical image made incident into the first imaging
device (front focus) and on image pickup of an optical image which
is focused at the rear thereof (rear focus) and also to secure the
amount of light on image pickup by the first imaging device.
Further, in the image capturing apparatus, based on a scanning
velocity v of the stage and an interval d between the first imaging
region and the second imaging region, a waiting time is set from
image pickup at the first imaging region to image pickup at the
second imaging region. As a result, light from the same position of
the sample is made incident into the first imaging region and the
second imaging region. Thus, it is possible to control a focus
position at high accuracy.
Inventors: |
Oishi; Hideshi;
(Hamamatsu-shi, JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
HAMAMATSU PHOTONICS K.K. |
Hamamatsu-shi, Shizuoka |
|
JP |
|
|
Assignee: |
HAMAMATSU PHOTONICS K.K.
Hamamatsu-shi, Shizuoka
JP
|
Prior
Publication: |
|
Document Identifier |
Publication Date |
|
US 20150116475 A1 |
April 30, 2015 |
|
|
Family ID: |
51209204 |
Appl. No.: |
14/398029 |
Filed: |
January 17, 2013 |
PCT Filed: |
January 17, 2013 |
PCT NO: |
PCT/JP2013/050852 PCKC 00 |
371 Date: |
October 30, 2014 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 5/232123 20180801;
G02B 21/365 20130101; G02B 21/244 20130101; H04N 5/23212
20130101 |
International
Class: |
G02B 21/36 20060101
G02B021/36; H04N 5/232 20060101 H04N005/232 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 19, 2011 |
JP |
2011-277532 |
Claims
1-12. (canceled)
13. An apparatus for capturing an image of sample, the apparatus
comprising: a stage configured to support the sample; a stage
control unit configured to move the stage at a moving speed; an
objective lens configured to face to the sample; a light dividing
unit optically coupled to the objective lens and configured to
divide an optical image of at least a portion of the sample though
the objective lens into a first optical image and a second optical
image; a first imaging unit configured to capture the at least a
portion of the first optical image; a second imaging unit
configured to capture the at least a portion of the second optical
image and provide an image data; a focus control unit configured to
analyze the image data so as to control a focus position of the
objective lens based on the analysis result; a region control unit
configured to set at an imaging area of the second imaging unit a
first imaging region and a second imaging region for capturing the
at least the portion of the second optical image; and an optical
path difference producing member configured to give an optical path
difference to the second optical image, wherein the region control
unit sets waiting time from image pickup at the first imaging
region to image pickup at the second imaging region based on the
moving speed and an interval between the first imaging region and
the second imaging region.
14. The image capturing apparatus of claim 13, wherein the second
imaging unit is an area sensor.
15. The image capturing apparatus of claim 14, wherein the optical
path difference producing member is a flat plate member which is
disposed so as to overlap at least on a part of the imaging area,
and the region control unit sets the first imaging region and the
second imaging region respectively to give a region which will
overlap on the flat plate member and a region which will not
overlap on the flat plate member in order to avoid a shadow of the
second optical image by an edge part of the flat plate member.
16. The image capturing apparatus of claim 14, wherein the optical
path difference producing member is a member which has a part
undergoing a continuous change in thickness along an in-plane
direction of the imaging area, and the region control unit sets the
first imaging region and the second imaging region so as to overlap
on a part of the optical path difference producing member which is
different in thickness.
17. The image capturing apparatus of claim 13, wherein each of the
first imaging region and the second imaging region is constituted
with a separate line sensor.
18. The image capturing apparatus of claim 13, further comprising:
an objective lens control unit configured to control a position of
the objective lens relatively with respect to the sample based on
control by the focus control unit, wherein the objective lens
control unit will not actuate the objective lens during analysis of
the focus position which is being performed by the focus control
unit and will allow the objective lens to move with respect to the
sample in one direction during analysis of the focus position which
is not being performed by the focus control unit.
19. A method of capturing an image of a sample, the method
comprising: by an objective lens, acquiring a optical image of at
least a portion of a sample supported on a stage; moving the stage
at a moving speed; dividing the optical image into a first optical
image and a second optical image; capturing at least a portion of
the first optical image; capturing at least a portion of the second
optical image at least twice and providing image data; and
analyzing the image data so as to control a focus position of the
objective lens based on the analysis result in an analysis period,
wherein, waiting time for the twice capturing is set.
20. The method of claim 19, wherein the waiting time is set based
on at least the moving speed.
21. The method of claim 19, further comprising: setting a first
imaging region and a second imaging region for capturing the at
least the portion of the second optical image.
22. The method of claim 21, wherein the waiting time is set based
on at least a distance between the first imaging region and the
second imaging region.
23. The method of claim 19, wherein an area sensor is used for
capturing the at least the portion of the second optical image.
24. The method of claim 19, wherein a line sensor is used for
capturing the at least the portion of the second optical image.
25. The method of claim 19, further comprising: giving an optical
path difference to the at least the portion of the second optical
image.
26. The method of claim 19, further comprising: controlling the
focus point of the objective lens in a control period, wherein the
analysis period and the control period happen alternately.
27. An apparatus for capturing an image of a sample, the apparatus
comprising: a stage configured to support the sample; a stage
control unit configured to move the stage at a moving speed; an
objective lens configured to face to the sample; a light dividing
unit optically coupled to the objective lens and configured to
divide an optical image of at least a portion of the sample though
the objective lens into a first optical image and a second optical
image; a first imaging unit configured to capture at least a
portion of the first optical image; a second imaging unit
configured to capture at least a portion of the second optical
image at least twice and provide image data; a focus control unit
configured to analyze the image data so as to control a focus
position of the objective lens based on the analysis result,
wherein waiting time is set for the twice capturing.
28. The apparatus of claim 27, wherein the waiting time is set
based on at least the moving speed.
29. The apparatus of according to claim 28, further comprising: a
region control unit configured to set a first imaging region and a
second imaging region for capturing the at least the portion of the
second optical image.
30. The apparatus of claim 29, wherein the waiting time is set
based on at least a distance between the first imaging region and
the second imaging region.
31. The apparatus of claim 27, wherein an area sensor is used for
capturing the at least the portion of the second optical image.
32. The apparatus of claim 27, wherein a line sensor is used for
capturing the at least the portion of the second optical image.
33. The apparatus of claim 27, further comprising: a prism-like
optical path difference producing member configured to give an
optical path difference to the at least the portion of the second
optical image.
34. The apparatus of claim 27, further comprising: an objective
lens control unit configured to control the focus point of the
objective lens in a control period, wherein the analysis period and
the control period are happen alternately.
35. A method of capturing an image of a sample, the method
comprising: by an objective lens, acquiring an optical image of at
least a portion of a sample supported on a stage; moving the stage
at a moving speed; dividing the optical image into a first optical
image and a second optical image; capturing at least a portion of
the first optical image; capturing at least a portion of the second
optical image given an optical path difference and providing image
data; analyzing the image data so as to control a focus position of
the objective lens based on the analysis result in an analysis
period; and controlling the focus position of the objective lens
based on the analysis result in a control period, wherein the
analysis period and the control period are happen alternately.
Description
TECHNICAL FIELD
[0001] The present invention relates to an image capturing
apparatus which is used for capturing images of a sample, etc., and
also relates to a focusing method thereof.
BACKGROUND ART
[0002] Image capturing apparatuses include a virtual microscope
apparatus in which, for example, an imaging region of a sample is
in advance divided into a plurality of regions to image the divided
regions at a high magnification and, thereafter, to synthesize the
regions. In capturing images by using the virtual microscope as
described above, conventionally, as conditions for picking up
images of a sample such as a biological sample, a focus map which
covers an entire region of the sample is set to capture images of
the sample, while focus control is performed based on the focus
map.
[0003] In preparation of the focus map, at first, an image
capturing apparatus equipped with a macro optical system is used to
capture an entire sample as a macro image. Next, the thus captured
macro image is used to set an image pickup range of the sample and
also the range is divided into a plurality of regions to set a
focus obtaining position for each of the divided regions. After the
focus obtaining position has been set, the sample is transferred to
the image capturing apparatus equipped with a micro optical system
to obtain a focus position at the thus set focus obtaining
position, thereby preparing a focus map with reference to the focus
position.
[0004] However, in preparation of the above-described focus maps,
there has been a problem that processing needs time. Further,
suppression of intervals and the number of focuses to be obtained
would reduce the time necessary for the processing. In this case,
however, there has been a problem of reduction in focus accuracy.
Therefore, development of dynamic focus for capturing images of a
sample at a high magnification, with a focus position being
obtained, is now underway. The dynamic focus is a method in which a
present direction of the focus position deviating from the height
of an objective lens is detected based on a difference in light
intensity or a difference in contrast between an optical image
which is focused at the front of an optical image made incident
into an imaging device for capturing an image (front focus) and an
optical image which is focused at the rear thereof (rear focus),
thereby allowing the objective lens to move in a direction at which
the deviation is cancelled to capture an image.
[0005] A microscope system disclosed, for example, in Patent
Document 1, is provided with a second imaging unit which images a
region at the front of a region imaged by a first imaging unit, an
automatic focusing control unit which adjusts a focusing position
of an objective lens at an imaging position of the first imaging
unit based on an image picked up by the second imaging unit, and a
timing control unit which synchronizes timing at which a divided
region moves from an imaging position of the second imaging unit to
the imaging position of the first imaging unit with timing at which
an image forming position of the divided region imaged by the
second imaging unit is positioned at an imaging area of the first
imaging unit depending on a distance between the divided regions
and a speed at which a sample moves. Further, in a microscope
apparatus disclosed, for example, in Patent Document 2 or Patent
Document 3, a glass member is used to make a difference in optical
path length inside a light guiding optical system for focus
control.
CITATION LIST
Patent Literature
[0006] [Patent Document 1] Japanese Patent Application Laid-Open
No. 2011-081211 [0007] [Patent Document 2] Japanese Patent
Publication No. WO2005/114287 [0008] [Patent Document 3] Japanese
Patent Publication No. WO2005/114293
SUMMARY OF INVENTION
Technical Problem
[0009] In the microscope system described in Patent Document 1, a
half mirror and a mirror are used to form an optical path
difference optical system, by which light different in optical path
length is made incident into each of two imaging regions of the
second imaging unit. In the conventional microscope system, for
example, a line sensor is used to constitute a first imaging unit
and a second imaging unit. In the line sensor, it is important to
secure an amount of light for capturing a clear image due to short
exposure time. However, in the conventional microscope system,
light is divided by the optical path difference optical system.
Thus, there is posed such a problem that it is difficult to secure
an amount of light. Further, with exposure time of the line sensor
taken into consideration, it is necessary to set timing for
capturing an image by the first imaging unit and timing by the
second imaging unit.
[0010] The present invention has been made in view of solving the
above problems, an object of which is to provide an image capturing
apparatus capable of securing an amount of light on image pickup
and also detecting a focus position of a sample at high accuracy as
well as to provide a focusing method thereof.
Solution to Problem
[0011] In order to solve the above problems, an image capturing
apparatus of the present invention is characterized by having a
stage on which a sample is placed, a stage control unit which scans
the stage at a predetermined speed, a light source which radiates
light to the sample, a light guiding optical system including a
light dividing unit which divides an optical image of the sample
into a first optical path for capturing an image and a second
optical path for focus control, a first imaging unit which captures
a first image by a first optical image divided into the first
optical path, a second imaging unit which captures a second image
by a second optical image divided into the second optical path, a
focus control unit which analyzes the second image to control a
focus position of the image pickup by the first imaging unit based
on the analysis result, a region control unit which sets at an
imaging area of the second imaging unit a first imaging region and
a second imaging region for capturing a partial image of the second
optical image, and an optical path difference producing member
which is disposed on the second optical path to give an optical
path difference to the second optical image along an in-plane
direction of the imaging area, in which the region control unit
sets waiting time from image pickup at the first imaging region to
image pickup at the second imaging region based on a scanning speed
of the stage and an interval between the first imaging region and
the second imaging region.
[0012] In the image capturing apparatus, the optical path
difference producing member is disposed on the second optical path.
Thereby, at the first imaging region and the second imaging region
of the second imaging unit, it is possible to image respectively an
optical image which is focused at the front of an optical image
made incident into the first imaging unit (front focus) and an
optical image which is focused at the rear thereof (rear focus).
The image capturing apparatus is able to make a difference in
optical path length without dividing light on the second optical
path for focus control. Therefore, an amount of light at the second
optical path necessary for obtaining information on a focus
position can be suppressed to secure an amount of light on image
pickup at the first imaging unit. Further, the image capturing
apparatus sets waiting time from image pickup at the first imaging
region to image pickup at the second imaging region based on a
scanning speed of the stage and an interval (distance) between the
first imaging region and the second imaging region. Therefore,
since light from the same position of the sample is made incident
into the first imaging region and the second imaging region, it is
possible to control a focus position at high accuracy.
[0013] Further, it is preferable that the second imaging unit is an
area sensor. In this case, it is possible to set the first imaging
region and the second imaging region favorably. The unit can also
be made simple in structure.
[0014] Still further, it is preferable that the optical path
difference producing member is a flat plate member which is
disposed so as to overlap at least on a part of the imaging area
and that the region control unit sets the first imaging region and
the second imaging region respectively to give a region which will
overlap on the flat plate member and a region which will not
overlap on the flat plate member in order to avoid a shadow of the
second optical image by an edge part of the flat plate member. In
this case, use of the flat plate member enables the optical path
difference producing member to be simple in configuration. Further,
the edge part of the flat plate member forms the shadow of the
second optical image at the imaging area of the second imaging
device. Therefore, the first imaging region and the second imaging
region are set so as to avoid the shadow, thus making it possible
to secure accurate control of the focus position.
[0015] It is also preferable that the optical path difference
producing member is a member having a part which undergoes a
continuous change in thickness along an in-plane direction of the
imaging area and that the region control unit sets the first
imaging region and the second imaging region so as to overlap on
the part of the optical path difference producing member which is
different in thickness. In this case, adjustment of a position of
the first imaging region and that of the second imaging region
makes it possible to adjust freely an interval between the front
focus and the rear focus. Thereby, it is possible to detect a focus
position of the sample at high accuracy.
[0016] It is preferable that each of the first imaging region and
the second imaging region is constituted with a separate line
sensor. In this case, it is possible to shorten the time necessary
for setting an imaging region of the first imaging region and that
of the second imaging region.
[0017] It is also preferable that there are provided an objective
lens which faces to a sample and an objective lens control unit
which controls a position of the objective lens relatively with
respect to the sample based on control by the focus control unit,
in which the objective lens control unit will not actuate the
objective lens during analysis of the focus position which is being
performed by the focus control unit and will allow the objective
lens to move with respect to the sample in one direction during
analysis of the focus position which is not being performed by the
focus control unit. Since no change in positional relationship will
take place between the objective lens and the sample during
analysis of the focus position, it is possible to secure analysis
accuracy of the focus position.
[0018] Further, the focusing method of the image capturing
apparatus in the present invention is a focusing method of an image
capturing apparatus which is characterized by having a stage on
which a sample is placed, a stage control unit which scans the
stage at a predetermined speed, a light source which radiates light
to the sample, a light guiding optical system including a light
dividing unit which divides an optical image of the sample into a
first optical path for capturing an image and a second optical path
for focus control, a first imaging unit which captures a first
image by a first optical image divided into the first optical path,
a second imaging unit which captures a second image by a second
optical image divided into the second optical path, and a focus
control unit which analyzes the second image to control a focus
position of the image pickup by the first imaging unit based on the
analysis result, and the focusing method of the image capturing
apparatus in which, at an imaging area of the second imaging unit,
there are set a first imaging region and a second imaging region
for capturing a partial image of the second optical image, an
optical path difference producing member which gives an optical
path difference to the second optical image along an in-plane
direction of the imaging area is disposed on the second optical
path, and based on a scanning speed of the stage and an interval
between the first imaging region and the second imaging region,
waiting time from image pickup at the first imaging region to image
pickup at the second imaging region is set by the region control
unit.
[0019] In the focusing method, the optical path difference
producing member is disposed on the second optical path, by which,
at the first imaging region and the second imaging region of the
second imaging unit, it is possible to image respectively an
optical image which is focused at the front of an optical image
made incident into the first imaging unit (front focus) and an
optical image which is focused at the rear thereof (rear focus). In
the focusing method, a difference in optical path length can be
made without dividing light on the second optical path for focus
control. Therefore, the amount of light at the second optical path
necessary for obtaining information on a focus position can be
suppressed to secure the amount of light on image pickup by the
first imaging unit. Further, in the focusing method, based on a
scanning speed of the stage and an interval (distance) between the
first imaging region and the second imaging region, a waiting time
is set from image pickup at the first imaging region to image
pickup at the second imaging region. Therefore, since light from
the same position of the sample is made incident into the first
imaging region and the second imaging region, it is possible to
control the focus position at high accuracy.
[0020] It is also preferable that an area sensor is used as the
second imaging unit. In this case, it is possible to set the first
imaging region and the second imaging region favorably. The unit
can also be made simple in structure.
[0021] It is also preferable that as the optical path difference
producing member, there is used a flat plate member which is
disposed so as to overlap at least on a part of the imaging area
and in order to avoid a shadow of the second optical image by an
edge part of the flat plate member, the first imaging region and
the second imaging region are set by the region control unit
respectively so as to give a region which will overlap on the flat
plate member and a region which will not overlap on the flat plate
member. In this case, use of the flat plate member enables the
optical path difference producing member to be made simple in
configuration. Further, the edge part of the flat plate member
forms the shadow of the second optical image at an imaging area of
the second imaging device. Therefore, the first imaging region and
the second imaging region are set so as to avoid the shadow, thus
making it possible to secure accurate control of the focus
position.
[0022] It is also preferable that as the optical path difference
producing member, there is used a member which has a part
undergoing a continuous change in thickness along an in-plane
direction of the imaging area and that the first imaging region and
the second imaging region are set by the region control unit so as
to overlap on the part of the optical path difference producing
member which is different in thickness. In this case, adjustment of
a position of the first imaging region and that of the second
imaging region makes it possible to freely adjust an interval
between the front focus and the rear focus. It is, thereby,
possible to detect a focus position of the sample at high
accuracy.
[0023] It is also preferable that each of the first imaging region
and the second imaging region is constituted with a separate line
sensor. In this case, it is possible to shorten the time necessary
for setting the first imaging region and the second imaging
region.
[0024] It is also preferable that the image capturing apparatus is
provided with an objective lens which faces to a sample and an
objective lens control unit which controls a position of the
objective lens relatively with respect to the sample based on
control by the focus control unit, in which the objective lens
control unit will not actuate the objective lens during analysis of
the focus position which is being performed by the focus control
unit and will allow the objective lens to move with respect to the
sample in one direction during analysis of the focus position which
is not being performed by the focus control unit. In this case,
since no change in positional relationship will take place between
the objective lens and the sample during analysis of the focus
position, it is possible to secure analysis accuracy of the focus
position.
Advantageous Effects of Invention
[0025] According to the present invention, it is possible to secure
an amount of light on image pickup and also to detect a focus
position of a sample at high accuracy.
BRIEF DESCRIPTION OF DRAWINGS
[0026] FIG. 1 is a drawing which shows one embodiment of a macro
image capturing device which constitutes an image capturing
apparatus of the present invention.
[0027] FIG. 2 is a drawing which shows one embodiment of a micro
image capturing device which constitutes the image capturing
apparatus of the present invention.
[0028] FIG. 3 is a drawing which shows a second imaging device.
[0029] FIG. 4 is a drawing which shows one example of a combination
of an optical path difference producing member with the second
imaging device.
[0030] FIG. 5 is a drawing which shows another example of a
combination of the optical path difference producing member with
the second imaging device.
[0031] FIG. 6 is a drawing which shows another example of a
combination of the optical path difference producing member with
the second imaging device.
[0032] FIG. 7 is a drawing which shows a further modified example
of the optical path difference producing member.
[0033] FIG. 8 is a block diagram which shows functional components
of the image capturing apparatus.
[0034] FIG. 9 is a drawing which shows an analysis result of a
contrast value where a distance to the surface of a sample is in
agreement with the focal length of an objective lens.
[0035] FIG. 10 is a drawing which shows an analysis result of a
contrast value where a distance to the surface of the sample is
longer than the focal length of the objective lens.
[0036] FIG. 11 is a drawing which shows an analysis result of a
contrast value where a distance to the surface of the sample is
shorter than the focal length of the objective lens.
[0037] FIG. 12 is a drawing which shows a relationship of the
distance between the objective lens and the surface of the sample
with respect to scanning time of a stage.
[0038] FIG. 13 is a drawing which shows control of a scanning
direction of the stage by a stage control portion.
[0039] FIG. 14 is a drawing which shows control of a scanning speed
of the stage by the stage control portion.
[0040] FIG. 15 is a flow chart which shows motions of the image
capturing apparatus.
DESCRIPTION OF EMBODIMENTS
[0041] Hereinafter, a description will be given in detail of
preferred embodiments of the image capturing apparatus and the
focusing method of the image capturing apparatus in the present
invention with reference to drawings.
[0042] FIG. 1 is a drawing which shows one embodiment of the macro
image capturing device which constitutes the image capturing
apparatus of the present invention. FIG. 2 is a drawing which shows
one embodiment of the micro image capturing device which
constitutes the image capturing apparatus of the present invention.
As shown in FIG. 1 and FIG. 2, an image capturing apparatus M is
constituted with a macro image capturing device M1 for capturing a
macro image of a sample S and a micro image capturing device M2 for
capturing a micro image of the sample S. The image capturing
apparatus M is an apparatus which sets, for example, a plurality of
line-shaped divided regions 40 with respect to the macro image
captured by the macro image capturing device M1 (refer to FIG. 13)
and produces a virtual micro image by capturing and synthesizing
each of the divided regions 40 by the micro image capturing device
M2 at a high magnification.
[0043] As shown in FIG. 1, the macro image capturing device M1 is
provided with a stage 1 which supports the sample S. The stage 1 is
an XY stage which is actuated in a horizontal direction by a motor
or an actuator such as a stepping motor (pulse motor) or a piezo
actuator, for example. The sample S which is observed by using the
image capturing apparatus M is, for example, a biological sample
such as cells and placed on the stage 1 in a state of being sealed
on a slide glass. The stage 1 is actuated inside the XY plane, by
which an imaging position with respect to the sample S is allowed
to move.
[0044] The stage 1 is able to move back and forth between the macro
image capturing device M1 and the micro image capturing device M2
and provided with functions to deliver the sample S between the
devices. It is acceptable that when a macro image is captured, an
entire image of the sample S is picked up at one time or the sample
S is divided into a plurality of regions to pick up each of the
images. It is also acceptable that the stage 1 is installed both on
the macro image capturing device M1 and on the micro image
capturing device M2.
[0045] A light source 2 which radiates light to the sample S and a
condensing lens 3 which concentrates light from the light source 2
at the sample S are disposed on a bottom of the stage 1. It is
acceptable that the light source 2 is disposed so as to radiate
light obliquely to the sample S. Further, a light guiding optical
system 4 which guides an optical image from the sample S and an
imaging device 5 which images the optical image of the sample S are
disposed on an upper face of the stage 1. The light guiding optical
system 4 is provided with an image forming lens 6 which forms the
optical image from the sample S at an imaging area of the imaging
device 5. Still further, the imaging device 5 is an area sensor
which is capable of capturing, for example, a two-dimensional
image. The imaging device 5 captures an entire image of the optical
image of the sample S made incident into the imaging area via the
light guiding optical system 4 and is housed at a virtual micro
image housing portion 39 to be described later.
[0046] As shown in FIG. 2, the micro image capturing device M2 is
provided on the bottom of the stage 1 with a light source 12 and a
condensing lens 13, as with the macro image capturing device M1.
Further, a light guiding optical system 14 which guides an optical
image from the sample S is disposed on the upper face of the stage
1. The optical system which radiates light from the light source 12
to the samples may include an excitation light radiating optical
system which radiates excitation light to the sample S and a
dark-field illuminating optical system which captures a dark-field
image of the sample S.
[0047] The light guiding optical system 4 is provided with an
objective lens 15 disposed so as to face to the sample S and a beam
splitter (light dividing unit) 16 disposed at a rear stage of the
objective lens 15. The objective lens 15 is provided with a motor
and an actuator such as a stepping motor (pulse motor) and a piezo
actuator for actuating the objective lens 15 in a Z direction
orthogonal to a face on which the stage 1 is placed. A position of
the objective lens 15 in the Z direction is changed by these
actuation units, thus making it possible to adjust a focus position
of image pickup when an image of the sample S is captured. It is
acceptable that the focus position is adjusted by changing a
position of the stage 1 in the Z direction or by changing positions
of both the objective lens 15 and the stage 1 in the Z
direction.
[0048] The beam splitter 16 is a portion which divides an optical
image of the sample S into a first optical path L1 for capturing an
image and a second optical path L2 for focus control. The beam
splitter 16 is disposed at an angle of approximately 45 degrees
with respect to an optical axis from the light source 12. In FIG.
2, an optical path passing through the beam splitter 16 is given as
the first optical path L1, while an optical path reflected at the
beam splitter 16 is given as the second optical path.
[0049] On the first optical path L1, there are disposed an image
forming lens 17 which forms the optical image of the sample S
(first optical image) which has passed through the beam splitter 16
and a first imaging device (first imaging unit) 18 in which an
imaging area is disposed at an image forming position of the image
forming lens 17. The first imaging device 18 is a device which is
capable of capturing a one-dimensional image (first image) by the
first optical image of the sample S, including, for example, a
two-dimension CCD sensor and a line sensor capable of realizing TDI
(time delay integration) actuation. Further, in a method which
captures images of the sample S sequentially, with the stage 1
controlled at a constant speed, the first imaging device 18 may be
a device which is capable of capturing a two-dimensional image such
as a CMOS sensor and a CCD sensor. First images picked up by the
first imaging device 18 are sequentially stored in a temporary
storage memory such as a lane buffer, thereafter, compressed and
output at an image producing portion 38 to be described later.
[0050] On the other hand, on the second optical path L2, there are
disposed a view-field adjusting lens 19 which contracts an optical
image of a sample reflected by the beam splitter 16 (second optical
image) and a second imaging device (second imaging unit) 20.
Further, at a front stage of the second imaging device 20, there is
disposed an optical path difference producing member 21 which gives
an optical path difference to the second optical image. It is
preferable that the view-field adjusting lens 19 is constituted in
such a manner that the second optical image is formed at the second
imaging device 20 in a dimension similar to that of the first
optical image.
[0051] The second imaging device 20 is a device which is capable of
capturing a two-dimensional image (second image) by the second
optical image of the sample S, including, for example, sensors such
as a CMOS (complementary metal oxide semiconductor) and a CCD
(charge coupled device). It is also acceptable that a line sensor
is used.
[0052] An imaging area 20a of the second imaging device 20 is
disposed so as to be substantially in alignment with an XZ plane
orthogonal to the second optical path L2. As shown in FIG. 3, a
first imaging region 22A and a second imaging region 22B which
capture a partial image of the second optical image are set on the
imaging area 20a. The first imaging region 22A and the second
imaging region 22B are set in a direction perpendicular to a
direction (scanning direction: Z direction) at which the second
optical image moves on the imaging area 20a in association with
scanning of the sample S. The first imaging region 22A and the
second imaging region 22B are set, with a predetermined interval
kept, and both of them capture a part of the second optical image
in a line shape. Thereby, an optical image at the same region as
that of the first optical image of the sample S captured by the
first imaging device 18 can be captured as the second optical image
at the first imaging region 22A and the second imaging region 22B.
It is acceptable that each of the first imaging region 22A and the
second imaging region 22B is set by using a separate line sensor.
In this case, each of the line sensors is controlled separately,
thus making it possible to shorten the time necessary for setting
the first imaging region 22A and the second imaging region 22B.
[0053] The optical path difference producing member 21 is a glass
member which gives an optical path difference to the second optical
image along an in-plane direction of the imaging area 20a. In an
example shown in FIG. 4, the optical path difference producing
member 21A is formed in the shape of a prism having a triangular
cross section and disposed in such a manner that an apex thereof is
substantially in alignment with a central part of the imaging area
20a in the Z direction. Therefore, the second optical image which
is made incident into the imaging area 20a is longest in optical
path at the central part of the imaging area 20a in the Z direction
and becomes shorter in optical path when moving toward both ends of
the imaging area 20a in the Z direction. Further, it is preferable
that the optical path difference producing member 21 is disposed in
such a manner that a face which faces to the second imaging device
20 is parallel with the imaging area (light receiving face) 20a of
the second imaging device. Thereby, it is possible to reduce
deflection of light by the face which faces to the second imaging
device 20 and also to secure the amount of light which is received
by the second imaging device 20.
[0054] Accordingly, the second imaging device 20 is able to capture
an optical image which is focused at the front of a first optical
image made incident into the first imaging device 18 (front focus)
and an optical image which is focused at the rear thereof (rear
focus) based on a position of the first imaging region 22A and that
of the second imaging region 22B. In the present embodiment, the
position of the first imaging region 22A and that of the second
imaging region 22B are set in such a manner that, for example, the
first imaging region 22A is given as the front focus and the second
imaging region 22B is given as the rear focus. A focus difference
between the front focus and the rear focus is dependent on a
difference between a thickness t1 and an index of refraction of the
optical path difference producing member 21A through which the
second optical image made incident into the first imaging region
22A passes and a thickness t2 and an index of refraction of the
optical path difference producing member 21A through which the
second optical image made incident into the second imaging region
22B passes.
[0055] It is noted that the optical path difference producing
member may include not only a member having a part which changes in
thickness along the in-plane direction of the imaging area 20a but
also, as shown in FIG. 5(a), include an optical path difference
producing member 21B formed with a flat-plate like glass member. In
this case, as shown in FIG. 5(b), a lower half region of the
imaging area 20a in the Z direction is to overlap on the optical
path difference producing member 21B, the first imaging region 22A
set at an upper half region of the imaging area 20a, and the second
imaging region 22B set at a lower half region of the imaging area
20a. It is, thereby, possible to make a focus difference between
the front focus and the rear focus, depending on the thickness and
the index of refraction of the optical path difference producing
member 21B.
[0056] Further, in this case, there is a fear that an edge part E
of the optical path difference producing member 21B may form a
shadow 23 of the second optical image at the imaging area 20a.
Thus, as shown in FIG. 5(b), it is preferable that an interval d
between the first imaging region 22A and the second imaging region
22B is made wider than the width of the shadow 23 and that the
first imaging region 22A and the second imaging region 22B are set
at a position so as to avoid the shadow 23.
[0057] Further, as shown in FIG. 6(a), it is also possible to use
an optical path difference producing member 21C prepared by
laminating a plurality of flat-plate like glass members which are
different in length in the Z direction. In this case as well, as
shown in FIG. 6(b), a lower half region of the imaging area 20a in
the Z direction is to overlap on the optical path difference
producing member 21C, the first imaging region 22A set at an upper
half region of the imaging area 20a, and the second imaging region
22B set at a lower half region of the imaging area 20a. It is,
thereby, possible to make a focus difference between the front
focus and the rear focus, depending on the thickness and the index
of refraction of the optical path difference producing member
21C.
[0058] In this case as well, there is a fear that an edge part E of
the optical path difference producing member 21C may form a shadow
23 of the second optical image at the imaging area 20a. Therefore,
as shown in FIG. 6(b), it is preferable that an interval d between
the first imaging region 22A and the second imaging region 22B is
made wider than the width of the shadow 23 and that the first
imaging region 22A and the second imaging region 22B are set at a
position so as to avoid the shadow 23.
[0059] Moreover, as shown in FIG. 7(a), it is acceptable that a
prism-like optical path difference producing member 21D having a
triangular cross section is disposed so as to increase in thickness
when moving in the Z direction. As shown in FIG. 7(b), it is
acceptable that a flat-plate like optical path difference producing
member 22E as with that given in FIG. 5 is disposed so as to be in
alignment with the center of the imaging area 20a in the Z
direction. In the case shown in FIG. 7(b), two edge parts E of the
optical path difference producing member 22E are projected on the
imaging area 20a. Therefore, it is preferable that the interval d
between the first imaging region 22A and the second imaging region
22B is made wider than the width of the shadow 23 and that the
first imaging region 22A and the second imaging region 22B are set
so as to avoid the two shadows 23.
[0060] FIG. 8 is a block diagram which shows functional components
of the image capturing apparatus. As shown in the diagram, the
image capturing apparatus M is provided with a computer system
having housing portions such as a CPU, a memory, a communication
interface and a hard disk, an operation portion 31 such as a
keyboard, a monitor 32 etc. The functional components of the
control portion 33 include a focus control portion 34, a region
control portion 35, an objective lens control portion 36, a stage
control portion 37, an image producing portion 38 and a virtual
micro image housing portion 39.
[0061] The focus control portion 34 is a portion which analyzes a
second image captured by the second imaging device 20 to control a
focus position of an image picked up by the first imaging device 18
based on the analysis result. More specifically, the focus control
portion 34 first determines a difference between a contrast value
of the image obtained at the first imaging region 22A and a
contrast value obtained at the second imaging region 22B in the
second imaging device 20.
[0062] Here, as shown in FIG. 9, where a focus position of the
objective lens 15 is in alignment with the surface of the sample S,
an image contrast value of the front focus obtained at the first
imaging region 22A is substantially in agreement with an image
contrast value of the rear focus obtained at the second imaging
region 22B. Thereby, a difference value between them is almost
zero.
[0063] On the other hand, as shown in FIG. 10, where a distance to
the surface of the sample S is longer than a focal length of the
objective lens 15, an image contrast value of the rear focus
obtained at the second imaging region 22B is greater than an image
contrast value of the front focus obtained at the first imaging
region 22A. Therefore, a difference value between them is a
positive value. In this case, the focus control portion 34 outputs
instruction information to the objective lens control portion 36 so
as to be actuated in a direction at which the objective lens 15 is
brought closer to the sample S.
[0064] Further, as shown in FIG. 11, where a distance to the
surface of the samples is shorter than a focal length of the
objective lens 15, an image contrast value of the rear focus
obtained at the second imaging region 22B is smaller than an image
contrast value of the front focus obtained at the first imaging
region 22A. Therefore, a difference value between them is a
negative value. In this case, the focus control portion 34 outputs
instruction information to the objective lens control portion 36 so
as to be actuated in a direction at which the objective lens 15 is
brought away from the sample S.
[0065] The region control portion 35 is a portion which controls a
position of the first imaging region 22A and a position of the
second imaging region 22B at the imaging area 20a of the second
imaging device 20. The region control portion 35 sets at first the
first imaging region 22A at a predetermined position based on
operation from the operation portion 31 and releases the setting of
the first imaging region 22A after image pickup at the first
imaging region 22A. Then, the region control portion 35 sets the
second imaging region 22B, with a predetermined interval kept in
the Z direction from the first imaging region 22A (scanning
direction), and releases the setting of the second imaging region
22B after image pickup at the second imaging region 22B.
[0066] At this time, waiting time W from image pickup at the first
imaging region 22A to image pickup at the second imaging region 22B
is set based on an interval d between the first imaging region 22A
and the second imaging region 22B and a scanning velocity v of the
stage 1. For example, where the waiting time W is given as time W1
from the start of image pickup at the first imaging region 22A to
the start of image pickup at the second imaging region 22B, it is
possible to determine the waiting time with reference to a formula
of W1=d/v-e1-st, with consideration given to exposure time e1 of
image pickup at the first imaging region 22A and time st from
release of the setting of the first imaging region 22A to the
setting of the second imaging region 22B.
[0067] Further, where the waiting time W is given as waiting time
W2 from the start of image pickup at the first imaging region 22A
to completion of image pickup at the second imaging region 22B, it
is possible to determine the waiting time with reference to a
formula of W2=d/v-st, with consideration given to time st from
release of the setting of the first imaging region 22A to setting
of the second imaging region 22B. Still further, an interval d
between the first imaging region 22A and the second imaging region
22B is set based on a difference in optical path length made by the
optical path difference producing member 21. However, the interval
d actually corresponds to a distance of the sample S on a slide.
Eventually, it is necessary to convert the interval d to the number
of pixels at the second imaging region 22B. Where a pixel size of
the second imaging device 20 is expressed in terms of AFpsz and
magnification is expressed in terms of AFmag, the number of pixels
dpix corresponding to the interval d can be determined with
reference to a formula of dpix=d/(AFpsz/AFmag).
[0068] Further, the region control portion 35 is able to change at
least one of a position of the first imaging region 22A and that of
the second imaging region 22B along an in-plane scanning direction
of the imaging area 20a (here, the Z direction) based on operation
from the operation portion 31. In this case, it is acceptable to
change only one of the position of the first imaging region 22A and
that of the second imaging region 22B or both of the position of
the first imaging region 22A and that of the second imaging region
22B. It is also acceptable to change both of the position of the
first imaging region 22A and that of the second imaging region 22B,
with the interval d between the first imaging region 22A and the
second imaging region 22B being kept.
[0069] The first imaging region 22A and the second imaging region
22B are changed in position, by which, for example, use of a
prism-like optical path difference producing member 21A as shown in
FIG. 4 makes it possible to change the thickness t1 of the optical
path difference producing member 21A through which the second
optical image made incident into the first imaging region 22A
passes and the thickness t2 of the optical path difference
producing member 21A through which the second optical image made
incident into the second imaging region 22B passes. Thereby, an
interval between the front focus and the rear focus is changed,
thus making it possible to adjust resolution on determination of a
difference in contrast value. Further, where there is used, for
example as shown in FIG. 6, the optical path difference producing
member 21C in which flat-plate like glass members are laminated,
the position of the second imaging region 22B is switched to a
position which is different in thickness of glass. It is, thereby,
possible to switch a focus difference between the front focus and
the rear focus in a step-wise manner.
[0070] The objective lens control portion 36 is a portion which
controls actuation of the objective lens 15. Upon receiving
instruction information output from the focus control portion 34,
the objective lens control portion 36 actuates the objective lens
15 in the Z direction in accordance with contents of the
instruction information. It is, thereby, possible to adjust a focus
position of the objective lens 15 with respect to the sample S.
[0071] The objective lens control portion 36 will not actuate the
objective lens 15 during analysis of the focus position which is
being performed by the focus control portion 34 and will actuate
the objective lens 15 only in one direction along the Z direction
until start of analysis of a next focus position. FIG. 12 is a
drawing which shows a relationship of the distance between the
objective lens and the surface of the sample with respect to
scanning time of the stage. As shown in the drawing, during
scanning of the sample S, there will take place alternately an
analysis period A of the focus position and an objective lens
actuation period B based on an analysis result thereof. As
described so far, no change in positional relationship takes place
between the objective lens 15 and the sample S during analysis of
the focus position, thus making it possible to secure analysis
accuracy of the focus position.
[0072] The stage control portion 37 is a portion which controls
actuation of the stage 1. More specifically, the stage control
portion 37 allows the stage 1 on which the sample S is placed to
scan at a predetermined speed based on operation from the operation
portion 31. By the scanning of the stage 1, an imaging field of the
sample S moves relatively and sequentially at the first imaging
device 18 and the second imaging device 20. It is acceptable that,
as shown in FIG. 13(a), a scanning direction of the stage 1 is
one-directional scanning in which upon every completion of scanning
of one divided region 40, a position of the stage 1 is returned to
a start position of scanning and then a next divided region 40 is
subjected to scanning in the same direction. It is also acceptable
that as shown in FIG. 13(b), the scanning direction is
bi-directional scanning in which after completion of scanning of
one divided region 40, the stage 1 is allowed to move in a
direction orthogonal to the scanning direction and a next divided
region 40 is subjected to scanning in an opposite direction.
[0073] Although the stage 1 is scanned at a constant speed while
images are captured, actually, immediately after the start of
scanning, there is a period during which the scanning speed is
unstable due to influences of vibrations of the stage 1 etc. Thus,
as shown in FIG. 14, it is preferable that there is set a scanning
width longer than the divided region 40 and an acceleration period
C for accelerating the stage 1, a stabilization period D for
stabilizing a scanning speed of the stage 1 and a slowing-down
period F for slowing down the stage 1 are allowed to take place
individually when scanning is performed outside the divided region
40. It is, thereby, possible to capture an image in synchronization
with a constant speed period E during which the stage 1 is scanned
at a constant speed. It is acceptable that image pickup is started
during the stabilization period D and a data part obtained during
the stabilization period D is deleted after the image has been
captured. The above-described method is desirable when used for an
imaging device which requires void reading of data.
[0074] The image producing portion 38 is a portion at which an
captured image is synthesized to produce a virtual micro image. The
image producing portion 38 receives sequentially first images
output from the first imaging device 18, that is, images of
individual divided regions 40, synthesizing these images to produce
an entire image of the sample S. Then, based on the thus
synthesized image, prepared is an image, the resolution of which is
lower than that of the synthesized image, and housed in a virtual
micro image housing portion 39 by associating a high resolution
image with a low resolution image. It is acceptable that an image
captured by the macro image capturing device M1 is also associated
at the virtual micro image housing portion 39. It is also
acceptable that the virtual micro image is housed as one image or
plurally divided images.
[0075] Next, a description will be given of motions of the
above-described image capturing apparatus M.
[0076] FIG. 15 is a flow chart which shows motions of the image
capturing apparatus M. As shown in the flow chart, at the image
capturing apparatus M, at first, a macro image of the sample S is
captured by the macro image capturing device M1 (Step S01). The
thus captured macro image is subjected to binarization by using,
for example, a predetermined threshold value and, thereafter,
displayed on a monitor 32. A scope for capturing micro images from
macro images is set by automatic setting based on a predetermined
program or manual setting by an operator (Step S02).
[0077] Then, the sample S is transferred to the micro image
capturing device M2 and focusing conditions are set (Step S03).
Here, as described above, based on a scanning velocity v of the
stage 1 and an interval d between the first imaging region 22A and
the second imaging region 22B, a waiting time W is set up to the
start of image pickup at the second imaging region 22B. It is more
preferable that consideration is given to exposure time e1 of image
pickup at the first imaging region 22A, time st from release of
setting of the first imaging region 22A to setting of the second
imaging region 22B etc.
[0078] After the focusing conditions have been set, scanning of the
stage 1 is started to capture a micro image for each of the divided
regions 40 of the sample S by the micro image capturing device M2
(Step S04). In capturing the micro image by the first imaging
device 18, at the second imaging device 20, a deviating direction
of the objective lens 15 with respect to the sample S is analyzed
based on a difference in contrast value between the front focus and
the rear focus by the first imaging region 22A and the second
imaging region 22B, thereby adjusting a position of the objective
lens 15 in real time. After micro images have been captured
completely for all the divided regions 40, the thus captured micro
images are synthesized to produce a virtual micro image (Step
S05).
[0079] As described so far, at the image capturing apparatus M, the
optical path difference producing members 21 (21A to 21E) are
disposed on the second optical path L2. Thereby, at the first
imaging region 22A and the second imaging region 22B of the second
imaging device 20, it is possible to image respectively an optical
image which is focused at the front of an optical image made
incident into the first imaging device 18 (front focus) and an
optical image which is focused at the rear thereof (rear focus). In
the image capturing apparatus M, a difference in optical path
length can be made without dividing light on the second optical
path L2 for focus control. Therefore, it is possible to suppress
the amount of light at the second optical path necessary for
obtaining information on a focus position and to sufficiently
secure the amount of light on image pickup at the first imaging
device 18. Further, in the image capturing apparatus M, based on a
scanning velocity v of the stage and an interval d between the
first imaging region 22A and the second imaging region 22B, a
waiting time W is set from image pickup at the first imaging region
22A to image pickup at the second imaging region 22B. As a result,
light from the same position of the sample S is made incident into
the first imaging region 22A and the second imaging region 22B.
Thus, it is possible to control a focus position of the objective
lens 15 at high accuracy.
[0080] Where, as the optical path difference producing member of
the present embodiment, there are used optical path difference
producing members 21 (21A, 21D) composed of a glass member which
has a part changing in thickness along an in-plane direction of the
imaging area 20a at the imaging device 20, the region control
portion 35 is used to adjust a position of the first imaging region
22A and a position of the second imaging region 22B. Thereby, it is
possible to freely adjust an interval between the front focus and
the rear focus. Accordingly, for example, where a plural number of
contrast peaks are found in an image picked up by the second
imaging device 20 or where a peak is flat in shape, a focus
difference between the front focus and the rear focus is adjusted,
thus making it possible to detect a focus position of the sample S
at high accuracy.
[0081] Further, where, as the optical path difference producing
member of the present embodiment, there are used optical path
difference producing members 21 (21B, 21C, 21E) composed of a
flat-plate like glass member, the optical path difference producing
member 21 can be made simple in structure. In this case, an edge
part E of the flat plate member forms a shadow 23 of the second
optical image at the imaging area 20a of the second imaging device
20. Therefore, the first imaging region 22A and the second imaging
region 22B are set so as to avoid the shadow 23, by which it is
possible to secure a focus position of the objective lens 15 at
high accuracy.
[0082] In the above-described embodiment, there is exemplified an
apparatus for producing a virtual micro image. The image capturing
apparatus of the present invention is, however, applicable to
various types of apparatuses, as long as the apparatuses are those
in which an image is captured by scanning a sample at a
predetermined speed by a stage etc.
REFERENCE SIGNS LIST
[0083] 1 . . . stage, 12 . . . light source, 14 . . . light guiding
optical system, 15 . . . objective lens, 16 . . . beam splitter
(light dividing unit), 18 . . . first imaging device (first imaging
unit), 20 . . . second imaging device (second imaging unit), 20a .
. . imaging area, 21 (21A to 31E) . . . optical path difference
producing member, 22A . . . first imaging region, 22B . . . second
imaging region, 34 . . . focus control portion (focus control
unit), 35 . . . region control portion (region control unit), 36 .
. . objective lens control portion (objective lens control unit), E
. . . edge part, L1 . . . first optical path, L2 . . . second
optical path, M . . . image capturing apparatus, M1 . . . macro
image capturing device, M2 . . . micro image capturing device, S .
. . sample.
* * * * *