U.S. patent application number 13/832120 was filed with the patent office on 2013-10-10 for optical arrangements for use with an array camera.
This patent application is currently assigned to Pelican Imaging Corporation. The applicant listed for this patent is Jacques Duparre, Dan Lelescu, Kartik Venkataraman. Invention is credited to Jacques Duparre, Dan Lelescu, Kartik Venkataraman.
Application Number | 20130265459 13/832120 |
Document ID | / |
Family ID | 49292006 |
Filed Date | 2013-10-10 |
United States Patent
Application |
20130265459 |
Kind Code |
A1 |
Duparre; Jacques ; et
al. |
October 10, 2013 |
OPTICAL ARRANGEMENTS FOR USE WITH AN ARRAY CAMERA
Abstract
A variety of optical arrangements and methods of modifying or
enhancing the optical characteristics and functionality of these
optical arrangements are provided. The optical arrangements being
specifically designed to operate with camera arrays that
incorporate an imaging device that is formed of a plurality of
imagers that each include a plurality of pixels. The plurality of
imagers include a first imager having a first imaging
characteristics and a second imager having a second imaging
characteristics. The images generated by the plurality of imagers
are processed to obtain an enhanced image compared to images
captured by the imagers. In many optical arrangements the MTF
characteristics of the optics allow for contrast at spatial
frequencies that are at least as great as the desired resolution of
the high resolution images synthesized by the array camera, and
significantly greater than the Nyquist frequency of the pixel pitch
of the pixels on the focal plane, which in some cases may be 1.5, 2
or 3 times the Nyquist frequency.
Inventors: |
Duparre; Jacques; (Jena,
DE) ; Lelescu; Dan; (Morgan Hill, CA) ;
Venkataraman; Kartik; (San Jose, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Duparre; Jacques
Lelescu; Dan
Venkataraman; Kartik |
Jena
Morgan Hill
San Jose |
CA
CA |
DE
US
US |
|
|
Assignee: |
Pelican Imaging Corporation
Mountain View
CA
|
Family ID: |
49292006 |
Appl. No.: |
13/832120 |
Filed: |
March 15, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13536520 |
Jun 28, 2012 |
|
|
|
13832120 |
|
|
|
|
61502158 |
Jun 28, 2011 |
|
|
|
Current U.S.
Class: |
348/218.1 |
Current CPC
Class: |
H04N 13/271 20180501;
G06T 7/50 20170101; H04N 5/2258 20130101; G02B 13/004 20130101;
H04N 5/232 20130101; G02B 3/0006 20130101; G02B 13/0035 20130101;
G02B 3/0068 20130101; H04N 5/2254 20130101; H04N 5/23238 20130101;
G02B 13/0045 20130101; H04N 5/23232 20130101; G02B 13/002
20130101 |
Class at
Publication: |
348/218.1 |
International
Class: |
H04N 5/232 20060101
H04N005/232 |
Claims
1. A camera array, comprising: A plurality of cameras, where each
camera includes a separate optics, and a plurality of light sensing
elements, and each camera is configured to independently capture an
image of a scene; wherein the optics of each camera are configured
so that each camera has a field of view that is shifted with
respect to the field-of-views of the other cameras so that each
shift includes a sub-pixel shifted view of the scene; wherein the
light sensing elements have a pixel pitch defining the Nyquist
frequency, and where the optics of each camera have a modular
transfer function (MTF) such that the optics optically resolve,
with sufficient contrast, spatial frequencies larger than the
Nyquist frequency (Ny); wherein the camera array is a monolithic
integrated module comprising a single semiconductor substrate on
which all of the sensor elements are formed, and optics including a
plurality of lens elements, where each lens element forms part of
the separate optics for one of the cameras; wherein each of the
cameras includes one of a plurality of different types of filer;
and wherein cameras having the same type of filter are uniformly
distributed about the geometric center of the camera array.
2. The camera array of claim 1, wherein the cut-off MTF of the
optics is at least 1.5 times the Ny.
3. The camera array of claim 1, wherein the cut-off MTF of the
optics is at least 2 times the Ny.
4. The camera array of claim 1, wherein the cut-off MTF of the
optics is at least 3 times the Ny.
5. The camera array of claim 1, wherein the optics of each camera
comprise a three-surface optical arrangement comprising: a first
lens element having a first convex proximal surface and a first
concave distal surface, wherein the diameter of the first convex
surface is larger than the diameter of the first concave surface; a
second lens element having a substantially flat second proximal
surface and a second convex distal surface, wherein the diameter of
the flat second proximal surface is smaller than the diameter of
the second convex surface, and wherein the diameter of the second
convex surface is intermediate between the diameters of the first
convex surface and the first concave surface; and wherein the first
and second lens elements are arranged sequentially in optical
alignment with an imager positioned at the distal end thereof.
6. The camera array of claim 1, wherein the optics of each camera
comprise a five-surface optical arrangement comprising: a first
lens element having a first convex proximal surface and a first
concave distal surface, wherein the diameter of the first convex
surface is larger than the diameter of the first concave surface; a
second lens element having a second concave proximal surface and a
second convex distal surface, wherein the diameter of the second
concave proximal surface is smaller than the diameter of the second
convex surface; a third lens element having a third concave
proximal surface and a third planar distal surface, wherein the
diameter of the third concave proximal surface is larger than the
diameters of any of the surfaces of the first and second lens
elements; and wherein the first, second and thirds lens elements
are arranged sequentially in optical alignment with an imager
positioned at the distal end thereof.
7. The camera array of claim 1, wherein the optics of each camera
comprise a substrate embedded hybrid lens optical arrangement
comprising: a substrate having proximal and distal sides; a first
monolithic lens element having first proximal and distal surfaces
disposed on the proximal side of said substrate; a second
monolithic lens element having second proximal and distal surfaces
disposed on the distal side of said substrate; at least one
aperture disposed on said substrate in optical alignment with said
first and second lens elements; and wherein the first and second
lens elements are arranged sequentially in optical alignment with
an imager positioned at the distal end thereof.
8. The camera array of claim 1, wherein the optics of each camera
comprise a monolithic lens optical arrangement comprising: at least
one lens element comprising: a first monolithic lens having first
proximal and distal surfaces, wherein the first proximal surface of
the first monolithic lens has one of either a concave or convex
profile, and wherein the first distal surface of the first
monolithic lens has a plano profile; at least one aperture disposed
on the first distal surface of the first monolithic lens and in
optical alignment therewith; a second monolithic lens having second
proximal and distal surfaces, wherein the second proximal surface
of the second monolithic lens has a plano profile, and wherein the
second distal surface of the second monolithic lens has one of
either a concave or convex profile, and wherein the second
monolithic lens is arranged in optical alignment with said
aperture; and wherein the first monolithic lens element is in
direct contact with the aperture and the second monolithic
lens.
9. The camera array of claim 1, wherein the optics of each camera
comprise a three-element monolithic lens optical arrangement
comprising: a first lens element having a first convex proximal
surface and a first plano distal surface; a second lens element
having a second concave proximal surface and a second convex distal
surface; a third menisci lens element having a third concave
proximal surface and a third convex distal surface; at least one
aperture disposed on the first plano distal surface; and wherein
the first, second and third lens elements are arranged sequentially
in optical alignment with the aperture stop and an imager.
10. A camera array, comprising: a plurality of cameras, where each
camera includes a separate optics, and a plurality of light sensing
elements each having a pixel pitch defining the Nyquist frequency,
and each camera is configured to independently capture a low
resolution image of a scene; a processor configured to synthesize a
higher resolution image from the plurality of lower resolution
images, the high resolution image has a characteristic MTF; wherein
the optics of each camera are configured so that each camera has a
field of view that is shifted with respect to the field-of-views of
the other cameras so that each shift includes a sub-pixel shifted
view of the scene; wherein the optics of each camera have a modular
transfer function (MTF) at least as large of the MTF of the high
resolution image; wherein the camera array is a monolithic
integrated module comprising a single semiconductor substrate on
which all of the sensor elements are formed, and optics including a
plurality of lens elements, where each lens element forms part of
the separate optics for one of the cameras; wherein each of the
cameras includes one of a plurality of different types of filer;
and wherein cameras having the same type of filter are uniformly
distributed about the geometric center of the camera array.
11. The camera array of claim 10, wherein the MTF of the optics is
at least 1.5 times the Ny.
12. The camera array of claim 10, wherein the MTF of the optics is
at least 2 times the Ny.
13. The camera array of claim 10, wherein the MTF of the optics is
at least 3 times the Ny.
14. The camera array of claim 10, wherein the optics of each camera
comprise a three-surface optical arrangement comprising: a first
lens element having a first convex proximal surface and a first
concave distal surface, wherein the diameter of the first convex
surface is larger than the diameter of the first concave surface; a
second lens element having a substantially flat second proximal
surface and a second convex distal surface, wherein the diameter of
the flat second proximal surface is smaller than the diameter of
the second convex surface, and wherein the diameter of the second
convex surface is intermediate between the diameters of the first
convex surface and the first concave surface; and wherein the first
and second lens elements are arranged sequentially in optical
alignment with an imager positioned at the distal end thereof.
15. The camera array of claim 10, wherein the optics of each camera
comprise a five-surface optical arrangement comprising: a first
lens element having a first convex proximal surface and a first
concave distal surface, wherein the diameter of the first convex
surface is larger than the diameter of the first concave surface; a
second lens element having a second concave proximal surface and a
second convex distal surface, wherein the diameter of the second
concave proximal surface is smaller than the diameter of the second
convex surface; a third lens element having a third concave
proximal surface and a third planar distal surface, wherein the
diameter of the third concave proximal surface is larger than the
diameters of any of the surfaces of the first and second lens
elements; and wherein the first, second and thirds lens elements
are arranged sequentially in optical alignment with an imager
positioned at the distal end thereof.
16. The camera array of claim 10, wherein the optics of each camera
comprise a substrate embedded hybrid lens optical arrangement
comprising: a substrate having proximal and distal sides; a first
monolithic lens element having first proximal and distal surfaces
disposed on the proximal side of said substrate; a second
monolithic lens element having second proximal and distal surfaces
disposed on the distal side of said substrate; at least one
aperture disposed on said substrate in optical alignment with said
first and second lens elements; and wherein the first and second
lens elements are arranged sequentially in optical alignment with
an imager positioned at the distal end thereof.
17. The camera array of claim 10, wherein the optics of each camera
comprise a monolithic lens optical arrangement comprising: at least
one lens element comprising: a first monolithic lens having first
proximal and distal surfaces, wherein the first proximal surface of
the first monolithic lens has one of either a concave or convex
profile, and wherein the first distal surface of the first
monolithic lens has a plano profile; at least one aperture disposed
on the first distal surface of the first monolithic lens and in
optical alignment therewith; a second monolithic lens having second
proximal and distal surfaces, wherein the second proximal surface
of the second monolithic lens has a plano profile, and wherein the
second distal surface of the second monolithic lens has one of
either a concave or convex profile, and wherein the second
monolithic lens is arranged in optical alignment with said
aperture; and wherein the first monolithic lens element is in
direct contact with the aperture and the second monolithic
lens.
18. The camera array of claim 10, wherein the optics of each camera
comprise a three-element monolithic lens optical arrangement
comprising: a first lens element having a first convex proximal
surface and a first plano distal surface; a second lens element
having a second concave proximal surface and a second convex distal
surface; a third menisci lens element having a third concave
proximal surface and a third convex distal surface; at least one
aperture disposed on the first plano distal surface; and wherein
the first, second and third lens elements are arranged sequentially
in optical alignment with the aperture stop and an imager.
Description
RELATED APPLICATION
[0001] This application is a continuation in part of U.S. patent
application Ser. No. 13/536,520, filed Jun. 28, 2012, which claims
priority to U.S. Provisional Application No. 61/502,158 filed Jun.
28, 2011. The disclosures of both these applications are
incorporated herein by reference.
FIELD OF THE INVENTION
[0002] The present invention is related to novel optical
arrangements, designs and elements for use in an array camera, and
more specifically to optical arrangements of varying configurations
having modulation transfer function (MTF) characteristics capable
of implementing super resolution for use with arrays of image
sensors.
BACKGROUND OF THE INVENTION
[0003] Image sensors are used in cameras and other imaging devices
to capture images. In a typical imaging device, light enters at one
end of the imaging device and is directed to an image sensor by an
optical element such as a lens. In most imaging devices, one or
more layers of optical elements are placed before and after the
aperture stop to focus light onto the image sensor. Recently array
cameras having many imagers and lenses have been developed. In most
cases, multiple copies of the optical elements must be formed
laterally for use in array cameras.
[0004] Conventionally, optical arrays can be formed by molding or
embossing from a master lens array, or fabricated by standard
lithographic or other means. However, the standard polymer-on-glass
WLO and monolithic lens WLO manufacturing techniques have so far
not been adapted for the specific high performance requirements of
array cameras. In particular, some technical limitations of
conventional WLO-processes need to be reduced, such as, for
example, minimum substrate thickness requirements, inflexibility of
where to place the aperture stop, accuracy, etc. The flexibility of
such choices or processes needs to be increased in order to meet
the high demands by array cameras otherwise such WLO techniques
cannot be used to manufacture array cameras. Accordingly, a need
exists for fabrication processes capable of accurately forming
these arrays and for optical arrangements that give an increased
flexibility in manufacturing so that the image processing software
of these new types of array-type cameras can take advantage to
deliver superior image quality at the system level.
[0005] The optical transfer function (OTF) of an imaging system
(camera, video system, microscope etc.) is considered the true
measure of an imaging system's performance, i.e., the resolution
(minimum feature size or maximum spatial frequency that can be
imaged with sufficient contrast) or image sharpness (the contrast
at a given spatial frequency) obtainable by an imaging system.
While optical resolution, as commonly used with reference to camera
systems, describes only the number of pixels in an image, and hence
the potential to show fine detail, the transfer function describes
the ability of adjacent pixels to change from black to white in
response to patterns of varying spatial frequency, and hence the
actual capability to show fine detail, whether with full or reduced
contrast. The optical transfer Function (OTF) consists of two
components: the modular transfer function (MTF), which is the
magnitude of the OTF, and the phase transfer function (PTF), which
is the phase component.
[0006] In cameras, the MTF is the most relevant measurement of
performance, and is generally taken as an objective measurement of
the ability of an optical system to transfer various levels of
detail from an object to an image. The MTF is measured in terms of
contrast (degrees of gray), or of modulation, produced from a
perfect source of that detail level (thus it is the ratio of
contrast between the object and the image). The amount of detail in
an image is given by the resolution of the optical system, and is
customarily specified in line pairs per millimeter (Ip/mm). A line
pair is one cycle of a light bar and dark bar of equal width and
has a contrast of unity. Contrast is defined as the ratio of the
difference in maximum intensity (I.sub.max) and minimum intensity
(I.sub.min) over the sum of I.sub.max and I.sub.min, where
I.sub.max is the maximum intensity produced by an image (white) and
I.sub.min is the minimum intensity (black). The MTF then is the
plot of contrast, measured in percent, against spatial frequency
measured in Ip/rm. This graph is customarily normalized to a value
of 1 at zero spatial frequency (all white or black).
SUMMARY
[0007] The current invention is directed to optical arrangements
for use with an array of cameras where the MTF of each of the
optical arrangements or stacks for each camera of the array of
cameras is at least as high as the desired MTF of the super
resolution image synthesized from the combined images of the of
cameras of the camera array.
[0008] In many embodiments, the camera array includes a plurality
of cameras, where each camera includes a separate optics, and a
plurality of light sensing elements, and each camera is configured
to independently capture an image of a scene;
[0009] wherein the optics of each camera are configured so that
each camera has a field of view that is shifted with respect to the
field-of-views of the other cameras so that each shift includes a
sub-pixel shifted view of the scene;
[0010] wherein the light sensing elements have a pixel pitch
defining a Nyquist frequency, and where the optics of each camera
have a modular transfer function (MTF) such that the optics
optically resolve, with sufficient contrast, spatial frequencies
larger than the Nyquist frequency (Ny);
[0011] wherein the camera array is a monolithic integrated module
comprising a single semiconductor substrate on which all of the
sensor elements are formed, and optics including a plurality of
lens elements, where each lens element forms part of the separate
optics for one of the cameras;
[0012] wherein each of the cameras includes one of a plurality of
different types of filer; and
[0013] wherein cameras having the same type of filter are uniformly
distributed about the geometric center of the camera array.
[0014] In other embodiments the camera array, includes:
[0015] a plurality of cameras, where each camera includes a
separate optics, and a plurality of light sensing elements each
having a pixel pitch defining a Nyquist frequency (Ny), and each
camera is configured to independently capture a low resolution
image of a scene;
[0016] a processor configured to synthesize a higher resolution
image from the plurality of lower resolution images, the high
resolution image has a characteristic MTF;
[0017] wherein the optics of each camera are configured so that
each camera has a field of view that is shifted with respect to the
field-of-views of the other cameras so that each shift includes a
sub-pixel shifted view of the scene;
[0018] wherein the optics of each camera have a modular transfer
function (MTF) at least as large of the desired MTF of the high
resolution image;
[0019] wherein the camera array is a monolithic integrated module
comprising a single semiconductor substrate on which all of the
sensor elements are formed, and
[0020] optics including a plurality of lens elements, where each
lens element forms part of the separate optics for one of the
cameras;
[0021] wherein each of the cameras includes one of a plurality of
different types of filer; and
[0022] wherein cameras having the same type of filter are uniformly
distributed about the geometric center of the camera array.
[0023] In still other embodiments, the cut-off MTF of the optics is
at least 1.5 times the Ny, at least 2 times the Ny, or at least 3
times the Ny.
[0024] In yet other embodiments the optics of each camera include a
three-surface optical arrangement includes: [0025] a first lens
element having a first convex proximal surface and a first concave
distal surface, wherein the diameter of the first convex surface is
larger than the diameter of the first concave surface; [0026] a
second lens element having a substantially flat second proximal
surface and a second convex distal surface, wherein the diameter of
the flat second proximal surface is smaller than the diameter of
the second convex surface, and wherein the diameter of the second
convex surface is intermediate between the diameters of the first
convex surface and the first concave surface; and [0027] wherein
the first and second lens elements are arranged sequentially in
optical alignment with an imager positioned at the distal end
thereof.
[0028] In still yet other embodiments the optics of each camera
include a five-surface optical arrangement including: [0029] a
first lens element having a first convex proximal surface and a
first concave distal surface, wherein the diameter of the first
convex surface is larger than the diameter of the first concave
surface; [0030] a second lens element having a second concave
proximal surface and a second convex distal surface, wherein the
diameter of the second concave proximal surface is smaller than the
diameter of the second convex surface; [0031] a third lens element
having a third concave proximal surface and a third planar distal
surface, wherein the diameter of the third concave proximal surface
is larger than the diameters of any of the surfaces of the first
and second lens elements; and [0032] wherein the first, second and
thirds lens elements are arranged sequentially in optical alignment
with an imager positioned at the distal end thereof.
[0033] In still yet other embodiments the optics of each camera
include a substrate embedded hybrid lens optical arrangement
including: [0034] a substrate having proximal and distal sides;
[0035] a first monolithic lens element having first proximal and
distal surfaces disposed on the proximal side of said substrate;
[0036] a second monolithic lens element having second proximal and
distal surfaces disposed on the distal side of said substrate;
[0037] at least one aperture disposed on said substrate in optical
alignment with said first and second lens elements; and [0038]
wherein the first and second lens elements are arranged
sequentially in optical alignment with an imager positioned at the
distal end thereof.
[0039] In still yet other embodiments the optics of each camera
include a monolithic lens optical arrangement including: [0040] at
least one lens element comprising: [0041] a first monolithic lens
having first proximal and distal surfaces, wherein the first
proximal surface of the first monolithic lens has one of either a
concave or convex profile, and wherein the first distal surface of
the first monolithic lens has a plano profile; [0042] at least one
aperture disposed on the first distal surface of the first
monolithic lens and in optical alignment therewith; [0043] a second
monolithic lens having second proximal and distal surfaces, wherein
the second proximal surface of the second monolithic lens has a
plano profile, and wherein the second distal surface of the second
monolithic lens has one of either a concave or convex profile, and
wherein the second monolithic lens is arranged in optical alignment
with said aperture; and [0044] wherein the first monolithic lens
element is in direct contact with the aperture and the second
monolithic lens.
[0045] In still yet other embodiments, the optics of each camera
include a three-element monolithic lens optical arrangement
including: [0046] a first lens element having a first convex
proximal surface and a first plano distal surface; [0047] a second
lens element having a second concave proximal surface and a second
convex distal surface; [0048] a third menisci lens element having a
third concave proximal surface and a third convex distal surface;
[0049] at least one aperture disposed on the first plano distal
surface; and [0050] wherein the first, second and third lens
elements are arranged sequentially in optical alignment with the
aperture stop and an imager.
[0051] In one embodiment, the invention is directed to a
three-surface optical arrangement for an array camera. In such an
embodiment, the optical arrangement includes: [0052] a first lens
element having a first convex proximal surface and a first concave
distal surface, where the diameter of the first convex surface is
larger than the diameter of the first concave surface, [0053] a
second lens element having a substantially flat second proximal
surface and a second convex distal surface, where the diameter of
the flat second proximal surface is smaller than the diameter of
the second convex surface, and where the diameter of the second
convex surface is intermediate between the diameters of the first
convex surface and the first concave surface; and [0054] wherein
the first and second lens elements are arranged sequentially in
optical alignment with an imager positioned at the distal end
thereof.
[0055] In one embodiment of the three-surface optical arrangement,
the surfaces of the first element are separated by a first
substrate, and the surfaces of the second element are separated by
a second substrate. In another such embodiment, the flat second
proximal surface is formed by the second substrate. In still
another such embodiment, an aperture stop is disposed on the flat
second proximal surface. In yet another such embodiment, at least
one aperture is disposed on at least one of the first or second
substrates. In still yet another such embodiment, an aperture
structure is disposed between said first and second lens elements,
comprising at least one aperture substrate having at least one
aperture disposed thereon. In still yet another such embodiment,
the first and second lens elements and the second lens element and
the imager are separated by spacers. In still yet another such
embodiment, a filter is disposed on at least one of the first or
second substrates. In still yet another such embodiment, at least
two of the surfaces of the lens elements are formed from materials
having different Abbe-numbers. In still yet another such
embodiment, the convex surfaces are formed from crown-like
materials, and the concave surfaces are formed from flint-like
materials.
[0056] In another embodiment of the three-surface optical
arrangement, an array of such arrangements are described, where the
array is designed to image a selected wavelength band, and where
the profile of at least one of the lens surfaces within each
optical arrangement is adapted to optimally image only a
narrow-band portion of the selected wavelength band such that in
combination the plurality of arrangements within the array image
the entirety of the selected wavelength band.
[0057] In another embodiment, the invention is directed to a
five-surface optical arrangement for an array camera. In such an
embodiment, the optical arrangement includes: [0058] a first lens
element having a first convex proximal surface and a first concave
distal surface, where the diameter of the first convex surface is
larger than the diameter of the first concave surface; [0059] a
second lens element having a second concave proximal surface and a
second convex distal surface, where the diameter of the second
concave proximal surface is smaller than the diameter of the second
convex surface; [0060] a third lens element having a third concave
proximal surface and a third planar distal surface, where the
diameter of the third concave proximal surface is larger than the
diameters of any of the surfaces of the first and second lens
elements; and [0061] where the first, second and thirds lens
elements are arranged sequentially in optical alignment with an
imager positioned at the distal end thereof.
[0062] In one such embodiment of the five-surface optical
arrangement the surfaces of the first element are separated by a
first substrate, and the surfaces of the second element are
separated by a second substrate. In another such embodiment, the
third planar distal surface is in contact with one of either the
image sensor or a cover glass disposed over the image sensor. In
still another such embodiment, an aperture stop is disposed on the
first concave distal surface. In yet another such embodiment, an
aperture stop is disposed on the first substrate adjacent to the
first concave distal surface. In yet another such embodiment, at
least one aperture is disposed within the first lens element. In
still yet another such embodiment, an aperture structure is
disposed between at least two of said lens elements, the aperture
structure comprising at least one aperture substrate having at
least one aperture disposed thereon. In still yet another such
embodiment, the first and second lens elements, and the second and
thirds lens elements are separated by spacers. In still yet another
such embodiment, a filter is disposed within at least one of the
first and second lens elements. In still yet another such
embodiment, at least two of the surfaces of the lens elements are
formed from materials having different Abbe-numbers. In still yet
another such embodiment, the convex surfaces are formed from
crown-like materials, and the concave surfaces are formed from
flint-like materials. In still yet another such embodiment, an
air-gap is positioned between the third lens element and the image
sensor. In still yet another such embodiment, at least one
substrate is disposed between the surfaces of at least one of the
lens elements. In still yet another such embodiment, a substrate is
disposed between the third lens element and the imager. In still
yet another such embodiment, at least one aperture is disposed on
at least one substrate within the lens elements. In still yet
another such embodiment, at least one aperture is embedded within
the first lens element.
[0063] In another embodiment of the five-surface optical
arrangement, a plurality of the five-surface optical arrangements
is provided in an array. In such embodiment, the array is designed
to image a selected wavelength band, and wherein the profile of at
least one of the lens surfaces within each optical arrangement is
adapted to optimally image only a narrow-band portion of the
selected wavelength such that in combination the plurality of
arrangements within the array image the entirety of the selected
wavelength band.
[0064] In another embodiment, the invention is directed to a
substrate embedded hybrid lens optical arrangement for an array
camera. In such an embodiment, the optical arrangement includes:
[0065] a substrate having proximal and distal sides; [0066] a first
monolithic lens element having first proximal and distal surfaces
disposed on the proximal side of the substrate; [0067] a second
monolithic lens element having second proximal and distal surfaces
disposed on the distal side of the substrate; [0068] at least one
aperture disposed on said substrate in optical alignment with the
first and second lens elements; and [0069] wherein the first and
second lens elements are arranged sequentially in optical alignment
with an imager positioned at the distal end thereof.
[0070] In another embodiment of the substrate embedded hybrid lens
optical arrangement at least two axially aligned apertures are
disposed on said substrate. In another such embodiment, the at
least two axially aligned apertures are one of either the same or
different sizes. In still another such embodiment, at least one
coating is disposed on said substrate in optical alignment with
said at least one aperture. In yet another such embodiment, the at
least one coating is selected from the group consisting of a
polarization filter, a color filter, an IRCF filter, and a NIR-pass
filter. In still yet another such embodiment, the substrate is
formed from a material that acts as a filter selected from the
group consisting of a polarization filter, a color filter, an IRCF
filter, and a NIR-pass filter. In still yet another such
embodiment, the substrate further comprises an adaptive optical
element. In still yet another such embodiment, at least two of the
lens elements are formed from materials having different
Abbe-numbers.
[0071] In still another embodiment of the substrate embedded hybrid
lens optical arrangement, such an arrangement is part of a wafer
stack comprising a plurality of the substrate embedded hybrid lens
optical arrangements including: [0072] a plurality of wafer
surfaces formed from the elements of the arrangements; and [0073]
at least two alignment marks formed in relation to each wafer
surface, each of said alignment marks being cooperative with an
alignment mark on an adjacent wafer surface such that said
alignment marks when cooperatively aligned aide in the lateral and
rotational alignment of the lens surfaces with the corresponding
apertures.
[0074] In yet another embodiment of the substrate embedded hybrid
lens optical arrangement, the arrangement is part of an array
comprising a plurality of the substrate embedded hybrid lens
optical arrangements, where the array is designed to image a
selected wavelength band, and wherein the profile of at least one
of the lens surfaces within each optical arrangement is adapted to
optimally image only a narrow-band portion of the selected
wavelength such that in combination the plurality of arrangements
within the array image the entirety of the selected wavelength
band.
[0075] In still another embodiment, the invention is directed to a
monolithic lens optical arrangement for an array camera. In such an
embodiment, the optical arrangement includes: [0076] at least one
lens element itself comprising: [0077] a first monolithic lens
having first proximal and distal surfaces, where the first proximal
surface of the first monolithic lens has one of either a concave or
convex profile, and where the first distal surface of the first
monolithic lens has a plano profile; [0078] at least one aperture
disposed on the first distal surface of the first monolithic lens
and in optical alignment therewith; [0079] a second monolithic lens
having second proximal and distal surfaces, where the second
proximal surface of the second monolithic lens has a plano profile,
and where the second distal surface of the second monolithic lens
has one of either a concave or convex profile, and where the second
monolithic lens is arranged in optical alignment with said
aperture; and [0080] where the first monolithic lens element is in
direct contact with the aperture and the second monolithic
lens.
[0081] In another embodiment the monolithic optical arrangement
includes at least one filter disposed on said plano surface in
optical alignment with said at least one aperture. In still another
such embodiment, the monolithic lenses are formed from materials
having different Abbe-numbers. In yet another such embodiment, at
least two lens elements are formed.
[0082] In still another embodiment the monolithic lens optical
arrangement is part of an array comprising a plurality of the
monolithic optical arrangements, where the array is designed to
image a selected wavelength band, and wherein the profile of at
least one of the lens surfaces within each optical arrangement is
adapted to optimally image only a narrow-band portion of the
selected wavelength such that in combination the plurality of
arrangements within the array image the entirety of the selected
wavelength band.
[0083] In yet another embodiment of the monolithic lens optical
arrangement, such an arrangement is part of a wafer stack
comprising: [0084] a plurality of wafer surfaces formed from the
elements of the arrangements; and [0085] at least two alignment
marks formed in relation to each wafer surface, each of the
alignment marks being cooperative with an alignment mark on an
adjacent wafer surface such that the alignment marks when
cooperatively aligned aide in the lateral and rotational alignment
of the lens surfaces with the corresponding apertures.
[0086] In yet another embodiment, the invention is directed to a
three-element monolithic lens optical arrangement for an array
camera. In such an embodiment, the optical arrangement includes:
[0087] a first lens element having a first convex proximal surface
and a first plano distal surface; [0088] a second lens element
having a second concave proximal surface and a second convex distal
surface; [0089] a third menisci lens element having a third concave
proximal surface and a third convex distal surface; [0090] at least
one aperture disposed on the first plano distal surface; and [0091]
wherein the first, second and third lens elements are arranged
sequentially in optical alignment with the aperture stop and an
imager.
[0092] In another embodiment the three-element monolithic optical
arrangement includes first and second lens elements that are formed
from low dispersion materials and the third lens element is formed
from a high dispersion material. In still another such embodiment,
at least one filter is disposed on the first plano distal surface
in optical alignment with the first lens element. In yet another
such embodiment, the first lens element further comprises a
substrate disposed on the distal surface thereof. In still yet
another such embodiment, at least one aperture is disposed on the
distal surface of said substrate. In still yet another such
embodiment, at least one filter is disposed on the distal surface
of said substrate. In still yet another such embodiment, the second
lens element further comprises a substrate disposed between the
proximal and distal surfaces thereof.
[0093] In still another embodiment the three-element monolithic
lens optical arrangement is part of an array comprising a plurality
of the three-element monolithic optical arrangements, where the
array is designed to image a selected wavelength band, and wherein
the profile of at least one of the lens surfaces within each
optical arrangement is adapted to optimally image only a
narrow-band portion of the selected wavelength such that in
combination the plurality of arrangements within the array image
the entirety of the selected wavelength band.
[0094] In still yet another embodiment, the invention is directed
to a plurality of optical arrangements for an array camera
including: [0095] a lens element array stack formed from a
plurality of lens element arrays, each of the lens element arrays
being in optical alignment with each other and a corresponding
imager; and [0096] where each of the individual lens elements of
each of the lens element stacks is formed from one of either a high
or low Abbe number material, and where the sequence in which one of
either a high or low Abbe number material is used in any individual
lens element stack depends upon the spectral band being detected by
the imager related thereto.
[0097] In still yet another embodiment, the invention is directed
to a plurality of optical arrangements for an array camera
including: [0098] a lens element array stack formed from a
plurality of lens element arrays, each of said lens element arrays
being formed of a plurality of lens elements; [0099] a plurality of
structural features integrated into each of the lens element
arrays; [0100] where the structural features ensure alignment of
the lens element arrays in relation to each other within the lens
element array stack in at least one dimension.
[0101] In one such embodiment of a plurality of optical
arrangements the structural features are selected from the group
consisting of lateral and rotational alignment features, spacers
and stand-offs.
[0102] In still yet another embodiment, the invention is directed
to a method of compensation for systematic fabrication errors in an
array having a plurality of optical channels comprising: [0103]
preparing a design incorporating a nominal shape of one of either a
waveplate or a multilevel diffractive phase element used for only
channel-wise color aberration correction of the optical channels of
the array; [0104] fabricating an array lens module based on the
design; [0105] experimentally determining the systematic deviation
of the lens module from the design based on at least one parameter
selected from the group consisting of lens metrologies, centering,
distance and optical performance; [0106] redesigning only the
channel-wise color aberration correcting surfaces of the lens
module based on the results of the experiment; [0107] refabricating
the lens module based on the redesign; and [0108] compensating for
any of the systematic deviations remaining using a back focal
length of the lens module.
[0109] In still yet another embodiment, the invention is directed
to an optical arrangement comprising a plurality of optical
channels, each optical channel including at least one optical
element, comprising at least two optical surfaces, wherein one of
the optical surfaces of each of the plurality of optical channels
is a channel specific surface having a wavefront deformation
sufficient solely to adapt the optical channel to a selected
waveband of light. In one such embodiment, the channel specific
surface is selected from the group consisting of waveplates,
kinoforms, and radial symmetric multilevel diffractive phase
elements.
[0110] The features and advantages described in the specification
are not all inclusive and, in particular, many additional features
and advantages will be apparent to one of ordinary skill in the art
in view of the drawings, specification, and claims. Moreover, it
should be noted that the language used in the specification has
been principally selected for readability and instructional
purposes, and may not have been selected to delineate or
circumscribe the inventive subject matter.
BRIEF DESCRIPTION OF DRAWINGS
[0111] These and other features and advantages of the present
invention will be better understood by reference to the following
detailed description when considered in conjunction with the
accompanying data and figures, wherein:
[0112] FIG. 1 is a plan view of a conventional camera array with a
plurality of imagers.
[0113] FIG. 2A is a perspective view of a camera module in
accordance with embodiments of the invention.
[0114] FIG. 2B is a cross-sectional view of a conventional module
in accordance with embodiments of the invention.
[0115] FIG. 3A illustrates a typical plot of the MTF of the optics
for a legacy camera cutting-off below the corresponding image
sensor's Ny spatial frequency.
[0116] FIG. 3B illustrates a plot of MTF of the optics for a camera
array having bandlimited optical channels at the Nyquist
frequency.
[0117] FIG. 3C illustrates a plot of MTF of the optics for a camera
array in accordance with embodiments of the invention.
[0118] FIG. 4A is a schematic of a three surface two-lens optical
arrangement according to one embodiment of the invention.
[0119] FIG. 4B is a table of exemplary lenses in accordance with
one embodiment of the optical arrangement of FIG. 4A.
[0120] FIG. 5A1 is a schematic of a five surface three-lens optical
arrangement according to one embodiment of the invention.
[0121] FIG. 5A2 is a table of exemplary lenses in accordance with
one embodiment of the optical arrangement of FIG. 5A1.
[0122] FIGS. 5B to 5H are data plots presenting characteristic
performance indicators of the optical arrangement of FIG. 5A1.
[0123] FIG. 5I1 is a schematic of a five surface three-lens optical
arrangement according to one embodiment of the invention.
[0124] FIG. 5I2 is a table of exemplary lenses in accordance with
one embodiment of the optical arrangement of FIG. 5I1.
[0125] FIG. 5J1 is a schematic of a five surface three-lens optical
arrangement according to one embodiment of the invention.
[0126] FIG. 5J2 is a table of exemplary lenses in accordance with
one embodiment of the optical arrangement of FIG. 5J1.
[0127] FIGS. 6A and 6B are schematics of conventional monolithic
lens and aperture arrangements.
[0128] FIG. 6C is a schematic of a monolithic optical arrangement
according to one embodiment of the invention.
[0129] FIGS. 6D1 to 6D6 are schematics of monolithic optical
arrangements according to various embodiments of the invention.
[0130] FIG. 7A is a schematic of a process flow for manufacturing a
monolithic optical arrangement according to one embodiment of the
invention.
[0131] FIGS. 7B to 7D are schematics of monolithic optical
arrangements according to various embodiments of the invention.
[0132] FIGS. 7E to 7G are schematics of optical arrangements which
incorporate monolithic lens elements according to the embodiments
of the invention shown in FIGS. 7B to 7D.
[0133] FIGS. 8A to 8D are schematics of three element monolithic
optical arrangements according to various embodiments of the
invention.
[0134] FIGS. 8E to 8J are data graphs of characteristic performance
indicators of the three-element monolithic optical arrangements
according to one embodiment of the invention.
[0135] FIGS. 9A and 9B are schematics of conventional injection
molded optical arrangement formed of two materials.
[0136] FIG. 9C is a schematic of an injection molded optical
arrangement formed of two materials according to one embodiment of
the invention.
[0137] FIG. 9D is a schematic of a conventional polymer on glass
wafer level optical arrangement formed of two materials.
[0138] FIG. 9E is a schematic of a polymer on glass wafer level
optical arrangement formed of two materials according to one
embodiment of the invention.
[0139] FIG. 10A is a schematic of a conventional polymer on glass
wafer level optical arrangement having an integrated aperture
stop.
[0140] FIG. 10B is a schematic of a polymer on glass wafer level
optical arrangement having an integrated aperture stop according to
one embodiment of the invention.
[0141] FIGS. 11A and 11B are schematics of optical arrangements
having preformed spacing and alignment elements according to one
embodiment of the invention.
[0142] FIG. 12 is a flowchart of a process for manufacturing an
optical arrangement according to one embodiment of the
invention.
DETAILED DESCRIPTION
[0143] Turning now to the drawings, novel optical arrangements for
use in an array camera that captures images using a distributed
approach using a plurality of imagers (cameras) of different
imaging characteristics are illustrated. In many embodiments, each
imager (camera) of such a camera array may be combined with
separate optics (lens stacks) with different filters and operate
with different operating parameters (e.g., exposure time). As will
be described, in some embodiments these distinct optical elements
may be fabricated using any suitable technique, including, for
example, injection molding, precision glass molding,
polymer-on-glass wafer level optics (WLO), or monolithic-lens WLO
technologies (polymer or glass). In other embodiments, the various
lens stacks of the individual cameras and camera array are
implemented such that the MTF characteristics of the optics include
contrast at a spatial frequency that is at least as large as the
resolution of the high resolution images to be synthesized by the
array camera from the low resolution images formed from the
individual cameras, and significantly greater than the Nyquist
frequency of the pixels in the focal plane.
[0144] Array cameras including camera modules that can be utilized
to capture image data from different viewpoints (i.e. light field
images) are disclosed in U.S. patent application Ser. No.
12/935,504 entitled "Capturing and Processing of Images using
Monolithic Camera Array with Heterogeneous Imagers" to Venkataraman
et al. In many instances, fusion and super-resolution processes
such as those described in U.S. patent application Ser. No.
12/967,807 entitled "Systems and Methods for Synthesizing High
Resolution Images Using Super-Resolution Processes" to Lelescu et
al., can be utilized to synthesize a higher resolution 2D image or
a stereo pair of higher resolution 2D images from the lower
resolution images in the light field captured by an array camera.
The terms high or higher resolution and low or lower resolution are
used here in a relative sense and not to indicate the specific
resolutions of the images captured by the array camera. The
disclosures of U.S. patent application Ser. No. 12/935,504 and U.S.
patent application Ser. No. 12/967,807 are hereby incorporated by
reference in their entirety.
[0145] Each two-dimensional (2D) image in a captured light field is
from the viewpoint of one of the cameras in the array camera. Due
to the different viewpoint of each of the cameras, parallax results
in variations in the position of foreground objects within the
images of the scene. Processes such as those disclosed in U.S.
Provisional Patent Application No. 61/691,666 entitled "Systems and
Methods for Parallax Detection and Correction in Imaged Captured
Using Array Cameras" to Venkataraman et al. can be utilized to
provide an accurate account of the pixel disparity as a result of
parallax between the different cameras in an array. The disclosure
of U.S. Patent Application Ser. No. 61/691,666 is hereby
incorporated by reference in its entirety. Array cameras can use
disparity between pixels in images within a light field to generate
a depth map from a reference viewpoint. A depth map indicates the
distance of the surfaces of scene objects from the reference
viewpoint and can be utilized to determine scene dependent
geometric corrections to apply to the pixels from each of the
images within a captured light field to eliminate disparity when
performing fusion and/or super-resolution processing.
[0146] The ultimate spatial resolution limit of a camera is
inversely proportional to the pixel size or pitch of the imaging
sensor of the camera and is defined as the Nyquist frequency limit.
The Nyquist frequency states that the maximum resolution, R, of a
system is equal to the inverse of two times the pixel pitch, x,
(R=/(2*x)). For a legacy camera very little contrast is desired at
spatial frequencies larger than the sensor's Ny, in order to avoid
aliasing in the final output image. Accordingly, in a legacy camera
the pixel pitch determines the spatial sampling rate, and the
corresponding Nyquist frequency (Ny), which is simply one half of
the reciprocal of the center-to-center pixel spacing. As such, in a
mobile imaging legacy camera, the required MTF for the optics
arrangement is usually specified at Ny/4, Ny/2, or Ny, as
illustrated in the plot of FIG. 3A, i.e., to be at least optically
limited by the pixel pitch.
[0147] The challenge in implementing optics for array cameras
results from the requirements necessary to achieve the
super-resolution processes described above. Of importance is that
super resolution should be able to recover a higher resolution
final output image than the intrinsic resolution in the input
component images from the individual cameras. Generally this
requires that the camera optically resolve, with sufficient
contrast, spatial frequencies that are actually larger than the
Nyquist frequency of the individual sensors. The spatial resolution
of a lens may be specified in terms of the modulation transfer
function (MTF) curve over a range of spatial frequencies. As
previously described, the MTF is a spatial frequency response (SFR)
of output signal contrast with input spatial frequency. Performance
is measured in terms of contrast or modulation at a particular
spatial frequency which is customarily specified in line pairs per
millimeter. At low line frequencies, the imaging system typically
passes the signal unattenuated, which implies a contrast of 100%.
At higher line frequencies, the signal is attenuated and the degree
of attenuation in the output signal is expressed as a percentage
with respect to that of the input signal, normalized to unity (or
100%) contrast at zero spatial frequency. In other words, the MTF
is a measure of the ability of an optical system to transfer
various levels of detail from object to image. Accordingly,
embodiments of array cameras have more stringent MTF requirements
than for legacy cameras, and in particular use optics having MTF
characteristics that exceed the spatial resolution (Nyquist
frequency) of the pixel pitch of the pixels on a focal plane.
[0148] In particular, when multiple copies of an aliased signal are
present, as in embodiments of a camera array, it is possible to use
the information that is inherently present in the aliasing to
reconstruct a higher resolution signal. However, there are slight
differences between the aliasing patterns in the different camera
images due to the array's sampling diversity. This sampling
diversity is the result of slightly different viewing directions of
the different cameras within the array, which are either
intentionally introduced or result from (positional) manufacturing
tolerances. Typically, filters would be introduced to create a
bandlimited signal having an MTF near the Ny of the sensor as
illustrated in FIG. 3B. However, in embodiments of a camera array
aliasing to create a super resolution image is necessary.
Accordingly, to provide sufficient contrast in the aliased LR
images, the lens MTF needs to be as high as the desired high
resolution output MTF from the super resolution processing. As
illustrated in FIG. 3C, this requires that the cameras capture
content above Ny such that the super-resolution process can then
recover the higher resolution information. To address this optics
challenge, in many embodiments, the MTF characteristics of the
optics in camera arrays are implemented such that images formed
include contrast at a spatial frequency that is at least as great
as the resolution of the high resolution images synthesized by the
array camera, and significantly greater than the Nyquist frequency
of the pixel pitch of the pixels on the focal plane (e.g., in some
embodiments from 1.5 to 3 times Ny). In many embodiments, the
specific MTF requirement of the optics of the camera array may be
determined by the ratio of the resolution of the high resolution
image and Nyquist resolution of the individual camera. In other
words, in some embodiments optics lens are implemented having a
contrast of at least 10%, in some embodiments at least 20%, and in
other embodiments at least 30% at a spatial frequency given by the
ratio of the number of line-pairs resolved in the final synthesized
high resolution image, and the physical size of the low resolution
camera image in the same dimension, where the physical size of the
low resolution image is a function of the size and number of pixels
in the individual camera along the relevant dimension.
DEFINITIONS
[0149] A sensor element or pixel refers to an individual
light-sensing element in a camera array. The sensor element or
pixel includes, among others, traditional CIS (CMOS Image Sensor),
CCD (charge-coupled device), quantum dot films, high dynamic range
pixel, multispectral pixel and various alternatives thereof. The
pixel pitch of these sensor elements defines the Nyquist
frequency.
[0150] An imager refers to a focal plane formed from a two
dimensional array of pixels associate with a lens stack formed from
a set of optical elements. The sensor elements or pixels of each
imager or focal plane have similar physical properties and receive
light through the same set of optical components or lens stack.
Further, the sensor elements in the each imager/focal plane may be
associated with the same color filter.
[0151] An imager or camera array refers to a collection of
imagers/cameras designed to function as a unitary component. The
imager or camera array may be fabricated on a single chip for
mounting or installing in various devices.
[0152] A lens stack refers to an axial arrangement of several
optical components/lens elements.
[0153] An optical channel refers to the combination of a lens stack
and an imager or focal plane.
[0154] A lens or optical array refers to a lateral arrangement of
individual lens elements stacks.
[0155] An optics or lens stack array refers to a lateral array of
lens stacks, or an axial arrangement of multiple lens arrays.
[0156] A camera array module refers to the combination of an optics
array and an imager array, and can also be defined as an array of
optical channels.
[0157] Image characteristics of an imager refer to any
characteristics or parameters of the imager associated with
capturing of images. The imaging characteristics may include, among
others, the size of the imager, the type of pixels included in the
imager, the shape of the imager, filters associated with the
imager, the exposure time of the imager, aperture size of the
optics associated with the imager, the configuration of the optical
element associated with the imager, gain of the imager, the
resolution of the imager, and operational timing of the imager. The
characteristics of the optics of a camera refer to at least the
field of view (FOV), F-number (F/#), resolution (MTF), effective
focal length or magnification, color or waveband, distortion, and
relative illumination.
[0158] These defined aspects of the embodiments will be described
in greater detail below.
Structure of Array Camera
[0159] Array cameras in accordance with embodiments of the
invention can include a camera module and a processor. FIG. 1 is a
plan view of a generic array camera 100, which includes a camera
module (110 with an array of cameras orimagers 1A through NM. As
shown, a camera module of the type shown is fabricated to include a
plurality or array of cameras 1A through NM. In turn, each of the
cameras 1A through NM may include a plurality of focal planes and
light sensing pixels (e.g., 0.32 Mega pixels). Although the imagers
1A through NM are shown as arranged into a grid format, it should
be understood that they may be arranged in any suitable
configuration. For example, in other embodiments, the imagers may
be arranged in a non-grid format, such as in a circular pattern,
zigzagged pattern or scattered pattern.
[0160] These array cameras may be designed as a drop-in replacement
for existing camera image sensors used in cell phones and other
mobile devices. For this purpose, the camera array may be designed
to be physically compatible with conventional camera modules of
approximately the same resolution although the achieved resolution
of the camera array may exceed conventional image sensors in many
photographic situations. Taking advantage of the increased
performance, the array camera of the embodiment may include an
imager with fewer pixels to obtain equal or better quality images
compared to conventional image sensors. Alternatively, the size of
the pixels in the imager may be reduced compared to pixels in
conventional image sensors while achieving comparable results. In
some embodiments, the array camera replaces a conventional image
sensor of M megapixels. The array camera has (N.times.N) individual
imagers or cameras, each camera including pixels of M/N.sup.2. Each
camera in the camera array may also have the same aspect ratio as
the conventional image sensor being replaced.
Array Camera Modules
[0161] Camera modules in accordance with embodiments of the
invention can be constructed from an imager array and an optic
array. Camera modules in accordance with embodiments of the
invention are illustrated in FIGS. 2A and 2B. The camera module 200
includes an imager array 230 including an array of focal planes 240
along with a corresponding optic array 210 including an array of
lens stacks 220. Within the array of lens stacks, each lens stack
220 creates an optical channel that forms an image of the scene on
an array of light sensitive pixels 242 within a corresponding focal
plane 240. As is described further below, the light sensitive
pixels 242 can be formed from quantum films. Each pairing of a lens
stack 220 and focal plane 240 forms a single camera 104 within the
camera module. Each pixel within a focal plane 240 of a camera 104
generates image data that can be sent from the camera 104 to the
processor 108. In many embodiments, the lens stack within each
optical channel is configured so that pixels of each focal plane
240 sample the same object space or region within the scene. In
several embodiments, the lens stacks are configured so that the
pixels that sample the same object space do so with sub-pixel
offsets to provide sampling diversity that can be utilized to
recover increased resolution through the use of super-resolution
processes. The camera module may be fabricated on a single chip for
mounting or installing in various devices.
[0162] In several embodiments, an array camera generates image data
from multiple focal planes and uses a processor to synthesize one
or more images of a scene. In certain embodiments, the image data
captured by a single focal plane in the sensor array can constitute
a low resolution image (the term low resolution here is used only
to contrast with higher resolution images), which the processor can
use in combination with other low resolution image data captured by
the camera module to construct a higher resolution image through
Super Resolution processing, as previously described. Where super
resolution is performed then multiple copies of an aliased signal
are present, such as in multiple images from the focal planes 240,
and the information inherently present in the aliasing may be used
to reconstruct the higher resolution signal. One skilled in the art
will note that the aliasing patterns from the different focal
planes 240 will have slight differences due to the sampling
diversity of the focal planes. These slight differences result from
the slightly different viewing directions of the cameras used to
capture the low resolution images that are either intentionally
introduced or result from positional manufacturing tolerances of
the individual focal planes. Thus, in accordance with some
embodiments of this invention, the MTFs of the lens stacks 220 need
to be at least as high as the desired high resolution output MTF to
provide sufficient contrast. Accordingly, in many embodiments of an
array camera, optics in the lens stack are implemented that have an
MTF at least as high as the desired MTF of the super resolution
image, i.e., an MTF at which the individual optic channels of the
camera array are capable of resolving spatial frequencies above the
Nyquist frequency of the pixels on the focal plane. In many such
embodiments, the optics have an MTF at which they are capable of
resolving spatial frequencies at least 1.5, 2 and/or 3 times the
Nyquist frequency of the pixels to allow the super-resolution
process to recover higher resolution information unavailable from
the individual low resolution images captured at the individual
cameras.
Imager Arrays
[0163] Imager arrays in accordance with embodiments of the
invention can be constructed from an array of focal planes formed
of arrays of light sensitive pixels. As discussed above in relation
to FIG. 2A, in many embodiments the imager array 230 is composed of
multiple focal planes 240, each of which have a corresponding lens
stack 220 that directs light from the scene through optical channel
and onto a plurality of light sensing elements (the pixel pitch of
which define Ny) formed on the focal plane 240. In many embodiments
the light sensing elements are formed on a CMOS device using
photodiodes formed in the silicon where the depleted areas used for
photon to electron conversion are disposed at specific depths
within the bulk of the silicon. In some embodiments, a focal plane
of an array of light sensitive pixels formed from a quantum film
sensor may be implemented. The formation, composition, performance
and function of various quantum films, and their use in optical
detection in association with semiconductor integrated circuits are
described in U.S. Patent Publication US/2009/0152664, entitled
"Materials, Systems and Methods for Optoelectronic Devices",
published Jun. 18, 2009, the disclosure of which is incorporated by
reference herein in its entirety.
[0164] A focal plane 240 in accordance with an embodiment of the
invention includes a focal plane array core that includes an array
of light sensitive pixels 242 disposed at the focal plane of the
lens stack 220 of a camera on a semiconducting integrated circuit
substrate 230, such as a CMOS or CCD. The focal plane can also
include all analog signal processing, pixel level control logic,
signaling, and analog-to-digital conversion (ADC) circuitry used in
the readout of pixels. The lens stack 220 of the camera directs
light from the scene and onto the light sensitive pixels 242. The
formation, architecture and operation of imager arrays and light
sensitive pixel arrays, and their use in optical detection in
association with array cameras are described in U.S. patent
application Ser. No. 13/106,797, entitled "Architectures for Imager
Arrays and Array Cameras", filed May 12, 2011, the disclosure of
which is incorporated by reference herein in its entirety.
[0165] Imager arrays of this design may include two or more types
of heterogeneous imagers, each imager or camera including two or
more sensor elements or pixels. Each one of the imagers may have
different imaging characteristics. Alternatively, there may be two
or more different types of imagers where the same type of imagers
shares the same imaging characteristics. For example, each imager
1A through NM in FIG. 1 may be associated with its own filter
and/or optical element (e.g., lens). Specifically, each of the
imagers 1A through NM or a group of imagers may be associated with
spectral color filters to receive certain wavelengths of light.
Example filters include a traditional filter used in the Bayer
pattern (R, G, B), an IR-cut filter, a near-IR filter, a polarizing
filter, and a custom filter to suit the needs of hyper-spectral
imaging. In addition, some imagers may have no filter to allow
reception of both the entire visible spectra and near-IR, which
increases the imager's signal-to-noise ratio. The number of
distinct filters may be as large as the number of cameras in the
camera array. Embodiments where filter groups are formed is further
discussed in U.S. Provisional Patent Application No. 61/641,165
entitled "Camera Modules Patterned with pi Filter Groups" filed May
1, 2012, the disclosure of which is incorporated by reference
herein in its entirety. These cameras can be used to capture data
with respect to different colors, or a specific portion of the
spectrum. In other words, instead of applying color filters at the
pixel level of the camera, color filters in many embodiments of the
invention are included in the lens stack of the camera. For
example, a green color camera can include a lens stack with a green
light filter that allows green light to pass through the optical
channel. In many embodiments, the pixels in each focal plane are
the same and the light information captured by the pixels is
differentiated by the color filters in the corresponding lens stack
for each filter.
[0166] It will be understood that such imager arrays may include
other related circuitry. The other circuitry may include, among
others, circuitry to control imaging parameters and sensors to
sense physical parameters. The control circuitry may control
imaging parameters such as exposure times, gain, and black level
offset. The sensor may include dark pixels to estimate dark current
at the operating temperature. The dark current may be measured for
on-the-fly compensation for any thermal creep that the substrate
may suffer from.
Optic Arrays
[0167] To provide lenses and other optical elements for
implementation in the lens stacks of the optical arrays any
suitable optical technology may be employed capable of forming a
lens stack with a suitable MTF, i.e., an optic MTF at least as high
as the MTF of the high resolution image to be obtained via super
resolution. To determine the suitability of optical elements and
lens stacks, each lens stack 220 may be specified in terms of its
MTF curve over a range of spatial frequencies. As the MTF is a
Spatial Frequency Response (SFR) of the output signal contrast with
the input spatial frequency, it is possible to determine the
frequencies that can be optically resolved with sufficient
frequency by the individual optical elements and lens stacks.
[0168] One skilled in the art will understand that any lens system
will demonstrate a number of MTF curves depending on the operating
conditions of the camera (such as aperture size) and the spatial
frequency being resolved. The MTF might also be impacted by the
type of scene being imaged (for example, there is often a spread in
the MTF curve between a lenses ability to resolve meridional and
sagittal lines). Finally, a lens system may demonstrate various
levels of spatial resolution as you proceed from the center of
image outward. In many embodiments of an array camera, optics are
selected such that the individual cameras of the camera array are
able to spatially resolve at frequencies above the Nyquist
frequency of the pixels across all imaging conditions and locations
on the lens to allow the super-resolution process to recover higher
resolution information under all conditions and camera settings. In
other embodiments, however, lens stacks and optical elements are
contemplated that demonstrate an MTF sufficiently high to allow for
the camera to spatially resolve at frequencies above the Nyquist
frequency of the pixels across only some imaging conditions and
camera settings. A detailed description of various optical elements
for use in camera arrays is provided below.
[0169] FIG. 2A illustrates a perspective view of one array camera
assembly 200 that incorporates an optics array 210 with an imager
array 230. As shown, the optics array 210 generally includes a
plurality of lens stacks 220 (which furthermore may consist of
several axially aligned lens elements), each lens stack 220
covering (in the shown example) one of twenty-five imagers 240 in
the imager array 230.
[0170] FIG. 2B illustrates a sectional view of a camera array
assembly 250. As shown, in such a design the camera assembly 250
would comprise an optics array including a top lens wafer 262 and a
bottom lens wafer 268, and an imager array including a substrate
278 with multiple sensors and associated light sensing elements
formed thereon. Spacers 258, 264 and 270 are also included to
provide proper positioning to the various elements. In this
embodiment the camera array assembly 250 is also packaged within an
encapsulation 254. Finally, an optional top spacer 258 may be
placed between the encapsulation 254 and the top lens wafer 262 of
the imager array; however, it is not essential to the construction
of the camera assembly 250. Within the imager array, individual
lens elements 288 are formed on the top lens wafer 262. Although
these lens elements 288 are shown as being identical in FIG. 2B, it
should be understood that within the same camera array different
types, sizes, and shapes of elements may be used. Another set of
lens elements 286 is formed on the bottom lens wafer 268. The
combination of the lens elements on the top lens wafer and bottom
lens wafer form the lens stacks 220 shown in FIG. 2A.
[0171] In these types of camera arrays, through-silicon vias 274
may also be provided to paths for transmitting signal from the
imagers. The top lens wafer 262 may be partially coated with light
blocking materials 284 (e.g., chromium, oxidized ("black")
chromium, opaque photoresist) to block of light. In such
embodiment, the portions of the top lens wafer 262 of the optics
array not coated with the blocking materials 284 serve as apertures
through which light passes to the bottom lens wafer 268 and the
imager array. Although only a single aperture is shown in the
embodiment provided in FIG. 2B, it should be understood that, in
these types of camera arrays, additional apertures may be formed
from opaque layers disposed on any and all of the substrate faces
in the camera assembly to improve stray light performance and
reduced optical crosstalk. In the example shown in FIG. 2B, filters
282 are formed on the bottom lens wafer 268 of the optics array.
Light blocking materials 280 may also be coated on the bottom lens
wafer 268 of the optics array to function as an optical isolator. A
light blocking material 280 may also be coated on the substrate 278
of the imager array to protect the sensor electronics from incident
radiation. Spacers 283 can also be placed between the bottom lens
wafer 268 of the optics array and the substrate 278 of the imager
array, and between the lens wafers 262 and 268 of the optics array.
In such array cameras, each layer of spacers may be implemented
using a single plate.
[0172] Although not illustrated in FIG. 2B, many such camera arrays
also include spacers between each optical channel located on top of
the top lens wafer 262 of the optics array that are similar to, or
implemented in single layer with, the spacer 258 shown at the edge
of the lens stack array. As is discussed further below the spacers
can be constructed from and/or coated in light blocking materials
to isolate the optical channels formed by the wafer level optics.
Suitable light blocking materials may include any opaque material,
such as, for example, a metal material like Ti and Cr, or an oxide
of these materials like black chromium (chrome and chrome oxide),
or dark silicon, or a black particle filled photoresist like a
black matrix polymer (PSK2000 from Brewer Science).
[0173] There are a number of advantages that can be realized by
using smaller lens elements with these array cameras. First, the
smaller lens elements occupy much less space compared to a single
large lens covering the entire camera array 230. In addition, some
of the natural consequences of using these smaller lens elements
include, improved optical properties by reduced aberrations, in
particular chromatic aberrations, reduced cost, reduced amount of
materials needed, and the reduction in the manufacturing steps. A
full discussion of these advantages may be found in U.S. Patent
Publication No. US-2011-0080487-A1, the disclosure of which is
incorporated herein by reference.
[0174] Because of the distributed approach of the array camera, and
the resultant relaxed total track length requirement (since the
array camera by its nature is much shorter than a comparable
classical objective), it is possible to adopt novel optical design
approaches for the lens channels of an array camera rather than
just using arrays of conventional optical designs. In particular,
many embodiments are directed to optic arrangements capable of MTF
characteristics such that images formed on a focal plane include
contrast at a spatial frequency that is at least greater than the
resolution of high resolution images synthesized by the array
camera during super resolution, and significantly greater than the
Nyquist frequency of the pixel pitch of the pixels on the focal
plane, and in some cases as much as 1.5 to 3 times the Nyquist
frequency. These novel arrangements will be described in detail
below, however, it should be understood that other optical
arrangements that incorporate the improvements set forth herein may
be used with the camera arrays described herein.
Embodiment 1
Three-Surface WLO Design
[0175] Traditional wafer level optics (WLO) is a technology where
polymer lenses are molded on glass wafers, potentially on both
sides, stacked with further such lens wafers by spacer wafers, and
diced into lens modules (this is called "polymer on glass WLO")
followed by packaging of the optics directly with the imager into a
monolithic integrated module. As will be described in greater
detail below, the WLO procedure may involve, among other
procedures, using a wafer level mold to create the polymer lens
elements on a glass substrate. Usually this involves incorporating
apertures, and in particular the aperture stop by providing
openings centered with the later lens channels in an otherwise
opaque layer onto the substrate before lens molding.
[0176] In a first embodiment, a three-surface optical arrangement
suitable for the fabrication by wafer level optics technology, and,
in particular, to be used for the optics (as one of the multiple
channels) of an array camera is described in reference to FIG. 4A.
More specifically, in a standard two-element lens, as shown in
FIGS. 2 and 3, there are typically four lens surfaces (front and
back for the top and the bottom lenses). In contrast, in this
total-track-length-relaxed but MTF-performance optimized design,
the third surface (first side of second element) has very low
refractive power. As a result, it is possible to omit it from the
design entirely. The result is a three surface design, which leads
to a less expensive lens, due to less required process steps, and
improved yield because of less contributors to the yield
multiplication. In addition, since only lenses that have shapes
which appear close to spherical- or parabolic profiles (monotonous
profiles, no wings) are applied in a specific axial arrangement,
centered around the aperture stop, only weak ray bending occurs at
all the refractions on air-lens or lens-air interfaces. The result
of this arrangement is a relaxed sensitivity with respect to
centering-, thickness-, distance-, or lens shape form error
tolerances. As shown in FIG. 4A, the rays for the different field
heights more or less transmit perpendicularly, and are thus not
strongly refracted through the lens surfaces. However, in such an
arrangement it is very important to find an optimum position where
the angle of incidence (AOI) on the glass substrate is minimal so
that the shift of the band edges due to the AOI is minimized for
any dielectric filter system (e.g. for IR cut-off), which is
applied on substrates within the lens stack.
[0177] As shown FIG. 4A, the three-surface optical arrangement is
identified by first 400 and second lens elements 402, which are
arranged sequentially along a single optical path 403. It should be
understood that for construction purposes, each lens element may
optionally be associated with a corresponding supporting substrate
404 and 406, made from example of glass, upon which the polymer
lens surfaces are formed. In addition, spacer elements (not shown)
that can serve to mechanically connect the lens elements to each
other and/or to the image sensor may also be included in the
construction. Although any suitable material may be used, in one
embodiment the lens surfaces are made from a (UV- or thermally
curable) polymer.
[0178] Turning to the construction of the lens elements themselves,
in the first lens element 400, there is a convex surface 408 of a
first diameter on the first side of the first element and a concave
surface 410 of a second diameter on the second side of the first
element. Preferably the diameter of the first side is larger than
the diameter of the second side of the first lens element. In the
second element 402, there is a shallow or flat surface 412 on the
first side of the second element, and a convex surface 414 on the
second side of the second element. Preferably, the diameter of the
first side of the second element is smaller than the diameter of
the second surface of the second element, and that the diameter of
the second side of the second element is intermediate between the
diameters of the first and second sides of the first element. In
addition, the system aperture or stop (not shown) is preferably
disposed on the first side of the second element.
[0179] Although not shown in the diagram, a (thin) first spacing
structure (not shown) is placed in between the two lens elements,
which can be either incorporated into the respective lens surfaces
("stand-offs"), or can be an additional element. Likewise, a
(thick) second spacing structure connecting the second side of the
second lens element with the cover glass or package 416 of the
image sensor 417 may also be provided. Both spacing structures are
preferably opaque, or have opaque surfaces, at least at the inner
side-walls, and provide partial optical isolation between adjacent
optical channels. FIG. 4B provides a lens table for an exemplary
embodiment of the three-surface optical arrangement in accordance
with the current invention.
[0180] Although the basic construction of the three-side optical
arrangement is described above, it should be understood that other
features and structures may be incorporated into the lens elements
to provide additional or enhanced functionality, including: [0181]
The inclusion of additional apertures within the lens stack (in
particular on the glass substrates underneath the polymer lenses).
[0182] Channel specific filters, such as, for example, organic
color filter arrays "CFA" and/or structured dielectric filters,
such as, for example, IR cut-off, NIR-pass, interference color
filters. These filters may be arranged within the stack of the
first and the second lens element, preferably in a surface close to
the system aperture. [0183] Partial achromatization of the
individual narrow-spectral-band-channels may be accomplished by
combining different Abbe-number materials for the different lens
surfaces, such as, for example, "crown-like" materials for the two
convex surfaces on the outsides of the optical arrangement, and
"flint-like" materials for the potentially two (concave) surfaces
on the inner sides of the two lens elements (as is further
described in Embodiment 6, below). [0184] Optimization of different
color channels to their specific narrow spectral band may be
accomplished by adapting (at least) one lens surface profile within
the optical arrangement to that color to correct for chromatic
aberrations. (For a full discussion see, e.g., U.S. patent
application Ser. No. 13/050,429, the disclosure of which is
incorporated herein by reference.)
[0185] There are several features of this novel three-surface
optical arrangement that render it particularly suitable for use in
array cameras. First, the optical arrangement is designed in such a
way that very high contrast at the image sensor's Nyquist spatial
frequency is achieved, which at the same time (for gradual fall-off
of contrast with increasing spatial frequency) provides sufficient
contrast at 1.5.times. or 2.times. the sensor's Nyquist frequency
to allow the super-resolution image information recovery to work
effectively. Second, the optical arrangement is optimized for
allowing a small lateral distance between adjacent optical channels
in order to economically exploit the die real-estate area,
consequently the lens diameters should be small, as should the
wall-thickness of the (opaque) spacer structures. Third, optical
channels within one array dedicated to imaging different "colors"
(parts of the overall wavelength spectrum to be captured) may
differ with regard to the particular surface profile of at least
one lens surface. The differences in the surface profiles of those
lenses in one array can be minor, but are very effective in keeping
the back focal length ("BFL") color-independent, and consequently
allowing (almost) equally sharp images for the different colors
without the costly need for wide-spectral-band achromatization.
Moreover, after computational color-fusion a high-resolution
polychromatic image can be achieved. In particular, preferably the
last surface in the lens stack (here--second surface of second
element) is specifically optimized for the narrow spectral band of
the respective color channel. Fourth, the above design approaches
result in a (partial--as far it can be the case for the given
non-symmetry between object- and image space) symmetry of the lens
system around the aperture stop, which helps to reduce certain
types of aberrations, including, distortion, coma and lateral
color.
[0186] The benefits of this array-dedicated design of the single
channels of the array camera include: [0187] The ability to provide
high resolution with as few as two elements, and with only three
surfaces of refractive power within the two elements, [0188]
Increased simplicity of the lens shapes, [0189] Reduced fabrication
tolerance sensitivity due to reduced ray bending. [0190] Low CRA,
due to relaxed total track length requirement. [0191] Low color
cross-talk, due to color filters differentiating the different
optical channels rather than having a Bayer pattern on the pixel
level performing this task. [0192] Low pixel cross-talk, due to the
smaller pixel stack height. [0193] Reduced color inhomogeneity, due
to the color filters being farther from the image plane. [0194] Low
inter-channel cross-talk, because of the long back focal length,
which allows a thick opaque second spacer compared to thin
transparent substrate. [0195] Fewer monochromatic aberrations, as a
result of the smaller lenses, because many of these aberrations
scale with lens size 2- 4. [0196] Separate color channels that only
need to be optimized for their respective spectral bands, resulting
in higher overall polychromatic resolution while minimizing the
achromatization requirements within the individual channels, again
resulting in simpler overall aberration balancing or correction
processes and simpler lenses, and/or better MTF, and/or lower F/#.
In many embodiments the MTF characteristics of the three-surface
optical arrangement allow for contrast at spatial frequencies that
are at least as great as the resolution of the high resolution
images synthesized by the array camera, and significantly greater
than the Nyquist frequency of the pixel pitch of the pixels on the
focal plane, which in some embodiments may be 1.5, 2 or 3 times the
Nyquist frequency. [0197] Higher yield during manufacture, because
the smaller lenses mean smaller sag (i.e., vertex height) of the
lenses, which leads to less shrinkage and the ability to use less
complex replication technology.
Embodiment 2
Five-Surface WLO Design
[0198] In a second embodiment, a five-surface optical arrangement
suitable for the fabrication by wafer level optics technology, and,
in particular, to be used for the optics (as one of the multiple
channels) of an array camera is described in reference to FIGS. 5A
to 5J. More specifically, this embodiment is directed to a
five-surface high-resolution wafer level lens/objective having a
field-flattening element close to the image plane. Again, in a
standard two-element lens, as shown in FIGS. 2 and 3, there are
typically four lens surfaces (front and back for the top and the
bottom lenses). However, these optical arrangements are non-ideal
for use in high resolution array cameras. Ideally, there would be a
reduced requirement on small maximum CRA of the optics by the image
sensor (allowing much larger angles of incidence on the same), for
example, by using BSI or quantum film sensors (quantum film having
the additional advantage of an increased fill factor not requiring
microlenses, which otherwise require an air gap in front of the
image sensor in order for the microlenses to provide refractive
power). Finally, regular lens designs have stronger small total
track length requirements, which would render the overall camera
length shorter than necessary for an array camera, since the array
camera is much shorter than a comparable classical objective by
concept. In contrast, in this design five surfaces are used. The
large number of degrees of freedom in the five-surface design
allows for achromatization for the full visible spectral band (or
other band of interest), so that channel-specific lens profiles are
not necessarily required. However, even though the back focal
length may be kept constant over the spectral band of interest, the
effective focal length and with it the magnification may vary.
[0199] As shown in FIG. 5A, in one embodiment the five-surface
optical arrangement is identified by first 500, second 502, and
third 504 lens elements arranged sequentially along a single
optical path 505. It should be understood that for construction
purposes, each lens element may optionally be associated with a
corresponding supporting substrate 506, 508 & 510, made from
example of glass, upon which the polymer lens surfaces are formed.
In addition, spacer elements (not shown) that can serve to
mechanically connects the lens elements to each other and/or to the
image sensor may also be included in the construction. Although any
suitable material may be used, in one embodiment the lens surfaces,
i.e., the volume between the surface of the lens and the underlying
substrate, are made from a (UV- or thermally curable) polymer.
[0200] Turning to the construction of the lens elements themselves,
the first lens element 500 has a convex surface 512 having a first
diameter, and a concave surface 514 having a second diameter.
Preferably the diameter of the convex surface is greater than the
diameter of the concave surface on this lens element. The second
lens element 502 has a concave surface 516 on the first side of the
second lens element, and a convex surface 518 on the second side of
the second element. In this second lens element, preferably the
convex surface thereof is of a larger diameter when compared to the
concave surface thereof. The third lens element 504 has a concave
surface 520 on the first side of the third lens element, and a
second planar side 522 that is adjoined to the substrate that
serves as the image sensor cover 510. Typically, the diameter of
the concave surface of the third lens element is larger than the
diameters of any of the surfaces of the first and second lens
elements.
[0201] In terms of arrangement, a first spacing structure (not
shown) is disposed in between the first 500 and the second 502 lens
elements. Likewise, a second spacing structure (not shown) is
disposed in between the second 502 and the third 504 lens elements.
Either of these spacers may be either incorporated (also split)
into the respective lens surfaces ("stand-offs" in which the lenses
then can be glued directly together), or can be an additional
element. In addition, both of these spacing structures are
preferably opaque (or have opaque surfaces, at least at the (inner)
side-walls), and provide (partial) optical isolation between
adjacent optical channels. The third lens element 504 is disposed
comparatively close to the image surface 524, and the second side
of the third lens element is preferably connected with the image
sensor or image sensor cover glass by a transparent areal bond or a
local bond (e.g. UV- and/or thermally curing adhesive), or even a
(-n opaque) spacing structure with transparent openings as
described above.
[0202] In summary, FIG. 5A1, discussed above, demonstrates a
five-surface optical arrangement disposed on a large-sag field
flattener on a regular-thickness image sensor cover glass. In
particular, there is no air gap between the field flattener 504 (or
its substrate, or the sensor cover glass, respectively) and the
image sensor 524. In front of the field flattener is the actual
focusing objective comprised of first 500 and second 502 lens
elements, ideally containing two concave surfaces close to the
system aperture more or less symmetrically surrounded by two convex
surfaces. An exemplary lens table associated with this design is
provided in FIG. 5A2.
[0203] FIGS. 5B-H present some characteristic performance
indicators of the five-surface optical arrangement shown in FIG.
5A1. In particular, FIG. 5B provides a data graph of the Strehl
ratio showing that the lens is diffraction limited over the full
field height (@ F/2.4 and diagonal full FOV of 56.degree.). FIG. 5C
provides a data graph showing that in a comparison of MTF vs. field
there is virtually no loss of performance with increasing field
height. FIG. 5D provides a data graph of the polychromatic
diffraction encircled energy, and demonstrates that most of the
focused light energy is within the Airy disk. FIG. 5E provides a
spot diagram demonstrating that the lens almost appears to be
isoplanatic where there is little change of spot size and shape
with field height. FIG. 5F provides a data graph of MTF vs. spatial
frequency, and shows that for small and intermediate field heights
there is still 15-20% contrast even at 500 LP/mm. FIG. 5G provides
data graphs showing that the lens design demonstrates acceptable
and monotonous distortion. Finally, FIG. 5H provides a relative
illumination plot, demonstrating that the optical arrangement shows
the usual vignetting behavior.
[0204] These data results show that for this particular design
family, due to the strong degree of achromatization, there is very
little performance loss using the green channel for red and blue
spectra. In other words, the system is already well achromatized
for the full visible spectrum. Accordingly, there is not much
benefit when optimizing the green and red channels specifically
rather than just using the green one. Explicit full visible
optimization is very promising as well, i.e., no differences
between the channels are required. Furthermore, from these results
it is possible to recognize that optimizing a lens for the full
visible spectrum, but using only the red, green and blue bands
separately will improve performance even beyond what is seen from
the visible-polychromatic MTF, Strehl-ratio and Encircled energy
plots. This is because fewer wavelengths contribute to each color
channel's polychromatic blur. Moreover, this effect becomes more
significant the more lateral color dominates the polychromatic blur
over axial color, since then the difference between colors is
mostly reflected in a difference of magnification or focal length
as described above rather than different blur sizes. In short, the
largest benefit of these features is that all channels could be the
same, simplifying the array mastering considerably.
[0205] Although the above discussion has focused on an embodiment
of a five-surface optical arrangement with no air gap between the
field flattener and the imager, it should be understood that
alternative embodiments incorporating air gaps may be made in
accordance with this invention. These embodiments are advantageous
because they may be combined with regular image sensors which can
have fill factor enhancing microlenses and a limited maximum CRA of
around 30.degree.. For Example, FIGS. 5I1 and J1 provide diagrams
of two such embodiments, which will be described below.
[0206] The embodiment shown in FIG. 5I1 is a five-surface optical
arrangement that allows for an air gap 526 between the sensor cover
glass 510 (on which the field flattener 528 is positioned) and the
image sensor 524. This is usually required when fill factor
enhancing .mu.lenses are applied on top of the image sensor 524. As
a result of the presence of the air gap, the chief ray angle needs
to be reduced over the embodiment shown in FIG. 5A1. Although there
are no constraints made on lens vertex heights and minimum glass
thicknesses, lens TTL increases and image performance reduces due
to the requirement of a reduced maximum CRA. However, ray
calculations indicate that even in this embodiment the CRA of the
inventive optical arrangement meets regular sensor specifications
(in the order of magnitude of 27-28.degree. in air). An exemplary
lens table associated with this design is provided in FIG. 5I2.
[0207] FIG. 5J1 provides a schematic of an embodiment of a
five-surface optical arrangement optimized for best possible
manufacturability. In particular, in this embodiment the lens sags
are decreased, and lens material planar base layers 532, 534, 536,
538 and 542 having suitable thicknesses are provided. It should be
understood that for construction purposes, each lens element may
optionally be associated with a corresponding supporting substrate
533 & 533', 537 & 543 made from example of glass, upon
which the polymer lens surfaces and base layers may be formed. In
addition, spacer elements (not shown) that can serve to
mechanically connect the lens elements to each other and/or to the
image sensor may also be included in the construction.
[0208] In addition, the system aperture 540 is sandwiched (or
"embedded") between two glass substrates 533 and 533' in order to
decrease the necessary polymer thickness of the adjacent lens
surfaces. Finally, a glass substrate 543 is provided between the
field flattener lens surface 544 and the imager package, including
the image sensor glass cover 545 and the image sensor 546 (with air
gap 548). Although an even split of 50/50 is shown in FIG. 5J1, the
thickness between the glass substrate 543 and the image sensor
cover glass 545 may be shared by any reasonable ratio (which allows
sufficient thickness for both). Cover glass as needed for the
imager may also be provided. All of these elements may then be
immersed and bonded together by a suitable adhesive during
manufacture. Again, in this embodiment the CRA meets regular sensor
specifications (in the order of magnitude of 27-28.degree. in air).
An exemplary lens table associated with this design is provided in
FIG. 5J2.
[0209] Although the basic construction of the five-surface optical
arrangement is described above, it should be understood that other
features and structures may be incorporated into the lens elements
to provide additional or enhanced functionality (references are to
FIG. 5A1), including: [0210] The system aperture (Stop) may be
disposed either on the second side of the first element 500, or on
the corresponding side of the respective glass substrate, or
embedded within the first lens element 500 (e.g. sandwiched between
two thinner glass substrates that have been structured with an
aperture array on the inner side, then glued together). In such an
embodiment, the lenses of the element would be replicated on this
aperture sandwich. [0211] As discussed above, other implementations
of this general design may have either no air gap or a thin air gap
between the third lens element 504 and the photosensitive surface
(or interface thereto) of the image sensor 524. Such a design
allows this optical arrangement to operate with regular image
sensors, specifically those with fill factor-enhancing microlenses,
and image sensors with conventional CRA. However, this results in
longer TTL and only moderate image quality compared to versions
without the discussed air gap. In particular, the CRA needs to be
moderate when there is an air gap at this location, because
otherwise there can be (partly) total internal reflection at the
interface between the higher refractive index third lens element
504 and the air gap, or such a strong refraction outwards that
strong aberrations occur, i.e., rays may be fanned out rather than
focused. [0212] Several additional apertures may also be disposed
within the stack, and, in particular, on the glass substrates
underneath the polymer lenses where applicable. [0213] Channel
specific filters may also be arranged within the stack of the first
500 and the second 502 lens element, preferably in a surface close
to the system aperture. Such filters may include, for example,
organic color filter array "CFA" and/or structured dielectric
filters, such as, IR cut-off or NIR-pass interference color
filters. [0214] Partial achromatization of the individual
narrow-spectral-band-channels may be accomplished by combining
different Abbe-number materials for the different lens surfaces.
Preferably "crown-like" materials would be used for the two convex
surfaces on the outsides of the two first lens elements and
"flint-like" materials for the two concave surfaces on the inner
sides of the two first lens elements (See Embodiment 6). [0215]
Optimization of the different color channels to their specific
narrow spectral band may also be accomplished by adapting at least
one lens surface profile within the optical arrangement to that
color to correct for chromatic aberrations. (For a full discussion
see, e.g., U.S. patent application Ser. No. 13/050,429, the
disclosure of which is incorporated herein by reference.)
[0216] There are several features of this novel five-surface
optical arrangement that render it particularly suitable for use in
array cameras. First, the optical arrangement is designed in such a
way that very high contrast at the used image sensor's Nyquist
spatial frequency is achieved, which at the same time (for gradual
fall-off of contrast with increasing spatial frequency) provides
sufficient contrast at 1.5.times. or 2.times. the sensor's Nyquist
frequency to allow the super-resolution image information recovery
to work effectively. Second, the optical arrangement is optimized
for allowing a small lateral distance between adjacent optical
channels in order to economically exploit the die real-estate area,
consequently the lens diameters and the wall-thickness of (opaque)
spacer structures may be reduced. However, for the field flattening
structure itself this is sometimes difficult to achieve. The reason
for this is that the proximity of this lens surface to the image
sensor requires a lens having a diameter on the order of magnitude
of the image circle (scaled by the distance between the two). In
order to relax this requirement the field flattener can be designed
and implemented in a non-rotational-symmetric way. This results in
a rectangular rather than circular footprint of this lens surface.
Thus the lens would have a large lateral extension along the
corners (=image sensor diagonal), thereby allowing multiple lens
surfaces within one array to be situated much closer together in
x-y and thus allowing an overall smaller pitch between the
channels. Third, optical channels within one array dedicated to
imaging different "colors" (parts of the overall wavelength
spectrum to be captured) may differ in the particular surface
profile of at least one lens surface. The differences in the
surface profiles of those lenses in one array can be minor, but are
very effective in order to keep the back focal length ("BFL")
color-independent, and consequently allow (almost) equally sharp
images for the different colors without the costly need for
wide-spectral-band achromatization. Moreover, after computational
color-fusion a high-resolution polychromatic image can still be
achieved. Here, preferably the first surface of the first lens
element would be specifically optimized for the narrow spectral
band of the respective color channel.
[0217] The benefits of this array-dedicated design of the single
channels of the array camera include: [0218] Extremely high image
quality, both in terms of resolution and contrast, and image
quality homogeneity over the field of view (close to diffraction
limited performance and close to isoplanatically); [0219] A reduced
TTL; [0220] A reduced fabrication tolerance sensitivity due to
reduced ray bending; [0221] Low color cross-talk, due to color
filters now being differentiating the different optical channels
rather than having a Bayer pattern on the pixel level; [0222] Low
pixel cross-talk, due to smaller pixel stack height); and [0223]
Reduced color inhomogeneity, due to the color filters being far
from the image plane.
[0224] Finally, of particular note in this design is the fact that
the separate color channels only need to be optimized for their
respective spectral bands. This results in overall higher
polychromatic resolution, while minimizing the need for
achromatization correction within the individual channels. This in
turn leads to the ability to implement simpler overall aberration
balancing or correction process, and therefore have simpler lenses
and lens manufacturing processes, and/or better MTF, and/or lower
F/#. In many embodiments the MTF characteristics of the
five-surface optical arrangement allow for contrast at spatial
frequencies that are at east as great as the resolution of the high
resolution images synthesized by the array camera, and
significantly greater than the Nyquist frequency of the pixel pitch
of the pixels on the focal plane, which in some embodiments may be
1.5, 2 or 3 times the Nyquist frequency.
Embodiment 3
Monolithic Lens Design with Embedded Substrate
[0225] The embodiments previously discussed dealt with lenses made
in accordance with a polymer on glass WLO process. In the following
embodiment optical arrangements and designs using a monolithic lens
WLO process are provided. In particular, in a first embodiment a
monolithic lens stacked with planar substrates for use in forming
apertures and filters is described.
[0226] FIG. 6A shows the current state of the art of monolithic
lens systems. More or less the same conceptual approach is taken as
in creating injection-molded lenses and their packaging. In the
state of the art of monolithic lens WLO, many lenses are fabricated
on a wafer scale. These replicated lenses 600 are stacked with
other previously replicated lens wafers of different topology, the
sandwich is diced, and the lens cubes are packaged into an opaque
housing 602 with the image sensor 604, which contains the aperture
stop 606 at the front as shown in FIG. 6A. This very much limits
the degrees of freedom available for the optical design of the
objective. In addition, it makes it difficult to accurately
replicate and align the lenses with respect to each other,
particularly as it is difficult to determine precisely the
placement of the aperture stop. Moreover, from the standpoint of
optical design it is very desirable to have the aperture stop
between the two lens elements, not in front of the first lens
element as shown in FIG. 6A. Currently, as shown in FIG. 6B, the
only method for forming apertures of this type on monolithic lenses
is to use a highly imprecise screen-printing method in which
apertures 608 in opaque resins are printed onto the flat portions
of the lens interfaces. The lateral accuracy of those apertures is
unsuitable for their use as a system stop, which must be precisely
aligned with the lenses.
[0227] In short, although monolithic lens WLO is potentially an
attractive means to manufacture cheap miniaturized optics for array
cameras, the current monolithic systems are directly adapted from
the methods used to form lenses by injection molding. As a result,
many of the techniques used in conventional polymer-on-glass WLO to
ensure proper alignment are not applied, leading to alignment
accuracy problems as well as to a limited lens design space. The
current embodiment is directed to a novel method of forming
monolithic lenses that combines the monolithic WLO lenses with
substrates that hold apertures and additional structures in precise
alignment, thereby reducing the limitations of conventional
monolithic lens WLO.
[0228] An exemplary embodiment of the method of monolithic lenses
formed in accordance with the invention is shown in FIG. 6C. As
shown, in this embodiment, monolithic lenses 612 & 614,
fabricated by an independent replication process, are stacked with
a substrate or sheet 616 that holds apertures 618 & 620. (As
discussed previously, it will be understood that the monolithic
lenses may be formed of glass or polymer.) Because the apertures
can be formed on the substrate with lithographic precision, it is
possible to align the elements with sufficient lateral precision to
function as the aperture stop. In addition, although not shown in
FIG. 6C, the accuracy of the alignment in such a system is
increased by cooperative alignment marks, which are disposed in the
opaque layer(s) where the transparent openings for the apertures
are structured, to provide a guide for the precision alignment of
the lenses and apertures. In particular, in a wafer stack formed
from a series of wafer surfaces, themselves formed from the
elements of a number of optical arrangements, alignment marks would
be formed in relation to each wafer surface. Each of the alignment
marks would be cooperative with an alignment mark on an adjacent
wafer surface such that when cooperatively aligned the alignment
marks would aide in the lateral and rotational alignment of the
lens surfaces with the corresponding apertures. Using these
alignment marks results in a very high lateral alignment accuracy
(on the order of a few .mu.m) compared to having the aperture stop
in the external housing, which results in an accuracy of several
10-20 .mu.m.
[0229] In addition to apertures, the current method of providing a
substrate embedded into monolithic lenses provides a base onto
which any number of different structures, coatings, kinds of
substrates or sheets can be applied in order to achieve a desired
optical functionality. A number of these possibilities are shown in
FIG. 6D, these include where there are two apertures on the front
and back of the substrate that are the same size (6D1) or different
sizes (6D2); where an additional IRCF coating, such as a homogenous
IR cut-off filter made by a dielectric interference coating, is
applied on either one or both sides of the substrate (6D3); where
an additional color filter array material coating is applied to the
substrate (6D4); where the sheet or substrate contains an adaptive
refractive optical element allowing for the adjustment of the
optical power of the element by changing an applied voltage, which
can allow for the focusing of the whole lens stack, accounting for
fabrication tolerances (such as BFL variations)(6D5); or where the
sheet or substrate is made from an opaque material (6D6).
[0230] Other alternative designs that may be incorporated into the
substrates and monolithic lenses of the instant invention, but that
are not shown in the figures may include: [0231] A substrate that
is made of a material that is itself an absorptive IRCF (or
combined with a dielectric coating); [0232] A structured dielectric
IRCF complemented by a structured dielectric NIR-pass filter for
extended color camera modules; [0233] A polarization filter
disposed on the surface of the substrate or that is preformed into
the sheet; [0234] A thin diffractive lens applied to the surface of
the thin substrate by replication of an additional thin polymer
layer, or also by etching the diffractive structure into the glass,
front and/or backside of the substrate surface (See Embodiment 9);
and/or [0235] Standoffs or spacing structures integrated into the
monolithic lenses in addition to the actual lens surfaces in order
to provide the correct positioning between the lens surfaces and
the system aperture on the thin substrate (See Embodiment 8).
[0236] Although the above has focused on specific substrate
structures and additional optical elements that the substrate
embedded monolithic lenses of the instant inventions can
incorporate, it should be understood that there are other features
unique to the embedded substrates of the invention. For example,
unlike conventional polymer on glass WLO where substrates or sheets
must be sufficiently thick to allow replication of lenses thereon,
the embedded substrates or sheets of the instant invention can be
thin in comparison to wafer level optics standards since there is
no need to replicate lenses on them. As a result, the mechanical
stability and stress applied to the substrate is not an issue. In
contrast, the independently replicated monolithic lenses can
themselves serve to stabilize the glass substrate. Moreover, this
holds true even for a singlet lens construct (i.e., one monolithic
lens and one thin substrate).
[0237] Finally, while individual modifications to the basic
embedded substrate monolithic lens optical array are described
above, it should be understood that all or some of these features
may be applied in various combinations to the substrates to obtain
the desired functionality of the optical arrangement. In
particular, these structures may allow for the implementation of
optical arrangements that allow for contrast at spatial frequencies
that are at least as great as the resolution of the high resolution
images synthesized by the array camera, and significantly greater
than the Nyquist frequency of the pixel pitch of the pixels on the
focal plane, which in some embodiments may be 1.5, 2 or 3 times the
Nyquist frequency.
Embodiment 4
Monolithic Lens Design with Embedded Aperture Stop
[0238] This embodiment of the invention provides yet another
alternative for aperture and filter placement within the lens stack
of polymer or glass WLO monolithic lenses. As described above with
respect to Embodiment 3, the current state of the art for producing
monolithic lens optical arrays is to stack the independently
replicated monolithic lens wafers, dice the sandwich and package
the lens cubes into an opaque housing which contains the aperture
stop as an integral part at the front of the array. This
methodology limits the degrees of freedom for the optical design of
the objective, as well as making it extremely difficult to
accurately align the lenses with respect to of the aperture
stop.
[0239] Embodiment 3 of the invention described a polymer or glass
monolithic lens stacked with substrates for the placement of
apertures and filters. In that embodiment of the invention, a
substrate, such as glass, having aperture and/or filters thereon is
disposed between separately fabricated monolithic lenses. This
novel optical arrangement provides addition degrees of freedom for
the optical design, and increases the lateral precision of the
lens-aperture-alignment. The invention described in this embodiment
embeds apertures and filters directly within a monolithic lens (See
FIGS. 7A to 7G), providing even more and different degrees of
freedom for the optical design, while maintaining a high
lithographic precision for the lateral aperture placement.
[0240] As discussed with respect to Embodiments 1 to 3,
lithographic procedures for producing apertures and/or filters are
well known for polymer on glass WLO, e.g., spin on photoresist,
expose desired areas through a correspondingly structured
photomask, develop unexposed or exposed--depending on whether a
positive or negative photoresist is used--areas away; either the
photoresist itself is the opaque layer the apertures are structured
in, or the (CFA) filter; or the photoresist is a protective layer
for a previously applied metal or dielectric coating, which
prevents the etching away of that material at the desired areas
when the wafer is placed into an etchant. However, for a monolithic
lens typically the monolithic lenses are replicated as double-sided
lenses. As a result of the unusual topography, these WLO techniques
cannot be applied since a plano surface is needed for
lithography.
[0241] The current invention is directed to an optical arrangement
and process for producing such monolithic lenses formed of either
polymer or glass with embedded apertures and filters. One
embodiment of the invention is shown schematically in FIG. 7A. As
shown, in this embodiment, first the thick front-side of the lens
702 is replicated as a plano-convex or plano-concave element.
Preferably, the front-side stamp, which also holds the lens
profiles, additionally contains alignment marks (as described above
in Embodiment 3) that are further used in the other manufacturing
steps to aide in the precise alignment of the various elements to
the overall optical arrangement. Because the backside 704 stamp in
this initial step may be simply a highly flat and/or highly
polished plate, no precise lateral alignment of the two stamps is
required, only wedge error compensation as well as the correct
thickness needs to be ensured. These modest requirements simplify
this initial process step considerably.
[0242] Once the first lens element of the arrangement is complete,
apertures 706 and filters 707 are applied on the plano back-side
704 of this lens element. As these apertures must be precisely
aligned, it is preferable if the front-side of the lens element is
provided with alignment features (not shown) that can be used
during manufacture to assist in positioning the apertures with
respect to the lens by aligning the alignment marks in the
photomask of the apertures and/or filters to the complementary
alignment marks within the first lens layer. Alignment marks, which
may be of any suitable design, provide the benefit of allowing much
higher lateral alignment accuracy (few .mu.m) compared to having
the aperture stop in the external housing, which has a typical
lateral alignment precision of several 10-20 .mu.m.
[0243] Once the apertures/filters 706/707 are positioned on the
back-side 704 of the first lens surface, the second lens surface
708 is replicated on the plano back-side 704, of the first lens
surface 700. This second lens surface can be aligned either based
on the alignment features in the first lens front-side 702, or
based on alignment features within the aperture layer. However, it
should be understood that aligning the second lens surface to the
front-side of the first lens surface is preferred since the
precision is expected to be better due to reduced error propagation
when referring to this initial surface.
[0244] Although FIG. 7A provides an embodiment of a desired lens
element with an embedded aperture in accordance with the current
invention, a number of modifications or additional elements may be
incorporated into the invention. For example, multiple filters 712,
even having different physical natures, can be stacked on each
other as shown in FIG. 7B. It should be understood that any desired
filter may be applied in this manner, including, for example, a CFA
filter or a structured IRCF filter.
[0245] In addition, although the materials used in forming the
first 714 and second lens 716 surfaces have not been specified, it
should be understood that the different replications may be formed
from any suitable material, and that the material may be the same
for both replications (as shown in FIG. 7C) or two different
materials (as shown in FIG. 7D). If the same lens materials are
used for the first and second replication, the inside lens surface
optically vanishes (in other words: it is not visible to the light
and thus provides no refraction and consequently is free of any
Fresnel reflection losses). However, making the two replications
from two different materials provides yet another degree of freedom
in manufacturing the optical arrangements, especially for
achromatization correction if the Abbe numbers of the two materials
are different (See Embodiment 6).
[0246] Although only a single lens element of a single lens channel
of a potential array camera or wafer arrangement is shown, it
should be understood that the monolithic lenses (polymer or glass)
formed in accordance with the current invention may be duplicated
as necessary to form the plurality of lens stacks needed for the
array camera, and that the monolithic lenses may be combined with
other lens elements to realize an optical arrangement having the
desired characteristics. For example, FIGS. 7E, F and G provide
schematic diagrams for monolithic lens arrangements suitable for
array camera architectures. FIGS. 7E and 7F show two different
monolithic doublet designs, while FIG. 7G shows a triplet design.
The arrows in the diagrams indicate where an embedded system
aperture or "stop" has been disposed between the monolithic lenses
on one of the planar surfaces of the monolithic lenses.
[0247] In summary, while there is no doubt that monolithic lens WLO
is very attractive for manufacturing optics for cheap miniaturized
cameras, current methods are adapted directly from techniques used
for injection molded lenses. As a result, several benefits of
polymer-on-glass WLO are not used, leading to alignment accuracy
problems as well as to limited lens design degrees of freedom. The
combination of monolithic lenses and lithographic technologies
described in the current embodiment allows for the manufacture of
precise apertures and additional structures for monolithic lenses
and their alignment to the monolithic lenses. This, in turn, allows
for greater flexibility in the choice of the z-position for
aperture stop and filters, increased lateral accuracy of the
lens-aperture alignment when compared to conventional stops that
are integrated into the lens housing, and the plano intermediate
surface of the monolithic lens allows application of lithographic
technologies for structuring the apertures while maintaining the
benefits of the monolithic lens over the polymer on glass WLO. In
particular, these structures may allow for the implementation of
optical arrangements that allow for contrast at spatial frequencies
that are at least as great as the resolution of the high resolution
images synthesized by the array camera, and significantly greater
than the Nyquist frequency of the pixel pitch of the pixels on the
focal plane, which in some embodiments may be 1.5, 2 or 3 times the
Nyquist frequency.
Embodiment 5
Three-Element Monolithic Lens Design
[0248] This embodiment of the invention provides yet another
alternative for aperture and filter placement within the lens stack
of polymer or glass WLO monolithic lenses. As described above with
respect to Embodiments 3 and 4, the current state of the art for
producing monolithic lens optical arrays is to stack the
independently replicated monolithic lens wafers, dice the sandwich,
and package the lens cubes into an opaque housing which contains
the aperture stop as an integral part at the front of the array.
This methodology limits the degrees of freedom for the optical
design of the objective, as well as making it extremely difficult
to accurately align the lenses with respect to the aperture
stop.
[0249] As described in both Embodiment 3 and 4, a major problem of
the monolithic lens process is that there is no suitable method to
provide a precise system aperture (array) as well as (color- or IR
cut-off-) filters within the lens stack of WLO monolithic lenses.
The current embodiment provides another alternative to bring a
lithographically fabricated aperture (stop), as well as filters
into a polymer or glass WLO monolithic lens stack. In particular,
this embodiment builds on the design introduced in Embodiment 4, in
which one element of the lens design is forced to have a plano
surface where the aperture and filters can be lithographically
structured. As described above, in such an embodiment the plane
side of the either plano-convex or plano-concave element can be
used as substrate for the subsequent lithography step. The current
embodiment provides a three-element optical arrangement using this
plano-element monolithic design.
[0250] As shown schematically in FIG. 8A, the basic three-element
design of the instant embodiment is characterized by the following
properties: [0251] A first plano-convex lens element 800 that has a
convex first surface 802 as well as a plane second side 804
carrying the system aperture stop as well as required filter
structures. Preferably, this first element is made from a first
(low dispersion, low refractive index) lens material. [0252] A
second concave-convex lens element 806 that has a concave first
surface 808, bent towards the object side and a convex second
surface 810, where the concave first surface 808 is very shallow
and this concave surface is very close to the plane (second)
surface 804 of the first element 800. Again, preferably this lens
element is made from a first (low dispersion, low refractive index)
lens material. In addition, in a preferred embodiment, the surface
profile of this shallow concave surface 808 close to the system
aperture stop 804 is the one optimized/adapted to the specific
narrow spectral band of the different color channels of an array
camera. [0253] A third menisc-lens element 812 that has a concave
first surface 813 and a convex second surface 814, both bent
towards the object side. This lens is preferably a strongly bent
concave-convex lens that is made from a second (high dispersion,
high refractive index) lens material. This third lens element is
disposed adjacent to the image sensor cover glass 816, which itself
is placed in above the image sensor 817.
[0254] This design has two significant advantages, first, a plane,
substrate-interface-like, surface (e.g., surface 804 in FIG. 8A) is
introduced in the lens stack. This plane,
substrate-interface-like-surface can be used to apply a highly
accurate (sub- or few-micron centering tolerance) aperture stop by
photolithography. This is a major improvement in precision as the
current state-of-the-art (screen printing)) has a centering
tolerance of around 20 .mu.m, which is insufficient for high image
quality array cameras. In addition, color filters (CFA) and/or
dielectric filters (IRCF) or other structures, which need a planar
substrate, can be applied to this planer surface.
[0255] Second, the design provides a surface which is very close to
the aperture stop (first surface of the second element) whose
surface profile can be optimally adapted to the specific narrow
spectral band of the different color channels of an array camera,
as will be described in greater detail below with reference to the
data-plots in FIGS. 8E to 8J, below.
[0256] Although one specific embodiment of the three-element
monolithic lens design is shown in FIG. 8A, it should be understood
that there are many different implementations of the above general
design principle, as shown and described in FIGS. 8B to 8D, below.
In particular, FIG. 8B shows a modification of the basic optical
arrangement in which the curvature at edges 818 of the second
surface 820 of the second lens element 806, and at the edges 820 of
both surfaces of the third lens element 812 quickly change slope
towards these edges. Such a design has the benefit of allowing for
a decrease in the steepness at the edge of the first surface of the
third element. FIG. 8C shows a modification of the basic optical
arrangement in which the first element 800 is thinner and the third
element 812 is in consequence made thicker. Such a design, however,
requires an increase in the steepness of the first surface 822 of
the third element 812. FIG. 8D shows a modification of the basic
optical arrangement in which the first element 800 is thinner and
the third element 812 is thicker, and where the curvature of the
surface at the edge 818 of the second surface 820 of the second
element 806 quickly changes slope towards the edge formed thereof.
Again, this design has the benefit of decreasing the steepness of
the first surface of the third element 812.
[0257] In another alternative embodiment that can be applied to any
of the arrangements described above, the first lens element can be
made as a polymer-on-glass wafer level lens instead of a (polymer
or glass) monolithic lens. This would mean that there would be a
(comparatively thick) glass substrate where the aperture stop and
filters would be lithographically applied to the second side
thereof, and the first lens surface would be replicated on the
first side. This "hybrid lens" would then be stacked with the
second and third lens elements, which would both be fabricated by a
monolithic lens process. Alternatively, the second lens element
could be a hybrid lens in which the polymer lens surfaces would be
replicated on both sides of a thinner glass substrate. However, the
third lens element would always be monolithic due to the
menisc-nature of this lens. There are several advantages of this
combination of technologies, namely: [0258] The first lens is a
comparatively thick element with a plane backside and a shallow
front lens surface, so little is lost functionally by inserting the
glass substrate. [0259] The use of the substrate provides
additional robustness/stability/planarity during the application of
the aperture and filters due to the presence of the glass
substrate. In addition, the first lens surface quality can be
improved due to the stable glass substrate it is replicated on.
[0260] There is less (especially lateral) thermal expansion than
with a purely monolithic lens since the thick glass substrate with
about 1/10.sup.th of the CTE of the polymer serves as a permanent
carrier of the overall lens stack providing the majority of the
mechanical integrity.
[0261] FIGS. 8E to 8J provide data plots showing the optical
properties of these novel three-element monolithic optical
arrangements. In particular, FIGS. 8E to 8H, provide plots of MTF
vs field (8E), Strehl ratio vs. field (8F), distortion and field
curvature (8G) as well as MTF vs. spatial frequency (8H) of the
lens design shown in FIG. 8A for a green channel. Meanwhile, FIGS.
81 & 8J provide plots of MTF vs field (81) and Strehl ratio vs.
field plots (8J) of the corresponding blue channel of the design
shown in FIG. 8A. It should be noted that only the surface profile
of the first surface 808 of the second element 806 needs to be
altered to optimize the optical arrangement for a different color
channel. As can be seen from this data, the three-element
monolithic optical arrangement provides high image quality (See,
e.g., FIGS. 8E to 8H) comparable to that of a design using a
field-flattening element (such as e.g. applied in Embodiment 2
above). Moreover, because only three lens elements need to be
stacked in the current design it is much more suitable for
manufacture using a monolithic method compared to complex
conventional multi-element optical arrangements.
[0262] The lens material sequence (i.e., in the above embodiment
high Abbe number, high Abbe number, low Abbe number) for the
positive, positive, negative elements provides an efficient way of
achromatization for each considered channel's spectral band (See
Embodiment 6). For example, even for regular dispersion materials
the blue channel performance seen in the exemplary embodiment is
much better than can be obtained for regular designs (See, e.g.,
FIGS. 8I and 8J). Moreover, even though for array cameras each
channel only has to perform well for a comparatively narrow
spectral band, this achromatization still increases the performance
since both the central wavelength and the wavelengths at the sides
of the used spectral band of the considered channels are imaged
sharply.
[0263] In many embodiments the MTF characteristics of the three
element monolithic optical arrangement allow for contrast at
spatial frequencies that are at least as great as the resolution of
the high resolution images synthesized by the array camera, and
significantly greater than the Nyquist frequency of the pixel pitch
of the pixels on the focal plane, which in some embodiments may be
1.5, 2 or 3 times the Nyquist frequency.
Embodiment 6
Different Lens Material Sequences for Channels that Work with
Different Spectral Bands
[0264] Although the above embodiments have focused on specific
optical arrangements, it will be understood that the current
invention is also directed to novel methods and materials for
modifying the optical properties of the various lens elements of
these novel optical arrangements. For example, in a first such
embodiment, the invention is directed to the use of different lens
materials (or combinations thereof) for different color
channels.
[0265] As shown in FIGS. 9A, 9B and 9D, using conventional array
optics, channel specific color-focusing in an array camera so far
is limited to adjusting at least one surface, (i.e., the surface
profiles of front- and/or backside for lens 900 or lens 902) for
channel-specific correction of the back-focal-length (BFL) for
axial color. However, the material sequence for the different
elements of the lens channels is always the same, independently of
the color the considered channel is supposed to work for. It should
be noted that the only difference between the lens arrays in FIGS.
9A and 9B is that in FIG. 9B the supporting structure of the array
is opaque and going through the full length of a channel. (For
reference the specific color channel red "R", green "G", or blue
"B" is indicated by the letter in the schematics provided.)
However, while there are several materials with high refractive
index, which are beneficial in achieving strong refractive power
with shallow lens profiles, these materials usually also show high
spectral dispersion. In particular, typically one has a choice
between high refractive index and low Abbe number (high dispersion)
materials ("flint-like"), and low refractive index and high Abbe
number (low dispersion) materials ("crown-like"). Indeed, for
(lens-) polymers the above connection is always valid, dispersion
always increases with increasing refractive index. If this physical
connection of the two material properties was not the case, from an
optical design standpoint the choice would usually be made to use a
high index material (so that the surface of the lens can be
shallow, while still maintaining strong optical power) with low
dispersion (so that the difference in refractive power for
different wavelengths would be small). However, as stated above,
such polymer materials are not available, so one has to choose if
the priority is on either one of the two properties.
[0266] While using a flint-like material can be acceptable for the
green and red channels, it can impact the blue spectral band
disproportionately, because dispersion is related to the change of
refractive index with wavelength and usually this change is
stronger in the blue spectral band than in the green and red ones.
In short, while green and red channels would profit from the use of
such a high index material, the blue channel would show too strong
axial color aberration due to the related large dispersion. The
current embodiment takes advantage of the array nature of the
camera to allow the use of a different material sequence in the
blue channel (as shown in FIG. 9C), which may be less optimal with
regard to refractive index, but shows much less spectral
dispersion. Using such a method makes it possible to adapt one or
more lens profiles to optimize a channel to its respective spectral
band, and to optimize the material sequence used, e.g., here
changing the material sequence for the blue channel.
[0267] It should be understood that the ability to modify the
material sequence to optimize it for a specific color channel may
be used in injection molded lenses (as shown in FIG. 9C) or with a
specific type of polymer on glass "WLO" lenses (as shown in FIGS.
9D and 9E) (where the lens material is dispensed in separated
islands prior to the replication (e.g. by some device similar to an
ink jet) other than with the wafer scale puddle dispense). For
example, in an injection molding process a "crown-like" polymer
material would, e.g., be PMMA, Zeonex (COP) and Topas (COC), and a
"flint-like" material would be Polycarbonate (PC) and Polystyrene
(PS). Finally, as described in reference to the invention more
broadly, the material sequence may also be modified in glass molded
lenses as well.
Embodiment 7
Polymer on Glass WLO Novel Aperture Stop
[0268] Again, although the above embodiments have focused on
specific optical arrangements, it will be understood that the
current invention is also directed to novel methods and materials
for modifying the optical properties of the various lens elements
of these novel optical arrangements. In a second such embodiment,
the invention is directed to a novel arrangement that could be used
in any polymer on glass WLO, in which the aperture stop is disposed
on a separate substrate in the air spacing between lenses.
[0269] As shown schematically in FIG. 10A, in the conventional
polymer on glass WLO, apertures 1000, and in particular the
aperture stop, is structured on the supporting glass substrate
1002, and then the lenses 1004 and 1006 are replicated above the
aperture. In the current embodiment, an additional layer 1010 is
introduced between the lens substrates 1012 and 1014, upon which
the aperture stop 1016 is disposed. In such an embodiment, the
apertures may be made using any suitable technique, such as, for
example, transparent openings in an opaque layer (e.g. metal, metal
oxide or opaque photoresist) on a thin (glass) substrate, be
(metal) etch mask, etc. Positioning the aperture in the air space
between the lenses as an additional diaphragm, or as an aperture on
very thin (glass) sheet, rather than forcing it to be on the
substrate under the polymer lens yields a number lens designs
benefits in terms of MTF performance. In contrast, constraining the
apertures to the substrate surfaces for a large variety of lens
designs reduces performance by 5-10% over the full field.
Embodiment 8
Polymer Injection- or Precision Glass Molded Lens Arrays
[0270] Again, although the above embodiments have focused on
specific optical arrangements, it will be understood that the
current invention is also directed to novel methods of
manufacturing the various lens elements of these novel optical
arrangements. In a third such embodiment, the invention is directed
to a novel method of manufacturing optical arrangements for use in
camera arrays in which stand-offs and mechanical self-alignment
features for assembly are included in the manufacture of the
lenses.
[0271] In conventional polymer injection- or precision glass
molding techniques, a cavity for producing one lens array (front
and back side) is provided. The mold cavity is filled with a
suitable material, such as, for example, PMMA or polycarbonate for
polymer injection molding or preferably "low-Tg-glasses" such as
e.g. P-BK7 or P-SF8 for precision glass molding. Then for
conventional camera assembly alignment barrels are used in which
the molded lenses are stacked and glued together. In an array
camera this method does not provide sufficient alignment precision.
The current invention proposes a method in which mechanical
alignment features are provided in the lens mold. In other words,
during the polymer injection- or precision glass molding process,
not only the lens features are replicated into the array, nor even
optical alignment marks, but also small mechanical features are
formed into the front- and back-faces of the elements, which allow
mechanical self-alignment with the adjacent array, such as, for
example, complementary rings and spherical segments, pins and
holes, cones and pyramids with complementary (and correspondingly
shaped) cavities on the opposing element.
[0272] Two such embodiments are shown in FIGS. 11A and 11B. For
example, FIG. 11A shows an example in which spacing
structures/stand-offs 1100 are included in injection molding
process of the lens array 1102. For polymer injection molding it is
desirable in such an embodiment that the material combination
choice and spacer thickness provide athermalization. In short, it
is desirable that the do/dT of the lens material is compensated by
the CTE of spacer. Alternatively, the same technique may be used in
independently fabricated spacer or hole matrix structures. In such
an embodiment, as shown in FIG. 11B an (opaque) cavity array 1104
is used as the supporting structure into which the lenses 1106 are
replicated.
Embodiment 9
Waveplate or Multilevel Diffractive Phase Elements
[0273] In yet another embodiment, the invention is directed to a
waveplate or multilevel diffractive phase element ("kinoform") for
channelwise correction of chromatic aberrations in an array camera,
and an iterative fabrication process tolerance compensation.
[0274] Currently one of the three or four lens surfaces in the
objective is channel-wisely optimized in order to correct for
chromatic aberrations of the specific channel (see, e.g., U.S Pat.
Pub. No. US-2011-0069189-A1, the disclosure of which is
incorporated herein by reference). For this a special mastering
regime for the array tool is required since slightly different
lenses need to be fabricated within one array. The overall lens
property can be considered as the sum of the average required
shape, and the individual color correction. However, the total
profile has to be implemented by machining, which adds difficulty
for diamond turning mastering techniques. (See, e.g., U.S. patent
application Ser. No. 13/050,429, the disclosure of which is
incorporated herein by reference.)
[0275] Lens design experiments show that it could be beneficial to
separate the channel-averaged optical power from the
channel-specific optical power, which is then related to the color
correction. The current embodiment is directed to an optical
arrangement that accomplishes this channel-wise correction using a
channel-specific surface that introduces only a minor wavefront
deformation of exactly the size needed to distinguish the channels
from each other so that they are perfectly adapted to their
individual waveband. The wavefront deformation required for this is
typically on the order of only several wavelengths. As a result,
this surface can either be a very shallow refractive surface
("waveplate"), a "low frequency" diffractive lens ("kinoform") or
radial symmetric multilevel diffractive phase element. As a result,
there is no longer a need to machine slightly different lenses
(e.g. by diamond turning), but all the lens surfaces within one
array could be equal. In addition, different technologies can be
used for the origination of arrays of such channel specific
surfaces, including, "classical" lithographic microoptics
fabrication technologies such as laser beam writing, gray scale
lithography, E-beam lithography, binary lithography, etc. Moreover,
these techniques are more suitable for manufacturing slight
differences in the surfaces comprised in the array, they have much
higher lateral precision than mechanical origination means, and
they provide much higher thickness precision (i.e., phase accuracy
of the surface).
[0276] In addition, it is possible to use the above advantages to
compensate for the effects of systematic fabrication errors on
image quality. A flow chart of this manufacturing method is
provided in FIG. 12. As shown, in a first step the optical channels
of the array camera are designed. This at this time includes the
nominal shape of the waveplate or multilevel diffractive phase
element which is used for channel-wise color aberration correction
only. In a second step the array lens module is fabricated by a
suitable means (as described above). Then in step three, the
systematic deviation of the lens prescriptions from design
expectations are determined by lens surface metrology, centering-
and distances-measurements are performed, as well as the systematic
deviation of optical performance from design expectation are
experimentally determined. The module is then redesigned (Step
Four) by adapting the above aberration correcting surfaces in order
to compensate for all determined systematic errors elsewhere in the
stack (profile, xy-position, thickness, etc.). In step six the
array lens module is re-fabricated. And finally, the back focal
length is used as a last compensator for all remaining systematic
deviations (Step Seven). The advantage of this method is that there
are more degrees of freedom, rather then being able to change back
focal length only, as is the case in conventional system. This
leads to better overall performance, potentially without impacting
optical magnification.
GENERAL CONSIDERATIONS
[0277] Finally, it will be understood that in any of the above
embodiments, multiple identical or slightly varied versions of such
optical arrangements may be collocated next to each other in an
array. The variation of the optical arrangements within an optical
array is related to e.g. one of the following optical performance
parameters of the considered channel: "color" (identifying the
narrow spectral band the considered optical channel is supposed to
image of the overall spectral band the whole system shall image),
e.g. RGB (and NIR), field of view (FOV), F/#, resolution, object
distance, etc. Most typical is the differentiation into different
colors, but different FOVs, for example, would allow for different
magnifications while different F/#s would allow for different
sensitivities and so forth. In many embodiments the MTF
characteristics of the optical arrangements are configured to allow
for contrast at spatial frequencies that are at least as great as
the resolution of the high resolution images synthesized by the
array camera, and significantly greater than the Nyquist frequency
of the pixel pitch of the pixels on the focal plane, which in some
embodiments may be 1.5, 2 or 3 times the Nyquist frequency.
DOCTRINE OF EQUIVALENTS
[0278] While particular embodiments and applications of the present
invention have been illustrated and described herein, it is to be
understood that the invention is not limited to the precise
construction and components disclosed herein and that various
modifications, changes, and variations may be made in the
arrangement, operation, and details of the methods and apparatuses
of the present invention without departing from the spirit and
scope of the invention as it is defined in the appended claims.
* * * * *