U.S. patent application number 12/826313 was filed with the patent office on 2011-12-29 for image sensor with dual layer photodiode structure.
This patent application is currently assigned to Aptina Imaging Corporation. Invention is credited to Yingjun Bai, Qun Sun.
Application Number | 20110317048 12/826313 |
Document ID | / |
Family ID | 45352206 |
Filed Date | 2011-12-29 |
![](/patent/app/20110317048/US20110317048A1-20111229-D00000.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00001.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00002.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00003.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00004.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00005.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00006.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00007.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00008.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00009.png)
![](/patent/app/20110317048/US20110317048A1-20111229-D00010.png)
View All Diagrams
United States Patent
Application |
20110317048 |
Kind Code |
A1 |
Bai; Yingjun ; et
al. |
December 29, 2011 |
IMAGE SENSOR WITH DUAL LAYER PHOTODIODE STRUCTURE
Abstract
An image system with a dual layer photodiode structure is
provided for processing color images. In particular, the image
system can include an image sensor that can include photodiodes
with a dual layer photodiode structure. In some embodiments, the
dual layer photodiode can include a first layer of photodiodes
(e.g., a bottom layer), an insulation layer disposed on the first
layer of photodiodes, and a second layer of photodiodes (e.g., a
top layer) disposed on the insulation layer. The first layer of
photodiodes can include one or more suitable pixels (e.g., green,
blue, clear, luminance, and/or infrared pixels). Likewise, the
second layer of photodiodes can include one or more suitable pixels
(e.g., green, red, clear, luminance, and/or infrared pixels). An
image sensor incorporating dual layer photodiodes can gain light
sensitivity with additional clear pixels and maintain luminance
information with green pixels.
Inventors: |
Bai; Yingjun; (San Jose,
CA) ; Sun; Qun; (San Jose, CA) |
Assignee: |
Aptina Imaging Corporation
Grand Cayman
KY
|
Family ID: |
45352206 |
Appl. No.: |
12/826313 |
Filed: |
June 29, 2010 |
Current U.S.
Class: |
348/294 ;
257/443; 257/E27.133; 348/E5.091 |
Current CPC
Class: |
H01L 27/1461 20130101;
H04N 9/04557 20180801; H01L 27/14652 20130101; H01L 27/14605
20130101; H04N 9/045 20130101; H04N 5/332 20130101; H04N 5/369
20130101; H01L 27/14621 20130101; H04N 9/04515 20180801; H01L
27/14607 20130101; H01L 27/14647 20130101; H04N 5/33 20130101 |
Class at
Publication: |
348/294 ;
257/443; 348/E05.091; 257/E27.133 |
International
Class: |
H04N 5/335 20060101
H04N005/335; H01L 27/146 20060101 H01L027/146 |
Claims
1. A plurality of photodiodes for receiving light, the plurality of
photodiodes comprising: a bottom layer of photodiodes; a top layer
of photodiodes, wherein at least one pixel in the top layer of
photodiodes has a different spectral response than a pixel at the
same position in the bottom layer of photodiodes; and an insulation
layer disposed between the bottom layer of the photodiodes and the
top layer of the photodiodes.
2. The plurality of photodiodes of claim 1, wherein the bottom
layer of the photodiodes comprises a bottom pixel at a first
position, and wherein the top layer of the photodiodes comprises a
top pixel stacked on top of the bottom pixel at the first
position.
3. The plurality of photodiodes of claim 2, wherein the bottom
pixel is a red pixel and the top pixel is a blue pixel.
4. The plurality of photodiodes of claim 2, wherein the bottom
pixel is a first green pixel and the top pixel is a second green
pixel.
5. The plurality of photodiodes of claim 2, wherein the bottom
pixel is a first clear pixel and the top pixel is a second clear
pixel.
6. The plurality of photodiodes of claim 2, wherein the bottom
pixel is a first luminance pixel and the top pixel is a second
luminance pixel.
7. The plurality of photodiodes of claim 2, wherein the bottom
pixel is a first infrared pixel and the top pixel is a second
infrared pixel.
8. The plurality of photodiodes of claim 2, wherein the bottom
pixel is an infrared pixel and the top pixel is one of a color
pixel, a clear pixel, or a luminance pixel.
9. The plurality of photodiodes of claim 1, wherein the first array
of the photodiodes and the second array of the photodiodes are
arranged in an adamantine pattern.
10. The plurality of photodiodes of claim 1, wherein the bottom
layer of the photodiodes and the top layer of the photodiodes are
arranged in a rectilinear pattern, wherein at least one line of the
bottom layer and the top layer comprises a plurality of pixels of
the same color.
11. The plurality of photodiodes of claim 10, wherein the at least
one line is at least one row or at least one column.
12. The plurality of photodiodes of claim 1, wherein the bottom
layer, the insulation layer, and the top layer have a
pre-determined orientation.
13. The plurality of photodiodes of claim 1, wherein the
pre-determined orientation is 45.degree..
14. An image sensor, the image sensor comprising: a plurality of
photodiodes with a dual layer photodiode pixel structure; and an
infrared ("IR") cutoff filter, wherein the IR cutoff filter is
positioned over at least a portion of the plurality of
photodiodes.
15. The image sensor of claim 14, further comprising a color filter
array ("CFA"), and wherein the CFA comprises one or more of a
magenta filter element and a green filter element.
16. The image sensor of claim 14, wherein the plurality of
photodiodes comprises one or more infrared pixels, and wherein the
IR cutoff filter covers the one or more infrared pixels.
17. The image sensor of claim 14, wherein the plurality of
photodiodes comprises one or more color pixels and one or more
clear pixels.
18. The image sensor of claim 17, wherein the IR cutoff filter
covers only the one or more color pixels.
19. The image sensor of claim 14, wherein the dual layer photodiode
pixel structure comprises a bottom layer of photodiodes and a top
layer of photodiodes.
20. The image sensor of claim 19, wherein the bottom layer
comprises a set of red pixels at one or more positions and the top
layer comprises a set of blue pixels stacked on top of the set of
red pixels at the one or more positions.
21. The image sensor of claim 19, wherein the bottom layer
comprises a set of green pixels at one or more positions and the
top layer comprises a set of blue pixels stacked on top of the set
of green pixels at the one or more positions.
22. The image sensor of claim 19, wherein the bottom layer
comprises a set of red pixels at one or more positions and the top
layer comprises a set of green pixels stacked on top of the set of
red pixels at the one or more positions.
23. The image sensor of claim 19, wherein the bottom layer
comprises a set of red pixels and a first set of clear pixels, and
the top layer comprises a set of blue pixels and a second set of
clear pixels.
24. The image sensor of claim 19, wherein the bottom layer
comprises a set of red pixels and a first set of green pixels, and
the top layer comprises a set of blue pixels and a second set of
green pixels.
25. The image sensor of claim 19, wherein the bottom layer
comprises a set of red pixels, a first set of green pixels, and a
first set of clear pixels, and the top layer comprises a set of
blue pixels, a second set of green pixels, and a second set of
clear pixels.
26. The image sensor of claim 14 further comprising at least one
photodiode with a single layer photodiode pixel structure, wherein
the plurality of photodiodes is positioned adjacent to the at least
one photodiode.
27. A method for binning a plurality of photodiodes, the method
comprising: binning a set of first color pixels of the plurality of
photodiodes along a first direction to form a first pixel group,
wherein the plurality of photodiodes comprises a dual layer
photodiode pixel structure; binning a set of second color pixels
and third color pixels of the plurality of photodiodes along a
second direction to form a second pixel group; combining the first
pixel group and the second pixel group to form a binned pixel
cluster; and interpolating the binned pixel cluster to output a
pixel image.
28. The method of claim 27, wherein the dual layer photodiode pixel
structure is arranged in an adamantine pixel arrangement.
29. The method of claim 27, wherein the first color pixels are
green pixels, the second color pixels are blue pixels, and the
third color pixels are red pixels.
30. The method of claim 27, wherein the pixel image comprises a
rectilinear grid.
31. The method of claim 27, wherein the first direction is
orthogonal to the second direction.
32. The method of claim 27, wherein the first direction is along a
horizontal direction and the second direction is along a vertical
direction.
33. The method of claim 27, wherein the first direction is along a
-45.degree. direction and the second direction is along 45.degree.
direction.
Description
FIELD OF THE INVENTION
[0001] This is directed to an image sensor with a dual layer
photodiode structure.
BACKGROUND OF THE DISCLOSURE
[0002] Image sensors are used in many different types of electronic
devices to capture an image. For example, modern cameras (e.g.,
video cameras and digital cameras) and other image capturing
devices use image sensors to capture an image.
[0003] Image sensors typically have color processing capabilities.
For example, an image sensor can include a color filter array
("CFA") that can separate various colors from a color image. The
resulting output from the image sensor can then be interpolated to
form a full color image.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 is a schematic view of an illustrative electronic
device configured in accordance with embodiments of the
invention.
[0005] FIG. 2A is a cross-sectional view of a first illustrative
image sensor in accordance with embodiments of the invention.
[0006] FIG. 2B is a cross-sectional view of a second illustrative
image sensor in accordance with embodiments of the invention.
[0007] FIG. 3 is a cross-sectional view of a third illustrative
image sensor in accordance with embodiments of the invention.
[0008] FIG. 4 is a cross-sectional view of a fourth illustrative
image sensor in accordance with embodiments of the invention.
[0009] FIG. 5 is a representation of a typical color filter array
("CFA") in a Bayer pattern.
[0010] FIGS. 6A-6L are representations of pixel arrangements in
accordance with embodiments of the invention.
[0011] FIG. 7 is a flowchart of an illustrative process for binning
photodiodes in accordance with embodiments of the invention.
DETAILED DESCRIPTION OF THE DISCLOSURE
[0012] FIG. 1 is a schematic view of an illustrative electronic
device configured in accordance with embodiments of the invention.
Electronic device 100 can be any type of user device that utilizes
an image sensor (embodied here as image sensor 110) and is
controlled generally by control circuitry 120. For example,
electronic device 100 can include a camera, such as a computer
camera, still camera, or portable video camera. Electronic device
100 can also include any other components in a typical camera (or
otherwise), which are not depicted in FIG. 1 to avoid any
distractions from embodiments of the invention.
[0013] Image sensor 110 can capture image data corresponding to a
streaming image. For example, image sensor 110 can include any
combination of lenses and arrays of cells (e.g., charge-coupled
devices ("CCDs") or complementary metal oxide semiconductor
("CMOS") sensor cells) for capturing light.
[0014] Control circuitry 120 may process data generated by image
sensor 110, and may perform any suitable operations based on this
data. For example, control circuitry 120 can obtain multiple color
pixels (e.g., red, green, and/or blue pixels) generated by image
sensor 110. Upon obtaining the color pixels, control circuitry 120
can optionally bin the color pixels in one or more dimensions to
form one or more pixel groups (e.g., one or more red, green, and/or
blue pixel groups). After binning the multiple color pixels,
control circuitry 120 can interpolate the one or more pixel groups
to output a pixel image. As used herein, a "pixel" can refer to a
photodiode and/or a filter that is capable of responding to one or
more wavelengths of light (e.g., responding to one or more colors).
For example, a "color pixel" can correspond to a photodiode or a
photodiode and filter combination capable of absorbing green, blue,
or red wavelengths of light. As another example, a "clear pixel"
can correspond to a photodiode or a photodiode and filter
combination capable of absorbing all visible light or both visible
light and near infrared signals. As yet another example, an
"infrared pixel" can correspond to a photodiode or a photodiode and
filter combination capable of absorbing near infrared and infrared
signals.
[0015] Image sensor 110 and control circuitry 120 may be
implemented using any suitable combination of hardware and
software. In some embodiments, image sensor 110 can be implemented
substantially all in hardware (e.g., as a system-on-a-chip
("SoC")). This way, image sensor 110 can have a small design that
minimizes the area occupied on electronic device 100. In addition,
image sensor 110 may have circuit components designed to maximize
the speed of operation. Control circuitry 120 may include, for
example, one or more processors, microprocessors, ASICS, FPGAs, or
any suitable combination of hardware and software.
[0016] Turning now to FIGS. 2-4, these figures show illustrative
image sensors in accordance with embodiments of the inventions. It
will be understood that, for the sake of simplicity, not all of the
layers of an image sensor are shown in FIGS. 2-4. For example,
there may be metal interconnect layers formed between the layers
shown and additional dielectric layers for insulation purposes. It
will also be understood that image sensors of FIGS. 2-4 can include
additional photodiodes not shown in FIGS. 2-4. It will further be
understood that FIGS. 2-4 are not drawn to scale.
[0017] Turning first to FIG. 2A, a cross-sectional view of image
sensor 200 is shown. Image sensor 200 can be the same as or
substantially similar to image sensor 110 of FIG. 1. Image sensor
200 can include substrate 202 that can incorporate multiple
photodiodes 204 (e.g., photodiodes 205-210). In some embodiments,
image sensor 200 can include a color filter array ("CFA") (not
shown in FIG. 2A). Photodiodes 204 can convert light into an
electrical light signal that can then be processed by control
circuitry (e.g., control circuitry 120 of FIG. 1) of an electronic
device (e.g., electronic device 100 of FIG. 1). In particular, the
level of intensity of light striking photodiodes 204 can affect the
magnitude of the electronic signal that can be read by the control
circuitry. For example, a higher intensity of light striking
photodiodes 204 can generate an increase in the amount of charge
collected by photodiodes 204.
[0018] In some embodiments, photodiodes 204 can have a dual layer
photodiode pixel structure. Thus, photodiodes 204 can include a
first layer (e.g., a bottom layer) of photodiodes such as, for
example, photodiodes 205, 207, and 209. In addition, photodiodes
204 can include a second layer (e.g., a top layer) of photodiodes
such as, for example, photodiodes 206, 208, and 210.
[0019] In some embodiments, the second layer of photodiode may not
be disposed uniformly on top of all photodiodes in a first layer of
photodiodes. For example, FIG. 2B shows a cross-sectional view of
image sensor 230, which can include photodiodes 240. Image sensor
230 can be the same as or substantially similar to image sensor 110
of FIG. 1 and image sensor 200 of FIG. 2A.
[0020] At some photodiode position(s) in image sensor 230,
photodiodes 240 may include only one thick integrated layer of
photodiodes. For instance, as shown in FIG. 2B, photodiodes 242 and
244 can have a single layer photodiode pixel structure. However, at
other photodiode position(s) in image sensor 230, a second layer of
photodiodes may be present, which may be positioned adjacent to
photodiodes 242 and 244. For instance, as shown in FIG. 2B, a
second layer of photodiodes (e.g., photodiode 246) can be disposed
on top of a first layer of photodiodes (e.g., photodiode 248). In
some cases, the thickness of each of photodiodes 242 and 244 may be
the same as or substantially similar to the combined thickness of
photodiodes 246 and 248.
[0021] In some embodiments, an image sensor (e.g., image sensor 200
of FIG. 2A and/or image sensor 230 of FIG. 2B) can include
insulation layer 212, which can be disposed between the first layer
of the photodiodes and the second layer of the photodiodes.
Insulation layer 212, which can provide insulation between the two
layers, can be formed using any suitable dielectric material such
as, for example, paper, plastic, glass, rubber-like polymers, and
or any other suitable material.
[0022] One or more photodiodes 204 (e.g., a dual layer pair of
photodiodes such as photodiodes 205 and 206, photodiodes 207 and
208, or photodiodes 209 and 210) or phototodiodes 240 (e.g., a dual
layer pair of photodiodes such as photodiodes 246 and 248) can
correspond to color pixels, which can respond to different
wavelengths of light based on the depths that each wavelength of
light can reach inside the silicon. Accordingly, each photodiode of
photodiodes 204 and/or photodiodes 240 may have a different
spectral sensitivity curve.
[0023] The wavelengths of light from longest to shortest can be
ranked as follows: red light, green light, and blue light.
Physically, longer wavelengths of light can reach greater depth
within the silicon. As such, the usual arrangement of photodiodes
204 can include photodiodes in one or more higher layers (e.g., a
top layer) that can respond to shorter wavelengths of light (e.g.,
blue light). Photodiodes 204 can also include photodiodes in one or
more lower layers (e.g., a bottom layer) that can respond to longer
wavelengths of light (e.g., red light). Because longer wavelengths
can be absorbed at greater depth by photodiodes 204 and 240, the
first layer of photodiodes (e.g., photodiodes 205, 207, and 209 of
FIG. 2A and photodiode 246 of FIG. 2B) can absorb red and/or green
wavelengths of light, and the second layer of photodiodes (e.g.,
photodiodes 206, 208, and 210 of FIG. 2A and photodiode 248 of FIG.
2B) can absorb blue and/or green wavelengths of light. Thus,
photodiodes 205, 207, and 209 of FIG. 2A and photodiode 246 of FIG.
2B can correspond to red and/or green pixels and photodiodes 206,
208, and 210 of FIG. 2A and photodiode 248 of FIG. 2B can
correspond to blue and/or green pixels.
[0024] In some embodiments, in addition to or instead of color
pixels, one or more photodiodes 204 and/or photodiodes 240 (e.g., a
dual layer pair of photodiodes such as photodiodes 205 and 206,
photodiodes 207 and 208, or photodiodes 209 and 210 of FIG. 2A
and/or photodiodes 246 and 248 of FIG. 2B) can include one or more
clear pixels. The one or more clear pixels can absorb a combination
of various wavelengths of light (e.g., red, green, and blue
wavelengths of light). In some cases, a single layer of photodiodes
(e.g., a top or bottom layer of photodiodes) can include a
combination of clear pixels and color pixels.
[0025] Because clear pixels can be more sensitive to light as
compared to color pixels, clear pixels can improve the imaging
performance of image sensor 200 under low light conditions. In
addition, clear pixels can enable high speed imaging under normal
lighting conditions.
[0026] In other embodiments, in addition to or instead of color
pixels, one or more photodiodes 204 and/or photodiodes 240 (e.g., a
dual layer pair of photodiodes such as photodiodes 205 and 206,
photodiodes 207 and 208, or photodiodes 209 and 210 of FIG. 2A
and/or photodiodes 246 and 248 of FIG. 2B) can include one or more
luminance pixels (e.g., pixels with brightness information
corresponding to a color image). Luminance pixels can have a
spectral response that is wider than the spectral response of green
pixels, but the spectral response of the luminance pixels can be
narrower than a full response.
[0027] In further embodiments, in addition to or instead of color
pixels, one or more photodiodes 204 and/or photodiodes 240 (e.g., a
dual layer pair of photodiodes such as photodiodes 205 and 206,
photodiodes 207 and 208, or photodiodes 209 and 210 of FIG. 2A
and/or photodiodes 246 and 248 of FIG. 2B) can include one or more
infrared pixels (e.g., pixels capable of absorbing near infrared
and infrared signals).
[0028] In some embodiments, for image devices that use visible
light responses, image sensor 200 can include an infrared ("IR")
cutoff filter 220, which can be positioned over a CFA (not shown in
FIG. 2A or FIG. 2B) or photodiodes (e.g., photodiodes 204 of FIG.
2A and/or photodiodes 240 of FIG. 2B) to ensure correct color
imaging. IR cutoff filter 220 can limit the effects of IR light on
the responses of photodiodes 204. For example, IR cutoff filter 220
can block undesirable IR light from reaching the photodiodes. In
some cases, lens 222, which can be positioned over IR cutoff filter
220, can focus light on the photodiodes. For example, light passing
through lens 222 can pass through IR cutoff filter 220 and fall on
the photodiodes. Lens 222 can include any suitable lens such as,
for example, a single lens or multiple micro-lenses. Thus, in one
configuration, each micro-lens can be formed over one or more
corresponding photodiodes.
[0029] In some embodiments, if one or more photodiodes are infrared
pixels, IR cutoff filter 220 can cover the one or more infrared
pixels, and can thereby behave as a dual band IR cutoff filter. For
example, IR cutoff filter 220 can pass both visible light and
certain portions of the IR light by creating an extra window in the
IR band.
[0030] Turning now to FIG. 3, a cross-sectional view of image
sensor 300 is shown. Image sensor 300 can be the same as or
substantially similar to image sensor 110 of FIG. 1. Image sensor
300 can include substrate 302, photodiodes 304 (e.g., photodiodes
305-310), insulation layer 312, color filter array ("CFA") 314, IR
cutoff filter 316, and lens 318. In one configuration, CFA 314 can
be positioned over photodiodes 304, and IR cutoff filter 316 can be
positioned over CFA 314. Persons skilled in the art will appreciate
that photodiodes 304 can include one or more single layer
photodiode pixel structures, one or more dual layer photodiode
pixel structures, and/or any combination thereof. Persons skilled
in the art will also appreciate that photodiodes 304 can include
any suitable number of layers in a particular photodiode pixel
structure.
[0031] CFA 314 can include one or more filter elements that may
correspond to the desired color responses required for a color
system. For example, as shown in FIG. 3, CFA 314 can include
magenta filter element 320, which can be positioned over one or
more photodiodes 304 (e.g., a dual layer pair of photodiodes such
as photodiodes 307 and 308). In some cases, photodiodes 307 and 308
can correspond to red and blue pixels, respectively. In some
embodiments, in order to produce a desired color reproduction
quality in photodiodes 304, CFA 314 can include magenta filer
element 320 if photodiodes 304 are unable to achieve a desired
quantum efficiency response for red and blue pixels.
[0032] As another example, CFA 314 can include green filter element
322, which can be positioned over one or more photodiodes 304
(e.g., a dual layer pair of photodiodes such as photodiodes 305 and
306). In some cases, photodiodes 305 and 306 can correspond to
green pixels. Persons skilled in the art will appreciate that, in
addition to or instead of magenta filter element 320 and/or green
filter element 322, CFA 314 can include one or more blue, red,
yellow, and/or cyan filter elements.
[0033] For the one or more photodiodes, one or more filter elements
(e.g., filter elements 320 and 322) of CFA 314 can separate out a
particular spectral response of light (e.g., a combination of red
and blue spectral responses or green spectral responses) by, for
example, blocking passage of other spectra of light. For instance,
light passing through lens 318 can pass through IR cutoff filter
316 and filter elements 320 and 322. The filtered light can then
fall on a dual layer pair of photodiodes such as photodiodes 307
and 308 or photodiodes 305 and 306.
[0034] In some embodiments, clear pixels of photodiodes 304 (e.g.,
a dual layer pair of photodiodes such as photodiodes 309 and 310)
can be those pixels that are associated with no CFA coating on CFA
314.
[0035] Turning next to FIG. 4, a cross-sectional view of image
sensor 400 is shown. Image sensor 400 can be the same as or
substantially similar to image sensor 110 of FIG. 1. Image sensor
400 can include substrate 402, photodiodes 404 (e.g., photodiodes
405-410), insulation layer 412, CFA 414, IR cutoff filter 416, and
lens 418. Persons skilled in the art will appreciate that
photodiodes 404 can include one or more single layer photodiode
pixel structures, one or more dual layer photodiode pixel
structures, and/or any combination thereof.
[0036] The imaging capabilities of image sensor 400 can vary
depending on the types of pixels that correspond to one or more of
photodiodes 404. For example, photodiodes 405-408 can correspond to
color pixels (e.g., green, red, and/or blue pixels), and
photodiodes 409 and 410 can correspond to clear pixels. As shown
for image sensor 400, IR cutoff filter 416 can cover only a subset
of photodiodes 404 corresponding to color pixels (e.g., photodiodes
405-408). The remaining photodiodes 404 that are uncovered can
correspond to clear pixels (e.g., photodiodes 409 and 410) and can
bring additional IR sensing capabilities and resolution to image
sensor 400. For example, photodiodes 409 and 410 can sense an
infrared signal that is beyond the visible light wavelength, which
can thus be used to enable night vision application for image
sensor 400. Furthermore, because the color pixels can still be
filtered by IR cutoff filter 416, image sensor 400 can continue to
render high-quality color images.
[0037] As another example, photodiodes 405-408 can correspond to
color pixels, and photodiodes 409 and 410 can correspond to
luminance pixels. Thus, if IR cutoff filter 416 only covers
photodiodes 405-408, photodiodes 409 and 410 can remain uncovered.
Similar to an image sensor incorporating uncovered clear pixels,
the luminance pixels can also provide additional IR sensing
capabilities for image sensor 400. However, the resulting image
sensor configuration may be less sensitive to light because
luminance pixels may have a lower sensitivity to light as compared
to clear pixels.
[0038] As discussed in connection with FIG. 3, a typical image
sensor (e.g., image sensor 110 of FIG. 1) can use a CFA (e.g., CFA
314 of FIG. 3 or CFA 414 of FIG. 4) to generate a Bayer pattern
from a color image. For example, FIG. 5 shows a representation of
CFA 500 in a Bayer pattern in accordance with embodiments of the
invention. As shown in FIG. 5, CFA 500 can have a repeating
patterns of alternating rows of green and red filters (e.g., row
502) with rows of blue and green filters (e.g., row 504).
[0039] Turning now to FIGS. 6A-6L, these figures show illustrative
representations of pixel arrangements in accordance with
embodiments of the invention. These pixel arrangements can
represent arrangements of photodiodes with a dual layer photodiode
pixel structure. These photodiodes can be the same as or similar to
photodiodes 204 of FIG. 2A, photodiodes 240 of FIG. 2B, photodiodes
304 of FIG. 3, and/or photodiodes 404 of FIG. 4. In some
embodiments, at least one pixel in a second layer (e.g., top layer)
of the photodiodes can have a different color than a pixel at the
same position in a first layer (e.g., bottom layer) of the
photodiodes.
[0040] As shown in FIGS. 6A-6L, "R/B" pixels can correspond to dual
layer photodiode pixels with red and blue pixels stacked together
at the same positions (e.g., a red pixel at a particular position
in a bottom layer of the photodiodes and a blue pixel stacked on
top of the red pixel at the same position in a top layer of the
photodiodes). As another example, "C" pixels can correspond to dual
layer photodiode pixels with two clear pixels stacked together at
the same position (e.g., clear pixels at the same positions in both
a top layer and a bottom layer of the photodiodes). As yet another
example, "IR" pixels can correspond to dual layer photodiode pixels
with two infrared pixels stacked together at the same position
(e.g., infrared pixels at the same positions in both a top layer
and a bottom layer of the photodiodes). As a further example, "G"
pixels can correspond to dual layer photodiode pixels with two
green pixels stacked together at the same position (e.g., green
pixels at the same positions in both a top layer and a bottom layer
of the photodiodes). As yet a further example, "G" pixels can
correspond to only one integrated layer of green pixels. For
instance, a top layer of photodiodes can be combined with a bottom
layer of photodiodes without any insulation in between the two
layers (e.g., photodiodes 242 and 244 of FIG. 2B).
[0041] Persons skilled in the art will appreciate that the
representations in FIGS. 6A-6L are merely exemplary, and that an
image sensor (e.g., image sensor 110 of FIG. 1, image sensor 200 of
FIG. 2A, image sensor 230 of FIG. 2B, image sensor 300 of FIG. 3,
and/or image sensor 400 of FIG. 4) can include any suitable pixel
arrangement. For example, one or more clear pixels in a pixel
arrangement can be replaced with one or more green pixels, one or
more luminance pixels (e.g., luminance pixels at the same positions
in both a top layer and a bottom layer of the photodiodes), one or
more infrared pixels (e.g., infrared pixels at the same positions
in both a top layer and a bottom layer of the photodiodes), one or
more cyan pixels (e.g., a green pixel at a particular position in a
bottom layer of the photodiodes and a blue pixel stacked on top of
the green pixel at the same position in a top layer of the
photodiodes), one or more yellow pixels (e.g., a red pixel at a
particular position in a bottom layer of the photodiodes and a
green pixel stacked on top of the red pixel at the same position in
a top layer of the photodiodes), any other suitable pixels, and/or
any combination thereof. As another example, one or more green
pixels in a pixel arrangement can be replaced with one or more
clear pixels, one or more luminance pixels, one or more infrared
pixels, one or more cyan pixels, one or more yellow pixels, any
other suitable pixels, and/or any combination thereof. As yet
another example, one or more pixels in these pixel arrangements can
be replaced with stack IR pixels. Stack IR pixels can correspond to
dual layer photodiode pixels with an infrared pixel and either a
color, clear, or luminance pixel stacked together at the same
position (e.g., an infrared pixel at a particular position in a
bottom layer of the photodiodes and either a color, clear, or
luminance pixel stacked on top of the infrared pixel at the same
position in a top layer of the photodiodes).
[0042] Turning first to FIGS. 6A and 6B, pixel arrangements 600 and
602 show two illustrative pixel arrangements for G and R/B pixels.
For example, pixel arrangement 600 of FIG. 6A can provide an
adamantine (e.g., a checkerboard) pattern of pixels. As another
example, pixel arrangement 602 of FIG. 6B can provide a rectilinear
pattern of pixels. For the rectilinear pattern of pixel arrangement
602, at least one line (e.g., every row and column) of the top and
bottom layers of pixel arrangement 602 have pixels of the same
color.
[0043] Pixel arrangement 600 can provide for efficient binning
(e.g., downsampling) of the one or more pixels. The binning can be
executed by control circuitry of an image system (e.g., control
circuitry 120 of FIG. 1). For example, the control circuitry can
separately bin one or more pixel clusters of pixel arrangement
600.
[0044] For instance, for pixel cluster 604 of pixel arrangement
600, control circuitry can bin a set of pixels (e.g., G pixels 605
and 606) along a first direction in order to form a first pixel
group. The first direction can, for example, be along a diagonal
direction of -45.degree.. In addition, for the same pixel cluster
604, the control circuitry can bin another set of pixels (e.g., R/B
pixels 607 and 608) along a second direction (e.g., a diagonal
direction of 45.degree.) to form a second pixel group. Because
pixels in the first and second pixel groups (e.g., G pixels 605 and
606 and R/B pixels 607 and 608) have a shorter distance relative to
one another as compared to pixels that are positioned vertically or
horizontally, the control circuitry can bin pixel arrangement 600
without generating as many artifacts as would be generated from
binning pixels arranged in a Bayer pattern (e.g., a Bayer pattern
as shown in FIG. 5).
[0045] Binning can similarly be performed by the control circuitry
for pixel arrangement 602 of FIG. 6B. For example, for pixel
cluster 610 of pixel arrangement 602, the control circuitry can bin
a set of pixels (e.g., G pixels 611 and 612) along a first
direction (e.g., a horizontal direction) to form a first pixel
group. In addition, for the same pixel cluster 610, the control
circuitry can bin another set of pixels (e.g., R/B pixels 613 and
614) along a second direction (e.g., a vertical direction) to form
a second pixel group.
[0046] After binning, the control circuitry can combine first pixel
group and the second pixel group to form a binned pixel cluster.
Due to the dual layer photodiode pixel structure of pixel
arrangement 600 of FIG. 6A and pixel arrangement 602 of FIG. 6B,
the center pixel of both binned pixel clusters will have a
red-green-blue ("RGB") triplet without requiring any demosaic
processing. In some embodiments, for pixel arrangement 600 of FIG.
6A, the control circuitry can interpolate the binned pixel cluster
to output a pixel image after binning. For example, the control
circuitry can output a 2.times. pixel image with a rectilinear
pattern.
[0047] Turning next to FIGS. 6C and 6D, pixel arrangements 620 and
622 show two illustrative pixel arrangements for C, G, and R/B
pixels. For example, pixel arrangement 620 of FIG. 6C can provide
an adamantine pattern of pixels. As another example, pixel
arrangement 622 of FIG. 6D can provide a rectilinear pattern of
pixels. For the rectilinear pattern of pixel arrangement 622, at
least one line (e.g., rows 626-630) of the top and bottom layers of
pixel arrangement 622 can have pixels of the same color.
[0048] In some embodiments, by using pixel arrangement 620 of FIG.
6C, an image sensor can obtain at least as much luminance
resolution as with a regular Bayer pattern of a CFA (e.g., CFA 500
of FIG. 5). For example, for any particular pixel cluster (e.g.,
pixel cluster 624), pixel arrangement 620 can have the same amount
of green pixels as compared to a pixel cluster (e.g., pixel cluster
510 of FIG. 5) of a Bayer pattern. For instance, for both of the
configurations shown in FIGS. 5 and 6C, there are 2 green pixels
for each 2.times.2 pixel cluster.
[0049] Because of the addition of C pixels (e.g., C pixels 625) in
pixel arrangement 620, an image sensor employing pixel arrangement
620 has the additional advantage of higher sensitivity to light as
compared to a regular Bayer pattern. Furthermore, because pixel
arrangement 620 can provide the same amount of red and blue pixels
as compared to a regular Bayer pattern, the additional sensitivity
is gained without sacrificing red and blue color resolution. In
other embodiments, by using pixel arrangement 622 of FIG. 6D, an
image sensor can completely avoid green imbalance issues because
the G pixels can be arranged along one or more lines of pixel
arrangement 622 (e.g., along rows 626-630).
[0050] Turning to FIGS. 6E and 6F, pixel arrangements 631 and 632
show two additional illustrative pixel arrangements for C, G, and
R/B pixels. For example, pixel arrangement 631 of FIG. 6E can
provide an adamantine pattern of pixels. As another example, pixel
arrangement 632 of FIG. 6F can provide a rectilinear pattern of
pixels. For the rectilinear pattern of pixel arrangement 632, at
least one line (e.g., alternate rows and columns) of the top and
bottom layers of pixel arrangement 632 have pixels of the same
color.
[0051] In comparison to pixel arrangements 620 of FIGS. 6C and 622
of FIG. 6D, pixel arrangements 631 and 632 have a greater number of
C pixels, thereby providing additional sensitivity to light.
Furthermore, because pixel arrangement 632 provides C pixels along
one or more lines (e.g., rows 633-637), pixel arrangement 632 can
allow for separate exposures of C pixels and RIB and G pixels if
per line control can be achieved. By separately exposing the C
pixels and the R/B and G pixels, pixel arrangement 632 can allow an
image sensor to simultaneously avoid saturation of the C pixels and
improve the dynamic range of image capture.
[0052] Turning next to FIGS. 6G and 6H, pixel arrangements 640 and
642 show two illustrative pixel arrangements for C and R/B pixels.
For example, pixel arrangement 640 of FIG. 6G can provide an
adamantine pattern of pixels. As another example, pixel arrangement
642 of FIG. 6H can provide a rectilinear pattern of pixels. For the
rectilinear pattern of pixel arrangement 642, at least one line
(e.g., alternate rows and columns) of the top and bottom layers of
pixel arrangement 632 can have pixels of the same color.
[0053] Because of additional C pixels in pixel arrangements 640 and
642, pixel arrangements 640 and 642 can provide even more
sensitivity to light as compared to pixel arrangements 631 of FIGS.
6E and 632 of FIG. 6F. In some embodiments, pixel arrangements 640
and 642 can be used for low light conditions because of lower color
resolution requirements.
[0054] As shown in FIGS. 6G and 6H, pixel arrangements 640 and 642
can provide C pixels along one or more lines (e.g., one or more
rows 643-651 and one or more columns 652-655), which can allow for
separate exposures of C pixels and R/B pixels if per line control
can be achieved. By separately exposing the C pixels and the R/B
pixels, pixel arrangements 640 and 642 can allow an image sensor to
simultaneously avoid saturation of the C pixels and improve the
dynamic range of image capture. This way, the image sensor can
capture additional information from images than would otherwise be
possible.
[0055] In some embodiments, G pixel values can be derived for pixel
arrangements 640 and 642 using any suitable approach. For example,
because clear pixels are a combination of green, red, and blue
pixels, control circuitry (e.g., control circuitry 120) can obtain
G pixel values by interpolating neighboring R/B pixel values and C
pixel values jointly.
[0056] Turning now to FIGS. 6I and 6J, pixel arrangements 660 and
662 show two illustrative pixel arrangements for G and R/B pixels.
For example, pixel arrangement 660 of FIG. 6G can provide a
non-rotated pattern of pixels. As another example, pixel
arrangement 662 of FIG. 6H can provide a rotated pattern of
pixels.
[0057] Pixel arrangements 660 and 662 can include one or more G
pixels that may have replaced one or more C pixels of pixel
arrangements 620 (FIG. 6C), 622 (FIG. 6D), 631 (FIG. 6E), 632 (FIG.
6F), 640 (FIG. 6G), and 642 (FIG. 6H). In some embodiments, the
number of G pixels in pixel arrangements 660 and 662 can exceed the
number of R/B pixels.
[0058] Pixel arrangements can have any suitable pre-determined
orientations (e.g., 10.degree., 20.degree., 30.degree., etc.). For
example, pixel arrangement 660 can have a pre-determined
arrangement of 0.degree.. As another example, pixel arrangement 662
can have a pre-determined orientation of 45.degree.. As a result of
its pre-determined orientation, the sampling frequency of pixel
arrangement 662 along the vertical and horizontal directions may be
different from the sampling frequency of pixel arrangement 660
along the vertical, horizontal, and 45.degree. directions.
[0059] Turning next to FIGS. 6K and 6L, pixel arrangements 670 and
672 show two illustrative pixel arrangements for IR, G, and R/B
pixels. Pixel arrangements 670 (FIG. 6K) and 672 (FIG. 6L) can be
substantially similar to pixel arrangements 620 (FIG. 6C) and 622
(FIG. 6D), respectively. In particular, the C pixels of pixel
arrangements 620 and 622 can be replaced with IR pixels in pixel
arrangements 670 and 672.
[0060] Pixel arrangement 670 of FIG. 6K can provide an adamantine
pattern of pixels. Moreover, pixel arrangement 672 of FIG. 6L can
provide a rectilinear pattern of pixels. For the rectilinear
pattern of pixel arrangement 672, at least one line (e.g., rows
673-677) of the top and bottom layers of pixel arrangement 672 can
have pixels of the same color.
[0061] In some embodiments, by using pixel arrangement 670 of FIG.
6K, an image sensor can obtain at least as much luminance
resolution as with a regular Bayer pattern of a CFA (e.g., CFA 500
of FIG. 5). For example, for any particular pixel cluster (e.g.,
pixel cluster 680), pixel arrangement 670 can have the same amount
of green pixels as compared to a pixel cluster (e.g., pixel cluster
510 of FIG. 5) of a Bayer pattern. For instance, for both of the
configurations shown in FIGS. 5 and 6K, there are 2 green pixels
for each 2.times.2 pixel cluster.
[0062] Because of the addition of IR pixels (e.g., IR pixels 682)
in pixel arrangement 670, an image sensor employing pixel
arrangement 670 can be capable of absorbing near infrared signals
and infrared signals. Furthermore, because pixel arrangement 670
can provide the same amount of red and blue pixels as compared to a
regular Bayer pattern, this additional capability is gained without
sacrificing red and blue color resolution. In other embodiments, by
using pixel arrangement 672 of FIG. 6L, an image sensor can
completely avoid green imbalance issues because the G pixels can be
arranged along one or more lines of pixel arrangement 672 (e.g.,
along rows 673-677).
[0063] As discussed previously, an image system (e.g., image system
100 of FIG. 1) can perform binning on one or more photodiodes. FIG.
7 is a flowchart of process 700 for binning photodiodes. In
particular, process 700 can be executed by control circuitry (e.g.,
control circuitry 120 of FIG. 1) of an image system configured in
accordance with embodiments of the invention. It should be
understood that process 700 is merely illustrative, and that any
steps can be removed, modified, combined, or any steps may be
added, without departing from the scope of the invention.
[0064] Process 700 begins at step 702. At step 704, the control
circuitry can bin a set of first color pixels (e.g., green pixels)
of multiple photodiodes along a first direction to form a first
pixel group, where the multiple photodiodes can include a dual
layer photodiode pixel structure (e.g., photodiodes 204 of FIG. 2A,
photodiodes 246 and 248 of FIG. 2B, photodiodes 304 of FIG. 3, or
photodiodes 404 of FIG. 4). For example, for a particular pixel
arrangement (e.g., pixel arrangements 600 and 602 of FIGS. 6A and
6B, respectively) of the multiple photodiodes, the control
circuitry can bin G pixels (e.g., G pixels 605 and 606 of FIG. 6A
or G pixels 611 and 612 of FIG. 6B) along a first direction (e.g.,
a diagonal direction of -45.degree. or a horizontal direction).
[0065] Then, at step 706, the control circuitry can bin a set of
second color pixels and third color pixels (e.g., red and blue
pixels) of the multiple photodiodes along a second direction to
form a second pixel group. For example, the control circuitry can
bin R/B pixels (e.g., R/B pixels 607 and 608 of FIG. 6A or R/B
pixels 612 and 613 of FIG. 6B) along a second direction (e.g., a
diagonal direction of 45.degree. or a vertical direction). In some
embodiments, the first direction can be orthogonal to the second
direction.
[0066] Continuing to step 708, the control circuitry can combine
the first pixel group and the second pixel group to form a binned
pixel cluster. After forming the binned pixel cluster, at step 710,
the control circuitry can interpolate the binned pixel cluster to
output a pixel image (e.g., a 2.times. pixel image with a
rectilinear pattern). Process 700 then ends at step 712.
[0067] In conclusion, an image system with a dual layer photodiode
structure is provided for processing color images. In particular,
the image system can include an image sensor that can include
photodiodes with a dual layer photodiode structure. In some
embodiments, the dual layer photodiode can include a first layer of
photodiodes (e.g., a bottom layer), a second layer of photodiodes
(e.g., a top layer), and an insulation layer disposed between the
bottom and top layers. The first layer of photodiodes can include
one or more suitable pixels (e.g., green, red, clear, luminance,
and/or infrared pixels). Likewise, the second layer of photodiodes
can include one or more suitable pixels (e.g., green, blue, clear,
luminance, and/or infrared pixels).
[0068] In contrast to conventional photodiodes, dual layer
photodiodes can include additional clear pixels and still maintain
the same or a greater number of green pixels. Thus, an image sensor
incorporating dual layer photodiodes can gain light sensitivity
with the additional clear pixels and maintain luminance information
with the green pixels. The additional resolution and light
sensitivity for such an image sensor can be advantageous for many
situations (e.g., normal lighting conditions, low light conditions,
and/or high speed imaging under normal lighting conditions).
[0069] In some embodiments, the image sensor can include a CFA that
can include a magenta filter element. The magenta filter element,
which can be positioned over one or more photodiodes, can allow the
image sensor to achieve a desired quantum efficiency response.
[0070] In some embodiments, the image sensor can include an IR
cutoff filter, which can cover only a subset of the photodiodes of
the image sensor. For example, if the IR cutoff filter only covers
photodiodes corresponding to color pixels, uncovered photodiodes
corresponding to clear pixels can provide greater sensitivity and
resolution to the image sensor. Furthermore, because the color
pixels can still be filtered by the IR cutoff filter, the image
sensor can continue to render high-quality color images.
[0071] In some embodiments, the image system can include control
circuitry for processing data generated by an image sensor. For
example, the control circuitry can bin pixels of multiple
photodiodes of the image sensor. For example, the control circuitry
can bin a set of first color pixels of the multiple photodiodes to
form a first pixel group. In addition, control circuitry can bin a
set of second color pixels and third color pixels of the multiple
photodiodes to form a second pixel group. The control circuitry can
then combine the first pixel group and the second pixel group to
form a binned pixel cluster. After forming the binned pixel
cluster, the control circuitry can interpolate the binned pixel
cluster to output a pixel image (e.g., a 2.times. pixel image with
a rectilinear pattern).
[0072] The described embodiments of the invention are presented for
the purpose of illustration and not of limitation.
* * * * *