U.S. patent number 6,487,321 [Application Number 09/662,774] was granted by the patent office on 2002-11-26 for method and system for altering defects in a digital image.
This patent grant is currently assigned to Applied Science Fiction. Invention is credited to Albert D. Edgar, Raymond S. Lee.
United States Patent |
6,487,321 |
Edgar , et al. |
November 26, 2002 |
Method and system for altering defects in a digital image
Abstract
One aspect of the invention is a method for altering defects in
a digital image. At least a first pixel of a first channel of a
digital image is filtered using digital circuitry to produce a
filtered pixel by averaging the intensity of the first pixel and a
plurality of additional pixels in the neighborhood of the first
pixel. At least one of the pixels is weighted in response to the
intensity value of at least one pixel in a defect channel
associated with the digital image. A corrected digital image is
produced using the digital circuitry in response to the filtered
pixel and the first channel of the digital image.
Inventors: |
Edgar; Albert D. (Austin,
TX), Lee; Raymond S. (Austin, TX) |
Assignee: |
Applied Science Fiction
(Austin, TX)
|
Family
ID: |
22550627 |
Appl.
No.: |
09/662,774 |
Filed: |
September 15, 2000 |
Current U.S.
Class: |
382/260; 358/463;
382/264; 382/275 |
Current CPC
Class: |
H04N
1/4097 (20130101); H04N 1/58 (20130101) |
Current International
Class: |
H04N
1/409 (20060101); H04N 1/56 (20060101); H04N
1/58 (20060101); G06T 005/20 (); G06K 009/40 ();
H04N 001/409 () |
Field of
Search: |
;382/261,262,260,264,275
;358/463 ;356/237.1,239.1,239.7,239.8,237.2,237.3
;250/341.8,339.11 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
28 21 868 |
|
Nov 1979 |
|
DE |
|
196 36 867 |
|
Jan 1998 |
|
DE |
|
1 547 811 |
|
Jun 1979 |
|
EP |
|
0 422 220 |
|
Apr 1991 |
|
EP |
|
0 482 790 |
|
Apr 1992 |
|
EP |
|
0 527 097 |
|
Feb 1993 |
|
EP |
|
0569142 |
|
Nov 1993 |
|
EP |
|
0 569 142 |
|
Nov 1993 |
|
EP |
|
0 624 848 |
|
Nov 1994 |
|
EP |
|
0 669 753 |
|
Aug 1995 |
|
EP |
|
0 699 753 |
|
Aug 1995 |
|
EP |
|
0 716 538 |
|
Jun 1996 |
|
EP |
|
0 751 670 |
|
Jan 1997 |
|
EP |
|
0768621 |
|
Apr 1997 |
|
EP |
|
0 768 621 |
|
Apr 1997 |
|
EP |
|
0 794 454 |
|
Sep 1997 |
|
EP |
|
0816833 |
|
Jan 1998 |
|
EP |
|
0 816 833 |
|
Jan 1998 |
|
EP |
|
0 816 833 |
|
Aug 1998 |
|
EP |
|
0 893 914 |
|
Jan 1999 |
|
EP |
|
2 283 633 |
|
May 1995 |
|
GB |
|
4-291139 |
|
Oct 1992 |
|
JP |
|
11185028 |
|
Jul 1999 |
|
JP |
|
2000-13604 |
|
Jan 2000 |
|
JP |
|
2000-196813 |
|
Jul 2000 |
|
JP |
|
WO 84/02019 |
|
May 1984 |
|
WO |
|
WO 89/06890 |
|
Jul 1989 |
|
WO |
|
90 01240 |
|
Feb 1990 |
|
WO |
|
WO 90/01240 |
|
Feb 1990 |
|
WO |
|
91 09493 |
|
Jun 1991 |
|
WO |
|
WO 91/09493 |
|
Jun 1991 |
|
WO |
|
WO 92/05469 |
|
Apr 1992 |
|
WO |
|
WO 95/15530 |
|
Jun 1995 |
|
WO |
|
WO 97/16028 |
|
May 1997 |
|
WO |
|
WO 98/31142 |
|
Jul 1998 |
|
WO |
|
WO 98/34397 |
|
Aug 1998 |
|
WO |
|
WO 99/40729 |
|
Aug 1999 |
|
WO |
|
WO 01/48694 |
|
Jul 2001 |
|
WO |
|
Other References
EP 0 569 142 A1: Inventor -Albert Durr Edgar, Applicant -IBM,
System and Method for Image Recovery -Application No. 93302897.9
(Date of Filing Apr. 14, 1993), Specification -12 pp. Drawings -6
pp.; Bulletin 93/45, Nov. 10, 1993. .
EP 0 816 833 A2: Inventor -Graham Bradburn, Applicant -Fujifilm
Electronic Imaging Limited, An Illumination Unit and Illumination
Method -Application No. 97304585.9 (Date of Filing: Jun. 26, 1997),
Specification -7 pp., Drawings -4 pp.; Bulletin Feb./1998. Jan. 7,
1998. .
WO 90/01240: Inventor -Martin Kaplan, Applicant -Eastman Kodak
Company, Film Noise Reduction by Application of Bayes Theorem to
Positive/Negative Film -Application No. PCT/US89/03052 (Date of
Filing: Jul. 17, 1989), Specification -45 pp., Drawings -7 pp. Feb.
8, 1990. .
WO 91/09493: Inventor -COK, David, Roy, Applicant -Eastman Kodak
Company, Method for Deriving Noise-Reduced Estimates of Colour
Signal Parameters From Multiple Colour/Luminance Image Sensor
Outputs -Application No. PCT/US90/07142, Specification -25 pp.,
Drawings -1 p. Jun. 27, 1991. .
New Adaptive Vector Filter Based on Noise Estimate, Mei Yu, et al.,
IEICE Trans Fundamentals, vol. E82 A, No. 6, Jun., 1999, pp.
911-919. .
A Robust Method for Parameter Estimation of Signal-Dependent Noise
Models in Digital Images, B. Aiazzi, et al., IEEE, 1997, pp. DSP 97
-601-604. .
A Regularized Iterative Image Restoration Algorithm, Aggelos K.
Katsaggelos, et al., IEEE, 1991, pp. 914-929. .
Adaptive Fourier Threshold Filtering: A Method to Reduce Noise and
Incoherent Artifacts in High Resolution Cardiac Images, M. Doyle,
et al., 8306 Magnetic Resonance in Medicine, May, 31, 1994, No. 5,
Baltimore, MD., pp. 546-550. .
Anisotropic Spectral Magnitude Estimation Filters for Noise
Reduction and Image Enhancement, Til Aach, et al., IEEE, 1996, pp.
335-338. .
Adaptive-neighborhood filtering of images corrupted by
signal-dependant noise, Rangaraj M. Rangayyan et al., Applied
Optics, vol. 37, No. 20, Jul. 10, 1998, pp. 4477-4487. .
Grayscale Characteristics, Photographic Negatives The Nature of
Color Images, pp. 163-168. .
Digital Imaging Equipment White Papers, Putting Damaged Film on
Ice, A Technical Discussion of Advances in Digital Imaging, Nikon
Corporation,
http://www.nikonusa.com/reference/whitepapers/imaging/ditechdisc.html,
Aug. 5, 1999. .
Local Cosine Transform -A Method for the Reduction of the Blocking
Effect in JPEG, Gil Aharoni, et al., Journal of Mathematical
Imaging and Vision, 3, pp. 7-38, 1993. .
Digital ICE.TM., Applied Science Fiction, Inc.,
http://www.asf.com/html/o_products/iceprod.html. Aug 5, 1999. .
About Digital ICE.TM. Technology, Applied Science Fiction, Inc.,
http://www.asf.com/html/o_products/icetech.html. Aug 5, 1999, 2 pp.
.
2-D Adaptive Volterra Filter for 2-D Nonlinear Channel Equalisation
and Image Restoration, J.N. Lin, et al., Electronics Letters, vol.
28, No. 2, Jan. 16, 1992, pp. 180-182..
|
Primary Examiner: Rogers; Scott
Attorney, Agent or Firm: Baker Botts L.L.P.
Parent Case Text
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims benefit of U.S. Provisional Application
Serial No. 60/154,255, filed Sep. 16, 1999 by Albert Edgar, et al.,
and entitled "Method and System for Altering Defects in a Digital
Image".
This application is related to U.S. application Ser. No.
08/999,421, filed on Dec. 29, 1997, by Albert Edgar and entitled,
"Defect Channel Nulling."
This application is related to U.S. application Ser. No.
09/156,271, filed on Sep. 16, 1998, by Albert Edgar and entitled,
"Method and Apparatus for Capturing Defect Data From Documents and
Films."
Claims
What is claimed is:
1. A method for altering defects in a digital image, comprising:
storing on a computer readable storage medium a digital image
comprising at least one channel, the at least one channel
comprising a plurality of pixels each having an intensity value;
storing on the computer readable storage medium a defect channel
comprising a plurality of pixels each having an intensity value, at
least some of the plurality of pixels in the defect channel having
an intensity value proportional to defects in the digital image;
filtering a first pixel of the at least one channel of the digital
image to produce a filtered pixel the filtering of the first pixel
responsive to the intensity of the first pixel and a plurality of
additional pixels in the neighborhood of the first pixel, wherein
at least one of the pixels used in filtering the first pixel is
weighted in response to the intensity value of at least one pixel
in the defect channel; and producing a corrected digital image in
response to the filtered pixel and the at least one channel.
2. The method of claim 1: wherein the digital image comprises three
channels, each channel comprising a plurality of pixels; wherein
the digital image represents a color image; and wherein the method
further comprises filtering at least a second pixel at the same
location in each of the three channels to produce a second filtered
pixel for each of the three channels, the filtering of the second
pixel in each channel, for a particular channel, responsive to the
intensity of the second pixel in the particular channel and a
plurality of additional pixels in the neighborhood of the second
pixel in the particular channel, wherein at least one of the pixels
used in filtering the second pixel in the particular channel is
weighted in response to the intensity value of at least one pixel
in the defect channel; producing a corrected digital image in
response to the filtered pixel, the second filtered pixel for each
channel, and the three channels.
3. The method of claim 1, wherein the digital image comprises a
single channel and represents a monochrome image.
4. The method of claim 1, wherein at least one of the pixels used
in filtering the first pixel is weighted in response to the
intensity value of a plurality of pixels in the defect channel.
5. The method of claim 1, wherein at least one of the pixels used
in filtering the first pixel is weighted in response to the
intensity value of a pixel in the defect channel which spatially
corresponds to the first pixel.
6. The method of claim 1, wherein the weighting of the at least one
of the pixels used in filtering the first pixel is further
responsive to red leakage in the defect channel.
7. The method of claim 1, wherein the weighting of the at least one
of the pixels used in filtering the first pixel is further
responsive to a clear film intensity measure.
8. The method of claim 1, wherein the filtering is performed in log
space.
9. The method of claim 8, wherein the corrected digital image is
further responsive to the difference between the digital image and
a filtered version of the digital image containing the filtered
pixel.
10. The method of claim 8, wherein the corrected digital image is
further responsive to the difference between (a) the difference
between the digital image and a filtered version of the digital
image containing the filtered pixel and (b) the difference between
the original defect channel and a filtered version of the defect
channel.
11. The method of claim 1, wherein the intensity of pixels in the
defect channel is responsive to the difference between at least two
reflected scans of an image from which the digital image was
derived, wherein the image is illuminated by at least one light
source at a different angle relative to the surface of the image
during each scan.
12. The method of claim 1, wherein filtering the first pixel
further comprises: filtering the first pixel a plurality of times,
wherein ones of the plurality of filtering operations cover a
different bandwidth, are responsive to the intensity of the first
pixel and a plurality of additional pixels in the neighborhood of
the first pixel, and wherein at least one of the pixels use in
filtering the first pixel is weighted in response to the intensity
value of at least one pixel in the defect channel, each of the
plurality of filtering operations producing a filtered pixel,
collectively comprising a plurality of filtered pixels; and wherein
the corrected digital image is further responsive to the plurality
of filtered pixels.
13. The method of claim 1, wherein the digital image comprises
three channels, each channel comprising a plurality of pixels;
wherein the digital image represents a color image; wherein the at
least one channel comprises one of the three channels; and wherein
the corrected digital image is produced in response to the filtered
pixel by applying a correction in all three channels in response to
the filtered pixel determined for the at least one channel.
14. The method of claim 1, wherein the digital image comprises
three channels, each channel comprising a plurality of pixels;
wherein the digital image represents a color image; wherein the
method further comprises forming a black and white channel based
upon the three channels, the black and white channel comprising the
at least one channel; and wherein the corrected digital image is
produced in response to the filtered pixel by applying a correction
in all three channels in response to the filtered pixel determined
for the black and white channel.
15. The method of claim 1, wherein the intensity of pixels in the
defect channel is proportional to infrared transmission through
film from which the digital image was derived.
16. The method of claim 15, further comprising: assigning a low
weight value to a pixel for the filtering operation if a spatially
corresponding pixel in the defect channel has an intensity less
than a first threshold.
17. The method of claim 15, further comprising: assigning a low
weight value to a pixel for the filtering operation if the average
intensity of a spatially corresponding pixel in the defect channel
and other pixels in the defect channel in the neighborhood of the
spatially corresponding pixel is less than a first threshold.
18. The method of claim 1, further comprising: dividing the digital
image into a plurality of segments; wherein the first pixel is in a
first segment comprising one of the plurality of segments of the at
least one channel of the digital image and wherein the plurality of
additional pixels in the neighborhood of the first pixel are in the
first segment; filtering a second pixel of the at least one channel
of the digital image to produce a second filtered pixel the
filtering of the second pixel responsive to the intensity of the
second pixel and a plurality of additional pixels in the
neighborhood of the second pixel, wherein at least one of the
pixels used in filtering the second pixel is weighted in response
to the intensity value of at least one pixel in the defect channel;
wherein the second pixel is in a second segment comprising one of
the plurality of segments of the at least one channel of the
digital image and wherein the plurality of additional pixels in the
neighborhood of the second pixel are in the second segment; and
wherein the corrected digital image is produced in response the
filtered pixel, the second filtered pixel, and the at least one
channel.
19. The method of claim 1, wherein the first pixel is filtered
using a square filter.
20. The method of claim 1, wherein the first pixel is filtered
using a circular filter.
21. A method for altering defects in a digital image, comprising:
filtering, using digital circuitry, a first pixel of a first
channel of the digital image to produce a filtered pixel by
averaging the intensity of the first pixel and a plurality of
additional pixels in the neighborhood of the first pixel, wherein
at least one of the pixels used for averaging is weighted in
response to the intensity value of at least one pixel in a defect
channel associated with the digital image; and producing a
corrected digital image, using digital circuitry, in response to
the filtered pixel and the first channel.
22. The method of claim 21, wherein the digital circuitry comprises
dedicated digital hardware.
23. The method of claim 21, wherein the digital circuitry comprises
a microprocessor executing computer software.
24. The method of claim 21, wherein the digital image comprises
three channels, each channel comprising a plurality of pixels;
wherein the digital image represents a color image; and wherein the
method further comprises filtering at least a second pixel at the
same location in each of the three channels to produce a second
filtered pixel for each of the three channels, the filtering of the
second pixel in each channel, for a particular channel, responsive
to the intensity of the second pixel in the particular channel and
a plurality of additional pixels in the neighborhood of the second
pixel in the particular channel, wherein at least one of the pixels
used in filtering the second pixel in the particular channel is
weighted in response to the intensity value of at least one pixel
in the defect channel; producing a corrected digital image in
response to the filtered pixel, the second filtered pixel for each
channel, and the three channels.
25. The method of claim 21, wherein at least one of the pixels used
in filtering the first pixel is weighted in response to the
intensity value of a plurality of pixels in the defect channel.
26. The method of claim 21, wherein at least one of the pixels used
in filtering the first pixel is weighted in response to the
intensity value of a pixel in the defect channel which spatially
corresponds to the first pixel.
27. The method of claim 21, further comprising: filtering the first
pixel a plurality of times, wherein ones of the plurality of
filtering operations cover a different bandwidth, are responsive to
the intensity of the first pixel and a plurality of additional
pixels in the neighborhood of the first pixel, and wherein at least
one of the pixels use in filtering the first pixel is weighted in
response to the intensity value of at least one pixel in the defect
channel, each of the plurality of filtering operations producing a
filtered pixel, collectively comprising a plurality of filtered
pixels; and wherein the corrected digital image is further
responsive to the plurality of filtered pixels.
28. The method of claim 21, further comprising: assigning a low
weight value to a pixel for the filtering operation if a spatially
corresponding pixel in the defect channel has an intensity
indicating a high probability that the pixel is defective.
29. The method of claim 21, further comprising: assigning a low
weight value to a pixel for the filtering operation if the average
intensity of a spatially corresponding pixel in the defect channel
and other pixels in the defect channel in the neighborhood of the
spatially corresponding pixel indicates a high probability that the
pixel is defective.
30. The method of claim 21, further comprising: performing a
filtering operation on each pixel in the at least one channel.
31. The method of claim 21, further comprising: dividing the
digital image into a plurality of segments; wherein the first pixel
is in a first segment comprising one of the plurality of segments
of the at least one channel of the digital image and wherein the
plurality of additional pixels in the neighborhood of the first
pixel are in the first segment; filtering a second pixel of the at
least one channel of the digital image to produce a second filtered
pixel the filtering of the second pixel responsive to the intensity
of the second pixel and a plurality of additional pixels in the
neighborhood of the second pixel, wherein at least one of the
pixels used in filtering the second pixel is weighted in response
to the intensity value of at least one pixel in the defect channel;
wherein the second pixel is in a second segment comprising one of
the plurality of segments of the at least one channel of the
digital image and wherein the plurality of additional pixels in the
neighborhood of the second pixel are in the second segment; and
wherein the corrected digital image is produced in response the
filtered pixel, the second filtered pixel, and the at least one
channel.
32. A digital image scanning system comprising: scanning hardware
operable to scan an image and convert the image into a digital
image having at least one channel and to produce a defect channel
responsive to defects in the image; and computer software
associated with the scanning hardware and operable to: filter a
first pixel of the at least one channel of the digital image to
produce a filtered pixel the filtering of the first pixel
responsive to the intensity of the first pixel and a plurality of
additional pixels in the neighborhood of the first pixel, wherein
at least one of the pixels used in filtering the first pixel is
weighted in response to the intensity value of at least one pixel
in the defect channel; and produce a corrected digital image in
response to the filtered pixel and the at least one channel.
33. The digital image scanning system of claim 32, wherein the
digital image comprises three channels, each channel comprising a
plurality of pixels; wherein the digital image represents a color
image; and wherein the computer software is further operable to
filter at least a second pixel at the same location in each of the
three channels to produce a second filtered pixel for each of the
three channels, the filtering of the second pixel in each channel,
for a particular channel, responsive to the intensity of the second
pixel in the particular channel and a plurality of additional
pixels in the neighborhood of the second pixel in the particular
channel, wherein at least one of the pixels used in filtering the
second pixel in the particular channel is weighted in response to
the intensity value of at least one pixel in the defect channel;
produce a corrected digital image in response to the filtered
pixel, the second filtered pixel for each channel, and the three
channels.
34. The digital image scanning system of claim 32, wherein at least
one of the pixels used in filtering the first pixel is weighted in
response to the intensity value of a plurality of pixels in the
defect channel.
35. The digital image scanning system of claim 32, wherein at least
one of the pixels used in filtering the first pixel is weighted in
response to the intensity value of a pixel in the defect channel
which spatially corresponds to the first pixel.
36. The digital image scanning system of claim 32, wherein the
computer software is further operable to: filter the first pixel a
plurality of times, wherein ones of the plurality of filtering
operations cover a different bandwidth, are responsive to the
intensity of the first pixel and a plurality of additional pixels
in the neighborhood of the first pixel, and wherein at least one of
the pixels use in filtering the first pixel is weighted in response
to the intensity value of at least one pixel in the defect channel,
each of the plurality of filtering operations producing a filtered
pixel, collectively comprising a plurality of filtered pixels; and
wherein the corrected digital image is further responsive to the
plurality of filtered pixels.
37. The digital image scanning system of claim 32, wherein the
computer software is further operable to: assign a low weight value
to a pixel for the filtering operation if a spatially corresponding
pixel in the defect channel has an intensity indicating a high
probability that the pixel is defective.
38. The digital image scanning system of claim 32, wherein the
computer software is further operable to: assign a low weight value
to a pixel for the filtering operation if the average intensity of
a spatially corresponding pixel in the defect channel and other
pixels in the defect channel in the neighborhood of the spatially
corresponding pixel indicates a high probability that the pixel is
defective.
39. The digital image scanning system of claim 32, wherein the
computer software is further operable to: divide the digital image
into a plurality of segments; wherein the first pixel is in a first
segment comprising one of the plurality of segments of the at least
one channel of the digital image and wherein the plurality of
additional pixels in the neighborhood of the first pixel are in the
first segment; filter a second pixel of the at least one channel of
the digital image to produce a second filtered pixel the filtering
of the second pixel responsive to the intensity of the second pixel
and a plurality of additional pixels in the neighborhood of the
second pixel, wherein at least one of the pixels used in filtering
the second pixel is weighted in response to the intensity value of
at least one pixel in the defect channel; wherein the second pixel
is in a second segment comprising one of the plurality of segments
of the at least one channel of the digital image and wherein the
plurality of additional pixels in the neighborhood of the second
pixel are in the second segment; and wherein the corrected digital
image is produced in response the filtered pixel, the second
filtered pixel, and the at least one channel.
40. A digital image processing system comprising: a computer
readable storage medium; computer software stored on the computer
readable storage medium and operable to: filter a first pixel of
the at least one channel of the digital image to produce a filtered
pixel the filtering of the first pixel responsive to the intensity
of the first pixel and a plurality of additional pixels in the
neighborhood of the first pixel, wherein at least one of the pixels
used in filtering the first pixel is weighted in response to the
intensity value of at least one pixel in the defect channel; and
produce a corrected digital image in response to the filtered pixel
and the at least one channel.
41. The digital image processing system of claim 40, wherein the
digital image comprises three channels, each channel comprising a
plurality of pixels; wherein the digital image represents a color
image; and wherein the computer software is further operable to
filter at least a second pixel at the same location in each of the
three channels to produce a second filtered pixel for each of the
three channels, the filtering of the second pixel in each channel,
for a particular channel, responsive to the intensity of the second
pixel in the particular channel and a plurality of additional
pixels in the neighborhood of the second pixel in the particular
channel, wherein at least one of the pixels used in filtering the
second pixel in the particular channel is weighted in response to
the intensity value of at least one pixel in the defect channel;
produce a corrected digital image in response to the filtered
pixel, the second filtered pixel for each channel, and the three
channels.
42. The digital image processing system of claim 40, wherein at
least one of the pixels used in filtering the first pixel is
weighted in response to the intensity value of a plurality of
pixels in the defect channel.
43. The digital image processing system of claim 40, wherein at
least one of the pixels used in filtering the first pixel is
weighted in response to the intensity value of a pixel in the
defect channel which spatially corresponds to the first pixel.
44. The digital image scanning system of claim 40, wherein the
computer software is further operable to: filter the first pixel a
plurality of times, wherein ones of the plurality of filtering
operations cover a different bandwidth, are responsive to the
intensity of the first pixel and a plurality of additional pixels
in the neighborhood of the first pixel, and wherein at least one of
the pixels use in filtering the first pixel is weighted in response
to the intensity value of at least one pixel in the defect channel,
each of the plurality of filtering operations producing a filtered
pixel, collectively comprising a plurality of filtered pixels; and
wherein the corrected digital image is further responsive to the
plurality of filtered pixels.
45. The digital image scanning system of claim 40, wherein the
computer software is further operable to: assign a low weight value
to a pixel for the filtering operation if a spatially corresponding
pixel in the defect channel has an intensity indicating a high
probability that the pixel is defective.
46. The digital image scanning system of claim 40, wherein the
computer software is further operable to: assign a low weight value
to a pixel for the filtering operation if the average intensity of
a spatially corresponding pixel in the defect channel and other
pixels in the defect channel in the neighborhood of the spatially
corresponding pixel indicates a high probability that the pixel is
defective.
47. The digital image scanning system of claim 40, wherein the
computer software is further operable to: divide the digital image
into a plurality of segments; wherein the first pixel is in a first
segment comprising one of the plurality of segments of the at least
one channel of the digital image and wherein the plurality of
additional pixels in the neighborhood of the first pixel are in the
first segment; filter a second pixel of the at least one channel of
the digital image to produce a second filtered pixel the filtering
of the second pixel responsive to the intensity of the second pixel
and a plurality of additional pixels in the neighborhood of the
second pixel, wherein at least one of the pixels used in filtering
the second pixel is weighted in response to the intensity value of
at least one pixel in the defect channel; wherein the second pixel
is in a second segment comprising one of the plurality of segments
of the at least one channel of the digital image and wherein the
plurality of additional pixels in the neighborhood of the second
pixel are in the second segment; and wherein the corrected digital
image is produced in response the filtered pixel, the second
filtered pixel, and the at least one channel.
48. An altered digital image derived from a digital image having at
least one channel, comprising: a computer readable storage medium;
an altered digital image stored on the computer readable storage
medium wherein the altered digital image was created by: filtering
a first pixel of the at least one channel of the digital image to
produce a filtered pixel the filtering of the first pixel
responsive to the intensity of the first pixel and a plurality of
additional pixels in the neighborhood of the first pixel, wherein
at least one of the pixels used in filtering the first pixel is
weighted in response to the intensity value of at least one pixel
in the defect channel; and producing a corrected digital image in
response to the filtered pixel and the at least one channel.
49. The altered digital image of claim 48, wherein the digital
image comprises three channels, each channel comprising a plurality
of pixels; and wherein the digital image represents a color image
and wherein the altered digital image was further created by:
filtering at least a second pixel at the same location in each of
the three channels to produce a second filtered pixel for each of
the three channels, the filtering of the second pixel in each
channel, for a particular channel, responsive to the intensity of
the second pixel in the particular channel and a plurality of
additional pixels in the neighborhood of the second pixel in the
particular channel, wherein at least one of the pixels used in
filtering the second pixel in the particular channel is weighted in
response to the intensity value of at least one pixel in the defect
channel; producing a corrected digital image in response to the
filtered pixel, the second filtered pixel for each channel, and the
three channels.
50. The altered digital image of claim 48, wherein the altered
digital image was further created by: filtering the first pixel a
plurality of times, wherein ones of the plurality of filtering
operations cover a different bandwidth, are responsive to the
intensity of the first pixel and a plurality of additional pixels
in the neighborhood of the first pixel, and wherein at least one of
the pixels use in filtering the first pixel is weighted in response
to the intensity value of at least one pixel in the defect channel,
each of the plurality of filtering operations producing a filtered
pixel, collectively comprising a plurality of filtered pixels; and
wherein the corrected digital image is further responsive to the
plurality of filtered pixels.
51. A method for altering defects in a digital image, comprising:
storing on a computer readable storage medium a digital image
comprising at least one channel, the at least one channel
comprising a plurality of pixels each having an intensity value;
storing on the computer readable storage medium a defect channel
comprising a plurality of pixels each having an intensity value, at
least some of the plurality of pixels in the defect channel having
an intensity value proportional to defects in the digital image;
creating, using digital circuitry, a plurality of filtered versions
of the at least one channel, wherein pixels of the filtered
versions of the at least one channel are determined by, for a
particular pixel in a particular filtered version, computing a
weighted average of the intensity values of the particular pixel
and pixels within a region adjacent to the particular pixel in the
at least one channel, wherein at least some of the intensity values
used in the computation of the weighted average are weighted in
response to the intensity value of at least one pixel in the defect
channel; and producing a corrected digital image in response to the
plurality of filtered versions of the at least one channel, the
original at least one channel, and the defect channel.
52. The method of claim 51, further comprising: creating, using
digital circuitry, a plurality of filtered versions of the defect
channel, wherein pixels of the filtered versions of the defect
channel are determined by, for a particular pixel in a particular
filtered version, computing a weighted average of the intensity
values of the particular pixel and pixels within a region adjacent
to the particular pixel, wherein at least some of the intensity
values used in the computation of the weighted average are weighted
in response to the intensity value of at least one pixel in the
defect channel; and wherein the corrected digital image is further
produced in response to the plurality of filtered versions of the
defect channel.
53. The method of claim 52, wherein the filtered versions of the at
least one channel are created in log space.
54. The method of claim 51, further comprising: dividing the at
least one channel into segments; creating, using digital circuitry,
a plurality of filtered versions of each segment of the at least
one channel wherein pixels of the filtered versions of each segment
of the at least one channel are determined by, for a particular
pixel in a particular filtered version, computing a weighted
average of the intensity values of the particular pixel and pixels
within a region adjacent to the particular pixel and within the
same segment as the particular pixel, and wherein at least some of
the intensity values used in the computation of the weighted
average are weighted in response to the intensity value of at least
one pixel in the defect channel; and wherein the the corrected
digital image is further produced in response to the plurality of
filtered versions of each segment of the at least one channel.
55. The method of claim 51, further comprising: prior to creating
the plurality of filtered versions of the at least one channel,
altering a plurality of pixels of the at least one channel by
subtracting, for a particular pixel, a first value responsive to
the intensity value of a pixel in the defect channel spatially
corresponding to the particular pixel's position in the at least
one channel from a second value responsive to the intensity value
of the particular pixel.
56. The method of claim 51, wherein the corrected digital image is
produced by: creating a series of frequency bands, the frequency
bands essentially contiguous and responsive to the at least one
channel and the plurality of filtered versions of the at least one
channel; and adding the frequency bands together to form the
corrected digital image.
57. The method of claim 55, wherein the corrected digital image is
produced by: creating a series of frequency bands, the frequency
bands essentially contiguous and responsive to the altered version
of the at least one channel and the plurality of filtered versions
of the altered version of the at least one channel; and adding the
frequency bands together to form the corrected digital image.
58. The method of claim 52, wherein the corrected digital image is
produced by: creating a first series of frequency bands, the first
series of frequency bands essentially contiguous and responsive to
the at least one channel and the plurality of filtered versions of
the at least one channel; creating a second series of frequency
bands, the second series of frequency bands essentially contiguous
and responsive to the defect channel and the plurality of filtered
versions of the defect channel and wherein each of the second
series of frequency bands corresponds to one of the frequency bands
of the first series of frequency bands; subtracting each
corresponding frequency band in the second series of frequency
bands from a frequency band in the first series of frequency bands
to form a third series of frequency bands; adding the third series
of frequency bands together to form the corrected digital
image.
59. The method of claim 58, wherein the subtracting comprises a
bounded subtraction wherein the bounded subtraction, for a
particular pixel in a particular frequency band, comprises the
selection of a result of a plurality of subtractions including a
first subtraction proportional to the difference between a first
value proportional to the intensity of the particular pixel in the
particular frequency band in the first series of frequency bands
and a second value proportional to the intensity of the particular
pixel in the particular frequency band in the second series of
frequency bands and a second subtraction proportional to the
difference between the first value and a third value proportional
to the intensity of the particular pixel in the particular
frequency band in the second series of frequency bands.
60. The method of claim 58, wherein the second series of frequency
bands is further responsive to red residue in the defect channel
such that the second series of frequency bands is adjusted to
reduce red residue in the defect channel.
61. The method of claim 58, wherein the weighted average comprise
an average selected from the group consisting of a median average,
an arithmetic average, a geometric average, a mean average, and a
mode average.
62. The method of claim 1, wherein the first pixel is filtered
using a window filter with feathered edges.
63. The method of claim 1, further comprising: filtering a second
pixel comprising a part of the defect channel to produce a second
filtered pixel the filtering of the second pixel responsive to the
intensity of the second pixel and a plurality of additional pixels
in the neighborhood of the second pixel, wherein at least one of
the pixels used in filtering the second pixel is weighted in
response to the intensity value of at least one pixel in the defect
channel.
64. The method of claim 63, wherein the corrected digital image is
produced in response to the filtered pixel, the second filtered
pixel, the at least one channel, and the defect channel.
65. The method of claim 1, further comprising: prior to filtering
the first pixel, altering at least a second pixel of the at least
one channel by subtracting a first value responsive to the
intensity value of a pixel in the defect channel spatially
corresponding to the second pixel's position in the at least one
channel from a second value responsive to the intensity value of
the second pixel.
Description
TECHNICAL FIELD OF THE INVENTION
This invention relates generally to image processing and more
particularly to a method and system for altering defects in a
digital image.
BACKGROUND OF THE INVENTION
Tangible images, such as photographic images, may have surface
defects such as scratches, fingerprints, or dust particles. Such
defects may occur, in the case of photographic images, in a
transparency or negative as well as in a photographic print of a
transparency or negative. Such defects often undesirably degrade a
photographic image.
In the field of image processing, digital images derived from
photographic images using a scanner most often include the defects
present in the underlying photographic image. Because digital
images are subject to mathematical manipulation, if image defects
may be identified and distinguished from image detail, then those
defects can be removed, either partially or completely.
A defect channel comprising a digital signal proportional to the
defects in a photographic image may be created by scanning the
photographic image using an infrared light source and an infrared
light sensor. Infrared light will tend to pass through developed
photographic film with nearly complete transmission because the dye
in various layers of the photographic film does not fully absorb
infrared light. On the other hand, where defects are present, a
portion of the infrared light will tend to be refracted from the
optical path before passing through the film. Thus, defects in the
photographic image will tend to show up in a defect channel
produced using an infrared light source and infrared sensor. In
reflective scanners, a defect channel may be obtained by examining
the difference between images obtained when the image being scanned
is illuminated by light sources at different angles. The challenge
is to use the defect channel to automatically alter defects in a
digital image, while making as few undesirable changes to the
digital image as possible.
SUMMARY OF THE INVENTION
One aspect of the invention is a method for altering defects in a
digital image. At least a first pixel of a first channel of a
digital image is filtered using digital circuitry to produce a
filtered pixel by averaging the intensity of the first pixel and a
plurality of additional pixels in the neighborhood of the first
pixel. At least one of the pixels is weighted in response to the
intensity value of at least one pixel in a defect channel
associated with the digital image. A corrected digital image is
produced using the digital circuitry in response to the filtered
pixel and the first channel of the digital image.
The invention has several important technical advantages. Various
embodiments of the invention may have none, one, some, or all of
these advantages without departing from the scope of the invention.
The invention allows automatic alteration of defects in a digital
image based upon a defect channel having a signal proportional to
defects in the digital image. The invention allows such alteration
in the image substantially with little or no damage to the
underlying image. Because the invention filters pixels of the
digital image with pixels weighted based upon the expected
reliability of each pixel, areas of strong defect are more easily
excised without causing significant damage to the area surrounding
the defect. Thus, for reasonably sized defects, the invention may
automatically alter defects in a way that will most often produce a
more pleasing image than the original image that included the
defect. The desirability of the resulting image may depend upon the
size of the defect and the amount of valid image information
remaining in the digital image.
BRIEF DESCRIPTION OF THE DRAWINGS
For a more complete understanding of the present invention and the
advantages thereof, reference is now made to the following
descriptions taken in conjunction with the accompanying drawings in
which:
FIG. 1 illustrates a block diagram of a general purpose computer
that may be used in accordance with the present invention;
FIG. 2 illustrates an example of a scanner that comprises an
embodiment of the present invention;
FIG. 3 illustrates a flow chart describing the alteration of a
defect in a digital image in accordance with one method of the
present invention;
FIG. 4 illustrates a flow chart describing adjustment of an image
in response to weighted averages produced by filtering an image in
accordance with the present invention;
FIG. 5 illustrates a flow chart describing a second method of
altering defects in a digital image in accordance with the present
invention; and
FIG. 6 illustrates a graph of an example weighting function that
may be used with the present invention.
DETAILED DESCRIPTION OF THE INVENTION
The preferred embodiment of the present invention and its
advantages are best understood by referring to FIGS. 1-5 of the
drawings, like numerals being used for like and corresponding parts
of the various drawings.
FIG. 1 illustrates a general purpose computer 10 that may be used
for image enhancement in accordance with the present invention.
Specifically, general purpose computer 10 may comprise a portion of
a digital image processing system and may be used to execute
applications comprising image enhancement software. General purpose
computer 10 may be adapted to execute any of the well known MS-DOS,
PC-DOS, OS2, UNIX, MAC-OS and Windows operating systems or other
operating system. General purpose computer 10 comprises processor
12, random access memory (RAM) 14, read only memory (ROM) 16, mouse
18, keyboard 20, and input/output devices such as printer 24, disk
drives 22, display 26 and communications link 28. The present
invention includes programs that may be stored in RAM 14, ROM 16,
or disk drives 22 and may be executed by processor 12.
Communications link 28 is connected to a computer network but could
be connected to a telephone line, an antenna, a gateway, or any
other type of communication link. Disk drive 22 may include a
variety of types of storage media such as, for example, floppy disk
drives, hard disk drives, CD ROM drives, or magnetic tape drives.
Although this embodiment employs a plurality of disk drives 22, a
single disk drive 22 could be used without departing from the scope
of the invention. FIG. 1 only provides one example of a computer
that may be used with the invention. The invention could be used
with computers other than general purpose computers as well as
general purpose computers without conventional operating
systems.
General purpose computer 10 further comprises scanner 30 that may
be used to scan images that are to be enhanced in accordance with
the teachings of the invention. In this embodiment, enhancement may
be performed by software stored and executed by scanner 30 with the
results stored in a storage medium comprising a part of scanner 30
and/or in any of the storage devices of general purpose computer
10. Alternatively, software for image enhancement may be stored in
any of the storage media associated with general purpose computer
10 and may be executed by processor 12 to enhance images scanned by
scanner 30. In addition, image enhancement could occur both
internally within scanner 30 and in general purpose computer 10
without departing from the scope of the invention. Scanner 30 may
comprise a film scanner or a flatbed scanner of any type without
departing from the scope of the invention. Image enhancement may
also be performed using special purpose digital circuitry contained
either in scanner 30, general purpose computer 10, or in a separate
device. Such dedicated digital circuitry which may include, for
example, state machines, fuzzy logic, etc.
FIG. 2 illustrates an exemplary scanner 34 constructed in
accordance with the invention. Scanner 34 comprises processor 36,
storage medium 38 and scanning hardware 40. Processor 36 controls
the operation of scanning hardware 40 by executing control software
44 stored in storage medium 38. Although a single storage medium
has been illustrated for simplicity, storage medium 38 may comprise
multiple storage mediums as well as comprising storage mediums of
different types. Thus, for example, control software 44 may be
stored in ROM memory, RAM memory, or on a disk drive. Scanning
hardware 40 is used to convert an analog image into a digital image
utilizing some type of optical circuitry. In addition, optical
circuitry may also be used to produce a defect channel proportional
to defects in the analog image. Any type of optical circuitry could
be used for scanning hardware 40 without departing from the scope
of the invention. For the defect channel, a light source comprised
mostly of energy outside the visible spectrum and a matching sensor
may be used to create the defect channel. For example, an infrared
light source and sensor such as those typically used in image
processing applications involving photographic images may be used
for this aspect of the scanning hardware.
If scanner 34 comprises a reflective scanner, then a defect channel
can be derived from a plurality of scanned versions of the image in
the visible spectrum. Such a defect channel may be derived by
illuminating the image being scanned at two or more angles and
calculating changes in the scanned image at the plurality of
angles. Defects will tend to affect the light differently when
illumination is made from different angles. Other types of scanning
hardware may be used to create a defect channel without departing
from the scope of the invention.
After scanning hardware 40 has scanned an image, that image may be
enhanced (by altering defects within it) in accordance with the
invention using image processing software 42, which is stored in
storage medium 38. Alternatively, rather than using software
running on a processor (one type of digital circuitry), the
invention may employ other types of digital circuitry comprising
any type of dedicated digital hardware to alter defects in the
digital image. This hardware may be a part of scanner 34 or general
purpose computer 10 as discussed above. Similarly, the scanned
image may be stored in storage medium 38 as may the enhanced image.
Alternatively, scanner 34 may not have any image processing
software 42. Such software instead may be provided on general
purpose computer 10 for enhancement of an image received from
scanner 34. Enhancement may occur both in scanner 34 and general
purpose computer 10 as well. Accordingly, a scanned image and/or an
enhanced scanned image may be provided by scanner 34 to the general
purpose computer 10 through a communications port (not explicitly
shown). The defect channel may be similarly provided. Although one
embodiment of an exemplary scanner 34 that may be used for image
enhancement in connection with the invention has been illustrated,
other scanners may be used without departing from the scope of the
invention.
FIG. 3 illustrates a flow chart describing a method employed by one
embodiment of the present invention to enhance a digital image. The
image enhancement described herein may be carried out using
computer software, as can any of the processes described below.
That software, as discussed above, may be executed by scanner 34,
by general purpose computer 10, or a combination thereof. A digital
image received from other than scanner 30 may be enhanced in
accordance with the invention.
The method described in FIG. 3 may be used to alter defects in many
types of images such as color photographic images (either negative
print or transparency), black and white images (either negative
print or transparency and including black and white images derived
from photographic film with multiple layers), other monochromatic
images, x-rays or any other type of image stored on film. The
invention may also be used. to alter defects in any image on a
tangible medium that may be scanned using a scanner.
In step 50, an image is scanned to create a digital image and
defect channel. As noted, however, this step could be omitted and
the invention carried out on a image that has been previously
scanned and has a defect channel associated with it. In the case of
a color image, the digital image will typically be comprised of
three channels: a red channel, a green channel, and a blue channel.
Each channel is comprised of a series of pixels with each pixel
having an intensity value associated with it corresponding to the
intensity of the particular color of light at that spatial location
in the original image. Other types of color images can be used
without departing from the scope of the invention. In addition, it
is within the scope of the invention to convert a color image into
a black and white image for alteration of defects in the image. The
enhanced image with the altered defects could then be converted
back into a color image. The methods of the invention could also be
applied to a single color channel of a digital image and the same
correction could be applied to all channels of the digital image. A
black and white image may comprise one or more channels similarly
made up of pixels. Other types of images may also comprise one or
more similar channels. The invention can be used for any of these
types of images.
The defect channel comprises a series of pixels, each having an
intensity value either directly proportional or inversely
proportional to defects (or the lack thereof) in the original
image. Such a defect channel may be created, for example, using an
infrared light source and infrared sensor of the kind commonly used
in image processing applications involving photographic film. Other
types of light sources and sensors may be used to create the defect
channel, such as described above in connection with reflective
scanners, for example. Any other method may be used to generate a
suitable defect channel without departing from the scope of the
invention. Such a defect channel will ordinarily produce a signal
having a stronger correlation to defects in the original image and
a weaker correlation to the visible image itself.
Steps 52-58 comprise a method for altering defects in a digital
image to produce an enhanced digital image. Before describing the
process in detail, it may be helpful to describe it generally. To
enhance a digital image by altering defects in the digital image,
the invention corrects defects in at least two frequency bands. The
term "correction" or "correct" when used in this application,
refers broadly to altering defects in a digital image. An image
defect does not need to be completely removed or completely
corrected to fall within the scope of the invention. Accordingly,
defect correction includes, but is not limited to, reduction or
other alteration of a defect in a digital image. The frequency
bands for the image are created using one or more filters which
average pixels in the neighborhood of a pixel in question with each
pixel weighted according to its expected reliability. The expected
reliability is determined by the intensity value in the defect
channel of the pixel in question or the pixel in question plus a
series of pixels in the neighborhood of the pixel in question.
Pixels that appear to be more reliable based upon the defect
channel are weighted more heavily than those with a low expected
degree of reliability. The defect channel may be similarly divided
into frequency bands.
The effect of each weighted averaging operation is to create a low
pass filtered version of the original image. Where multiple
averaging operations are performed, multiple low pass filtered
versions of the original image are derived, most often with
different bandwidths. These multiple lowpass filtered versions,
along with the original image may be used to derive a
representation of the original image in multiple contiguous
frequency bands. By subtracting one low pass filtered version from
another, a bandpass representation for one band may be derived
where common frequencies are eliminated. A series of bands can be
created similarly, as further described below.
The filtered representations of the digital image may then be
processed such that the original image is divided into a plurality
of frequency bands which collectively make up all or substantially
all frequencies present in the original image but overlap little,
if at all. These bands may then be recombined to produce an image
with defects removed.
The same filtering operation may optionally be performed on the
defect channel. Defects may then be further removed, frequency band
by frequency band, by subtracting the defect channel bands from the
image frequency bands. An enhanced image is then constructed by
combining the corrected individual frequency bands back together
again.
In step 52, data is optionally converted to log space for the
defect channel and for each channel of the digital image where
defect correction is to be performed. The conversion of data to
logarithmic space may enable the remaining steps of the method to
be performed more easily than if the conversion were not made. For
example, several of the operations described below involve
additions and subtractions in log space, but could involve division
by zero in non-log space. Because division by zero can lead to
erroneous results, it may be more convenient to carry out the
process of the invention in log space. However, the performance of
mathematically equivalent functions outside of log space could also
be used in connection with the invention.
Although any type of logarithmic calculation can be used, one
embodiment of the invention uses the conversion to log space
described by Formula 1. ##EQU1##
In Formula 1, "x" represents the intensity value of the pixel to be
converted to log space. Each such pixel in this embodiment
comprises a twelve-bit intensity value. Other numbers of bits can
be used without departing from the scope of the invention.
Depending upon how the invention is carried out, convenient
conversions may be made to take into account the capabilities of
digital hardware used to perform calculations in accordance with
the invention.
If data is converted to log space, and the defect channel is
suitable, then a first type of defect correction may be carried out
by subtracting the defect channel from each of the visible image
channels. Such a subtraction may be a direct subtraction or a
bounded subtraction, analogous to the bounded subtraction described
below in connection with FIG. 5. This subtraction may also take
into account red leakage and clear film values where the defect
channel is derived using an infrared source and sensor in a film
scanner. Any method of taking into account these effects (which are
described below) is within the scope of the invention. Defect
channels obtained using infrared light in film scanners will often
be suitable for this optional enhancement of the image. Even in
such a case, however, this step is optional. An alternative to this
optional subtraction is to subtract the defect channel from the
image channel, frequency band by frequency band, after enhancement
by weighted filtering as described below. This alternative is also
optional and may be done where a suitable defect channel is
available. The defect channel may be suitable where pixel intensity
values in the defect channel are proportional, either directly or
inversely, to defects and vary approximately linearly with the
amount of light blocked by the defect.
In step 54, the image and defect channel are filtered by taking
weighted averages of pixels within varying distances from specific
pixels. As noted above, each channel of the image may be filtered,
the channels may be combined into a single channel and the combined
channel filtered or a subset of the channels can be combined or
filtered individually and the defect correction applied based upon
results obtained from those channels without departing from the
scope of the invention. Depending upon the type of enhancement
used, the filtering of the defect channel is also optional.
The weighting applied to a specific pixel in calculating the
weighted averages may be based upon the expected reliability of
that pixel. The expected reliability of a pixel may be determined
by using the defect channel. As used herein, "a weighted average"
or "averaging" or any similar term refers to any type of average
such as a median average, mode average, mean average, arithmetic
average, or geometric average. The calculation of weighted averages
of pixels around the pixels in each channel of the digital image
has the effect of filtering each channel with different strengths
of low pass filters. The effect of weighting each pixel (or a
subset of the pixels) involved in the calculation of the averages
based upon the expected reliability of each pixel dampens the
effect of the defect in each filtered version of the original
channel of the digital image.
An example may illustrate the calculation of the weighted averages.
In this example, four filtering operations are performed: a
3.times.3 weighted average, a 5.times.5 weighted average, a
9.times.9 weighted average, and a 17.times.17 weighted average. In
this example, each of these four weighted averages is computed for
each pixel in the digital image. Any pixel outside the boundaries
of the image is set to zero for purposes of these calculations. For
a particular pixel, the 3.times.3 weighted average is computed
using a 3.times.3 matrix of pixels with the pixel in question at
the center of the matrix. Thus, pixels in the neighborhood of the
pixel in question are used to calculate the weighted average. The
weight applied to each pixel corresponds to its expected
reliability as determined using a corresponding pixel or pixels in
the defect channel. The weighted average is computed by summing the
product of the intensity of each pixel times the weight and
dividing the total sum by the sum of the weight values that were
applied to each pixel. The 5.times.5, 9.times.9, and 17.times.17
filters are calculated similarly.
The calculation of the four discussed weighted averages at each
pixel results in four low pass filtered versions of the original
channel of the digital image. The 3.times.3 weighted average
applied to each pixel of the image channel produces a low pass
filtered version of the image having spatial frequencies between
essentially zero and one-third of the maximum spatial frequency
possible in the image. Similarly, the 5.times.5 weighted average
produces a low pass filtered version of the original channel of the
digital image having frequencies predominantly between zero and
one-fifth of the maximum spatial frequency possible in the image.
The 9.times.9 and 17.times.17 weighted averages produce filtered
versions with frequencies predominantly between zero and one-ninth
the maximum spatial frequency possible in the image and
predominantly between zero and one-seventeenth the maximum spatial
frequency possible in the image. Due to the weighting applied to
each pixel during this filtering operation, the filtered versions
in each frequency band have had the effects of defects reduced. Low
pass filtering alone tends to dampen the effects of defects, but
may also blur image detail. The weighting may dampen the effects of
a defect more than low pass filtering alone would with
proportionally less reduction of image detail. This example will be
used below to further illustrate adjustment of the image and defect
channel. The same filtering operation may also be performed on the
defect channel, if desirable for use in further reduction of
defects.
Numerous options may be used for calculating the weighted average
of a pixel and other pixels in the neighborhood of the pixel. The
filter can have a square shape (as in the example above), a circle
shape, or any other type of shape without departing from the scope
of the invention. The filter may be symmetric or asymmetric and may
be centered or not centered on the pixel in question. The filter
may also be a window with feathered edges. In addition, any number
of filters can be used. The use of more filters will allow division
of the image into a greater number of frequency bands. The use of
additional filters, however, requires additional calculation. Less
filters than those used in the above example can also be used
without departing from the scope of the invention.
The filters in the example above had particular dimensions each of
which are approximately double the dimensions of the previous
filter. Any size filters can be used, however, without departing
from the scope of the invention. In the example above, any pixel
outside the boundaries of the image was set to an intensity value
of zero for purposes of the weighted average calculation. Other
boundary conditions could be used without departing from the scope
of the invention; for example, soft edge boundary conditions such
as a triangle edge or gaussian edge could be used.
In the example above, the weighted average was computed for each
pixel in each channel of the digital image and in the defect
channel. The weighted average could also be performed on a subset
of the pixels in a channel or on all of the pixels in a subset of
the channels without departing from the scope of the invention. In
addition, different weights could be used for each frequency band
and could be omitted in some frequency bands. Different weights
could also be used for different channels.
The weighting used for a particular pixel can be determined in
several ways. The weighting could depend upon the intensity of a
spatially corresponding pixel in the defect channel. Where the
weighting is applied to the defect itself, the weighting could be
determined based upon the intensity of each pixel involved in the
calculation. The same or different weightings could be used for the
defect channel and one or more image channels. The defect channel,
however, may be blurred compared to the visible channel due to
focal shifts, the nature of the defects, and inconsistent
registration between the visible channels of the digital image and
the defect channel, for example. Accordingly, it may be desirable
to estimate the reliability of a pixel and determine its weight by
taking into consideration a plurality of pixels surrounding a pixel
in the defect channel corresponding to the pixel in a visible image
channel that is the subject of the weighted average calculation.
Alternatively, if blurring effects are insignificant and
registration error is fairly constant, then the pixel used in the
defect channel could be chosen while compensating for the constant
registration error.
The particular weighting function chosen may depend upon the
characteristics of the particular scanner used to create the defect
channel. The weighting applied may be a function of the intensity
value of a pixel or pixels in the defect channel. Where multiple
pixels are used to determine a weight, an average or weighted
average of the pixels may be used to come up with an average
intensity used to determine a weight.
Any type of function may used to relate a weight to the intensity
of a pixel in the defect channel or average intensity of a series
of pixels in the defect channel. A straight line may be used to
establish this function or any other type of curve may be used. In
this example, a high threshold and a low threshold may be
determined based upon the characteristics of an infrared channel
produced by a scanner. For example, in one embodiment, a weight of
one may be assigned for all pixel values greater than a particular
high threshold where the threshold provides high predictability
that no defect is present. A weight of zero may be assigned when
the intensity value is below a certain low threshold indicating a
high probability that a defect is present with little or no image
detail remaining. For points with an intensity in between these
threshold values, a straight line or other curve can be used to
connect the two thresholds to establish a continuous function for
weights between zero and one. Pixels with intensity in this middle
region tend to be ones that are defective but some image detail
remains to allow enhancement of the digital image to remove the
defect but maintain some image detail. In the case of a defect
channel derived from a reflective scanner using illumination from
multiple angles, the intensity of the pixels in the defect channel
may lack a linear relationship to the amount of light that passes
through a defect. With such a defect channel, a discrete set of
weights such as 0, 1/2 and 1, may be used.
An example of a weight function is illustrated in FIG. 6. In FIG.
6, infrared intensity in the defect channel will be high when no
defect is present as most or all of the infrared energy is allowed
to pass through the film. Where a defect is present, however,
infrared intensity will be low. In this example, all likely
defective pixels (those with an infrared intensity in the defect
channel less than 20 percent) are set to a weight of zero. All
pixels with a high probability of being nondefective pixels (those
with an infrared intensity in the defect channel greater than 64
percent of the maximum intensity are assigned a weight of one).
Pixels in the region in between are assigned a weight based upon
the illustrated curve. Again, more complicated functions may be
used without departing from the scope of the invention.
The weighting applied may also compensate for effects such as
leakage from the red channel of a color image into the defect
channel. This leakage may result because infrared light sources
and/or sensors are residually sensitive to the cyan dye in an image
used to modulate the red region of the visible spectrum. This
effect manifests itself as an apparent leakage of the red image
into the infrared image. The effects of red leakage can be taken
into account when establishing a weight value. For example, an
overall red leakage value for an image may be calculated. This red
leakage value can then be used to establish a constant to be
multiplied times a pixel intensity value in the red channel. This
product may represent an estimate of the amount of the red channel
present in the defect channel for that particular pixel. Thus, in
calculating a weight, this product may be subtracted from the
intensity of the pixel in the defect channel.
Similarly, a portion of the intensity in the defect channel is
proportional to the intensity that would result if infrared light
was passed through clear film. Different types of film produce
different intensity values when infrared is passed through the
clear film. Accordingly, the weight may also be adjusted by
subtracting the average clear film value for a particular digital
image.
Depending upon the type of filtering employed, one could filter the
image once with one or more filters and then filter the corrected
image again with different filters. A decision could be made as to
whether to apply the second filtering step based upon the estimated
size of defects as determined by the defect channel.
In Step 56, the channels of the digital image and the defect
channel are adjusted in response to the weighted averages to lessen
the effects of defects in the image. The invention includes any use
of the weighted averages computed in Step 54 to adjust an image to
lessen the effects of defects in the image. A specific method for
adjusting the digital image in response to the weighted averages
will be discussed below in connection with FIG. 4. Such adjustment
could occur in the time domain or the frequency domain and, when in
the spatial domain, in log space or any other space. After the
image has been adjusted to lessen the effects of defects, the
process terminates in Step 58.
Besides the method described below in connection with FIG. 4,
another possible method of enhancement is to create a series of
essentially contiguous frequency bands based upon the plurality of
weighted averages of the original image and the original image
itself. The process of creating these bands may be as described
below in connection with step 60 of FIG. 4. After these bands have
been created, they may be added together to form an enhanced
version of the original image with the defects reduced. This method
may be used, for example, where the defect channel is unsuitable
for subtraction from the image. This method could, however, be used
even where a defect channel is suitable for correction. The
enhanced image may also be further enhanced by other operations
without departing from the scope of the invention. If the pixel
data was converted to log space in step 52, the enhanced pixel data
may be converted back to the space of the original image using a
suitable inverse formula such as an inverse to Formula 1.
FIG. 4 illustrates one example of a method for adjusting images in
response to weighted averages to lessen the effects of defects in
the digital image. In step 60, the filtered digital image (which
comprises the original digital image and the filtered versions of
the digital image calculated in Step 54) and the filtered defect
channel (which consists of the original defect channel and the
filtered versions of the defect channel obtained in step 54) are
separated into frequency bands in response to the calculated
weighted averages. In general, the filtered images and the original
image are used to create a series of frequency bands with little or
no overlap representative of the original digital image and
original defect channel with defective pixels suppressed due to the
weighting that took place during the filtering operation. This
method assumes that the defect channel was filtered in FIG. 3.
Using the example discussed above in connection with FIG. 3, five
frequency bands may be created using the four weighted averages
calculated. For purposes of the following, F.sub.m represents the
maximum spatial frequency that can exist in the digital image. For
a particular channel of the digital image or for the defect
channel, the five frequency bands corresponding to the channel in
question may be created using the weighted averages computed for
that channel. The frequency band from approximately 1/3 F.sub.m to
F.sub.m comprises the difference between the original channel and
the 3.times.3 weighted average filtered version of that channel.
The frequency band from approximately 1/5 F.sub.m through 1/3
F.sub.m may be calculated by subtracting the 5.times.5 weighted
average version of the channel from the 3.times.3 weighted average
version of the original channel. The frequency bands from
approximately 1/5 F.sub.m to 1/9 F.sub.m and approximately 1/17
F.sub.m to 1/9 F.sub.m may be determined by subtracting the
9.times.9 weighted average version of the original channel from the
5.times.5 version of the original image channel and by subtracting
the 17.times.17 weighted average version of the original channel
from the 9.times.9 weighted average version of the original
channel, respectively. Finally, the frequency band from
approximately zero to 1/17 F.sub.m is represented by the
17.times.17 weighted average version of the original image
channel.
In general, the low pass filtered versions of the image and defect
channel created using the weighted averages can be used to divide
each channel of the image as well as the defect channel into
contiguous frequency bands where some or all of the frequency bands
have had the effect of defects suppressed using the weighted
average calculations.
In Step 62, the red residue in each frequency band of the defect
channel may be removed. Optionally, such residue may be removed
using a bounded calculation to allow some variance. If infrared
sources and sensors (or other sources and sensors) are used that do
not leave red residue in the defect channel, then the step may be
omitted without departing from the scope of the invention. One
option for removing the red leakage from the defect channel is to
subtract from the intensity value of each pixel in the defect
channel, the product of the intensity of the corresponding pixel in
the red channel multiplied by an average red leakage constant
representing the average red leakage for the entire digital image.
This difference may be divided by the difference between one and
the red leakage value to properly normalize the result.
Alternatively, because red leakage may vary within regions of
particular images, it may be useful to use a bounded subtraction to
allow for some variance in localized red leakage values.
An example of a bounded subtraction for red residue will be
provided in connection with FIG. 5 below. In general, two or more
subtractions are performed from the intensity value in the defect
channel. The product of the intensity value for the pixel in
question in the red channel is multiplied by the red leakage
constant adjusted upward for one subtraction and adjusted downward
for a second subtraction from the intensity value in the defect
channel. These results are divided by the difference between one
and the appropriate adjusted red leakage constant. If both
calculations have results having the same sign (i.e., both are
positive or both are negative) then the smaller result is chosen.
If the results have opposite signs, then the pixel in question is
set to zero. Besides allowing variance in the red leakage value,
such a bounded subtraction may compensate for registration error
between visible channels of the digital image and the defect
channel as well as for blurring that may occur in the defect
channel.
In Step 64, defects are further removed from the image by
subtracting the relevant frequency band of the defect channel
obtained in Step 62 from the relevant frequency band in each image
channel obtained in Step 60. Optionally, a bounded calculation such
as that described in connection with Step 62 may be used to allow
for some variance caused by registration error, blurring, etc. in
the defect channel.
Next, in Step 66, the frequency bands are recombined through a
summation to form an enhanced image with the original defect
removed. In Step 67, the enhanced image is converted back from log
space to the original space from which it was derived using a
suitable inverse formula such as an inverse to Formula 1. If the
enhanced image was created in a space other than log space, then
Step 67 may be omitted without departing from the scope of the
invention. If Step 60 through 66 were carried out in the frequency
domain, then the enhanced image may be reconverted back to the time
domain in Step 67.
FIG. 5 illustrates a flow chart describing a second method of
enhancing an image by altering defects in the image in accordance
with the invention. In this embodiment, an image may be divided
into segments and defects processed within each individual segment.
By dividing the image into segments, the amount of storage space
used at any one time to enhance the image may be reduced, and the
amount of computation required may be reduced. Thus, in this
embodiment, the image is divided into segments and a process
similar to that discussed above in connection with FIGS. 3 and 4 is
applied to each segment individually as if that segment comprised
the entire image.
In step 68 an image is scanned to create a digital image comprising
one or more channels and a defect channel. All of the options
discussed above in connection with step 50 are available for this
embodiment of the invention. Next, in step 70 the data from the
defect channel and the channels of the digital image is converted
to log space. Again, any of the options discussed above in
connection with step 52 may be employed in step 70, including
optional subtraction of the defect channel from the visible
channel.
In step 72, each channel of the digital image and the defect
channel may be divided into segments. Any size or shape of segments
may be used without departing from the scope of the invention. In
one embodiment, each channel of the digital image and the defect
channel are divided into 8.times.8 segments. This example will be
used to illustrate the remainder of the steps of the method
illustrated in FIG. 5.
In step 74, weighted averages of pixels within varying distances
from specific pixels in a segment of the defect channel and a
segment of each channel of the image are computed. These weighted
averages are computed similarly to the weighted averages that were
computed in step 54. However, the weighted averages computed in
step 74 are computed for an individual segment of the image,
assuming that the segment comprises the entire image. Accordingly,
even if the 8.times.8 segment is surrounded by other 8.times.8
segments, the pixels beyond the boundaries of 8.times.8 segment in
question are treated as beyond the image boundary and any pixel
beyond those boundaries are treated as having an intensity of zero
(or other boundary condition).
All of the options for filtering and weighting discussed above in
connection with step 54 may be employed in step 74 for the
embodiment disclosed in FIG. 5 (including the option of not
filtering the defect channel). However, because the segment in this
example is 8.times.8, an 8.times.8 filter is the maximum sized
filter that will be used in this example. The weight function, in
this embodiment, takes into account red leakage and clear film
effects as discussed above in connection with step 54. The weight
for a particular pixel for the weighted average calculations may be
determined, for example, using Formula 2.
In Formula 2, x and y represent the coordinates of a particular
pixel within an 8.times.8 segment. D.sub.in represents the defect
channel received from the scanner in step 68. R.sub.L comprises a
constant value representative of average red leakage in either the
segment in question or in the entire digital image. R.sub.IN
represents the red channel of the digital image. C.sub.F is a
constant representing the average clear film value for either the
segment in question or the entire digital image. The weight
function of Formula 2 is constrained such that the weight varies
between zero and one. A plot of the weighting function versus the
intensity in the defect channel would look similar to the curve
illustrated in FIG. 6. Any other weight function could be used
without departing from the scope of the invention.
In this example, two different weighted averages are computed for
each channel. A 3.times.3 weighted average is computed. In
addition, an 8.times.8 weighted average is computed. The 8.times.8
average covers the entire segment, and as a result, a single scalar
value may be used to represent the result of this weighted average.
The 3.times.3 weighted average for the red, green, blue, and defect
channels may be computed using Formulas 3 through 6. ##EQU2##
In these Formulas, R.sub.in, G.sub.in, B.sub.in, D.sub.in,
respectively, represent the red, green, blue, and defect channels
of the digital image that was scanned in step 68.
The 8.times.8 weighted average for the red, green, blue, and defect
channels may be calculated using Formulas 7 through 10,
respectively. As noted, each of these Formulas produces a scalar
result. ##EQU3##
R.sub.3L represents a weighted low-pass filtered version of the red
channel of the digital image having a frequency band between
approximately zero and 1/3 F.sub.m, where F.sub.m is the maximum
spatial frequency possible for the segment of the digital image.
G.sub.3L, B.sub.3L, and D.sub.3L cover a similar frequency band for
the green, blue, and defect channels respectively. R.sub.8L
represents a weighted low-pass filtered version of the red channel
of the digital image having a frequency band between approximately
zero and 1/8 F.sub.m. G.sub.8L, B.sub.8L, and D.sub.8L represent
weighted low-pass filtered versions of the green, blue, and defect
channels, respectively in the same frequency band.
In step 76, the segment of the image is adjusted to lessen the
effects of defects in the image in response to the weighted
averages of each image channel and the defect channel obtained in
step 74 and the original image channels and defect channels. Any of
the options discussed above in connection with step 56 and steps 60
through 67 may be used in connection such adjustment.
In this example, a version of the original image and original
defect channel, with effects of the defect partially suppressed due
the weighted averages computed in step 74, is created in three
separate frequency bands. For each image channel and the defect
channel, a first frequency band is obtained having frequencies
between approximately zero and 1/8 F.sub.m. A second frequency band
extends from approximately 1/8 F.sub.m to 1/3 F.sub.m. A third
frequency band extends from approximately 1/3 F.sub.m to
F.sub.m.
The first frequency band may be obtained using Formulas 11 through
14.
B.sub.8B =B.sub.8L (13)
Because this frequency band was already calculated in step 74, it
can be used without any further computation.
The third frequency may be computed using Formulas 15 through 18.
In these Formulas, the weighted low-pass filtered version of each
channel in the frequency range of approximately zero to 1/3 F.sub.m
is subtracted from the unfiltered image channel to produce the
frequency band between approximately 1/3 F.sub.m and F.sub.m.
Finally, the second frequency band, the one between approximately
1/8 F.sub.m and 1/3 F.sub.m may be computed using Formulas 19
through 22. In Formulas 19 through 22, the 8.times.8 weighted
filtered version of each image channel is subtracted from the
3.times.3 weighted filtered version of the original image channel.
The process just described for creating a plurality of frequency
bands may employ any of the options discussed above in connection
with step 60.
G.sub.3B (x,y)=G.sub.3L (x,y)-G.sub.8L (20)
Next, the red residue may be removed from the defect channel in
each frequency band. Again, any of the options described above in
connection with step 62 may be used without departing from the
scope of the invention. In this embodiment, a bounded subtraction
is used for two of the frequency band and a simple subtraction is
used for the remaining frequency band. Formula 23 may be used to
subtract the red residue in the first frequency band of the defect
channel. ##EQU4##
In Formula 23, the constant, D.sub.K8, will normally be set to one,
but may vary depending upon the scanner in question. This constant
represents the amount of the red channel that should be subtracted
from the defect relative to the measured average red leakage for
the entire segment and/or the entire image.
Formulas 24 through 26 may be used to subtract the red residue from
the second frequency band of the defect channel. ##EQU5## D'.sub.3B
(x,y)=T1.sub.D3B (x,y) where .vertline.T1.sub.D3B
(x,y).vertline..ltoreq..vertline.T2.sub.D3B (x,y).vertline. and
T1.sub.D3B (x,y)and T.sup.2.sub.D3B (x,y)have the same sign
The bounded subtraction provided for in Formulas 24 through 26
allows for local variance in the red leakage value, as well as for
some registration error and/or blurring in the defect channel as
compared to the visible channel. Formula 24 multiplies a high value
constant, D.sub.K3H against the red leakage constant, R.sub.L.
Formula 25 multiplies a low-value constant, D.sub.K3L, by the red
leakage constant, R.sub.L. D.sub.K3H and D.sub.K3L are the high and
low ranges determining how much of the image content in the red
channel should be subtracted from the defect channel. These
constants may be determined experimentally.
In this embodiment, D.sub.K3H is chosen to be 1.3 while D.sub.K3L
is chosen to be 0.6. These constants will tend to average about
one. The average is a function of the resolution of the system for
the infrared channel versus the red channel. The spread between
these two constants is a function of the accuracy of the scanner
that produces the defect channel. If D.sub.K3H is chosen too large,
then small image patterns may cause large matching defects to be
erased, rendering defects uncorrected in middle frequencies when
they are next to image detail. If D.sub.K3H is chosen too small,
then image residue may remain in the defect record, causing middle
frequency image detail to be erased. If D.sub.K3L is chosen too
large, then too much image detail may be removed from the defect
causing a negative residue that interferes with the ability to
distinguish a defect. If D.sub.K3L is chosen too small, then
visible detail may not be removed in the presence of defect detail
of opposite polarity.
After the results of Formula 24 and 25 are calculated, the revised
version of the frequency band is calculated using Formula 26. If
the results of Formula 24 and Formula 25 have different signs, then
the intensity value is set to zero. If these signs are the same,
then the lesser value produced by either Formula 24 or Formula 25
is chosen. Thus, this bounded subtraction tends to drive the
revised value closer to zero.
Formulas 27 through 32 may be used to remove red residue in the
defect channel for the third frequency band created above. ##EQU6##
T3.sub.D1B (x,y)=T1.sub.D1B (x,y) where .vertline.T1.sub.D1B
(x,y).vertline..ltoreq..vertline.T2.sub.D1B (x,y).vertline. and
T1.sub.D1B (x,y)and T2.sub.D1B (x,y)have the same sign
Formulas 27 through 29 are similar to Formulas 24 through 26. For
this frequency band, in this example, the high-constant D.sub.K1H
is chosen to be 1.0 while the low-frequency constant, D.sub.K1L is
chosen to be 0.3. Similar considerations apply to the choice of
these constants, as applied to the choice of D.sub.K3H and
D.sub.K3L above. In this case, the effects of these constants will
be in the high frequency band. Here, the average of the two
constants is less than one and may be chosen as such to the extent
that heightened frequency defects are blurred relative to the high
frequency image. The range is also wider to accommodate a greater
variance that often occurs at higher frequencies.
The difference between Formulas 27 through 32 and Formulas 23
through 26 involves the addition of Formulas 30 through 31. These
Formulas make a minor adjustment in the value computed by Formula
29 to adjust for residual noise in the high-frequency portion of
the defect channel. In this embodiment, constant D.sub.K1A has a
value of 0.01. If the residual noise constant, D.sub.K1A is set too
high, then detail in small defects in the image may not be removed.
If the value is set too low, then too little residual electronic
noise will be removed from this frequency band of the defect
channel and such noise in the defect channel may appear in the
visible channel(s) as a negative of this noise, causing the
electronic noise in the defect channel to contaminate the processed
visible image.
Next, the defect is removed from the visible image frequency band
by frequency band. Similar bounded subtractions are performed as
were performed when the red residue was removed from the defect
channel above. In this case, the constant for the first frequency
band, R.sub.K8, is set to one, but could be adjusted. For this
frequency band, a direct subtraction is used, rather than a bounded
subtraction. Blurring effects tend not to affect this frequency
band or affect it insignificantly, but a bounded subtraction could
be employed if desired. The constants R.sub.K3H and R.sub.K3L are
set to 1.5 and 0.5 respectively. The constants R.sub.K1H and
R.sub.K1L are set to 1.7 and 0.5, respectively. These values are
also chosen for the corresponding constants for the green and blue
channels. These constants establish the bounds within which
correction is made. Again, one selects whichever number between the
bounds drives the corrected result closet to zero. The upper and
lower bounds will normally average about one, except that one may
set the lower bound a bit lower without damaging the image.
Finally, the defects in the original image may be suppressed even
further by multiplying the pixels resulting from a bounded
subtraction of the defect channel from the image channel by the
weight for a particular pixel calculated using Formula 2.
The defects may be removed from each frequency band of the visible
image channels, using Formulas 33 through 59. Formulas 33 through
35 may be used to remove defect information from the first
frequency band. Formulas 36 through 47 may be used to remove the
defect from the middle frequency band of each channel. Formulas 48
through 59 may be used to remove the defect from the third
frequency band of each channel.
T2.sub.R3B (x,y)=R.sub.3B (x,y)-D'.sub.3B (x,y)R.sub.K3L (37)
T2.sub.B3B (x,y)=B.sub.3B (x,y)-D'.sub.3B (x,y)B.sub.K3L (45)
T2.sub.G1B (x,y)=G.sub.1B (x,y)-D'.sub.1B (x,y)G.sub.K1L (53)
To obtain an enhanced image with the defect removed, the frequency
bands may be recombined for each channel using Formulas 60 through
62.
These enhanced images may be converted back to the original image
space using an appropriate inverse logarithmic function. Again, all
of the options discussed above in connection with step 67 are
applicable to this embodiment as well.
In step 78, it is determined whether there are any more segments of
the digital image to process. If so, then steps 74 and 76 are
repeated for each remaining segment of the digital image. If no
more segments are to be processed, then the method continues in
step 80.
In step 80, it is determined if there are more overlaps to perform.
If not, then the procedure terminates in step 84. If so, then each
channel of the enhanced digital image obtained by using steps 74
and 76 for each segment is divided into segments spatially
different from the earlier segments. Steps 74 and 76 are then
repeated for each of these segments. This aspect of this embodiment
may compensate for results that may be obtained for pixels near the
boundary of segments that were used when the image was first
divided into segments in step 72. By performing defect removal a
second time, the defect may be further suppressed. In this example,
a second overlap is performed using 8.times.8 segments comprising a
2.times.2 corner of four adjoining segments that touch one another
at a common point. These 8.times.8 segments are thus made up of
one-fourth of each of four adjoining segments. Any of the options
described above with respect to step 72 may be used in step 82 in
performing division of the enhanced image into different segments.
Different shapes can be used and different size segments can be
used without departing from the scope of the invention.
Although the inventions described herein involve calculations in
the spatial domain, analogous calculations in the frequency domain
could equivalently be used without departing from the scope of the
invention.
Although the present invention has been described in detail, it
should be understood that various changes, substitutions and
alterations can be made hereto without departing from the sphere
and scope of the invention as defined by the appended claims.
To aid the Patent Office, and any readers of any patent issued on
this application in interpreting the claims appended hereto,
applicants wish to note that they do not intend any of the appended
claims to invoke .paragraph. 6 of 35 U.S.C. .sctn. 112 as it exists
on the date of filing hereof unless "means for" or "step for" are
used in the particular claim.
* * * * *
References