U.S. patent application number 13/467909 was filed with the patent office on 2012-09-06 for system and method for producing and improving images.
This patent application is currently assigned to Avantis Medical Systems, Inc.. Invention is credited to Lex BAYER, Michael Stewart.
Application Number | 20120224026 13/467909 |
Document ID | / |
Family ID | 38668871 |
Filed Date | 2012-09-06 |
United States Patent
Application |
20120224026 |
Kind Code |
A1 |
BAYER; Lex ; et al. |
September 6, 2012 |
SYSTEM AND METHOD FOR PRODUCING AND IMPROVING IMAGES
Abstract
A method for displaying images includes adjusting at least one
characteristic of an image from a first imaging device of an
endoscope to match at least one corresponding characteristic of an
image from a second imaging device of the endoscope. The at least
one characteristic may be one or more of color, contrast and
brightness. An endoscopic system includes an endoscope including a
first imaging device and a second imaging device, and a display
device that displays an image from the first imaging device of the
endoscope and an image from the second imaging device of the
endoscope, wherein the images are sized so that an object, when
placed at the same distance from the imaging devices, appears to
have about the same size in the images.
Inventors: |
BAYER; Lex; (Palo Alto,
CA) ; Stewart; Michael; (Menlo Park, CA) |
Assignee: |
Avantis Medical Systems,
Inc.
Sunnyvale
CA
|
Family ID: |
38668871 |
Appl. No.: |
13/467909 |
Filed: |
May 9, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
11751596 |
May 21, 2007 |
8197399 |
|
|
13467909 |
|
|
|
|
60801748 |
May 19, 2006 |
|
|
|
Current U.S.
Class: |
348/45 ; 348/71;
348/E13.074; 348/E7.085 |
Current CPC
Class: |
A61B 1/0676 20130101;
A61B 1/00179 20130101; A61B 1/045 20130101; A61B 1/0051 20130101;
A61B 1/05 20130101; A61B 1/005 20130101; G02B 23/2423 20130101;
A61B 1/00181 20130101; A61B 1/0125 20130101; A61B 1/0005 20130101;
A61B 1/00009 20130101; A61B 1/00006 20130101 |
Class at
Publication: |
348/45 ; 348/71;
348/E07.085; 348/E13.074 |
International
Class: |
H04N 13/02 20060101
H04N013/02; H04N 7/18 20060101 H04N007/18 |
Claims
1-36. (canceled)
37. An endoscopic system comprising: a first imaging sensor of an
endoscope; a second imaging sensor, wherein the second imaging
sensor faces the first imaging sensor; a controller programmed with
an algorithm to reverse left for right, an image from either the
first imaging sensor or an image from the second imaging sensor;
and a display device that displays, side by side, the images from
the first imaging sensor and the second imaging sensor, wherein one
of the images is reversed left for right.
38. The endoscopic system of claim 37, wherein the controller
reverses the image from either the first imaging sensor or the
second imaging sensor such the location of an object being
simultaneously viewed by the first and second imaging sensors is
correlated in both images.
39. The endoscopic system of claim 38, wherein the object is in the
same general location in both images.
40. The endoscopic system of claim 38, wherein the movement of the
object is correlated in both images.
41. The endoscopic system of claim 37, wherein the controller is
programmed to reverse the image from the first imaging sensor.
42. The endoscopic system of claim 37, wherein the controller is
programmed to reverse the image from the second imaging sensor.
43. A method for displaying images, comprising: acquiring a first
image from a first imaging sensor of an endoscope; acquiring a
second image from a second imaging sensor, wherein the second
imaging sensor faces the first imaging sensor; placing, side by
side, the first image and the second image on a display device; and
using a controller that has been pre-programmed with an algorithm
to reverse one of the images left for right.
44. An endoscopic system comprising: a first imaging sensor; a
second imaging sensor, wherein the second imaging sensor faces that
first imaging sensor; a controller programmed with an algorithm to
size an image from the first imaging sensor and an image from the
second imaging sensor so that an object located at the same
distance from both the imaging sensors appears to have about the
same size in both the images; and a display device that displays
the images from the first imaging sensor and the second imaging
sensor, wherein the images are sized by the controller.
45. A method for sizing images, comprising: acquiring a first image
from a first imaging sensor of an endoscope; acquiring a second
image from a second imaging sensor, wherein the second imaging
sensor faces the first imaging sensor; placing the first image and
the second image on a display device; and using a controller that
has been pre-programmed with an algorithm to size the images so
that an object located at the same distance from both the imaging
sensors appears to have the same size in both images.
46. An endoscopic system comprising: a first imaging sensor of an
endoscope; a second imaging sensor, wherein the second imaging
sensor faces the first imaging sensor; and a controller that stores
image data from the first and second imaging sensors in one
computer file for simultaneous display on a display device.
47. The system of claim 46, wherein the image data from the imaging
sensors are time-correlated.
48. The system of claim 46, wherein the controller places patient
information data from only one image in the computer file for
simultaneous display with the images on a display device.
49. A method for processing images, comprising: acquiring a first
image from a first imaging sensor of an endoscope; acquiring a
second image from a second imaging sensor, wherein the second
imaging sensor faces the first imaging sensor; and using a
controller to place the first and second images in one computer
file for simultaneous display on a display device.
50. The method of claim 49, wherein the first and second images are
time-correlated.
51. The method of claim 49, further comprising: placing patient
information data from only one of the images in the computer file
for simultaneous display with the images on the display device.
Description
[0001] This application claims the benefit of U.S. Provisional
Patent Application No. 60/801,748, filed May 19, 2006, the entire
disclosure of which is incorporated herein by reference.
TECHNICAL FIELD OF THE INVENTION
[0002] The present invention relates to a system and method for
producing and improving images.
BACKGROUND OF THE INVENTION
[0003] Multiple endoscopic devices with multiple cameras and light
sources may be used for medical procedures, inspection of small
pipes, or remote monitoring. For example, such an endoscopic device
may be a medical endoscope comprising a flexible tube, and a camera
and a light source mounted on the distal end of the flexible tube.
The endoscope is insertable into an internal body cavity through a
body orifice to examine the body cavity and tissues for diagnosis.
The tube of the endoscope has one or more longitudinal channels,
through which an instrument can reach the body cavity to take
samples of suspicious tissues or to perform other surgical
procedures such as polypectomy.
[0004] There are many types of endoscopes, and they are named in
relation to the organs or areas with which they are used. For
example, gastroscopes are used for examination and treatment of the
esophagus, stomach and duodenum; colonoscopes for the colon;
bronchoscopes for the bronchi; laparoscopes for the peritoneal
cavity; sigmoidoscopes for the rectum and the sigmoid colon;
arthroscopes for joints; cystoscopes for the urinary bladder; and
angioscopes for the examination of blood vessels.
[0005] Each endoscope has a single forward viewing camera mounted
at the distal end of the flexible tube to transmit an image to an
eyepiece or video camera at the proximal end. The camera is used to
assist a medical professional in advancing the endoscope into a
body cavity and looking for abnormalities. The camera provides the
medical professional with a two-dimensional view from the distal
end of the endoscope. To capture an image from a different angle or
in a different portion, the endoscope must be repositioned or moved
back and forth. Repositioning and movement of the endoscope
prolongs the procedure and causes added discomfort, complications,
and risks to the patient. Additionally, in an environment similar
to the lower gastro-intestinal tract, flexures, tissue folds and
unusual geometries of the organ may prevent the endoscope's camera
from viewing all areas of the organ. The unseen area may cause a
potentially malignant (cancerous) polyp to be missed.
[0006] This problem can be overcome by providing an auxiliary
camera and an auxiliary light source. The auxiliary camera and
light source can be oriented to face the main camera and light
source, thus providing an image of areas not viewable by the
endoscope's main camera. This arrangement of cameras and light
sources can provide both front and rear views of an area or an
abnormality. In the case of polypectomy where a polyp is excised by
placing a wire loop around the base of the polyp, the camera
arrangement allows better placement of the wire loop to minimize
damage to the adjacent healthy tissue.
SUMMARY OF THE INVENTION
[0007] The present invention relates to devices and methods for
producing and improving video images generated by the imaging
devices of endoscopes.
[0008] In accordance with one aspect of the invention, a method for
displaying images includes adjusting at least one characteristic of
an image from a first imaging device of an endoscope to match at
least one corresponding characteristic of an image from a second
imaging device of the endoscope. The characteristic may be one or
more of color, contrast and brightness.
[0009] In a preferred embodiment, the adjusting step includes
creating a histogram for each of RGB colors for the image from the
first imaging device and a histogram for each of the RGB colors for
the image from the second imaging device; adjusting the gamut of
each histogram of the image from the first imaging device to match
the gamut of the corresponding histogram of the image from the
second imaging device; and using gamma coefficients to adjust a
color level of each histogram of the image from the first imaging
device to match a color level of the corresponding histogram of the
image from the second imaging device.
[0010] In accordance with another aspect of the invention, a method
for displaying images includes placing, side by side, an image from
a first imaging device of an endoscope and an image from a second
imaging device of the endoscope, wherein the imaging devices face
each other; and reversing one of the images left for right.
[0011] In accordance with still another aspect of the invention, a
method for sizing images includes placing an image from a first
imaging device of an endoscope and an image from a second imaging
device of the endoscope on a display device; and sizing the images
so that an object, when placed at the same distance from the
imaging devices, appears to have about the same size in the
images.
[0012] In accordance with yet another aspect of the invention, a
method for processing images includes placing image data from first
and second imaging devices of an endoscope in one computer file for
simultaneous display on a display device. Preferably, the image
data from the imaging devices are time-correlated.
[0013] In a preferred embodiment, patient information data is also
placed in the computer file for simultaneous display with the
images on the display device.
[0014] In a further preferred embodiment, a time stamp is placed in
the computer file for simultaneous display with the images and
patient information data on the display device.
[0015] In accordance with still yet another aspect of the
invention, an endoscopic system includes an endoscope that has a
first imaging device and a second imaging device, and a controller
that adjusts at least one characteristic of an image from the first
imaging device of the endoscope to match at least one corresponding
characteristic of an image from the second imaging device of the
endoscope. The at least one characteristic may be one or more of
color, contrast and brightness.
[0016] In a preferred embodiment, the controller creates a
histogram for each of RGB colors for the image from the first
imaging device and a histogram for each of the RGB colors for the
image from the second imaging device; adjusts the gamut of each
histogram of the image from the first imaging device to match the
gamut of the corresponding histogram of the image from the second
imaging device; and uses gamma coefficients to adjust a color level
of each histogram of the image from the first imaging device to
match a color level of the corresponding histogram of the image
from the second imaging device.
[0017] In accordance with a further aspect of the invention, an
endoscopic system includes an endoscope including a first imaging
device and a second imaging device, and a display device that
displays, side by side, an image from the first imaging device of
the endoscope and an image from the second imaging device of the
endoscope, wherein the imaging devices face each other, and wherein
one of the images is reversed left for right.
[0018] In accordance with a still further aspect of the invention,
an endoscopic system includes an endoscope including a first
imaging device and a second imaging device, and a display device
that displays an image from the first imaging device of the
endoscope and an image from the second imaging device of the
endoscope, wherein the images are sized so that an object, when
placed at the same distance from the imaging devices, appears to
have about the same size in the images.
[0019] In accordance with a yet further aspect of the invention, an
endoscopic system includes an endoscope including a first imaging
device and a second imaging device, and a controller that places
image data from the first and second imaging devices of the
endoscope in one computer file for simultaneous display on a
display device. Preferably, the image data from the imaging devices
are time-correlated.
[0020] In a preferred embodiment, patient information data is also
placed in the computer file for simultaneous display with the
images on the display device.
[0021] In a further preferred embodiment, a time stamp is placed in
the computer file for simultaneous display with the images and
patient information data on the display device.
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] FIG. 1 shows a perspective view of an endoscope with an
imaging assembly according to one embodiment of the present
invention.
[0023] FIG. 2 shows a perspective view of the distal end of an
insertion tube of the endoscope of FIG. 1.
[0024] FIG. 3 shows a perspective view of the imaging assembly
shown in FIG. 1.
[0025] FIG. 4 shows a perspective view of the distal ends of the
endoscope and imaging assembly of FIG. 1.
[0026] FIG. 5 shows a schematic representation of a display device
used with the endoscope of FIG. 1.
[0027] FIG. 6 shows a schematic representation of a screen showing
two images and patient information.
DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
[0028] FIG. 1 illustrates an exemplary endoscope 10 of the present
invention. This endoscope 10 can be used in a variety of medical
procedures in which imaging of a body tissue, organ, cavity or
lumen is required. The types of procedures include, for example,
anoscopy, arthroscopy, bronchoscopy, colonoscopy, cystoscopy, EGD,
laparoscopy, and sigmoidoscopy.
[0029] The endoscope 10 of FIG. 1 includes an insertion tube 12 and
an imaging assembly 14, a section of which is housed inside the
insertion tube 12. As shown in FIG. 2, the insertion tube 12 has
two longitudinal channels 16. In general, however, the insertion
tube 12 may have any number of longitudinal channels. An instrument
can reach the body cavity through one of the channels 16 to perform
any desired procedures, such as to take samples of suspicious
tissues or to perform other surgical procedures such as
polypectomy. The instruments may be, for example, a retractable
needle for drug injection, hydraulically actuated scissors, clamps,
grasping tools, electrocoagulation systems, ultrasound transducers,
electrical sensors, heating elements, laser mechanisms and other
ablation means. In some embodiments, one of the channels can be
used to supply a washing liquid such as water for washing. Another
or the same channel may be used to supply a gas, such as CO.sub.2
or air into the organ. The channels 16 may also be used to extract
fluids or inject fluids, such as a drug in a liquid carrier, into
the body. Various biopsy, drug delivery, and other diagnostic and
therapeutic devices may also be inserted via the channels 16 to
perform specific functions.
[0030] The insertion tube 12 preferably is steerable or has a
steerable distal end region 18 as shown in FIG. 1. The length of
the distal end region 18 may be any suitable fraction of the length
of the insertion tube 12, such as one half, one third, one fourth,
one sixth, one tenth, or one twentieth. The insertion tube 12 may
have control cables (not shown) for the manipulation of the
insertion tube 12. Preferably, the control cables are symmetrically
positioned within the insertion tube 12 and extend along the length
of the insertion tube 12. The control cables may be anchored at or
near the distal end 36 of the insertion tube 12. Each of the
control cables may be a Bowden cable, which includes a wire
contained in a flexible overlying hollow tube. The wires of the
Bowden cables are attached to controls 20 in the handle 22. Using
the controls 20, the wires can be pulled to bend the distal end
region 18 of the insertion tube 12 in a given direction. The Bowden
cables can be used to articulate the distal end region 18 of the
insertion tube 12 in different directions.
[0031] As shown in FIG. 1, the endoscope 10 may also include a
control handle 22 connected to the proximal end 24 of the insertion
tube 12. Preferably, the control handle 22 has one or more ports
and/or valves (not shown) for controlling access to the channels 16
of the insertion tube 12. The ports and/or valves can be air or
water valves, suction valves, instrumentation ports, and
suction/instrumentation ports. As shown in FIG. 1, the control
handle 22 may additionally include buttons 26 for taking pictures
with an imaging device on the insertion tube 12, the imaging
assembly 14, or both. The proximal end 28 of the control handle 22
may include an accessory outlet 30 (FIG. 1) that provides fluid
communication between the air, water and suction channels and the
pumps and related accessories. The same outlet 30 or a different
outlet can be used for electrical lines to light and imaging
components at the distal end of the endoscope 10.
[0032] As shown in FIG. 2, the endoscope 10 may further include an
imaging device 32 and light sources 34, both of which are disposed
at the distal end 36 of the insertion tube 12. The imaging device
32 may include, for example, a lens, single chip sensor, multiple
chip sensor or fiber optic implemented devices. The imaging device
32, in electrical communication with a processor and/or monitor,
may provide still images or recorded or live video images. The
light sources 34 preferably are equidistant from the imaging device
32 to provide even illumination. The intensity of each light source
34 can be adjusted to achieve optimum imaging. The circuits for the
imaging device 32 and light sources 34 may be incorporated into a
printed circuit board (PCB).
[0033] As shown in FIGS. 3 and 4, the imaging assembly 14 may
include a tubular body 38, a handle 42 connected to the proximal
end 40 of the tubular body 38, an auxiliary imaging device 44, a
link 46 that provides physical and/or electrical connection between
the auxiliary imaging device 44 to the distal end 48 of the tubular
body 38, and an auxiliary light source 50 (FIG. 4). The auxiliary
light source 50 may be an LED device.
[0034] As shown in FIG. 4, the imaging assembly 14 of the endoscope
10 is used to provide an auxiliary imaging device at the distal end
of the insertion tube 12. To this end, the imaging assembly 14 is
placed inside one of the channels 16 of the endoscope's insertion
tube 12 with its auxiliary imaging device 44 disposed beyond the
distal end 36 of the insertion tube 12. This can be accomplished by
first inserting the distal end of the imaging assembly 14 into the
insertion tube's channel 16 from the endoscope's handle 18 and then
pushing the imaging assembly 14 further into the assembly 14 until
the auxiliary imaging device 44 and link 46 of the imaging assembly
14 are positioned outside the distal end 36 of the insertion tube
12 as shown in FIG. 4.
[0035] Each of the main and auxiliary imaging devices 32, 44 may be
an electronic device which converts light incident on
photosensitive semiconductor elements into electrical signals. The
imaging sensor may detect either color or black-and-white images.
The signals from the imaging sensor can be digitized and used to
reproduce an image that is incident on the imaging sensor. Two
commonly used types of image sensors are Charge Coupled Devices
(CCD) such as a VCC-5774 produced by Sanyo of Osaka, Japan and
Complementary Metal Oxide Semiconductor (CMOS) camera chips such as
an OVT 6910 produced by OmniVision of Sunnyvale, Calif. Preferably,
the main imaging device 32 is a CCD imaging device, and the
auxiliary imaging device 44 is a CMOS imaging device.
[0036] When the imaging assembly 14 is properly installed in the
insertion tube 12, the auxiliary imaging device 44 of the imaging
assembly 14 preferably faces backwards towards the main imaging
device 32 as illustrated in FIG. 4. The auxiliary imaging device 44
may be oriented so that the auxiliary imaging device 44 and the
main imaging device 32 have adjacent or overlapping viewing areas.
Alternatively, the auxiliary imaging device 44 may be oriented so
that the auxiliary imaging device 44 and the main imaging device 32
simultaneously provide different views of the same area.
Preferably, the auxiliary imaging device 44 provides a retrograde
view of the area, while the main imaging device 32 provides a front
view of the area. However, the auxiliary imaging device 44 could be
oriented in other directions to provide other views, including
views that are substantially parallel to the axis of the main
imaging device 32.
[0037] As shown in FIG. 4, the link 46 connects the auxiliary
imaging device 44 to the distal end 48 of the tubular body 38.
Preferably, the link 46 is a flexible link that is at least
partially made from a flexible shape memory material that
substantially tends to return to its original shape after
deformation. Shape memory materials are well known and include
shape memory alloys and shape memory polymers. A suitable flexible
shape memory material is a shape memory alloy such as nitinol. The
flexible link 46 is straightened to allow the distal end of the
imaging assembly 14 to be inserted into the proximal end of
assembly 14 of the insertion tube 12 and then pushed towards the
distal end 36 of the insertion tube 12. When the auxiliary imaging
device 44 and flexible link 46 are pushed sufficiently out of the
distal end 36 of the insertion tube 12, the flexible link 46
resumes its natural bent configuration as shown in FIG. 3. The
natural configuration of the flexible link 46 is the configuration
of the flexible link 46 when the flexible link 46 is not subject to
any force or stress. When the flexible link 46 resumes its natural
bent configuration, the auxiliary imaging device 44 faces
substantially back towards the distal end 36 of the insertion tube
12 as shown in FIG. 5.
[0038] In the illustrated embodiment, the auxiliary light source 50
of the imaging assembly 14 is placed on the flexible link 46, in
particular on the curved concave portion of the flexible link 46.
The auxiliary light source 50 provides illumination for the
auxiliary imaging device 44 and may face substantially the same
direction as the auxiliary imaging device 44 as shown in FIG.
4.
[0039] The endoscope of the present invention, such as the
endoscope 10 shown in FIG. 1, may be part of an endoscope system
that may also include a controller 52 and a display device 54, as
shown in FIG. 5. In the preferred embodiment shown in FIG. 5, the
controller 52 is connected to the main and auxiliary imaging
devices 32, 44 to receive image data. The controller 52 may be used
to process the image data and transmit the processed image data to
the display device 54. The term "controller" as used in this
specification is broadly defined. In some embodiments, for example,
the controller may simply be a signal processing unit.
[0040] In the embodiment shown in FIG. 5, the display device 54
displays, side by side, the image 56 from the main imaging device
32 and the image 58 from the auxiliary imaging device 44. In the
present invention, the images may also be displayed on different
display devices, and the term "side by side" may simply mean that
the two images are positioned so that they can be viewed by the
same operator during a medical procedure. The controller 52
preferably incorporates the image data from the main and auxiliary
imaging devices 32, 44 into a single signal and sends the signal to
the display device 54. In some embodiments, the display device 54
includes a wide screen display with a 16:9 aspect ratio.
Preferably, the two images 56, 58 are sized appropriately for
display on the wide screen display. For example, the image 56 from
the main imaging device 32 may be displayed about 1.5 times larger
than the image 58 from the auxiliary imaging device 44. This sizing
ratio may also be used to balance the resolution of the two images,
as well as to take into account the different aspect ratios of the
two images 56, 58. For example, the image 56 from the main imaging
device 32 may be displayed with a 1:1 aspect ratio, while the image
58 from the auxiliary imaging device 44 may have a 4:3 aspect
ration. The images 56, 58 may also be sized so that the same
object, when placed at the same distance from the imaging devices
32, 44, appears to have about the same size in the images 56, 58.
The images 56, 58 shown in FIG. 5 are not drawn to scale.
[0041] As shown in FIG. 5, one of the images 56, 58 on the display
device 54 may be reversed from left for right. With this
arrangement, an object 60 that appears on the left side of one
image 56 also appears on the left side of the other image 58.
Similarly, an object 62 that appears on the right side of one image
56 also appears on the right side of the other image 58, 56.
Additionally, when an object moves from the left side of one of the
images 56, 58 to the right side, the same object also moves from
the left side of the other image 56, 58 to the right side. And, if
the object in one image appears to rotated clockwise, the object
will appear to rotate clockwise in the other image. Furthermore,
the movements of the imaging devices 32, 44 appear to be
coordinated. This arrangement makes it easier for an operator to
observe, identify and correlate the objects and their movements in
both images 56, 58.
[0042] Preferably, the data for the two images 56, 58 and possibly
other data 64, such as patient information data or a time stamp,
are stored in one computer file. In some cases, the patent
information may be associated with one of the two images 56, 58.
Preferably, the stored images 56, 58 and possibly other data 64 are
time-correlated (i.e., they are captured at the same time). For
example, as shown in FIG. 6, the two images 56, 58 and possibly
other data 64 may be incorporated into one screen 66 in an image
file. In some embodiments, the two images 56, 58 and possibly other
data 64 may be captured in one jpeg file.
[0043] In some preferred embodiments, one or more characteristics
of one image 56, 58 may be adjusted to match the same or similar
one or more characteristics of the other image 58, 56, so that the
images 56, 58 and the objects in the images 56, 58 have similar
appearances. The characteristics may include, for example, color,
contrast, and brightness. In one example, one or more
characteristics of the auxiliary imaging device's image 58 are
adjusted to match those of the main imaging device's image 56.
Matched images make it easier for an operator to observe, identify
and correlate the objects in the images.
[0044] In one preferred embodiment, the following technique is used
to adjust the characteristics of the auxiliary imaging device's
image 58 to match those of the main imaging device's image 56.
First, a histogram for each of the RGB colors is created for the
auxiliary imaging device's image 58 (called "current file"). The
image used to create the histograms may be an average of the past
images, such as the past two to ten images, preferably the past
four images. And a histogram for each of the RGB colors is created
also for the main imaging device's image 56 (called "master file").
This histogram may be the average of the histograms of the past
images, such as the histograms of the past two to ten images,
preferably the histograms of the past four images.
[0045] Second, a minimum and maximum is determined for each
histogram by means of thresholding. Then a clip and gain is set for
each histogram of the auxiliary imaging device's image to equalize
its color gamut to that of the corresponding histogram of the main
imaging device's image. In particular, the minimum and maximum for
each histogram of the auxiliary imaging device's image are adjusted
to match those for the corresponding histogram of the main imaging
device's image.
[0046] Finally, gamma coefficients are used to adjust the color
levels of the histograms of the auxiliary imaging device's image to
match those of the histograms of the main imaging device's image.
The equations for the gamma coefficients are: [0047]
red_gamma_color_balance=(current_profile.m_AverageRed*master
average)/(master_profile.m_AverageRed * current_average); [0048]
green_gamma_color_balance=(current_profile.m_AverageGreen*master_average)-
/(master_profile.m_AverageGreen * current_average); and [0049]
blue_gamma_color_balance
=(current_profile.m_AverageBlue*master_average)/(master_profile.m_Average-
Blue * current_average) Gamma coefficients are used because they
are simple and convenient and preserve black and white points and
because the code can be re-used for conventional gamma
correction.
[0050] Additional processing of the images, such as sharpening,
frame averaging, and noise reduction, may be performed.
[0051] The images described above may be still pictures or
continuous video images (such as television images). When the
images are video images, in the embodiment of the invention in
which one or more characteristics of one image are adjusted to
match those of another image, the characteristics are adjusted
continuously in real time (i.e., dynamically). For example, the
characteristics of the video image may be adjusted for every frame
of the image. The reason for real time adjustment is that the video
images are changing constantly as the lighting, object distance or
tissue color varies.
[0052] The implementation of the above-described features may be
performed digitally by software or firmware in the controller.
Alternately, the image manipulation can be performed by hardware
image chipsets, FPGAs or other electrical circuitry. These image
manipulation techniques are well known in the field of graphic and
video processing and will not be described in detail.
[0053] Although in the preferred embodiments described above, the
images are from the main and auxiliary imaging devices of the same
endoscope, the images may also come from imaging devices of
different endoscopes such as laparoscopes. For example, when two
laparoscopes are used during a procedure, the images from the
laparoscopes may have different characteristics due to, for
example, different imaging device types, different manufacturing
techniques, or differences in lighting sensitivities. The
controller that receives images from the laparoscopes may designate
any one of the images as a master and then match the second image
to the master image. In this way the operator is able to conduct a
procedure with consistent visualization across the laparoscopes.
Additionally, the present invention may be used with three or more
images from two or more endoscopes. For example, two images may be
adjusted to match a third image.
[0054] While particular embodiments of the present invention have
been shown and described, it will be obvious to those skilled in
the art that changes and modifications can be made without
departing from this invention in its broader aspects. Therefore,
the appended claims are to encompass within their scope all such
changes and modifications as fall within the true spirit and scope
of this invention.
* * * * *