U.S. patent number 9,270,919 [Application Number 14/035,661] was granted by the patent office on 2016-02-23 for simultaneous display of two or more different sequentially processed images.
This patent grant is currently assigned to Karl Storz Imaging, Inc.. The grantee listed for this patent is Karl Storz Imaging, Inc.. Invention is credited to Marc R. Amling, Helga Schemm.
United States Patent |
9,270,919 |
Amling , et al. |
February 23, 2016 |
Simultaneous display of two or more different sequentially
processed images
Abstract
A medical imaging system having a processor with software
executing thereon is provided for processing and display of
multiple bandwidths of video in multiple display areas. The system
receives a video signal with a plurality of portions and generates
at least two signals there from. Each of the two signals has a
bandwidth for display in a different display area. The two signals
are updated so that each component displays a different portion of
the input video signal, and the two signals may be combined for
display on a single display device having two display areas.
Inventors: |
Amling; Marc R. (Santa Barbara,
CA), Schemm; Helga (Tuttlingen, DE) |
Applicant: |
Name |
City |
State |
Country |
Type |
Karl Storz Imaging, Inc. |
Goleta |
CA |
US |
|
|
Assignee: |
Karl Storz Imaging, Inc.
(Goleta, CA)
|
Family
ID: |
51584953 |
Appl.
No.: |
14/035,661 |
Filed: |
September 24, 2013 |
Prior Publication Data
|
|
|
|
Document
Identifier |
Publication Date |
|
US 20150085186 A1 |
Mar 26, 2015 |
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N
21/47 (20130101); H04N 5/2624 (20130101); A61B
1/0005 (20130101); H04N 5/45 (20130101); H04N
21/4316 (20130101); H04N 9/12 (20130101); A61B
1/0638 (20130101); A61B 1/00009 (20130101); A61B
5/7425 (20130101) |
Current International
Class: |
G06F
3/033 (20130101); H04N 5/445 (20110101); H04N
5/45 (20110101); H04N 5/262 (20060101); A61B
1/00 (20060101); A61B 1/06 (20060101) |
Field of
Search: |
;345/7,8,30,44,59,82,156-173,207,419,473,629
;348/36,51,68,74,80,128,231.7,598 ;382/128,274
;600/160,382,437 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
Other References
FICE--Fuji Intelligent Chromo Endoscopy; Fuji Group; FICE 2005; 6
pages. cited by applicant .
European Search Report Application No. EP 14 18 4970 Completed:
Mar. 10, 2015; Mailing Date: Mar. 19, 2015 13 pages. cited by
applicant.
|
Primary Examiner: Dharia; Prabodh M
Attorney, Agent or Firm: Whitmyer IP Group LLC
Claims
What is claimed is:
1. A medical imaging system comprising: a video signal having a
plurality of portions; a processor for receiving said video signal;
a first signal having a first bandwidth generated by said processor
from a first one of the plurality of portions of said video signal;
and a second signal having a second bandwidth generated by said
processor from a second one of the plurality of portions of said
video signal; said processor alternately updating said first and
second signals with portions of said video signal, said first and
second signals each updated from a different one of the plurality
of portions of said video signal; said first signal for display in
a first display area, said second signal for display in a second
display area; one of said first and second signals updated with one
of the portions of said video signal to create an updated signal, a
previously updated one of said first and second signals for display
at the same time as the updated signal, the updated signal and said
previously updated signal for display in different display
areas.
2. The imaging system of claim 1 wherein said processor combines
said first and second signals for display on one display having the
first and second display areas.
3. The imaging system of claim 1 wherein the first and second
display areas are on a single monitor.
4. The imaging system of claim 3 wherein the first and second
display areas are configured as a picture-in-picture display.
5. The imaging system of claim 1 further comprising: an interface
provided by software executing on said processor; at least one
bandwidth selection received by said interface, said bandwidth
selection indicative of at least one the first and second
bandwidths.
6. The medical imaging system of claim 1 wherein each portion of
said video signal has red, green and blue color components.
7. A medical imaging system comprising: at least one input module
having a processor; a video signal received by each said input
module; a processed video signal generated by each said input
module from each said video signal, said processed video signal
having a plurality of portions; a control module having a processor
for receiving each said processed video signal; a first signal
having a first bandwidth generated by the processor of said control
module from a first one of the plurality of portions of said
processed video signal; and a second signal having a second
bandwidth generated by the processor of said control module from a
second one of the plurality of portions of said processed video
signal; the processor of said control module alternately updating
said first and second signals with portions of said processed video
signal, said first and second signals each updated from a different
one of the plurality of portions of said video signal; said first
signal for display in a first display area, said second signal for
display in a second display area; one of said first and second
signals updated with one of the portions of said video signal to
create an updated signal, a previously updated one of said first
and second signals for display at the same time as the updated
signal, the updated signal and said previously updated signal for
display in different display areas.
8. The medical imaging system of claim 7 wherein said processor
combines said first and second signals for display on one display
having the first and second display areas.
9. The medical imaging system of claim 7 wherein: at least a first
and a second video signals are received by said processor; said
processor generating first and second signals from each of said
first and second video signals; each said first signals
respectively for display in a first and third display areas; each
said second signals respectively for display in a second and fourth
display areas; the first and third display areas for display of the
first bandwidth; the second and fourth display areas for display of
the second bandwidth.
10. The imaging system of claim 9 wherein said first and second
signals are for display on separate display devices.
11. The imaging system of claim 7 further comprising: an interface
provided by software executing on the processor of said control
module; at least one bandwidth selection received by said
interface, said bandwidth selection indicative of at least one the
first and second bandwidths.
12. The medical imaging system of claim 7 wherein each portion of
said processed video signal has red, green and blue color
components.
13. A medical imaging system comprising: an input video signal
having a plurality of portions; a processor for receiving said
input video signal; an output video signal generated by said
processor from said input video signal; and a plurality of
components of said output video signal generated in an order by
said processor, each component generated according to one of a
plurality of bandwidths; each one of said plurality of components
generated by said processor from a different one of the plurality
of portions of said input video signal; each said component of said
output video signal updated by said processor with portions of said
input video signal according to the order; each one of said
plurality of components for display in one of a plurality of
display areas; one of said components of said output video signal
updated with one of the portions of said input video signal to
create an updated component, and a previously updated component of
said output video signal included in the output video signal for
display at the same time as the updated component, the updated
component and said previously updated component for display in
different display areas.
14. The imaging system of claim 13 wherein said plurality of
components are for display on a single display device.
15. The imaging system of claim 13 further comprising: an interface
provided by software executing on said processor; at least one
bandwidth selection received by said interface, said bandwidth
selection indicative of at least one the first and second
bandwidths.
16. The medical imaging system of claim 13 wherein each portion of
said input video signal has red, green and blue color
components.
17. A medical imaging system comprising: a video signal having a
plurality of portions; a processor for receiving said video signal;
a first signal having a first signal processing mode generated by
said processor from a first one of the plurality of portions of
said video signal; and a second signal having a second signal
processing mode generated by said processor from a second one of
the plurality of portions of said video signal; said processor
alternately updating said first and second signals with portions of
said video signal, said first and second signals each updated from
a different one of the plurality of portions of said video signal;
said first signal for display in a first display area, said second
signal for display in a second display area; one of said first and
second signals updated with one of the portions of said video
signal to create an updated signal, a previously updated one of
said first and second signals for display at the same time as the
updated signal, the updated signal and said previously updated
signal for display in different display areas.
18. The imaging system of claim 17 wherein said processor combines
said first and second signals for display on one display having the
first and second display areas.
19. The imaging system of claim 17 wherein the first and second
display areas are on a single monitor.
20. The imaging system of claim 19 wherein the first and second
display areas are configured as a picture-in-picture display.
21. The imaging system of claim 17 further comprising: an interface
provided by software executing on said processor; at least one
bandwidth selection received by said interface, said bandwidth
selection indicative of at least one of the signal processing
modes.
22. The medical imaging system of claim 17 wherein each portion of
said video signal has red, green and blue color components.
23. The imaging system of claim 17 wherein one or both of said
first and second signal processing modes is selected from the group
consisting of: a bandwidth, enhanced color, edge enhancement,
texture enhancement, sharpness adjustment, and combinations
thereof.
24. A medical imaging system comprising: at least one input module
having a processor; a video signal received by each said input
module; a processed video signal generated by each said input
module from each said video signal, said processed video signal
having a plurality of portions; a control module having a processor
for receiving each said processed video signal; a first signal
having a first signal processing mode generated by the processor of
said control module from a first one of the plurality of portions
of said processed video signal; and a second signal having a second
signal processing mode generated by the processor of said control
module from a second one of the plurality of portions of said
processed video signal; the processor of said control module
alternately updating said first and second signals with portions of
said processed video signal, said first and second signals each
updated from a different one of the plurality of portions of said
video signal; said first signal for display in a first display
area, said second signal for display in a second display area; one
of said first and second signals updated with one of the portions
of said video signal to create an updated signal, a previously
updated one of said first and second signals for display at the
same time as the updated signal, the updated signal and said
previously updated signal for display in different display
areas.
25. The medical imaging system of claim 24 wherein said processor
of said control module combines said first and second signals for
display on one display having the first and second display
areas.
26. The medical imaging system of claim 24 wherein: at least a
first and a second processed video signals are received by said
processor of said control module; said processor of said control
module generating said first and said second signals for each of
said first and second video signals; each said first signals
respectively for display in a first and third display areas; each
said second signals respectively for display in a second and fourth
display areas; the first and third display areas for display of the
first signal processing mode; the second and fourth display areas
for display of the second signal processing mode.
27. The imaging system of claim 26 wherein said first and second
signals are for display on separate display devices.
28. The imaging system of claim 24 further comprising: an interface
provided by software executing on the processor of said control
module; at least one bandwidth selection received by said
interface, said bandwidth selection indicative of at least one of
the signal processing modes.
29. The medical imaging system of claim 24 wherein each portion of
said processed video signal has red, green and blue color
components.
30. The imaging system of claim 24 wherein one or both of said
first and second signal processing modes is selected from the group
consisting of: a bandwidth, enhanced color, edge enhancement,
texture enhancement, sharpness adjustment, and combinations
thereof.
31. A medical imaging system comprising: an input video signal
having a plurality of portions; a processor for receiving said
input video signal; an output video signal generated by said
processor from said input video signal; and a plurality of
components of said output video signal generated in an order by
said processor, each component generated according to one of a
plurality of signal processing modes; each one of said plurality of
components generated by said processor from a different one of the
plurality of portions of said input video signal; each said
component of said output video signal updated by said processor
with portions of said input video signal according to the order;
each one of said plurality of components for display in one of a
plurality of display areas; one of said components of said output
video signal updated with one of the portions of said input video
signal to create an updated component, a previously updated
component of said output video signal included in the output video
signal for display at the same time as the updated component, the
updated component and said previously updated component for display
in different display areas.
32. The imaging system of claim 31 wherein said plurality of
components are for display on a single display device.
33. The imaging system of claim 31 further comprising: an interface
provided by software executing on said processor; at least one
bandwidth selection received by said interface, said bandwidth
selection indicative of at least one of the plurality of signal
processing modes.
34. The medical imaging system of claim 31 wherein each portion of
said input video signal has red, green and blue color
components.
35. The imaging system of claim 31 wherein one or all of said
plurality of signal processing modes is selected from the group
consisting of: a bandwidth, enhanced color, edge enhancement,
texture enhancement, sharpness adjustment, and combinations
thereof.
Description
FIELD OF THE INVENTION
The invention relates to image capture, processing and display
devices, and more particularly medical imaging and devices.
BACKGROUND OF THE INVENTION
During medical procedures, endoscopes and other imaging devices are
used to perform minimally invasive surgery and diagnostics. These
imaging devices typically use a broad band light source to
illuminate the tissue inside a cavity so that an image sensor can
capture the reflected light and send a signal to a processor for
display.
A difficulty with the use of a white light or wide band light
source is that hemoglobin absorbs the majority of optical light,
and the penetration depth of light is closely related to the
absorption spectrum of hemoglobin. In the visible spectrum,
hemoglobin shows the highest absorption of blue (.about.410-440 nm)
and green (.about.530-580 nm) wavelength regions. Therefore,
optical information obtained in the blue and green spectral region
can discriminate hemoglobin concentration an optimal way. Due to
the short penetration depth of blue light (.about.1 mm),
intermediate penetration depth of green light (.about.3 mm) and
high penetration depth of red light (.about.5 mm), the tissue
structures near the surface are easily identified, but information
in the red spectral region cannot be easily obtained due to the
high penetration depth.
There are some known imaging systems that are capable of reducing
the contribution of the red light region to a displayed image. For
example U.S. Pat. No. 7,420,151 to Fengler et al. discloses a
system for performing short wavelength imaging with a broadband
illumination source includes an image processor that receives
signals from a color image sensor. The image processor reduces the
contribution of red illumination light to an image by computing
blue, green, and blue-green (cyan) color components of display
pixels from the signals received from the image sensor. The blue,
green, and cyan color component values are coupled to inputs of a
color monitor for display to produce a false-color image of the
tissue.
U.S. Pat. No. 4,742,388 to Cooper et al. discloses a color video
endoscope system having a light source and a solid state image
sensor that transmits a signal to a video processor that converts
the signal from the image sensor in to a composite RGB video
signal, this RGB signal is received by the video processor and the
signal is filtered electronically to vary the color image. Cooper
discloses a number of potentiometers that allow the user to select
and change red, green and blue gains applied to the signal.
U.S. Pat. No. 6,147,705 to Krauter discloses a video colposcope
with a microcomputer having algorithms for color balance. A video
camera obtains an electronic image. A CCD sensor converts an image
into an analog electrical signal which is amplified and digitized.
Using an algorithm-driven digital signal processing circuitry,
color saturation, hue and intensity levels of the electronic image
are modified according to the DSP reference filter algorithm.
U.S. Pat. No. 7,050,086 to Ozawa discloses a system for displaying
a false-color image with reduced red component. The red, green and
blue ("RGB") signals are cyclically and sequentially read from a
frame memory, and the frames are used to generate a digital video
signal for display. The RGB components are emitted from the distal
end face of a light guide and these RGB signals are sequentially
and cyclically focused on the light receiving surface of a CCD
image sensor. These RGB signals are then sequentially used to
update a display or display memory. Optionally, the red component
may be reduced by a switching circuit to display a false-color
image.
Current systems synchronize the display of wide band and narrow
band images. When the wide and narrow band images are both
displayed on a monitor using a split screen, or on two monitors,
the images are updated at the same time. Further, the required
resolution for medical imaging devices may be rather high. Fengler
appears to disclose that the wide band and narrow band images can
be displayed at the same time, but the processor would need
sufficient processing speed to accomplish this task.
Cooper appears to disclose a processor including a series of
potentiometers that modify the RGB signal in a way that would allow
for the elimination of the red component. These potentiometers
allow for an adjustable filter that may be set or checked at the
beginning of each procedure
Ozawa appears to disclose cyclically and sequentially reading image
signals. However, wide and narrow band display regions are updated
at the same time. Thus if one were to display both wide band and
narrow band images on a split screen or two separate monitors, both
the wide band and narrow band images would be updated
simultaneously.
Improved visualization techniques can be used to provide a system
that uses less processing power for the same resolution. Likewise,
a higher resolution may be obtained with reduced processing power
requirements in comparison to prior art systems.
It is therefore an object of the present invention to provide a
system for display of wide and narrow band images that uses a cost
effective processing technology.
Yet another object of the present invention is to provide an
imaging system that can primarily display information obtained from
the blue and green wavelength regions that suppresses the red
region while reducing the required processing power in comparison
to prior art systems.
It is further an object of the present invention to provide an
imaging system with sufficient visibility of wide band and narrow
band images with reduced hardware costs.
It is yet another object of the present invention to provide a
narrow band imaging system that offers simplified settings for
display of narrow band images.
It is yet another object of the present invention to provide a
system with enhanced resolution without an increase in processing
power.
SUMMARY OF THE INVENTION
These and other objects are achieved by providing a medical imaging
system having a processor for receiving a video signal having a
plurality of portions. A first signal generated according to a
first signal processing mode, such as a first bandwidth. The first
signal is generated by the processor from a first one of the
plurality of portions of the video signal. A second signal is
generated according to a second signal processing mode, such as a
second bandwidth. The second signal is generated by the processor
from a second one of the plurality of portions of the video signal.
The processor alternately updates the first and second signals with
portions of the video signal, the first and second signals each
updated from a different one of the plurality of portions of the
video signal. The first signal is for display in a first display
area, the second signal is for display in a second display
area.
The processor can combine the first and second signals for display
on one display having the first and second display areas. The first
and second display areas can also be on a two separate monitors.
The first and second display areas can be configured as a
picture-in-picture display.
The imaging system can include an interface provided by software
executing on the processor. At least one bandwidth selection is
received by the interface, the bandwidth selection indicative of at
least one the first and second bandwidths.
In one aspect, one of the first and second signals is updated with
one of the portions of the video signal to create an updated
signal. A previously updated one of the first and second signals is
for display at the same time as the updated signal, the updated
signal and the previously updated signal for display in different
display areas.
Other objects are achieved by providing a medical imaging system
having at least one input module having a processor. A video signal
is received by each of the input modules. A processed video signal
is generated by each of the input modules from each of the video
signals, and the processed video signal has a plurality of
portions. A control module having a processor receives each of the
processed video signals. A first signal generated according to a
first signal processing mode, such as a first bandwidth. The first
signal is generated by the processor of the control module from a
first one of the plurality of portions of the processed video
signal. A second signal is generated according to a second signal
processing mode, such as a second bandwidth. The second signal is
generated by the processor of the control module from a second one
of the plurality of portions of the processed video signal. The
processor of the control module alternately updates the first and
second signals with portions of the processed video signal, the
first and second signals are each updated from a different one of
the plurality of portions of the video signal. The first signal is
for display in a first display area, the second signal is for
display in a second display area.
The processor of the control module can combine the first and
second signals for display on one display having the first and
second display areas. The first and second display areas can also
be on a single monitor. The first and second display areas can be
configured as a picture-in-picture display.
The imaging system can include an interface provided by software
executing on the processor of the control module. At least one
bandwidth selection is received by the interface, the bandwidth
selection indicative of at least one the first and second
bandwidths.
In one aspect, the first and second signals are updated with one of
the portions of the video signal to create an updated signal. A
previously updated one of the first and second signals is for
display at the same time as the updated signal, the updated signal
and the previously updated signal for display in different display
areas.
In another aspect at least a first and a second video signal are
received by the processor. The processor generating first and
second signals from each of the first and second video signals.
Each of the first signals are respectively for display in a first
and third display areas. Each of the second signals are
respectively for display in a second and fourth display areas. The
first and third display areas are for display of the first
bandwidth, the second and fourth display areas for display of the
second bandwidth.
Other objects are achieved by providing a medical imaging system
having a processor receiving an input video signal having a
plurality of portions. An output video signal is generated by the
processor from the output video signal. A plurality of components
of the output video signal are generated in an order by the
processor. Each component is generated according to one of a
plurality of bandwidths. Each one of the plurality of components is
generated by the processor from a different one of the plurality of
portions of the input video signal. Each component of the output
video signal is updated by the processor with portions of said
input video signal according to the order. Each one of the
plurality of components is for display in one of a plurality of
display areas.
The plurality of components can be for display on a single display
device or a single monitor.
The system can include an interface provided by software executing
on the processor. At least one bandwidth selection is received by
the interface, the bandwidth selection is indicative of at least
one the first and second bandwidths.
Further, one of the components of the output video signal can be
updated with one of the portions of the input video signal to
create an updated component. A previously updated component of the
output video signal can be included in the output video signal for
display at the same time as the updated component, where the
updated component and the previously updated component are for
display in different display areas.
Each portion of the input video signal can have red, green and blue
color components. Other color combinations are possible, and may
depend on the camera sensor. For example, a CMYG sensor can be used
in the camera, and the corresponding components can be reduced or
enhanced depending on filter characteristics.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1A is a schematic view of the medical imaging system according
to an exemplary embodiment.
FIG. 1B is another schematic view of another exemplary embodiment
of the system of FIG. 1A.
FIG. 1C is another schematic view of another exemplary embodiment
of the system of FIG. 1A.
FIG. 1D is another schematic view of another exemplary embodiment
of the system of FIG. 1A.
FIGS. 2A and 2B are schematic view of prior art medical imaging
systems.
FIG. 3A-3D are yet other schematic views of a medical imaging
system of FIG. 1A according to another exemplary embodiment.
FIG. 4 is a schematic view of an exemplary embodiment of the output
signal generation shown in FIGS. 1B, 1D and 3A, 3C.
FIG. 5 is a schematic view of an exemplary embodiment of the
generation of two signals as shown in FIGS. 1A, 1C and 3B, 3D.
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1A shows a medical imaging system having an image capture
device or camera, such as an endoscope 2. A light source 22
illuminates the body cavity where the endoscope 2 is inserted. The
light source 22 will typically be a broad band or white light
source. The endoscope 2 produces a video signal 20, and the video
signal has a plurality of portions etc. The video signal may come
to the processor already divided into the plurality of portions.
Alternately, the processor 4 can divide the video signal into the
portions for processing and display. The processor 4 can exist on a
camera control unit or other imaging system. The video signal 20 is
processed according to a pattern where different portions of the
video signal 20 are processed according to one or more signal
processing modes, for example the video signal may be processed
according to different bandwidth selections. The selections 60 are
received by the processor. These selections 60 can indicate, for
example, different signal processing modes such as bandwidth
ranges. For example, if it is desirable to reduce or eliminate the
red component, a selection of the appropriate bandwidths can be
received through an interface 6. The interface may exist on a
separate device, such as a computer or wireless device. The
interface may also be part of a camera control unit or imaging
system having the processor 4.
As shown in FIG. 1A, the selection 60 results in two bandwidth
ranges 462, 502 that are used to process the portions of the video
signal 20. Each bandwidth range 462, 502 the video signal forms a
signal 46, 50. The processor generates. The signals 46, 50 are
alternately updated 48 for displayed on monitor 8 and 8' having
display areas 80, 82.
FIG. 1B shows one aspect where the signals 46, 50 are displayed a
single monitor 8 having two display areas 80, 82. Also shown,
signals 46 and 50 combined to form an output video signal 400 for
display on the monitor 8.
FIG. 1C shows another aspect of FIG. 1A where each signal 46, 50 is
processed according to a signal mode 462' and 502'. FIG. 1D shows
another aspect of FIG. 1B where each signal 46, 50 is processed
according to a signal mode 462' and 502'. It is contemplated that
the signal mode can include many different image modification,
formatting, filtering and processing techniques. These signal modes
will modify the incoming signal, for example, to enhance those
aspects or structures that are important to be able to see during a
procedure. Optionally, aspects or structures that are less
important to see during a procedure are suppressed. Some signal
processing modes include a bandwidth selection as shown in FIGS. 1A
and 1B, other signal processing modes may include, for example,
edge enhancement, image sharpening or others. Combinations of
signal modes are contemplated. For example, a bandwidth mode for
signal 46 and an edge enhancement mode may be used for signal 50.
Other combinations and permutations are contemplated.
FIG. 2A shows a prior art imaging system with an endoscope 21
having a light source 1. A signal 10 produced by the endoscope 21
includes red 101, green 103 and blue 105 components. A processor
121 computes the signal 10 to reduce the red component 141. This
computed image signal 181 is sent to a display memory 161.
FIG. 2B shows another prior art imaging system with an endoscope 21
having a light source 1. A signal 10 produced by the endoscope 21
includes red, green and blue components. A processor 121 creates a
white light image signal 183 and a computed image signal 181. The
signals 183 and 181 are sent to a display 3, with two display areas
31, 33. As shown in the figures, the white light image signal 183
and the computed image signal 181 are both produced from the same
part of the image signal.
FIG. 3A shows another embodiment of the imaging system of the
present invention. In this case, multiple image capture devices,
such as endoscopes 2, 2' are each connected to an input module
2000, 2000'. The input modules each have a processor 2002, 2002'.
Each input module receives a video signal 20, 20' from the
endoscope. The input module processes the video signal 20, 20' to
create a processed video signal 2020, 2020'. The control module
receives the processed video signals and generates an output video
signal 4000, which is formatted 4042 for display. Additional
processing can take place after the alternate updating or after the
generation of the output video signal. The formatting 4042 can
prepare the signal(s) for the appropriate display, for example DVI,
VGA, S-Video, Composite, 3G-SDI. In some areas, digital video
formats and standards are currently being developed and adopted.
The Society of Motion Picture and Television Engineers (SMPTE) is
typically in the business of defining and adopting voluminous
digital video formal standards. As each is adopted, various
applications, and application improvements generally will also be
realized. Some digital video standards currently in use are:
IEEE-1394 FireWire.RTM., ISO/IEC IS 13818, International Standard
(1994), MPEG-2, and ITU-R BT.601-4 (1994) Encoding Parameters of
Digital Television for Studios.
FIG. 3B shows another aspect of the imaging system of the present
invention that is similar to FIG. 3A. In this case, the two signals
4110 and 4120 are not combined into a single output signal and the
signals 4110 and 4120 are separately sent to displays 8000, 8000'.
It should be understood that various combinations of combined or
un-combined signals are possible, for example, a video signal 20
may be processed into two signals that are combined to an output
video signal, and video signal 20' may be processed into two
signals that are not combined and are displayed on separate
monitors.
FIGS. 3C and 3D show other aspects where signal modes 4130' and
4140' are used to generate the signals 4110 and 4120. As previously
discussed, the signal modes may be bandwidth selections, for
example. Other signal modes as discussed herein are
contemplated.
The control module can format 4042 the signals and/or the output
video signal for display. As shown in FIGS. 3A and 3B, the two
signals 4110 and 4120 are generated from different portions of each
of the processed video signals 2020, 2020'. Each of the two signals
4110, 4120 is processed according to a bandwidth 4130, 4140. FIGS.
3C and 3D show a similar system where the two signals are processed
according to a signal processing mode 4130', 4140'. The output
video signal 4000 is alternately updated 4048 with the two signals
4110, 4120 so that the display areas 8002 display different
portions of the processed video signal 2020. Other embodiments can
include more than two signals, where each signal is processed
according to a bandwidth and each of the signals is updated
according to an order. For example, if there are three signals, the
update order could be update signal 1, update signal 2, update
signal 3, repeat. Other orders are envisioned and this example
should not be seen as limiting, however in many cases, each of the
updates is taken from a different portion of the processed video
signal. The example of the order can apply to an imaging system
that does not use an input and control module configuration,
similar to the system shown in FIG. 1A.
The bandwidths 4130 and 4140 can be selected through an interface
6' that can receive multiple selections 60'. Signal processing
modes 4130' and 4140' can also be selected through the interface
6'. The selections 60' may indicate the bandwidth selections or the
processing mode selection for processing and display, and these
selections are received by the processor. Although two bandwidths
4130, 4140 are shown in FIGS. 3A and 3B, more than two bandwidths
may be selected. The same is true for the two signal processing
modes 4130' and 4140' shown in FIGS. 3C and 3D. The interface 6'
can also select different numbers of bandwidths or signal
processing modes for each camera. As an example, the video signal
20 from endoscope 2 can be displayed in two display areas each with
a different bandwidth, and the video signal 20' of endoscope 2' can
be displayed in four display areas, each with a different
bandwidth. Therefore, the interface is configured to allow
selections specific to each endoscope. The interface can be
configured to have a number of pre-set filter characteristics that
adjust the red, green and/or blue components of the video signal.
There is also an option for customized settings that would allow
settings to be adjusted depending on the specific needs of the
physician. For example, customized filter settings. The interface
may also be arranged to allow modification to filter
characteristics or signal processing mode during a medical
procedure to modify the resulting image in a customized way.
In the case of two cameras and two bandwidth selections, there
would be four display areas used. The system can combine all four
components generated from the video signals for display on a single
monitor. Alternately, each camera can be associated with a
particular monitor, with each monitor displaying the selected
components or signals.
Each of FIGS. 1A-D and 3A-D show an imaging system that generates
two signals or components for each camera, each with a different
bandwidth. It may be desirable to generate more than two signals or
components for each camera. In this case, the interface 6 would
receive more than two selections 60. The system may also be
programmed with multiple signal processing modes and more than two
filter or bandwidth ranges. Each selection would indicate a
particular bandwidth or range of bandwidths for use in generating a
signal or component of an output video signal. Each signal or
component would be generated from a different portion of the video
signal or processed video signal, and each signal or component
would be associated with a display area. The interface may be, for
example, a touch screen, computer interface, buttons, switches,
knobs, software or other mechanical, electrical and digital systems
that may allow for human interaction with the system to set the
parameters of the signal processing mode. It is also understood
that a single signal processing mode may be selected for the one of
the signals (or components thereof) where the other signal (or
component thereof) is processed without modifying the content of
the displayed signal. For example, when a bandwidth selection is
received, the color components are modified to reduce or enhance a
particular color or colors. If not processed according to a signal
processing mode in the example of one signal being in false color
mode, the other signal could be displayed with no color
modification. Similar scenarios are contemplated with other
processing modes discussed herein.
It is contemplated that mixtures of combined and uncombined signals
can be displayed. For example endoscope 2 can have two signals
generated, each with a bandwidth or signal processing mode. The
signals of endoscope 2 are then combined for display on a single
monitor having two areas. Endoscope 2' can have two signals
generated, each with a bandwidth. The two signals can then be
displayed on two separate monitors. Thus in the present example,
there would be 3 monitors for a total of 4 display areas. Other
combinations are contemplated.
FIG. 4 shows an example of an output video signal having two
components alternately updated for display where the signal
processing mode is a false-color image having a bandwidth
selection. The first portion of the video signal 2100 is processed
according to a first bandwidth range 4100, to generate a first
component 4101 of a first portion of the output video signal 4001.
The second component 4102 of the first portion of the output video
signal 4001 as shown is generated from portion 0. Since portion 0
may not contain data, the first portion of the output video signal
4001 may only have one of the display areas showing content. As
shown, the first portion of the output video signal 4001 is
displayed 8100 on a monitor.
The second portion 2200 of the video signal is received by the
processor and this portion 2200 is processed according to a second
bandwidth range 4200 to generate the second component 4202 of the
second portion of the output video signal 4002. The first component
4201 of the second portion of the output video signal 4002 is
retained from the first portion of the output video signal 4001.
That is, component 4101 and 4201 display the same content, and both
are generated from the first portion 2100 of the video signal. The
second portion 2200 of the output video signal is used to update
the display 8200.
The third portion 2300 of the video signal is processed according
to the first bandwidth range 4300, the third portion of the output
video signal 4003 includes the component 4301, which is generated
from portion 2300. The second component 4302 of the third portion
of the output video signal 4003 is the same as component 4202, and
again, components 4302 and 4301 are generated from different
portions of the video signal. The third portion 2300 of the output
video signal is used to update the display 8300
The fourth portion of the video signal 2400 is received by the
processor and processed according to the second bandwidth range to
generate the second component 4402 of the fourth portion of the
output video signal 4004. The first component 4401 of the fourth
component of the output video signal 4004 is the same as the first
component 4301 of the third portion of the output video signal
4003. The fourth portion of the output video signal 4004 is used to
update the display 8400. The process is repeated with each
successive portion of the video signal being alternately processed
according to the first or second bandwidth range. The previously
processed portion is retained for the non-updated component.
Therefore, if the portions 2100, 2200, 2300 and 2400 are received
at 60 Hz, each component of the output video signal is updated at
30 Hz. Likewise, if there are three bandwidth selections, the
portions are received at 60 Hz, and each of the three components of
the output video signal is updated at 20 Hz.
Although FIG. 4 shows that the components are generated according
to bandwidth ranges, it would be understood by one of skill in the
art that the bandwidth ranges shown in the figures can be replaced
with other signal modes. For example, the first and/or second
components could be generated according to an edge enhancement
signal processing mode and the second component can be generated
according to a first bandwidth range. The system would alternately
update the signal as referenced above, but with the different
processing modes. The processing modes may be pre-set in some
cases, and in others, the system can receive a selection of
processing modes and characteristics of the processing modes. In
the case of a false-color processing mode, the selection could
first indicate a false-color mode and secondly indicate a
particular bandwidth or selection of bandwidth ranges for use with
the false-color mode.
FIG. 5 shows an example of two signals are alternately updated for
display. The first portion of the video signal 2100' is processed
according to a first bandwidth range 4100', to generate a first
portion 4101' the first signal 46. The second portion 4201' of the
first signal 46 is retained from the first portion 4101' of the
first signal 46. The third portion 4301' of the first signal 46 is
generated from the third portion 2300' of the video signal and
processed according to the first bandwidth range 4300'. The fourth
portion 4401' of the first signal 46 is retained from the third
portion 4301' of the first signal 46
The first portion 4102' of the second signal 50 as shown is
generated from portion 0. Since portion 0 may not contain data, the
first portion of the second signal may only have one of the display
areas showing content. The second portion 4202' of the second
signal 50 is generated from the second portion 2200' of the video
signal and processed according to a second bandwidth range 4200'.
The third portion 4302' of the second signal 50 is retained from
the second portion 4202' of the second signal 50 The fourth portion
4402' of the second signal 50 is generated from a fourth portion
2400' of the video signal and processed according to the second
bandwidth range 4400'.
As shown, the first portions of the respective signals are for
display in display areas 8101' and 8102'. The second, third and
fourth portions of the respective signals are for updating the
display 8202', 8301' and 8402' The non updated portion 8201', 8302'
and 8401' may be retained from the previously updated portion of
the signal. The updating may repeat continuously during display
according to the order shown.
Although FIG. 5 shows that the components are generated according
to bandwidth ranges, it would be understood by one of skill in the
art that the bandwidth ranges shown in the figures can be replaced
with other signal modes. For example, the first and/or second
signals could be generated according to an edge enhancement signal
processing mode and the second component can be generated according
to a first bandwidth range. The system would alternately update the
signal as referenced above, but with the different processing
modes. The processing modes may be pre-set in some cases, and in
others, the system can receive a selection of processing modes and
characteristics of the processing modes. In the case of a
false-color processing mode, the selection could first indicate a
false-color mode and secondly indicate a particular bandwidth or
selection of bandwidth ranges for use with the false-color
mode.
The process is repeated with each successive portion of the video
signal being alternately processed according to the first or second
bandwidth range. The previously processed portion is retained for
the non-updated component. Therefore, if the portions 2100', 2200',
2300' and 2400' are received at 60 Hz, the two signals 46', 50' are
each updated at 30 Hz. Likewise, if there are three bandwidth
selections, the portions are received at 60 Hz, and each of the
three signals are updated at 20 Hz. The display updating is
continuous according to the order shown, but other orders or
patterns are contemplated.
As discussed previously, it is often desirable to process a signal
with reduced red component to better visualize tissue structures.
The video signal can be processed to reduce or enhance different
color components. The system can also be adapted to process a video
signal from a CMYG color sensor. In such a case, the relevant color
components from the CMYG sensor can be reduced or enhanced
depending on the desired filter characteristics.
The present system includes a computed virtual chromoendoscopy
(CVC) system that provides for enhanced visibility between certain
structures with different hemoglobin concentrations and to enhance
visibility of surface structures to distinguish and classify types
of tissue.
The present system uses a broadband white-light illumination (light
source), and endoscope optics and video sensors, and a Camera
Control Unit (CCU) having a processor or a Modular Camera Control
Unit having a processor. The control unit is capable of a full
color conversion calculation using software-based image processing.
A Red-Green-Blue (RGB) color image sensor can be used. The image
processor utilizes matrices that transform acquired color channels
into a false-color image in order to display relevant tissue
features more clearly. The color channels may be, for example, CCD
or CMOS. Primarily, blue and green spectral wavelength regions are
utilized, while the red spectral wavelength region is suppressed or
attenuated. CMYG sensors can also be used to capture the video
signal and likewise, the relevant components from the CMYG sensor
can be enhanced, reduced or otherwise modified according to the
desired filter.
In the present system, the settings in the color conversion can be
chosen so that: a normal white-light image rendering (with natural
colors) is obtained; or a false-color image rendering is obtained,
in particular, where the signals from the blue and green input
channels are essentially used to generate the output image, while
the signal from the red color channel is strongly suppressed. The
system provides one or more different filter options for obtaining
a false-color image. Each filter may produce a different intensity
of the false-color scheme for assisting the practitioner in imaging
the tissue of interest.
One example of the color transformation coefficient matrices used
for the present filter modes are as follows, with the coefficients
represented by letters a-i, and SPIE representing the transformed
or false-color image:
.times..times. ##EQU00001## In one example, the filter coefficients
may be as follows:
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times. ##EQU00002##
The present system is implemented with matrix multiplication in a
color space where luminance and chrominance are combined. In this
design, the input color signal is a combined RGB signal. The output
is a RGB signal, which may have been color converted to a
false-color image rendering. Other filter coefficients are
contemplated and the example above should not be seen as
limiting.
Although aspects of the present system have been described with
reference to a reduced red component, the video signal may be
processed for reduced blue, green or other components. In this
case, the above example of the filter coefficients, reduced blue or
green component would require different filter characteristics. The
same holds true for a CMYG sensor or any other type of sensor in
that the filter can be selected to modify the image to show desired
characteristics.
As discussed previously, many signal processing modes display modes
are contemplated with the present system. The signal processing
modes modify the incoming image signal so that a modified image
signal can be displayed. Some of these include switching between a
normal white-light image or a computed mode image on a singular
display; displaying both the normal white-light image and the
computed mode image side-by-side on a singular display; a
picture-in-picture display featuring both the normal white-light
image and the computed mode image; and displaying the normal
white-light image and the computed mode image on two separate
displays. Further, switching from white-light image rendering to
computed mode may not require additional white balance. The system
can also update various other types of signal processing modes for
display. The types of signal processing modes can include, for
example, false or enhanced color, edge enhancement, texture
enhancement, sharpness adjustment, fiber image bundle. The fiber
image bundle may remove a honeycomb mosaic resulting from different
optical fiber bundles. This list should not be seen as exhaustive
as other signal processing modes can be used to modify the incoming
signal or portion of a signal for display.
Edge enhancement may include a signal processing technique that
recognizes certain tissue structures based on their reaction to the
light source. The edge enhancement technique would therefore modify
the signal based on a computation that would identify the location
of an edge of a particular tissue structure or type of tissue
structure. This may help a physician identify the tissue
structure.
In the present system, the white light and computed images are
processed by alternating portions of the video image. It is also
contemplated that different types of computed images may be used
where appropriate, the computed images may be processed according
to a signal processing mode. In some cases, it may be desirable to
have all displays showing computed images of different types. It is
also contemplated that different color enhancements can be used,
for example, red, blue and green components can all be attenuated,
enhanced or suppressed to create different false-color images. As
an example, the first captured portion is processed to display a
first white light image. The second captured portion is processed
to display a first computed image. The third captured portion is
processed to update the white light image. The fourth captured
portion is processed to update the computed image, and so on. As
discussed above, it is contemplated that the first white light
image may be replaced with a second computed image. It is also
contemplated that more than two processing modes can be displayed
and alternately updated. For example, a first portion is processed
to display a first computed image, a second portion processed to
display a second computed image, a third portion processed to
display a third computed image and a fourth portion processed to
display the first computed image, with the pattern repeating as
additional portions are processed for display. It is also
understood that different bandwidth selections within a false or
enhanced color mode can be considered different signal processing
modes. For example, a first signal processing mode could be a white
light or wide band mode and a second processing mode could be a
reduced-red light or narrow band mode. These examples provided are
not intended to be limiting as other combinations and updating
patterns can be used to display the computed image(s).
* * * * *