U.S. patent application number 09/875628 was filed with the patent office on 2001-10-04 for navigation apparatus and surgical operation image acquisition/display apparatus using the same.
This patent application is currently assigned to OLYMPUS OPTICAL CO., LTD.. Invention is credited to Asano, Takeo, Furuhashi, Yukihito, Kosaka, Akio, Matsuzaki, Hiroshi, Saito, Akito, Shibasaki, Takao.
Application Number | 20010027272 09/875628 |
Document ID | / |
Family ID | 26430826 |
Filed Date | 2001-10-04 |
United States Patent
Application |
20010027272 |
Kind Code |
A1 |
Saito, Akito ; et
al. |
October 4, 2001 |
Navigation apparatus and surgical operation image
acquisition/display apparatus using the same
Abstract
A navigation apparatus comprises a navigation-related
information generating section and a display section. The
navigation-related information generating section measures the
position and orientation of an object and a target in a
three-dimensional space and generate navigation-related information
to be used for navigating the object toward the target. The display
section displays the navigation-related information generated by
the navigation-related information generating section in any of
different modes depending on the relationship of the position and
orientation of the object and that of the target. A surgical
operation image acquisition/display apparatus comprises an
observation section, an image display section and a specifying
section. The observation section includes a plurality of
observation sections whose position and orientation is modifiable.
The image display section is adapted to alternatively display any
of the images obtained by the observation sections or synthetically
combine and display the combined images. The specifying section
specifies the image to be displayed to the image display section
according to the position and orientation of the observation
section.
Inventors: |
Saito, Akito; (Hino-shi,
JP) ; Shibasaki, Takao; (Tokyo, JP) ; Asano,
Takeo; (Kunitachi-shi, JP) ; Matsuzaki, Hiroshi;
(Hachioji-shi, JP) ; Furuhashi, Yukihito;
(Hachioji-shi, JP) ; Kosaka, Akio; (Hachioji-shi,
JP) |
Correspondence
Address: |
FRISHAUF, HOLTZ, GOODMAN, LANGER & CHICK, P.C.
25th Floor
767 Third Avenue
New York
NY
10017-2023
US
|
Assignee: |
OLYMPUS OPTICAL CO., LTD.
Tokyo
JP
|
Family ID: |
26430826 |
Appl. No.: |
09/875628 |
Filed: |
June 6, 2001 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09875628 |
Jun 6, 2001 |
|
|
|
09533651 |
Mar 22, 2000 |
|
|
|
Current U.S.
Class: |
600/426 ;
600/429 |
Current CPC
Class: |
A61B 2034/2072 20160201;
A61B 34/20 20160201; A61B 90/20 20160201; A61B 5/064 20130101; A61B
90/361 20160201; A61B 2034/2055 20160201; A61B 1/0005 20130101;
A61B 2090/3983 20160201; A61B 90/36 20160201 |
Class at
Publication: |
600/426 ;
600/429 |
International
Class: |
A61B 006/00 |
Foreign Application Data
Date |
Code |
Application Number |
Mar 30, 1999 |
JP |
11-089405 |
Jun 10, 1999 |
JP |
11-163964 |
Claims
What is claimed is:
1. A navigation apparatus comprising: a navigation-related
information generating section for generating navigation-related
information by measuring the relative position and orientation of
an object and a target in a three-dimensional space in order to
navigate said object to said target; and a display section for
displaying said navigation-related information generated by said
navigation-related information generating section in different
modes according to the relative position and orientation of said
object and said target.
2. A navigation apparatus comprising: a navigation-related
information generating section for generating navigation-related
information by measuring the relative position and orientation of
an object and a target in a three-dimensional space in order to
navigate said object to said target; and a display section for
displaying at least a model image of the object or the target,
information on the direction of navigation or information on the
distance between the object of navigation and the target when said
navigation-related information generating section can measure said
position and orientation in a three-dimensional space but
displaying information indicating a measurement incapable situation
when said navigation-related information generating section cannot
measure said position and orientation.
3. A navigation apparatus according to claims 1 and 2, wherein said
object has an image acquisition function; and said display section
can display the image acquired by the image acquisition function of
said object with other navigation-related information in an
overlaid.
4. A navigation apparatus adapted to generate information for
navigating an object to a target comprising: a three-dimensional
position and orientation measuring section for measuring the
three-dimensional position and orientation of said object and that
of said target; an information generating section for generating
information necessary for the navigation on the basis of the
obtained result of said three-dimensional position and orientation
measuring section; and a display section for displaying the
information generated by said information generating section;
wherein said display section displays navigation-related
information according to at least information on the distance
between said object and said target, information on the direction
toward said target as viewed from said object and information if
either said object or said target is located within the effective
area of measurement of said three-dimensional position and
orientation measuring section in a display mode selected out of a
plurality of display modes.
5. A navigation apparatus according to claim 4, wherein said
display section displays at least profile information on said
target or said object, internal tomographic information on said
target, information on the direction toward said target as viewed
from said object or vice versa or information on the distance
between said object and said target when said target and said
object is measurable by said three-dimensional position and
orientation measuring section but displays information indicating a
measurement incapable situation when neither said target nor said
object is measurable by said three-dimensional position and
orientation measuring section.
6. A navigation apparatus according to claim 4, wherein said object
has an image acquisition function; and said display section can
display the image acquired by the image acquisition function of
said object with other navigation-related information in an
overlaid.
7. A navigation apparatus according to claim 4, wherein said
navigation-related information displayed on the display section
changes its color as a function of the relative distance between
said target and said object or the direction toward said target as
viewed from said object.
8. A navigation apparatus according to claim 4, wherein the
thickness of the lines of the navigation-related information
displayed on the display section changes as a function of the
relative distance between said target and said object or the
direction toward said target as viewed from said object.
9. A navigation apparatus according to claim 4, wherein a profile
model of the target and an internal tomographic image are displayed
alternatively on said display section as a function of the distance
between said target and said object.
10. A navigation apparatus according to claim 4, wherein the
density of lines displayed on said display section for drawing said
target model image is varied as a function of the distance between
said target and said object.
11. A navigation apparatus according to claim 4, wherein an image
representing said target and directional information of said target
are displayed alternatively on said display section as a function
of the distance between said target and said object or the
direction toward said target as viewed from said object.
12. A navigation apparatus according to claim 4, wherein direction
information of said target is displayed on said display section in
the form of a symbol at least whose size or shape varies as a
function of the distance between said target and said object or the
direction toward said target as viewed from said object.
13. A navigation apparatus comprising: a target of navigation; an
object to be navigated to said target; a three-dimensional position
and orientation measuring section for measuring the
three-dimensional position and orientation of at least said target
or said object of navigation; a computational information
determining section for generating navigation-related information
on the basis of the information obtained by the measurement of said
three-dimensional position and orientation measuring section and
controlling the generated information; and an information display
section for display the navigation-related information generated
and controlled by said computational information determining
section; the relative distance between the target and the object of
navigation and their orientations in a three-dimensional space are
determined by the three-dimensional position and orientation
measuring section; wherein said computational information
determining section modifies either the attribute or the type of
said navigation-related information as a function of at least the
relative distance between said target and said object of navigation
or the direction toward said target as viewed from said object
measured by said three-dimensional position and orientation
measuring section or the immeasurability thereof so as to make it
visibly reflect the outcome of measurement.
14. A navigation apparatus according to claim 13, wherein said
navigation-related information includes a model image of the
profile of the target and/or that of the object of navigation, a
model image of the internal tomographic information of the target,
a symbol pattern indicating the direction in which the target
and/or the object of navigation will be found and/or a numerical
value or a symbol pattern indicating the distance between the
target and the object of navigation when said three-dimensional
position and orientation measuring section is operating normally
whereas it includes character information or a symbol pattern
indicating an immeasurable situation when the three-dimensional
position and orientation measuring section is inoperative.
15. A navigation apparatus according to claim 13, wherein said
object of navigation has an image acquisition function and the
acquired image obtained by means of the image acquisition function
is displayed with other navigation-related information obtained by
said computational information determining section in an overlaid
manner by means of said computational information determining
section.
16. A navigation apparatus according to claim 13, wherein the
navigation-related information displayed on the display section
changes its color as a function of the relative distance between
said target and said object of navigation or the direction toward
said target as viewed from said object of navigation.
17. A navigation apparatus according to claim 13, wherein the
thickness of the lines of the navigation-related information
displayed on the display section changes as a function of the
relative distance between said target and said object of navigation
and/or the direction toward the target as viewed from the object of
navigation.
18. A navigation apparatus according to claim 13, wherein the
profile model of said target and the internal tomographic image are
switched from one to the other on the display section as a function
of the relative distance between said target and said object of
navigation.
19. A navigation apparatus according to claim 13, wherein the
density of lines of the displayed target model image is modified as
a function of the relative distance between said target and said
object of navigation.
20. A navigation apparatus according to claim 13, wherein a model
image of at least either said target or said object of navigation
or a symbol pattern indicating the direction in which at least said
target or said object of navigation is found is displayed
alternatively as a function of the relative distance between said
target and said object of navigation and the direction toward the
target as viewed from the object of navigation.
21. A navigation apparatus according to claim 13, wherein the size
of the symbol pattern indicating the direction of said target as
viewed from the object of navigation is modified as a function of
the relative distance between said target and said object of
navigation.
22. A surgical operation image acquisition/display apparatus
comprising: an observation section having a plurality of
observation units and adapted to modify its position and
orientation; an image display section adapted to alternatively or
synthetically display the images obtained by said plurality of
observation units of said observation section; and a specifying
section for specifying the images to be displayed according to the
position and orientation of said observation section.
23. A surgical operation image acquisition/display apparatus
comprising: an observation section having a plurality of
observation units and adapted to modify its position and
orientation; an image display section adapted to display each of
the images obtained by said plurality of observation unit of said
observation section in a size either enlarged or reduced with an
independently selected magnitude; and a specifying section for
specifying the images to be synthetically combined according to the
position and orientation of said observation section.
24. A surgical operation image acquisition/display apparatus
comprising: an observation section having a plurality of
observation units and adapted to modify its position and
orientation; an image display section adapted to display an
selected from the images obtained by said plurality of observation
section of said observation section; and a specifying section for
specifying the image to be selected according to the position and
orientation of said observation section.
25. A surgical operation image acquisition/display apparatus
comprising: an observation section having a plurality of
observation units and adapted to modify its position and
orientation relative to an area of surgical operation; an image
display section adapted to synthetically display the images
obtained by said plurality of observation units of said observation
section in a mode selected out of a plurality of different modes;
and a specifying section for specifying a mode of image synthesis
to said image display section according to the position and
orientation of said observation section.
26. A surgical operation image acquisition/display apparatus
comprising: an observation section having a surgical microscope
adapted to modify its position and orientation and an endoscope
also adapted to modify its position and orientation; a detection
section for detecting the position and orientation of said
observation section; an image display section adapted to
synthetically combine the image observed by said surgical
microscope and the image observed by said endoscope and display the
images in a mode selected out of a plurality of different modes;
and a specifying section for specifying the mode of image synthesis
to said image display section according to the outcome of the
detection of said detection section.
27. A surgical operation image acquisition/display apparatus
comprising: an observation section having a surgical microscope
adapted to modify its position and orientation and an endoscope
also adapted to modify its position and orientation; a detection
section for detecting the position and orientation of said
observation section; an image display section adapted to
alternatively display the image observed by said surgical
microscope or the image observed by said endoscope; and a
specifying section for specifying the image to be displayed
according to the outcome of the detection of said detection
section.
28. A surgical operation image acquisition/display apparatus
according to claim 22, wherein said specifying section specifies in
accordance with the distance between the area of surgical operation
and the observation section.
29. A surgical operation image acquisition/display apparatus
comprising: a first observation section for observing an area of
surgical operation; a second observation section arranged apart
from said first observation section and adapted to observe at least
said area of surgical operation or the vicinity thereof; an
observed image synthesizing section for synthetically combining the
image observed by said first observation section and the image
observed by said second observation section; a display section for
display the image synthesized by said observed image synthesizing
section; a position and orientation detection section for detecting
at least either the position and orientation of said first
observation section of that of said second observation section; and
a state of synthesis modification specifying section for specifying
the modification made to the state of synthesis of the observed
images to said observed image synthesizing section according to the
position and orientation information from said position and
orientation detection section.
30. A surgical operation image acquisition/display apparatus
according to claim 29, wherein said state of synthesis modification
specifying section specifies for use either the image observed by
said first observation section or the image observed by said second
observation section according to the position and orientation
information of either said first observation section of said second
observation section fed from said position and orientation
detection section and either the relative distance between said
area of surgical operation and said first observation section or
the direction of said area of surgical operation as viewed from
said second observation section.
31. A surgical operation image acquisition/display apparatus
according to claim 29, wherein said state of synthesis modification
specifying section specifies reduction of either the image observed
by said first observation section or the image observed by said
second observation section according to the position and
orientation information of either said first observation section of
said second observation section fed from said position and
orientation detection section and either the relative distance
between said area of surgical operation and said first observation
section or the direction of said area of surgical operation as
viewed from said second observation section, and synthesis the
reduced image into the other image.
32. A navigation apparatus comprising: navigation-related
information generating means for generating navigation-related
information by measuring the relative position and orientation of
an object and a target in a three-dimensional space in order to
navigate said object to said target; and display means for
displaying said navigation-related information generated by said
navigation-related information generating means in different modes
according to the relative position and orientation of said object
and said target.
33. A navigation apparatus comprising: navigation-related
information generating means for generating navigation-related
information by measuring the relative position and orientation of
an object and a target in a three-dimensional space in order to
navigate said object to said target; and display means for
displaying at least a model image of the object or the target,
information on the direction of navigation or information on the
distance between the object of navigation and the target when said
navigation-related information generating means can measure said
position and orientation in a three-dimensional space but
displaying information indicating a measurement incapable situation
when said navigation-related information generating means cannot
measure said position and orientation.
34. A navigation apparatus adapted to generate information for
navigating an object to a target comprising: three-dimensional
position and orientation measuring means for measuring the
three-dimensional position and orientation of said object and that
of said target; information generating means for generating
information necessary for the navigation on the basis of the
obtained result of said three-dimensional position and orientation
measuring means; and display means for displaying the information
generated by said information generating means; wherein said
display means displays navigation-related information according to
at least information on the distance between said object and said
target, information on the direction toward said target as viewed
from said object and information if either said object or said
target is located within the effective area of measurement of said
three-dimensional position and orientation measuring means in a
display mode selected out of a plurality of display modes.
35. A navigation apparatus comprising: a target of navigation; an
object to be navigated to said target; three-dimensional position
and orientation measuring means for measuring the three-dimensional
position and orientation of at least said target or said object of
navigation; computational information determining means for
generating navigation-related information on the basis of the
information obtained by the measurement of said three-dimensional
position and orientation measuring means and controlling the
generated information; and information display means for display
the navigation-related information generated and controlled by said
computational information determining means; the relative distance
between the target and the object of navigation and their
orientations in a three-dimensional space are determined by the
three-dimensional position and orientation measuring means; wherein
said computational information determining means modifies either
the attribute or the type of said navigation-related information as
a function of at least the relative distance between said target
and said object of navigation or the direction of said target as
viewed from said object measured by said three-dimensional position
and orientation measuring means or the immeasurability thereof so
as to make it visibly reflect the outcome of measurement.
36. A surgical operation image acquisition/display apparatus
comprising: observation means having a plurality of observation
sections and adapted to modify its position and orientation; image
display means adapted to alternatively or synthetically display the
images obtained by said plurality of observation sections of said
observation means; and specifying means for specifying the images
to be displayed according to the position and orientation of said
observation section.
37. A surgical operation image acquisition/display apparatus
comprising: observation means having a plurality of observation
sections and adapted to modify its position and orientation; image
display means adapted to display each of the images obtained by
said plurality of observation sections of said observation section
in a size either enlarged or reduced with an independently selected
magnitude; and specifying means for specifying the images to be
synthetically combined according to the position and orientation of
said observation means.
38. A surgical operation image acquisition/display apparatus
comprising: observation means having a plurality of observation
sections and adapted to modify its position and orientation; image
display means adapted to display an selected from the images
obtained by said plurality of observation sections of said
observation means; and specifying means for specifying the image to
be selected according to the position and orientation of said
observation means.
39. A surgical operation image acquisition/display apparatus
comprising: observation means having a plurality of observation
means and adapted to modify its position and orientation relative
to an area of surgical operation; image display means adapted to
synthetically display the images obtained by said plurality of
observation means of said observation section in a mode selected
out of a plurality of different modes; and specifying means for
specifying a mode of image synthesis to said image display means
according to the position and orientation of said observation
section.
40. A surgical operation image acquisition/display apparatus
comprising: observation means having a surgical microscope adapted
to modify its position and orientation and an endoscope also
adapted to modify its position and orientation; a detection means
for detecting the position and orientation of said observation
means; image display means adapted to synthetically combine the
image observed by said surgical microscope and the image observed
by said endoscope and display the images in a mode selected out of
a plurality of different modes; and specifying means for specifying
the mode of image synthesis to said image display means according
to the outcome of the detection of said detection means.
41. A surgical operation image acquisition/display apparatus
comprising: observation means having a surgical microscope adapted
to modify its position and orientation, and an endoscope also
adapted to modify its position and orientation; detection means for
detecting the position and orientation of said observation means;
image display means adapted to alternatively display the image
observed by said surgical microscope or the image observed by said
endoscope; and specifying means for specifying the image to be
displayed according to the outcome of the detection of said
detection means.
42. A surgical operation image acquisition/display apparatus
comprising: first observation means for observing an area of
surgical operation; second observation means arranged apart from
said first observation means and adapted to observe at least said
area of surgical operation or the vicinity thereof; observed image
synthesizing means for synthetically combining the image observed
by said first observation means and the image observed by said
second observation means; display means for display the image
synthesized by said observed image synthesizing means; position and
orientation detection means for detecting at least either the
position and orientation of said first observation means of that of
said second observation means; and state of synthesis modification
specifying means for specifying the modification made to the state
of synthesis of the observed images to said observed image
synthesizing means according to the position and orientation
information from said position and orientation detection means.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is based upon and claims the benefit of
priority from the prior Japanese Patent Applications No. 11-089405,
filed Mar. 30, 1999; and No. 11-163964, filed Jun. 10, 1999, the
entire contents of which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] This invention relates to a navigation apparatus and, more
particularly, to a navigation apparatus adapted to modify
navigation-related information according to the relative position
and orientation of the object of navigation and the target within a
three-dimensional space.
[0003] This invention also relates to a surgical operation image
acquisition/display apparatus and, more particularly, to an
operation image acquisition/display apparatus adapted to
acquisition and display images of a plurality of observation
systems used in surgical operations.
[0004] Various navigation apparatus have been proposed for
applications in the field of surgical operations, including those
disclosed in Jpn. Pat. Appln. KOKAI Publication Nos. 9-173352 and
10-5245.
[0005] The medical navigation system disclosed in Jpn. Pat. Appln.
KOKAI Publication No. 9-173352 is adapted to display information
(profile information, medical image information) on the desired
part of the object of examination specified by a specifying section
for specifying a desired part of the object of examination.
[0006] It is also adapted to display video information obtained by
an appearance imaging section, profile information on the profile
measured by a profile measuring section and medical image
information obtained by a medical image acquisition section on an
image display section in an overlaid way.
[0007] The surgical operation assisting apparatus disclosed in Jpn.
Pat. Appln. KOKAI Publication No. 10-5245 is adapted to display the
current position of the surgical instrument being used in a
surgical operation and the blood vessel located closest to the
instrument on a tomographic image of the area of surgical operation
in an overlaid manner by using the image data on the tomographic
image, the surgical instrument being used, blood vessel detection
section for detecting the blood vessel located closest to the
surgical instrument, a position detection section for detecting the
current position of the surgical instrument, an arithmetic
computing section for computationally determines the position of
the front end of the surgical instrument and the direction in which
the surgical instrument is inserted, an image selection section for
selecting the image data on the image being acquired for the area
where the front end of the surgical instrument is located and an
image synthesizing section for synthetically combining the image
selected by the image selection section and a predetermined pattern
indicating the front end of the surgical instrument in an overlaid
manner.
[0008] The above described arrangement is intended to allow the
operator to visually confirm the position of the front end of the
surgical instrument inserted into the body of the patient on the
tomographic image being displayed.
[0009] However, the medical navigation system and the surgical
operation assisting apparatus as disclosed in the above patent
documents are accompanied by the following problems.
[0010] As for the medical navigation system disclosed in Jpn. Pat.
Appln. KOKAI Publication No. 9-173352, it simply displays
information on a desired part of the object of examination
specified by the section for specifying a desired part of the
object of examination and it is difficult to navigate the section
to the part desired by the user.
[0011] Additionally, this known medical navigation system provides
a difficulty with which the surgeon realizes distances in the
perspective of displayed information in the direction connecting
the eyes of the surgeon and the display screen that is
perpendicular to the latter.
[0012] Furthermore, this known medical navigation system provides
an additional difficulty with which the surgeon determines the
route of navigation on the basis of the displayed information when
both the object of examination and the section for specifying the
desired part of the object of examination are located at respective
positions that are found within the measurable area but outside the
displayable area of the system.
[0013] The surgical operation assisting system disclosed in Jpn.
Pat. Appln. KOKAI Publication No. 10-5245 is accompanied by a
problem of cumbersomeness that the user has to be constantly aware
of the distance between the position of the front end of the
surgical instrument on the displayed tomographic image and the
position of the detected blood vessel in order to know the distance
between the blood vessel and the surgical instrument.
[0014] In recent years, micro-surgery has become popular as a
result of the development of both surgical techniques and surgical
instruments.
[0015] In micro-surgery, generally a surgical microscope is used to
observe an enlarged view of the area of surgical operation.
[0016] Particularly, in the field of cranial nerve surgery and
otorhinolarygology, there arise occasions frequently where the area
of operation can hardly be observed because it is at the so-called
dead angle if the surgical microscope is handled elaborately when
the area is located deep in the body.
[0017] For observing an area at such a dead angle, normally a
mirror or an endoscope is used.
[0018] When using an endoscope for micro-surgery, it has to
manipulated and placed accurately at the right position that is
located deep in the body having an exquiarealy complicated
three-dimensional structure because the area of operation is always
at the dead angle of the surgical microscope.
[0019] The manipulation has to be conducted carefully by the
operator, while observing it through the surgical microscope so
that any normal tissues of the patient would not be inadvertently
damaged by the endoscope and, at the same time, the area of
operation has to be visually confirmed by means of the
endoscope.
[0020] While manipulating the endoscope, the operator has to select
instantaneously either the image taken by the surgical microscope
or the image acquired by way of the endoscope as object of
observation and the selection has to be correct.
[0021] As an attempt for aiding a surgeon manipulating the
endoscope, Jpn. Pat. Appln. KOKAI Publication No. 5-203881 proposes
an integrated image system comprising a plurality of CCD cameras
connected to respective observation systems, each including a
surgical microscope, an endoscope and other instruments, a CCD
camera controller for controlling the operation of selectively
using any of the observation systems and a view finder controller
so that the user may select any of the observation systems by means
of the CCD camera controller in the course of the ongoing surgical
operation.
[0022] Jpn. Pat. Appln. KOKAI Publication No. 7-261094 discloses a
surgical microscope with which the user can switch from the image
of the surgical microscope to that of the endoscope or vice versa
or overlay one on the other whenever necessary.
[0023] However, with the known technique disclosed in the above
described Jpn. Pat. Appln. KOKAI Publication No. 5-203881, the
operator has to carry out the switching or overlaid operation at
the cost of a smooth progress of the ongoing surgical
operation.
[0024] Additionally, while the above patent document describes that
the image may be switched from one to the other, it does not
describe specifically how the switching operation proceeds.
[0025] On the other hand, the known technique disclosed in Jpn.
Pat. Appln. KOKAI Publication No. 7-261094 involves the use of a
mode switch with which the surgical operator can switch the display
mode whenever necessary.
[0026] However, it is highly cumbersome for the operator to switch
from the image of the surgical microscope to that of the endoscope
or vice versa when he or she has to place the endoscope in a
position deep in the body of the patient having an exquiarealy
complicated three-dimensional structure. Additionally, such a
switching operation can obstruct the smooth progress of the
surgical operation.
BRIEF SUMMARY OF THE INVENTION
[0027] In view of the above identified problems of the prior art,
it is therefore the object of the present invention to provide a
navigation apparatus with which the user can easily and visually
realize the distance between a target and an object of navigation
by modifying the obtained navigation-related information according
to the relative position and orientation of the object of
navigation and the target within a three-dimensional space and the
user can easily obtain navigation-related information of the type
necessary for the user.
[0028] Another object of the invention is to provide an operation
image acquisition/display apparatus adapted to acquisition and
display images of a plurality of observation systems used in
surgical operations without requiring the operator to manually
switch from one observation system to another so that the ongoing
surgical operation may proceeds smoothly.
[0029] In the first aspect of the invention, the above first object
is achieved by providing a navigation apparatus comprising:
[0030] a navigation-related information generating section for
generating navigation-related information by measuring the relative
position and orientation of an object and a target in a
three-dimensional space in order to navigate the object to the
target; and
[0031] a display section for displaying the navigation-related
information generated by the navigation-related information
generating section in different modes according to the relative
position and orientation of the object and the target.
[0032] Thus, with a navigation apparatus according to the invention
adapted to display navigation-related information in different
modes according to the relative position and orientation of the
object and the target within a three-dimensional spatial.
[0033] A navigation apparatus according to the invention will be
described hereinafter in terms of the firs and second embodiments.
While the above target may normally be a patient, a tumor to be
surgically treated of a patient or an area of the body of a patient
requiring special attention during a surgical operation, it is by
no means limited to an existing object of examination and may
alternatively be a virtual target displayed as a two-dimensional or
three-dimensional image of a model synthesized by using the video
information of an existing target that is obtained in advance.
[0034] While the above object may normally refer to an endoscope 3,
it may alternatively refer to some other surgical instrument such
as a suction pipe or a pair of forceps.
[0035] While the above display section may normally refer to a
liquid crystal monitor, it may alternatively refer to some other
video information display such as a CRT display or a head mount
display.
[0036] For the purpose of the invention, the above expression "in
different modes" refers to differences in color, in the thickness
of line, in dimensions and in the density of drawing.
[0037] In the second aspect of the invention, the above second
object is achieved by providing a surgical operation image
acquisition/display apparatus comprising:
[0038] an observation section having a plurality of observation
section and adapted to modify its position and orientation;
[0039] an image display section adapted to alternatively or
synthetically display the images obtained by the plurality of
observation section of the observation section; and
[0040] an indication section for indicating the images to be
displayed.
[0041] Additional objects and advantages of the invention will be
set forth in the description which follows, and in part will be
obvious from the description, or may be learned by practice of the
invention. The objects and advantages of the invention may be
realized and obtained by means of the instrumentalities and
combinations particularly pointed out hereinafter.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING
[0042] The accompanying drawings, which are incorporated in and
constitute a part of the specification, illustrate presently
preferred embodiments of the invention, and together with the
general description given above and the detailed description of the
preferred embodiments given below, serve to explain the principles
of the invention.
[0043] FIG. 1 is a schematic illustration of the first embodiment
of the invention which is a navigation apparatus, showing its
configuration;
[0044] FIG. 2 is a schematic illustration of a distance map that
can be used for a navigation apparatus according to the
invention;
[0045] FIG. 3 is a schematic illustration of a distance map that
can be used for a navigation apparatus according to the
invention;
[0046] FIG. 4 is a schematic illustration of the relationship
between data on an object of examination and the object of
examination itself;
[0047] FIG. 5 is a schematic illustration of a coordinate
transformation matrix for correlating data on an object of
examination and the object of examination itself;
[0048] FIG. 6 is a schematic illustration of a coordinate
transformation matrix for transforming a coordinate system defined
by a sensing plate fitted to an endoscope into a coordinate system
to be used by a camera model expressing the optical system of the
endoscope and a coordinate transformation matrix for transforming
the coordinate system of a camera model into the coordinate system
of a liquid crystal monitor;
[0049] FIG. 7 is a schematic illustration of a coordinate
transformation matrix for transforming a coordinate system defined
by a sensing plate fitted to the head of an object of examination
into the coordinate system defined by a sensing plate fitted to an
endoscope.
[0050] FIG. 8 is a schematic illustration of a transformation using
a plurality of coordinate transformation matrices for transforming
data on a target area into positional data on a liquid crystal
monitor;
[0051] FIGS. 9A through 9D schematically illustrate examples of
images that may be displayed on a liquid crystal monitor, of which
FIG. 9A is an image obtained by overlaid a wireframe image as
navigation information on an image obtained by means of the optical
system of an endoscope, FIG. 9B is an image obtained by overlaid an
internal tomographic image of three-dimensional volume data as
navigation information on an image obtained by means of the optical
system of an endoscope, FIG. 9C is an image obtained when no target
area is found within the effective area of measurement of an
endoscope and FIG. 9D is an image obtained when the apparatus is
inoperative for measurement;
[0052] FIG. 10 is a flow chart of a display operation for
displaying an image as shown in FIG. 9C;
[0053] FIG. 11 is a schematic illustration of the second embodiment
of the invention which is also a navigation apparatus, showing its
configuration;
[0054] FIG. 12 is a schematic illustration of an example of an
image displayed by the second embodiment and a coordinate
transformation matrix that can be used for the display;
[0055] FIG. 13 is a schematic illustration of an operation of
modifying the thickness of lines of an orthogonally projected image
of an endoscope as a function of the relative distance between the
target area and the front end of an endoscope;
[0056] FIG. 14 is a schematic block diagram of the third embodiment
of the invention which is a surgical operation image
acquisition/display apparatus, showing its configuration;
[0057] FIGS. 15A through 15F are schematic illustrations of a
plurality of display modes that can be realized by the video mixer
143 of FIG. 14;
[0058] FIG. 16 is a schematic illustration of an operation of
correlating data on the operation area of a patient 146 and data on
the characteristic points of a model data coordinate system m;
[0059] FIG. 17 is a schematic illustration of a mode of
computationally obtaining a coordinate transformation matrix pHe
for transforming the patient coordinate system p defined by the
sensing plate 145b fitted to the head of a patient 146 to the
endoscope coordinate system e defined by the sensing plate 145c
fitted to an endoscope 142;
[0060] FIG. 18 is a flow chart of the operation of the image
controller 147 of FIG. 14; and
[0061] FIG. 19 is a flow chart of the operation of the image
controller 147 of the fourth embodiment of the invention which is
also a surgical operation image acquisition/display apparatus.
DETAILED DESCRIPTION OF THE INVENTION
[0062] Reference will now be made in detail to the presently
preferred embodiments of the invention as illustrated in the
several views of the accompanying drawing, in which like reference
numerals designates like or corresponding parts.
[0063] (Embodiment 1)
[0064] FIG. 1 is a schematic illustration of the first embodiment
of the invention which is a navigation apparatus, showing its
configuration.
[0065] Referring to FIG. 1, object of examination 1, or patient, is
lying flat on an operating table, facing upward.
[0066] A hard sensing plate 2 carrying three LEDs for emitting
infrared rays that are arranged at the respective corners of a
triangle is securely fitted to the head of the object of
examination 1 in such a way that its position and orientation
relative to the head would not change easily.
[0067] Another hard sensing plate 4 carrying three LEDs for
emitting infrared rays arranged at the respective corners of a
triangle is securely fitted to an endoscope 3.
[0068] The LEDs arranged on the sensing plate 2 and those arranged
on the sensing plate 4 do not change their positional
relationships.
[0069] The positions of the LEDS of each of the sensing plates 2
and 4 are observed and determined in advance in terms of the
coordinate system defined on the sensing plate and stored in sensor
information storage section 5 as LED definition data.
[0070] The sensor information storage section 5 is connected to
sensor control section 6.
[0071] Then, image acquisition type sensor assembly 7 is arranged
at a position where the sensing plates 2 and 4 are found within its
effective area of measurement.
[0072] Then, a three-dimensional position and orientation measuring
section is established as the sensing plates 2 and 4 and the sensor
assembly 7 are connected to sensor control section 6.
[0073] The three-dimensional position and orientation information
obtained by the three-dimensional position and orientation
measuring section is sent to navigation-related information control
section 8.
[0074] The information including profile information and internal
tomographic image information on the object of examination, the
tumor thereof to be surgically treated and the parts thereof
requiring special attention during a surgical operation and
obtained in advance by measurement using CT and/or MRI is divided
into low resolution information (e.g., for a resolution level of
32.times.32.times.32 voxels), medium resolution information (e.g.,
for a resolution level of 128.times.128.times.128 voxels) and high
resolution information (e.g., for a resolution level of
512.times.512.times.512) and then transformed into wireframe
three-dimensional model data 10 (high resolution wireframe
three-dimensional model data 10a, medium resolution wireframe
three-dimensional model data 10b, low resolution wireframe
three-dimensional model data 10c) and three-dimensional volume data
11 (high resolution three-dimensional volume data 11a, medium
resolution three-dimensional volume data 11b, low resolution
three-dimensional volume data 11c) and stored in the
navigation-related information storage section 9 as data.
[0075] The navigation-related information storage section 9
additionally stores in advance as distance map.
[0076] As shown in FIGS. 2 and 3, a distance map 12 contains a
three-dimensional array having values representing the shortest
distances from the surface of the target area (the object of
examination, the tumor to be surgically treated or the parts of the
body requiring special attention during a surgical operation), the
affixed numbers of the array being variables corresponding to the
three-dimensional positional coordinate system of the space where
the target area is located.
[0077] For example, when the smallest unit of division is 0.1 mm, a
{fraction (1/10)} of an index number represents a corresponding
coordinate value as expressed in terms of millimeter.
[0078] Assume that such a distance map is prepared for each target
area in advance by means of a distance map preparing computer and
stored in the navigation-related information storage section 9 as
data.
[0079] Note that all the wireframe three-dimensional model data 10,
the three-dimensional volume data 11 and the distance map 12 are
subjected to a coordinate transforming operation so that they are
be expressed in terms of a same coordinate system.
[0080] Then, the image obtained by way of the optical system of the
endoscope 3 is taken into the navigation-related information
control section 8 by way of a camera control unit and an image
input board (not shown).
[0081] The navigation-related information generated by the
navigation-related information control section 8 is display to the
user on information display section, which is a liquid crystal
monitor 13.
[0082] As shown in FIG. 4, data on the object of examination 1 and
the object of examination 1 itself are correlated by measuring the
coordinate value m of each characteristic point on the data and the
coordinate values p of the corresponding characteristic point as
defined by the sensing plate 2 and computing a coordinate
transformation matrix (pHm) 14.
[0083] The coordinate transformation matrix (pHm) 14 is stored in
the above navigation-related information storage section 9.
[0084] As shown in FIG. 5, a coordinate transformation matrix is a
4-row and 4-column matrix comprising a rotational component R
representing a rotary motion in a three-dimensional space, a
translational component T representing a translation in the
three-dimensional space and a constant component.
[0085] Additionally, as shown in FIG. 6 a coordinate transformation
matrix (cHe) 15 for transforming the coordinate system defined by
the sensing plate 4 into the coordinate system to be used by a
camera model expressing the optical system of the endoscope 3 and a
coordinate transformation matrix (f_ctos) 16 for transforming the
camera model coordinate system into the coordinate system on the
actual liquid crystal monitor 13 are also determined and stored in
the navigation-related information storage section 9.
[0086] Now, the operation of the first embodiment of navigation
apparatus according to the invention and having the above described
configuration will be discussed below.
[0087] During the operation of the navigation apparatus, the sensor
control section 6 that is a component of the three-dimensional
position and orientation measuring section measures the
three-dimensional position of each of the LEDs that are emitting
infrared rays of the sensing plates 2 and 4 and then
computationally determines the three-dimensional position and
orientation information of each of the sensing plates 2 and 4 in
terms of the coordinate values of the original point of the space
defined by the sensing plate 4 on the three-dimensional space
defined by turn by the sensing plate 2 and the values of the unit
vectors along the X, Y and Z axis of the space defined by the
sensing plate 4 by using the LED definition data stored in the
sensor information storage section 5.
[0088] Then, as shown in FIG. 7, the coordinate transformation
matrix (pHe) 17 from the sensing plate 2 attached to the head of
the object of examination 1 to the sensing plate 4 attached to the
endoscope 3 is computationally determined on the basis of the
obtained three-dimensional position and orientation
information.
[0089] Then, as shown in FIG. 8, the data of target area is
converted to the positional data on the liquid crystal monitor 13,
the navigation-related information control sections 8 generates
navigation-related information by using the obtained positional
data based on the coordinate transformation matrix 17 and the
coordinate transformation matrixes 14, 15 and 16.
[0090] As the image formed by the optical system of the endoscope 3
is input to the navigation-related information control section 8,
the navigation-related information and the image are displayed on
the liquid crystal monitor 13 in an overlaid manner as shown in
FIG. 9A.
[0091] Then, as shown in FIG. 7, the position of the front end of
the endoscope 3 is subjected to an operation of coordinate
transformation by using the above described coordinate
transformation matrices 14, 15 and 17 and the relative distance
between the target area and the front end of the endoscope 3 is
determined by referring to the distance map 12.
[0092] Then, as shown in FIG. 9A, when the object of examination 1
and the endoscope 3 are in a measurable state, the relative
distance between the target area and the front end of the endoscope
3 is displayed on the liquid crystal monitor 13 as the distance to
the tumor in terms of the length of a bar 30 and a numerical value
31.
[0093] In a surgical operation using an endoscope 3, the endoscope
3 has to be inserted toward the tumor from the outside of the
object of examination 1, paying attention to the parts that should
not be damaged, and then the tumor has to be surgically
treated.
[0094] When the endoscope 3 is located outside the object of
examination 1, the model image of the target area is generated as a
profiled, wireframe image 18 as shown in FIG. 9A.
[0095] In this embodiment, the color and the thickness of the lines
of the wireframe image 18 are made to vary as a function of the
relative distance between the front end of the endoscope 3 and the
surface of the target area as determined in a manner as described
above.
[0096] The color and the width of the bar 30 and those of the
numerical value 31 showing the distance to the tumor are also made
to vary along with those of the background.
[0097] For instance, when the relative distance is equal to or
greater than 10 mm, the color of the lines of the wireframe image
18 may be blue and the thickness of the lines may be equal to 1
pixel while both the color of the bar 30 representing the distance
and that of the background of the numerical value 31 may be equally
blue and the width of the bar 30 may equal to 20 pixels.
[0098] When, on the other hand, the relative distance is equal to
or greater than 0 mm and smaller than 10 mm, the color of the lines
of the wireframe image 18 may be yellow and the thickness of the
lines may be equal to 2 pixels while both the color of the bar 30
representing the distance and that of the background of the
numerical value 31 may be equally yellow and the width of the bar
30 may equal to 30 pixels.
[0099] If the front end of the endoscope 3 is inserted by a
distance equal to or greater than 0 mm and smaller than 10 mm, the
color of the lines of the wireframe image 18 may be purple and the
thickness of the lines may be equal to 2 pixel while both the color
of the bar 30 representing the distance and that of the background
of the numerical value 31 may be equally purple and the width of
the bar 30 may equal to 30 pixels.
[0100] In this way, when the front end of the endoscope 3 traveled
by a predetermined distance, both the color of the wireframe image
18 and the thickness of the lines of the wireframe image 18 may be
made to change so that the user can visually recognize the distance
between the surface of the target area and the front end of the
endoscope 3.
[0101] Additionally, the wireframe image 18 of an area requiring
special attention may be drawn with thick lines when the endoscope
3 is located close to the area and separated therefrom by a
distance smaller than a predetermined value so that the user may
visually recognize that the endoscope 3 is too close to the
area.
[0102] For instance, when the reference value of the distance map
12 for the area requiring special attention is less than 10 mm, the
lines of the area requiring special attention of the wireframe
image 18 may be made five times thicker than before.
[0103] The denseness or coarseness of the wireframe image 18 that
is drawn in correspondence to the relative distance between the
endoscope 3 and the surface of the target area is also made to
vary.
[0104] More specifically, a set of more detailed wireframe
three-dimensional model data 10a will be selected as the relative
distance is reduced, whereas a set of more scarce wireframe
three-dimensional model data 10c will be selected as the relative
distance is increased.
[0105] For instance, high resolution wireframe three-dimensional
model data 10a will be used when the distance to the target area is
less than 30 mm and medium resolution wireframe three-dimensional
model data 10b will be used when the distance to the target area is
between 30 mm and 100 mm, whereas low resolution wireframe
three-dimensional model data 10c will be used when the distance to
the target area is greater than 100 mm.
[0106] With this arrangement, the problem of the prior art that
coarse wireframe three-dimensional model data have to be used to
reduce the time until the completion of drawing a wireframe image
in order to save time when the endoscope is approaching the target
area whereas dense wireframe three-dimensional model data are used
to unnecessarily consume time before the completion of drawing a
wireframe image when the endoscope is remote from the target area
is successfully eliminated and a required level of detail and
drawing rate can be realized depending on the distance between the
endoscope and the target area.
[0107] Additionally, when the endoscope 3 is inserted into the
object of examination 1 by using the above described embodiment,
the model image of the object of examination 1 drawn by the
embodiment is switched from the wireframe image 18 to the internal
tomographic image 19 obtained by using the three-dimensional volume
data of the object of examination 1 as shown in FIG. 9B depending
on the relative distance between the endoscope 3 and the outermost
zone of the target area.
[0108] The relative distance between the front end of the endoscope
3 and the surface of the target is determined in a manner as
described above.
[0109] For instance, the model image of the object of examination
may be switched from the wireframe image 18 to an internal
tomographic image 19 obtained by using the three-dimensional volume
data of the object of examination that reflect the viewing
direction of the endoscope.
[0110] As a result of this switching operation, the user can easily
acquire the internal tomographic image 19 that is very important
after the insertion of the endoscope 3 in stead of the wireframe
image 18 of the object of examination that becomes unnecessary
after the insertion of the endoscope 3 without being required to
carrying out the switching operation by him- or herself.
[0111] Thus, the navigation-related information displayed to the
user when the endoscope 3 is inserted into the object of
examination includes the internal tomgraphic image obtained by
using the three-dimensional volume data of the object of
examination 1 and the wireframe image 20 of the area requiring
special attention.
[0112] As the endoscope 3 is brought close to the target and
separated from the latter by a distance smaller than a
predetermined value under this condition, not only the drawing
attributes of the wireframe image 20 but also the color of the
internal tomographic image 19 drawn by using the three-dimensional
volume data are made to change.
[0113] If, on the other hand, the target is not found within the
effective area of measurement of the endoscope 3, arrow 21
indicates the direction in which the target area will be found as
shown in FIG. 9C.
[0114] If the model image is found within the drawable
(displayable) range or not can be determined by checking if the
coordinate of each and every point of the model image as computed
when drawing the model image is found as a point on the monitor to
be used for displaying the image.
[0115] Referring to FIG. 10, the coordinate of the model image is
transformed into the coordinate of the display screen (Step S1) and
it is determined if the coordinate of a transformed point is found
within the displayable range of the display screen (Step S2).
[0116] If the coordinate is found within the displayable range, the
model image is displayed on the screen (Step S3).
[0117] If, on the other hand, the coordinate is not found within
the display range, the coordinate of a representative point of the
model image is transformed into the corresponding coordinate of the
display screen (Step S4) and, at the same time, the coordinate of
the front end of the endoscope 3 is transformed into the
corresponding coordinate of the display screen (Step S5). Then, the
distance and the direction of the line connecting the
representative point of the model image and the front end of the
endoscope 3 are computationally determined (Step S6).
[0118] Thus, the relative distance and the relative direction of
the line connecting the center 22 of the endoscope image, or the
front end of the endoscope 3, and the target can be determined by
transforming the coordinate values 23 on the model data coordinate
system of the representative point of the target into the
coordinate values on the liquid crystal monitor 13 by means of the
above described coordinate transformation matrices 14, 15, 16 and
17.
[0119] Then, the user can visually comprehend the extent to which
the endoscope 3 should be moved to bring the target into the
effective area of measurement of the endoscope 3 by modifying the
size of the arrow 21 indicating the target area in proportion to
the obtained distance.
[0120] When the apparatus is incapable of measuring the distance
and the direction, the sensor control section 6 outputs a message
telling the user that the apparatus is incapable of measuring the
distance and the direction in place of three-dimensional position
and orientation information.
[0121] Upon receiving this message, the navigation-related
information control section 8 erases the model image being
displayed as navigation-related information and generates character
information 28 of "unmeasurable condition" and a yellow pixel frame
29 having a width equal to 60 pixels, which are then displayed to
the user on the liquid crystal monitor 13 as shown in FIG. 9D.
[0122] Then, the user can easily comprehend that the endoscope 3 or
the object of examination 1 located at a position that makes the
intended measurement impossible so that the user can be effectively
protected against he risk of operating the endoscope 3 in a wrong
way according to navigation-related information that does not
reflect the reality.
[0123] It may be needless to say that the configuration of this
embodiment can be modified and/or altered in various different
ways.
[0124] For instance, the area that provides the object of
navigation is not limited to the target area and may alternatively
be a plurality of any areas which are defined by information on the
profile of the object of examination 1 or information on an
internal tomographic image.
[0125] It is also possible to carry out a simulation by fitting a
sensing plate to the head of a virtual object of examination
without using an actual object of examination 1.
[0126] The three-dimensional position and orientation measuring
section may be made to alternatively comprises a magnetic sensor or
a set of mechanical links and joints, encoders and potentiometers
popularly used in ordinary three-dimensional position and
orientation measuring systems.
[0127] When the object of examination is immobile, it is sufficient
to measure the three-dimensional position and orientation of the
object of examination 1, store the information in the sensor
information storage section and utilize it in the computational
operation for determining the relative three-dimensional position
and orientation of the object of examination and the object of
navigation in advance so that it is only necessary to measure the
three-dimensional position and orientation of the object of
navigation when the system is in operation.
[0128] The wireframe for expressing the profile information of the
target area may be replaced by any known technique for graphic
expression that is popularly used for three-dimensional computer
graphics including polygons.
[0129] Alternatively, contour lines and equidistant curves relative
to the viewing direction may be used.
[0130] The endoscope 3 that is the object of navigation may be
replaced by a plurality of endoscopes.
[0131] The object of navigation may be a surgical instrument that
is not provided with a section of observation.
[0132] The technique used for determining if the target area is
located within the effective area of measurement or not is limited
to the above described one.
[0133] The technique for determining the relative distance between
the object of navigation and the target area is not limited to the
above described one that uses a distance map and any known
technique for computationally determining the distance between two
points in a three-dimensional space may alternatively be used for
determining the distance between a representative point of the
object of navigation and a representative point of the target area
for the purpose of the invention.
[0134] Additionally, the color may be made to change continuously
as a function of the distance in stead of the above described use
of a single boundary value. Alternatively, the color may be made to
change stepwise by providing a plurality of boundary values.
[0135] Similarly, the line thickness may be made to change
continuously in stead of the above described use of a single
boundary value. Alternatively, the line thickness may be made to
change stepwise by providing a plurality of boundary values.
[0136] A situation where there is no navigation-related information
to be displayed may be indicated by making the color to be
transparent and the lines to be practically invisible.
[0137] The density of lines for drawing the model image that varies
as a function of the distance may be made to change continuously on
the basis of a single set of data in stead of selectively using a
plurality of sets of data with different levels of density that are
provided in advance as described above.
[0138] The pattern that is displayed when the target area is out of
the effective area of measurement is not limited to the arrow 21.
Alternatively, a triangle, a circle, a bar or some other figure may
be used. The distance may be expressed by the size of the
figure.
[0139] Furthermore, the size of the arrow 21 may be made to vary
stepwise by using a plurality of preselected values in stead of
making it vary continuously in a manner as described above.
[0140] When a section for determining the density of lines for
drawing the model image of the target area on the basis of a single
set of data is provided, it is no longer necessary to store in
advance a plurality of sets of data with different levels of
density.
[0141] The navigation-related information indicating a situation
where the apparatus is incapable of measuring the distance and the
direction may not require both character information 28 and a frame
29. It may be sufficient to use only either character information
28 or a frame 29 to convey the information.
[0142] It may be so arranged that the user can define the color and
the line thickness that are used as attributes of the
navigation-related information, the density of lines for drawing
the model image, the size of the displayed pattern, the boundary
values for changing the color and the line thickness as a function
of the distance and the character string of the characteristic
information 28 indicating that the incapability of measurement of
the apparatus.
[0143] [Embodiment 2]
[0144] Now, the second embodiment of the invention, which is a
navigation apparatus, will be discussed below.
[0145] FIG. 11 is a schematic illustration of the second embodiment
of the invention which is also a navigation apparatus, showing its
configuration.
[0146] This second embodiment has a configuration same as the above
described first embodiment except the following.
[0147] In this embodiment, the endoscope 3 is not required to pass
the imaging information obtained by the optical system to the
navigation-related information control section 8.
[0148] In this embodiment, the navigation-related information
storage section 9 stores in advance vectors 24 for expressing the
route along which the endoscope 3 is inserted (minimal invasive
route) as data.
[0149] Then, the coordinate values 25 for the front end and the
rear end of the endoscope 3 are determined in terms of the
coordinate system defined by the sensing plate 4 rigidly fitted to
the endoscope 3 and stored in the navigation-related information
storage section 9.
[0150] Now, the operation of the embodiment having the above
described configuration will be described below.
[0151] When the embodiment of navigation apparatus is in operation,
the sensor control section 6 measures the three-dimensional
position of each of the LEDs that are emitting infrared rays of the
sensing plates 2 and 4 and then computationally determines the
three-dimensional position and orientation of each of the sensing
plates 2 and 4, by using the LED definition data stored in the
sensor information storage section 5.
[0152] Then, the coordinate transformation matrix 17 from the
sensing plate 2 fitted to the head of the object of examination 1
to the sensing plate 4 fitted to the endoscope 3 is computationally
determined on the basis of the obtained three-dimensional position
and orientation information.
[0153] Then, the position and the orientation of the endoscope 3 is
determined in terms of the data on the target area by using the
coordinate transformation matrix 17 and the above described
coordinate transformation matrices 14, 15.
[0154] Then, the navigation-related information control section 8
generates a tri-sectional image 26 of the three-dimensional volume
data 11 including those of the tumor and an orthogonal projection
image 27 of the endoscope 3 projected on the cross section as
navigation-related information and displays it on the liquid
crystal monitor 13.
[0155] If, for instance, the coordinate of a representative point
of the tumor is expressed by (260, 180, 280), the tri-sectional
image 26 of the three-dimensional volume data 11 will have a YZ
plane with x=260, a ZX plane with y=180 and an XY plane with
z=280.
[0156] Then, the relative distance between the surface of the
target area and the front end of the endoscope 3 is determined by
referring to the above described distance map 12, using the
position of the front end of the endoscope 3.
[0157] The thickness of the lines of the orthogonal projection
image 27 of the endoscope 3 is continuously changed as a function
of the distance between the surface of the target area and the
front end of the endoscope 3.
[0158] Then, the user can easily comprehend a situation where the
endoscope 3 is approaching the target area.
[0159] The orientation of the endoscope 3 determined by the
coordinate values 25 of the position of the front end and that of
the rear end of the endoscope 3 is compared with the data of the
vector indicating the direction in which the endoscope 3 is to be
inserted and, if the orientation is inclined relative to the vector
by a predetermined value (e.g., 10 degrees), the color and the line
thickness of the orthogonal projection image 27 of the endoscope 3
will be changed.
[0160] Thus, the user can easily comprehend that the direction in
which the endoscope 3 is currently inserted is deviating from the
direction in which it is to be inserted.
[0161] It may be needless to say that the configuration of this
embodiment can be modified and/or altered in various different
ways.
[0162] For instance, the area that provides the object of
navigation is not limited to the target area and may alternatively
be a plurality of any areas which are defined by information on the
profile of the object of examination 1 or information on an
internal tomographic image.
[0163] It is also possible to carry out a simulation by fitting a
sensing plate to the head of a virtual object of examination
without using an actual object of examination 1.
[0164] The three-dimensional position and orientation measuring
section may be made to alternatively comprises a magnetic sensor or
a set of mechanical links and joints, an encoder and a
potentiometer popularly used in ordinary three-dimensional position
and orientation measuring systems.
[0165] When the object of examination is immovable, it is
sufficient to measure the three-dimensional position and
orientation of the object of examination 3, store the information
in the sensor information storage section and utilize it in the
computational operation for determining the relative
three-dimensional position and orientation of the object of
examination and the object of navigation in advance so that it is
only necessary to measure the three-dimensional position and
orientation of the object of navigation when the system is in
operation.
[0166] The endoscope 3 that is the object of navigation may be
replaced by a plurality of endoscopes.
[0167] While the above object of navigation may normally refer to
an endoscope 3, it may alternatively refer to some other surgical
instrument such as a suction pipe or a pair of forceps so long as
the mechanical profile thereof can be determined by
measurement.
[0168] The coordinate of the front end of the endoscope 3 does not
need to agree with the actual front end and data may be manipulated
to make the endoscope 3 virtually have an extended front end.
[0169] Alternatively, the coordinate of the front end of the
endoscope 3 may be defined by means of an operational formula using
the extension of the front end from the actual front end as
parameter so that the coordinate of the front end may be determined
successively on the basis of the extension specified by the
user.
[0170] Additionally, the color may be made to change on the basis
of a single boundary value instead of making it change continuously
as a function of the distance in a manner as described above.
Alternatively, the color may be made to change stepwise by
providing a plurality of boundary values.
[0171] Similarly, the line thickness may be made to change on the
basis of a single boundary value in stead of making it change
continuously in a manner as described above. Alternatively, the
line thickness may be made to change stepwise by providing a
plurality of boundary values.
[0172] The technique of determining the angle of inclination of the
endoscope 3 is not limited to the one described above.
[0173] It may be so arranged that the user can define the color and
the line thickness that are used as attributes of the
navigation-related information, the density of lines for drawing
the model image, the size of the displayed pattern, the boundary
values for changing the color and the line thickness as a function
of the distance and the character string of the characteristic
information 28 indicating that the incapability of measurement of
the apparatus.
[0174] The object of navigation may be a microscope.
[0175] Then, position of the focal point can be defined as object
of navigation by obtaining the focal length of the microscope from
the microscope main body and replacing the coordinate of the front
end of the endoscope 3 by that of the focal point.
[0176] While a navigation apparatus according to the invention is
described above in terms of the first and second embodiments, the
present invention is by no means limited thereto and the
embodiments can be modified and/or alterered in various different
ways without departing from the scope of the present invention.
[0177] With a navigation apparatus as set forth in claim 2 of the
appended claims, a model image of the object or the target,
information on the direction of navigation and/or information on
the distance between the object of navigation and the target are
displayed whenever the position and orientation of the object in a
three-dimensional space can be determined so that the user can
easily comprehend the position and orientation of the object of
navigation in the three-dimensional space.
[0178] Additionally, when the apparatus is incapable of measuring
the position and orientation, it displays so and, therefore, the
user can easily be aware of the situation.
[0179] A navigation apparatus according to claim 2 covers both the
above described first and second embodiments.
[0180] More specifically, while the above target may normally be a
patient, a tumor to be surgically treated of a patient or an area
of the body of a patient requiring special attention during a
surgical operation, it is by no means limited to an existing object
of examination and may alternatively be a virtual target displayed
as a two-dimensional or three-dimensional image of a model
synthesized by using the video information of an existing target
that is obtained in advance.
[0181] While the above object may normally refer to an endoscope 3,
it may alternatively refer to some other surgical instrument such
as a suction pipe or a pair of forceps.
[0182] While the above display section may normally refer to a
liquid crystal monitor, it may alternatively refer to some other
video information display such as a CRT display or a head mount
display.
[0183] While the above model image refers to wireframe model data
10 in the first embodiment, it may alternatively refer model data
adapted to express a profile, including a popular data structure to
be used for three-dimensional computer graphics.
[0184] Additionally, while the above model image refers to the
three-dimensional volume data 11 of the target area in the first
and second embodiments, it may alternatively take a form where a
plurality of two-dimensional pixel data exist.
[0185] While the above described information on the direction of
navigation refers to the arrow 21 in the first embodiment, it may
alternatively be a two-dimensional geometric figure such as a
triangle or a circle, a three-dimensional geometric figure such a
cone or a set of visually recognizable image data.
[0186] While the above described distance information refers to the
numeral 31 indicating the distance to the tumor in the first
embodiment, it may alternatively be a numeral indicating the
distance to an appropriate target.
[0187] While it also refers to the bar 30 indicating the distance
to the tumor in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0188] While the information indicating an unmeasurable condition
refers to the character information 28 of "unmeasurable condition"
in the first embodiment, it may alternatively include any character
information telling the user that the apparatus is in an
unmeasurable condition.
[0189] While it also refers to a yellow pixel frame 29 having a
width equal to 60 pixels in the first embodiment, it may
alternatively refer to any expression using symbols defined to
indicate an unmeasurable condition.
[0190] With a navigation apparatus as set forth in claim 3 of the
appended claims, an image of an object acquired by the imaging
section is displayed with other information on the object in an
overlaid manner so that the user can obtain an actual image of the
object and navigation-related information simultaneously and hence
comprehend the position, the profile and the condition of the
object that he or she cannot see on the basis of the
navigation-related information.
[0191] A navigation apparatus according to claim 3 covers both the
above described first and second embodiments.
[0192] While the object refers to the endoscope 3 in the first
embodiment, it may alternatively refer to a microscope or some
other object.
[0193] While the above display section normally refers to the
liquid crystal monitor in the first embodiment, it may
alternatively refer to some other video information display such as
a CRT display or a head mount display.
[0194] With a navigation apparatus as set forth in claim 3 of the
appended claims, the relative position and orientation of the
target and those of the object in a three-dimensional space are
measured by means of a three-dimensional position and orientation
measuring section.
[0195] Then, the information generating section of the navigation
apparatus generates information necessary for navigating the object
on the basis of the outcome of the measurement of the
three-dimensional position and orientation measuring section.
[0196] Then, the display section of the navigation apparatus
displays navigation-related information in a display mode selected
out of a plurality of different display modes according to at least
any of distance information on the distance between the object and
the target, direction information on the direction of the target as
viewed from the object or information telling if the object or the
target is found within the effective area of measurement of the
three-dimensional position and orientation measuring section or
not.
[0197] As a result, the user can easily comprehend the distance
between the target and the object, the direction of the target as
viewed from the object and if the object or the target is found
within the effective area of measurement of the three-dimensional
position and orientation measuring section or not.
[0198] A navigation apparatus according to claim 4 covers both the
above described first and second embodiments.
[0199] Thus, while the target is a patient 1, a tumor to be
surgically treated of a patient or an area of the body of a patient
requiring special attention during a surgical operation in the
above first and second embodiments, it is by no means limited to an
existing object of examination and may alternatively be a virtual
target displayed as a two-dimensional or three-dimensional image of
a model synthesized by using the video information of an existing
target that is obtained in advance.
[0200] While the above object refers to an endoscope 3, it may
alternatively refer to some other surgical instrument such as a
suction pipe or a pair of forceps.
[0201] While the above three-dimensional position and orientation
measuring section refers to sensors using LEDs for emitting
infrared rays (sensing plates 2, 4, sensor assembly 7, sensor
information storage section 5 and sensor control section 6), it may
alternatively refer to a sensing system using magnetic sensors or a
sensing system using a set of mechanical links and joints, an
encoder and a potentiometer popularly used in ordinary
three-dimensional position and orientation measuring systems.
[0202] The information generating section refers to the
navigation-related information storage section 9 and the
navigation-related information control section 8.
[0203] While the display section normally refers to the liquid
crystal monitor 13, it may alternatively refer to some other video
information display such as a CRT display or a head mount
display.
[0204] For the purpose of the invention, the expression of "a
plurality of different display modes" refers to differences in
color, in the thickness of line, in the dimensions of the drawing
and in the density of drawing lines.
[0205] With a navigation apparatus as set forth in claim 5 of the
appended claims, a display section as set forth in claim 4 displays
at least profile information on the target or the object, internal
tomographic information on the object, information on the direction
of the object as viewed from the object or vice versa or
information on the distance to the object when the target or the
object is measurable by the three-dimensional position and
orientation measuring section but it displays information telling
that neither the target nor the object can be measured.
[0206] A navigation apparatus according to claim 5 covers both the
above described first and second embodiments.
[0207] More specifically, while the above profile information
refers to wireframe model data 10 in the first embodiment, it may
also refer to model data adapted to express a profile, including a
popular data structure to be used for three-dimensional computer
graphics.
[0208] Additionally, while it refers to the lines drawing the
orthogonal projection image 27 of the endoscope 3 in the second
embodiment, it may also refer to expression techniques adapted to
express a profile, including a popular data structure to be used
for three-dimensional computer graphics.
[0209] Still additionally, while the internal tomographic
information refers to the three-dimensional volume data 11 of the
target area in the first and second embodiments, it may
alternatively take a form where a plurality of two-dimensional
pixel data exist.
[0210] While the above described direction of the target refers to
the arrow 21 in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0211] While the above described distance information refers to the
numeral 31 indicating the distance to the tumor in the first
embodiment, it may alternatively be a numeral indicating the
distance to an appropriate target.
[0212] While it also refers to the bar 30 indicating the distance
to the tumor in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0213] While the information indicating an unmeasurable condition
refers to the character information 28 of "unmeasurable condition"
in the first embodiment, it may alternatively include any character
information telling the user that the apparatus is in an
unmeasurable condition.
[0214] While it also refers to a yellow pixel frame 29 having a
width equal to 60 pixels in the first embodiment, it may
alternatively refer to any expression using symbols defined to
indicate an unmeasurable condition.
[0215] With a navigation apparatus as set forth in claim 6 of the
appended claims, the object has an imaging section and the image
acquired by the imaging section is displayed with other
navigation-related information obtained by the information
generating section in an overlaid manner.
[0216] Thus, the user can obtain an actual image of the object and
navigation-related information simultaneously and hence comprehend
the position, the profile and the condition of the object that he
or she cannot see on the basis of the navigation-related
information.
[0217] A navigation apparatus according to claim 6 covers the above
described first embodiment.
[0218] While the object having an imaging section refers to the
endoscope 3 in the first embodiment, it may alternatively refer to
a microscope or some other object.
[0219] While the above display section normally refers to the
liquid crystal monitor 13 in the first embodiment, it may
alternatively refer to some other video information display such as
a CRT display or a head mount display.
[0220] With a navigation apparatus as set forth in claim 7 of the
appended claims, the navigation-related information displayed on
the display section changes its color as a function of the relative
distance between the target and the object as measured by the
three-dimensional position and orientation measuring section so
that the user can visually comprehend with ease a situation where
the relative distance is made too small.
[0221] Alternatively, it may be so arranged that both the relative
distance and the direction toward the target as viewed from the
object are evaluated at the same time and the color of the
displayed information is changed depending on the situation. Then,
the user can visually comprehend both the relative distance and the
direction with ease.
[0222] A navigation apparatus according to claim 7 covers both the
above described first and second embodiments.
[0223] More specifically, while the color of the displayed
navigation-related information refers to that of the wireframe
image 18 of the target area and the internal tomographic image 19
displayed on the monitor in the first embodiment, it may also refer
to the color of the arrow 21 of the first embodiment.
[0224] It may additionally refers to the color of the tri-sectional
image 26 of the target area and the orthogonal projection image 27
of the endoscope 3 in the second embodiment.
[0225] With a navigation apparatus as set forth in claim 8 of the
appended claims, the thickness of the lines of the
navigation-related information displayed on the display section
changes as a function of the relative distance between the target
and the object as measured by the three-dimensional position and
orientation measuring section so that the user can visually
comprehend with ease a situation where the relative distance is
made too small.
[0226] Additionally, as the thickness of the lines of the
navigation-related information displayed on the display section
changes as a function of the direction of the target as viewed from
the object, the user can visually comprehend with ease a situation
where the relative direction is deviating from the right
direction.
[0227] Alternatively, it may be so arranged that both the relative
distance and the direction toward the target as viewed from the
object are evaluated at the same time and the line thickness of the
displayed information is changed depending on the situation. Then,
the user can visually comprehend both the relative distance and the
direction with ease.
[0228] A navigation apparatus according to claim 8 covers both the
above described first and second embodiments.
[0229] More specifically, while the line thickness of the displayed
navigation-related information refers to that of the wireframe
image 18 of the target area and the internal tomographic image 19
displayed on the monitor in the first embodiment, it may also refer
to the line thickness of the arrow 21 of the first embodiment.
[0230] It may additionally refers to line thickness of the
tri-sectional image 26 of the target area and the orthogonal
projection image 27 of the endoscope 3 in the second
embodiment.
[0231] With a navigation apparatus as set forth in claim 9 of the
appended claims, the profile model of the target and the internal
tomographic image are switched from one to the other on the display
section as a function of the relative distance between the target
and the object as measured by the three-dimensional position and
orientation measuring section so that the user can visually
comprehend with ease that the object is located close to the
target.
[0232] Thus, the user can get necessary information with ease
depending on if the distance between the object and the target is
smaller than a predetermined value or not.
[0233] A navigation apparatus according to claim 9 covers the above
described first embodiment.
[0234] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction. The above internal tomographic
image refers to the internal tomographic image 19 of the first
embodiment.
[0235] With a navigation apparatus as set forth in claim 10 of the
appended claims, the density of lines drawing the target model
image is finely lowered when the relative distance between the
target and the object is large and finely raised when the relative
distance is small so as to make the load of drawing the target
image and the quantity of information used for displaying the image
may be well balanced.
[0236] As a result, the user can obtain an adequate amount of
information that is displayed with an adequate drawing rate
depending as a function of the relative distance between the target
and the object.
[0237] A navigation apparatus according to claim 10 covers the
above described first embodiment.
[0238] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction.
[0239] With a navigation apparatus as set forth in claim 11 of the
appended claims, the information generating section computationally
determines the positional relationship of the image to be displayed
and the display area of the display section on the basis of the
relative distance between the target and the object and the
relative direction of the target as viewed from the object and
simply indicates the direction of the target when no image is
displayed in the display area for the target so that the user can
comprehend the relative positions of the target and the object and
the direction of the target as viewed from the object without
missing either of them by selecting the display of the direction
when no image is display for the target.
[0240] A navigation apparatus according to claim 11 covers the
above described first embodiment.
[0241] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction along with information on the
internal tomographic image.
[0242] While the above described information on the direction of
navigation refers to the arrow 21 in the first embodiment, it may
alternatively be a two-dimensional geometric figure such as a
triangle or a circle, a three-dimensional geometric figure such a
cone or a set of visually recognizable image data.
[0243] With a navigation apparatus as set forth in claim 12 of the
appended claims, the relative distance between the target and the
object is indicated by the size or the shape of a symbol so that
the user can visually comprehend with ease not only the distance
but also the direction of the target as viewed from the object.
[0244] A navigation apparatus according to claim 12 covers the
above described first embodiment.
[0245] More specifically, while the above described symbol refers
to the arrow 21 in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0246] With a navigation apparatus as set forth in claim 13 of the
appended claims, the relative distance between the target and the
object of navigation and their orientations in a three-dimensional
space are determined by the three-dimensional position and
orientation measuring section.
[0247] Then, an computational information determining section
generates navigation-related information such as information
three-dimensional position and orientation information on the
target and the object of navigation including the relative distance
between the target and the object of navigation and their
orientations and if they are measurable or not and controls the
generated information.
[0248] Then, the information display section displays the
navigation-related information generated by the computational
information determining section.
[0249] As a result, the user can easily comprehend the positional
relationship between the target and the object of navigation
including their orientations and if they are measurable or not.
[0250] A navigation apparatus according to claim 13 covers both the
first and second embodiments.
[0251] More specifically, while the above target may normally be a
patient, a tumor to be surgically treated of a patient or an area
of the body of a patient requiring special attention during a
surgical operation, it is by no means limited to an existing object
of examination and may alternatively be a virtual target displayed
as a two-dimensional or three-dimensional image of a model
synthesized by using the video information of an existing target
that is obtained in advance.
[0252] While the above object may normally refer to an endoscope 3,
it may alternatively refer to some other surgical instrument such
as a suction pipe or a pair of forceps.
[0253] While the above three-dimensional position and orientation
measuring section refers to sensors using LEDs for emitting
infrared rays (sensing plates 2, 4, sensor assembly 7, sensor
information storage section 5 and sensor control section 6), it may
alternatively refer to a sensing system using magnetic sensors or a
sensing system using a set of mechanical links and joints, an
encoder and a potentiometer popularly used in ordinary
three-dimensional position and orientation measuring systems.
[0254] The computational information determining section refers to
the navigation-related information storage section 9 and the
navigation-related information control section 8.
[0255] While the information display section refers to the liquid
crystal monitor 13, it may alternatively refer to some other video
information display such as a CRT display or a head mount
display.
[0256] The expression "attributes of navigation-related
information" as used herein refers to the color, the thickness of
line, the dimensions the drawing and the density of drawing
lines.
[0257] With a navigation apparatus as set forth in claim 14 of the
appended claims, the navigation-related information includes a
model image of the profile of the target and/or that of the object
of navigation, a model image of the internal tomographic
information of the target, a symbol pattern indicating the
direction in which the target and/or the object of navigation will
be found and/or a numerical value o a symbol pattern indicating the
distance between the target and the object of navigation when the
three-dimensional position and orientation measuring section is
operating normally.
[0258] When the three-dimensional position and orientation
measuring section is inoperative, the navigation-related
information refers to character information or a symbol pattern
indicating that the three-dimensional position and orientation
measuring section is in operative.
[0259] A navigation apparatus according to claim 14 covers both the
first and second embodiments.
[0260] More specifically, while the above model image of the
profile refers to wireframe model data 10 in the first embodiment,
it may also refer to model data adapted to express a profile,
including a popular data structure to be used for three-dimensional
computer graphics.
[0261] Additionally, while it refers to the lines drawing the
orthogonal projection image 27 of the endoscope 3 in the second
embodiment, it may also refer to expression techniques adapted to
express a profile, including a popular data structure to be used
for three-dimensional computer graphics.
[0262] Still additionally, while the model image of the internal
tomographic information refers to the three-dimensional volume data
11 of the target area in the first and second embodiments, it may
alternatively take a form where a plurality of two-dimensional
pixel data exist.
[0263] While the above described symbol pattern indicating the
direction in which the object of navigation will be found refers to
the arrow 21 in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0264] While the above described numerical value indicating the
distance to the target refers to the numeral 31 indicating the
distance to the tumor in the first embodiment, it may alternatively
be a numeral indicating the distance to an appropriate target.
[0265] While it also refers to the bar 30 indicating the distance
to the tumor in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0266] While the character information indicating an unmeasurable
condition refers to the character information 28 of "unmeasurable
condition" in the first embodiment, it may alternatively include
any character information telling the user that the apparatus is in
an unmeasurable condition.
[0267] While it also refers to a yellow pixel frame 29 having a
width equal to 60 pixels in the first embodiment, it may
alternatively refer to any expression using symbols defined to
indicate an unmeasurable condition.
[0268] With a navigation apparatus as set forth in claim 15 of the
appended claims, the object of navigation has an observational
function and the observed image obtained by means of the
observational function is displayed with other navigation-related
information obtained by the computational information determining
section in an overlaid manner.
[0269] Thus, the user can obtain an actual image of the object and
navigation-related information simultaneously and hence comprehend
the position, the profile and the condition of the object that he
or she cannot see on the basis of the navigation-related
information.
[0270] A navigation apparatus according to claim 15 covers the
above described first embodiment.
[0271] While the object of navigation having an observational
function refers to the endoscope 3 in the first embodiment, it may
alternatively refer to a microscope or some other object.
[0272] While the above information display section normally refers
to the liquid crystal monitor 13 in the first embodiment, it may
alternatively refer to some other video information display such as
a CRT display or a head mount display.
[0273] With a navigation apparatus as set forth in claim 16 of the
appended claims, the navigation-related information displayed on
the display section changes its color as a function of the relative
distance between the target and the object as measured by the
three-dimensional position and orientation measuring section so
that the user can visually comprehend with ease a situation where
the relative distance is made too small.
[0274] Alternatively, it may be so arranged that the
navigation-related information displayed on the display section
changes its color as a function of the relative direction of the
target and the object of navigation so that the user also can
visually comprehend a situation where the relative direction is
deviated from the right direction.
[0275] Still alternatively, the color may be made to change
simultaneously as a function of both the relative distance and the
relative direction. Then, the user can visually comprehend both the
relative distance and the direction with ease.
[0276] A navigation apparatus according to claim 16 covers both the
above described first and second embodiments.
[0277] More specifically, while the color of the displayed
navigation-related information refers to that of the wireframe
image 18 of the target area and the internal tomographic image 19
displayed on the monitor in the first embodiment, it may also refer
to the color of the arrow 21 of the first embodiment.
[0278] It may additionally refers to the color of the tri-sectional
image 26 of the target area and the orthogonal projection image 27
of the endoscope 3 in the second embodiment.
[0279] With a navigation apparatus as set forth in claim 17 of the
appended claims, the thickness of the lines of the
navigation-related information obtained by the three-dimensional
position and orientation measuring section is made to vary as a
function of the relative distance between the target and the object
of navigation so that the user can visually comprehend with ease a
situation where the relative distance has become too small.
[0280] Alternatively, it may be so arranged that the thickness of
the lines of the navigation-related information vary as a function
of the direction to the target as viewed from the object of
navigation so that the user can also visually comprehend with ease
a situation where the relative direction has deviated.
[0281] A navigation according to claim 17 covers both the first and
second embodiments.
[0282] While the thickness of the lines of navigation-related
information refers to the wireframe image 18 of the target area
drawn on the monitor in the first embodiment, it may also include
the arrow 21 in the first embodiment.
[0283] It refers to the orthogonal projection image 27 of the
endoscope 3 drawn on the monitor in the second embodiment.
[0284] With a navigation apparatus as set forth in claim 18 of the
appended claims, the profile model of the target and the internal
tomographic image are switched from one to the other on the display
section as a function of the relative distance between the target
and the object as measured by the three-dimensional position and
orientation measuring section so that the user can visually
comprehend with ease that the object is located close to the
target.
[0285] Thus, the user can get necessary information with ease
depending on if the distance between the object and the target is
smaller than a predetermined value or not.
[0286] A navigation apparatus according to claim 18 covers the
above described first embodiment.
[0287] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction.
[0288] The above internal tomographic image refers to the internal
tomographic image 19 of the first embodiment.
[0289] With a navigation apparatus as set forth in claim 19 of the
appended claims, the density of lines drawing the target model
image is finely lowered when the relative distance between the
target and the object is large and finely raised when the relative
distance is small so as to make the load of drawing the target
image and the quantity of information used for displaying the image
may be well balanced.
[0290] As a result, the user can obtain an adequate amount of
information that is displayed with an adequate drawing rate
depending as a function of the relative distance between the target
and the object.
[0291] A navigation apparatus according to claim 19 covers the
above described first embodiment.
[0292] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction.
[0293] With a navigation apparatus as set forth in claim 20 of the
appended claims, the computational information determining section
computationally determines the positional relationship of the image
to be displayed and the display area of the display section on the
basis of the relative distance between the target and the object of
navigation and the relative direction of the target as viewed from
the object and only a symbol pattern is displayed when no model
image is displayed in the display area so that the user can
comprehend the relative positions of the target and the object and
the direction of the target as viewed from the object without
missing either of them by selecting the display of the direction
when no image is display for the target.
[0294] A navigation apparatus according to claim 20 covers the
above described first embodiment.
[0295] More specifically, while the above profile model refers to
wireframe image 18 in the first embodiment, it may also refer to
various profiles used in three-dimensional computer graphics such
as polygon as well as contour lines and equidistant curves drawn
relative to the viewing direction along with information on the
internal tomographic image.
[0296] While the above described symbol pattern refers to the arrow
21 in the first embodiment, it may alternatively be a
two-dimensional geometric figure such as a triangle or a circle, a
three-dimensional geometric figure such a cone or a set of visually
recognizable image data.
[0297] With a navigation apparatus as set forth in claim 21 of the
appended claims, the relative distance between the target and the
object is indicated by the size of a pattern so that the user can
visually comprehend with ease not only the distance but also the
direction of the target as viewed from the object.
[0298] A navigation apparatus according to claim 21 covers the
above described first embodiment.
[0299] More specifically, while the above described symbol pattern
refers to the arrow 21 in the first embodiment, it may
alternatively be a two-dimensional geometric figure such as a
triangle or a circle, a three-dimensional geometric figure such a
cone or a set of visually recognizable image data.
[0300] As described above, both the above described first and
second embodiments of navigation apparatus according to the
invention are adapted to modify the navigation-related information
displayed on the display section as a function of the relative
three-dimensional positions and orientations of the target and the
object of navigation to make the user easily comprehend the
distance between the target and the object and obtain
navigation-related information of necessary type with ease.
[0301] Now, the third and fourth embodiments of the invention will
be described. They are surgical operation image acquisition/display
apparatus realized by applying a navigation apparatus according to
the invention.
[0302] (Embodiment 3)
[0303] FIG. 14 is a schematic block diagram of the third embodiment
of the invention which is a surgical operation image
acquisition/display apparatus, showing its configuration; The third
embodiment of surgical operation image acquisition/display
apparatus according to the invention has a configuration as
described below.
[0304] Referring to FIG. 14, the surgical operation image
acquisition/display apparatus comprises a surgical microscope 141
(first observation section) and an endoscope 142 (second
observation section) for observing an area located in the dead
angle of the surgical microscope 141.
[0305] The surgical microscope 141 includes an microscope optical
system 141a mounted on a stand (not shown), a microscope camera
141b attached to the microscope optical system 141a and a
microscope camera control unit (hereinafter referred to as
microscope CCU) 141c for converting the output of the microscope
camera 141b into a video signal.
[0306] The microscope optical system 141a is provided with
illumination light emitted from a light source (not shown) and
guided by a light guide (not shown) for the purpose of
observation.
[0307] On the other hand, the endoscope 142 includes an endoscope
optical system 142a, an endoscope camera 142b attached to the
endoscope optical system 142a and an endoscope camera control unit
(hereinafter referred to as endoscope CCU) 142 for converting the
output of the endoscope camera into a video signal.
[0308] The endoscope optical system 142a is provided with
illumination light emitted from a light source (not shown) and
guided by a light guide (not shown) for the purpose of
observation.
[0309] The video output of the microscope CCU 141c and that of the
endoscope CCU 142c are fed to a video mixer 143 (observed image
synthesizing section).
[0310] As illustrated in FIGS. 15A through 15F, the video mixer 143
has a plurality of display modes 0 through 5.
[0311] In the display mode 0 (FIG. 15A), the video output of the
microscope CCU 141c is displayed without being processed.
[0312] In the display mode 1 (FIG. 15B), the video output of the
endoscope CCU 142c is displayed without being processed.
[0313] In the display mode 2 (FIG. 15C), the video output of the
endoscope CCU 142c is dimensionally reduced and displayed in the
video output of the microscope CCU 141c.
[0314] In the display mode 3 (FIG. 15D), the video output of the
microscope CCU 141c is dimensionally reduced and displayed in the
video output of the endoscope CCU 142c.
[0315] The extent of dimensional reduction and the display position
of the dimensionally reduced output are variable both in the
display mode 2 and the display mode 3.
[0316] In the display mode 4 (FIG. 15E), which is a variation of
the display mode 2 and in which the video output of the endoscope
CCU 142c is dimensionally reduced and displayed at the right top
corner of the video output of the microscope CCU 141c, the video
output of the endoscope CCU 142c is dimensionally reduced to the
same extent and displayed at the left bottom corner of the video
output of the microscope CCU 141c.
[0317] Similarly, in the display mode 5 (FIG. 15F), which is a
variation of the display mode 3 and in which the video output of
the microscope CCU 141c is dimensionally reduced and displayed at
the right top corner of the video output of the endoscope CCU 142c,
the video output of the microscope CCU 141c is dimensionally
reduced to the extent smaller than that of FIG. 15D and displayed
at the right top corner of the video output of the endoscope CCU
142c.
[0318] The mode of display and the position and the size of the
displayed image (or each of the displayed images) as defined by the
video mixer 143 can be appropriately changed by means of an
externally applied control signal. More specifically, the displayed
image(s) can be enlarged or reduced independently with an
appropriately selected magnification factor.
[0319] Then, the output of the video mixer 143 is fed to a liquid
crystal display 144 (display section) and displayed to the surgical
operator for observation.
[0320] The combination of the video mixer 143 and the liquid
crystal display 144 corresponds to the image display section as
used in the appended claims.
[0321] Referring to FIG. 14, position and orientation sensor 145
(position and orientation detection section) comprises hard sensing
plates 145b and 145c, each having three infrared light emitting
diodes (LEDs) 145a for emitting arranged at the respective corners
of a triangle, a sensor assembly 145d for detecting the quantity of
light emitted from each of the infrared LEDs 145a for emitting and
a sensor controller 145e for computationally determining the
three-dimensional position and orientation of the sensing plate
145b and that of the sensing plate 145c from the output of the
sensor assembly.
[0322] Note that the position of each of the LEDs 145a for emitting
infrared is observed and determined in advance in terms of the
coordinate system defined on each of the sensing plates 145b and
145c and stored in the sensor controller 145e as LED definition
data.
[0323] Assume that the sensing plate 145b is attached to the head
of the patient 146 in such a way that its position and orientation
relative to the head would not change easily.
[0324] The other sensing plate 145c is attached to the endoscope
143 by a mount section (not shown).
[0325] The sensor controller 145e is connected to an image
controller 147 (state of synthesis modification specifying
section).
[0326] The image controller 147 is connected to the video mixer
143.
[0327] Assume that the area of operation to be surgically treated
and certain characteristic points on the patient are observed by
means of CT or MRI and the three-dimensional positions thereof are
computationally determined by an image processing computer (not
shown) and stored in the image controller 147 as data on the area
and the characteristic points.
[0328] At this time, as shown in FIG. 16, the data on the area of
surgical operation and the patient 146 are correlated by observing
the coordinate values of the characteristic points in the model
data coordinate system m and those of the characteristic points on
the patient 146 in the patient coordinate system p defined by the
sensing plate 145b and computing a coordinate transformation matrix
pHm.
[0329] The coordinate transformation matrix pHm is stored in the
storage section of the image controller 147.
[0330] Similarly, the coordinate values of the front end and those
of the rear end of the endoscope 142 are observed in terms of the
endoscope coordinate system e defined by the sensing plate 145c
attached to the endoscope 142 and stored in the storage section of
the image controller 147.
[0331] Now, the operation of the third embodiment will be described
below.
[0332] When the surgical operation image acquisition/display
apparatus is used, the surgical microscope 141 and the endoscope
142 are combined for use as shown in FIG. 14.
[0333] When the surgical operation image acquisition/display
apparatus is in operation, the sensor controller 145e drives the
infrared LEDs 145a to sequentially emit and determines the
three-dimensional position of each of the infrared LEDs 145a on the
basis of its output. At the same time, the sensor controller 145e
computationally determines the three-dimensional position and
orientation of the sensing plate 145b and that of the sensing plate
145c, using the LED definition data stored in the sensor controller
145e and outputs the obtained data to the image controller 147 upon
request.
[0334] The image controller 147 computes the coordinate
transformation matrix pHe from the sensing plate 145b of the
patient coordinate system p attached to the head of the patient 146
to the sensing plate 145c of the endoscope coordinate system e
attached to the endoscope 142 on the basis of the three-dimensional
position and orientation information.
[0335] The image controller 147 also computationally determines the
relative distance between the patient 146 and the endoscope 142 and
the relative direction of the patient 146 as viewed from the
endoscope 142 on the basis of the coordinate transformation matrix
pHe and the above described coordinate transformation matrix
pHm.
[0336] The processing operation of the image controller 147 will be
described further by referring to the flow chart of FIG. 18.
[0337] Firstly, the image controller 147 outputs a request signal
cyclically with a predetermined period (e.g., 33 msec) to the
sensor controller 145e to receive the three-dimensional position
and orientation information on the sensing plates 145b and 145c
from the sensor controller 145e (Step S101).
[0338] Then, the image controller 147 judges if the endoscope 142
is located close to the area of surgical operation (e.g., within a
range of 50 mm) on the basis of the received three-dimensional
position and orientation information (Step S102).
[0339] If the image controller 147 judges that the endoscope 142 is
located close to the area of surgical operation, it outputs an
instruction for switching from the microscope image to the
endoscope image (mode 1) to the video mixer 143 (Step S103).
[0340] If, on the other hand, the image controller 147 judges that
the endoscope 142 is not located close to the area of surgical
operation, it outputs an instruction for switching from the
endoscope image to the microscope image (mode 0) to the video mixer
143 (Step S104).
[0341] When, the endoscope 142 is moved away from the area of
surgical operation and the area is observed mainly by the surgical
microscope 141 as a result of the above judgment, the image
observed through the surgical microscope 141 is displayed on the
liquid crystal display 144. On the other hand, when the endoscope
142 is moved closer to the area of surgical operation, the image
observed through the endoscope 142 is displayed on the liquid
crystal display 144.
[0342] Because the image observed through the surgical microscope
is displayed on the liquid crystal display 144 when the endoscope
142 is moved away from the area of surgical operation, the surgeon
can see the desired image without paying effort for switching from
the microscope image to the endoscope image.
[0343] It may be needless to say that the configuration of this
embodiment can be modified and/or altered in various different ways
without departing from the scope of the invention.
[0344] For instance, while the first observation section refers to
the surgical microscope 141 in this embodiment, it may
alternatively be the endoscope 142 or some other observation
sections or units.
[0345] While the second observation section refers to the endoscope
142 in this embodiment, it may alternatively be the surgical
microscope 141 or some other observation sections or units.
[0346] While the image synthesizing section refers to the video
mixer 143 in this embodiment, it may be some other section for
externally modifying the synthesized state of a plurality of
images.
[0347] While the display section refers to the liquid crystal
display 144 in this embodiment, it may alternatively be a CRT
display, a head mounted display or a projector adapted to display
video signals.
[0348] While the position and orientation detection section is the
position and orientation sensor (comprising the infrared LEDs 145a
for emitting, the sensing plates 145b, 145c, the sensor assembly
145d and the sensor controller 145e) in this embodiment, it may
alternatively be any appropriate section for detecting the
three-dimensional position and orientation of an object such as a
magnetic sensor or a set of mechanical links and joints, encoders
and potentiometers.
[0349] While the state of synthesis modification specifying section
of this embodiment uses the distance between the area of surgical
operation and the endoscope 142 for its judgment, it may
alternatively use both the distance and the direction of the area
as viewed from the endoscope 142.
[0350] While the position and orientation detection section of this
embodiment detects the position and orientation of the endoscope
142, it may alternatively detect the position and orientation of
the microscope 141 or both the position and orientation of the
endoscope 142 and that of the microscope 141.
[0351] (Embodiment 4)
[0352] Now, the fourth embodiment of the invention, which is also a
surgical operation image acquisition/display apparatus, will be
described below.
[0353] The fourth embodiment of surgical operation image
acquisition/display apparatus has a configuration substantially
same as the above described third embodiment and hence the similar
components in the graphic illustrations thereof will be denoted
respectively by the same reference symbols and will not be
described any further.
[0354] Additionally, since the operational function of the fourth
embodiment of surgical operation image acquisition/display
apparatus is same as that of the above described third embodiment
except the processing operation of the image controller 147, only
the latter will be discussed hereinafter.
[0355] More specifically, in this embodiment, one of the obtained
two images is dimensionally reduced and synthetically combined with
the other image so that they may be displayed simultaneously on the
display screen for observation.
[0356] The processing operation of the image controller 147 will be
discussed below by referring to the flow chart of FIG. 19.
[0357] Firstly, the image controller 147 outputs a request signal
cyclically with a predetermined period (e.g., 33 msec) to the
sensor controller 145e to receive the three-dimensional position
and orientation information on the sensing plates 145b and 145c
from the sensor controller 145e (Step S201).
[0358] Then, the image controller 147 judges if the endoscope 142
is located close to the area of surgical operation (e.g., within a
range of 50 mm) on the basis of the received three-dimensional
position and orientation information (Step S202).
[0359] If the image controller 147 judges that the endoscope 142 is
located close to the area of surgical operation, it outputs an
instruction for dimensionally reducing the microscope image and
display it with the endoscope image (mode 3) to the endoscope image
(mode 1) to the video mixer 143 (Step S203).
[0360] If, on the other hand, the image controller 147 judges that
the endoscope 142 is not located close to the sit of surgical
operation, it outputs an instruction for dimensionally reducing the
endoscope image and display it with the microscope image (mode 2)
to the video mixer 143 (Step S204).
[0361] When, the endoscope 142 is moved away from the area of
surgical operation and the area is observed mainly by the surgical
microscope 141 as a result of the above judgment, the dimensionally
reduced image observed through the endoscope 142 is displayed with
the image observed through the surgical microscope 141 on the
liquid crystal display 144.
[0362] On the other hand, when the endoscope 142 is moved closer to
the area of surgical operation, the dimensionally reduced image
observed through the surgical microscope 141 is displayed with the
image observed through the endoscope 142 on the liquid crystal
display 144.
[0363] Because the dimensionally reduced image observed through the
endoscope 142 is displayed with the image observed through the
surgical microscope on the liquid crystal display 144 when the
endoscope 142 is moved away from the area of surgical operation,
the surgeon can see the desired plurality of images without paying
effort for switching from the microscope image to the endoscope
image.
[0364] It may be needless to say that the configuration of this
fourth embodiment can be modified and/or altered in various
different ways without departing from the scope of the
invention.
[0365] For instance, while the first observation section refers to
the surgical microscope 141 in these embodiments, it may
alternatively be the endoscope 142 or some other observation
sections or units.
[0366] While the second observation section refers to the endoscope
142 in these embodiments, it may alternatively be the surgical
microscope 141 or some other observation sections or units.
[0367] While the image synthesizing section refers to the video
mixer 143 in these embodiments, it is by no means limited thereto
and may be some other section for externally modifying the
synthesized state of a plurality of images.
[0368] While the display section refers to the liquid crystal
display 144 in these embodiments, it may alternatively be a CRT
display, a head mounted display or a projector adapted to display
video signals.
[0369] While the position and orientation detection section is the
position and orientation sensor (comprising the infrared LEDs 145a
for emitting, the sensing plates 145b, 145c, the sensor assembly
145d and the sensor controller 145e) in these embodiments, it may
alternatively be any appropriate section for detecting the
three-dimensional position and orientation of an object such as a
magnetic sensor or a set of mechanical links and joints, encoders
and potentiometers.
[0370] The state of synthesis modification specifying section
refers to the image controller in these embodiments.
[0371] As described above, both the third and fourth embodiments of
the invention provide a surgical operation image
acquisition/display apparatus that efficiently assists a surgeon to
smoothly carry out a surgical operation without requiring him or
her to switch the observation system from one to another by means
of a navigation apparatus when the surgical operation is conducted
by using a plurality of observation systems including a surgical
microscope and an endoscope.
[0372] Additional advantages and modifications will readily occur
to those skilled in the art. Therefore, the invention in its
broader aspects is not limited to the specific details and
representative embodiments shown and described herein. Accordingly,
various modifications may be made without departing from the spirit
or scope of the general inventive concept as defined by the
appended claims and their equivalents.
* * * * *