U.S. patent application number 14/531016 was filed with the patent office on 2015-05-28 for method and system for determining a shape of a lumen in an anatomical structure.
The applicant listed for this patent is Intuitive Surgical Operations, Inc.. Invention is credited to Prashant Chopra, Caitlin Q. Donhowe, Vincent Duindam, Giuseppe Maria Prisco.
Application Number | 20150148690 14/531016 |
Document ID | / |
Family ID | 47142305 |
Filed Date | 2015-05-28 |
United States Patent
Application |
20150148690 |
Kind Code |
A1 |
Chopra; Prashant ; et
al. |
May 28, 2015 |
METHOD AND SYSTEM FOR DETERMINING A SHAPE OF A LUMEN IN AN
ANATOMICAL STRUCTURE
Abstract
A medical system provides navigation assistance to a surgeon so
that the surgeon may navigate a flexible medical device through
linked passages of an anatomical structure to a target in or
adjacent to the anatomical structure. As the medical device moves
through the linked passages, images are captured by an image
capturing element at its distal end and pose and shape information
for the medical device are received from sensors disposed in the
medical device. A 4-D computer model of the anatomical structure is
registered to the medical device using one or both of 4-D shape
registration and virtual camera registration so that the captured
image and a virtual image generated from the perspective of a
virtual camera are registered to each other and displayed while
providing an indication of a navigational path to the target.
Inventors: |
Chopra; Prashant;
(Sunnyvale, CA) ; Donhowe; Caitlin Q.; (Sunnyvale,
CA) ; Duindam; Vincent; (Oakland, CA) ;
Prisco; Giuseppe Maria; (Mountain View, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Intuitive Surgical Operations, Inc. |
Sunnyvale |
CA |
US |
|
|
Family ID: |
47142305 |
Appl. No.: |
14/531016 |
Filed: |
November 3, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13107562 |
May 13, 2011 |
8900131 |
|
|
14531016 |
|
|
|
|
Current U.S.
Class: |
600/478 ;
600/481; 600/508; 600/529; 600/587; 600/593 |
Current CPC
Class: |
A61B 2017/00703
20130101; A61B 2576/023 20130101; A61B 5/6852 20130101; A61B
2090/3614 20160201; A61B 5/20 20130101; A61B 34/20 20160201; A61B
2576/026 20130101; A61B 2017/00323 20130101; G06T 19/003 20130101;
A61B 5/08 20130101; A61B 5/0037 20130101; A61B 5/7475 20130101;
A61B 2034/2061 20160201; A61B 1/00055 20130101; A61B 1/00009
20130101; A61B 2090/367 20160201; G06T 7/75 20170101; G06T
2207/10068 20130101; A61B 1/0051 20130101; A61B 5/02028 20130101;
A61B 2017/00699 20130101; A61B 1/2676 20130101; A61B 5/0084
20130101; A61B 5/02007 20130101; A61B 2034/107 20160201; A61B
5/4064 20130101; A61B 5/42 20130101 |
Class at
Publication: |
600/478 ;
600/587; 600/508; 600/593; 600/481; 600/529 |
International
Class: |
A61B 5/00 20060101
A61B005/00; A61B 5/08 20060101 A61B005/08; A61B 5/20 20060101
A61B005/20; A61B 5/02 20060101 A61B005/02 |
Claims
1-36. (canceled)
37. A method for determining a shape of a lumen in an anatomical
structure, the method comprising: positioning a medical device in
the lumen, wherein the medical device has a flexible body that
conforms to the shape of the lumen when the flexible body is
positioned in the lumen, and wherein the medical device has a
plurality of sensors disposed along a length of the flexible body;
concurrently reading information from the plurality of sensors; and
using a processor to computationally determine the shape of the
lumen by using the information concurrently read from the plurality
of sensors.
38. The method of claim 37, wherein the medical device has a fiber
optic cable extending along the length of the flexible body, and
wherein the plurality of sensors comprise a plurality of strain
sensors configured on the fiber optic cable.
39. The method of claim 38, wherein the plurality of strain sensors
employ Rayleigh scattering.
40. The method of claim 38, wherein the plurality of strain sensors
comprise a plurality of Fiber Bragg Gratings.
41. The method of claim 37, wherein the medical device has a
steerable tip which is electromechanically movable for positioning
the medical device in the lumen.
42. The method of claim 37, wherein the anatomical structure is
part of one of the heart, brain, digestive system, circulatory
system, respiratory system, and urinary system.
43. The method of claim 42, wherein the anatomical structure is a
lung and the lumen is an air passageway in the lung.
44. A medical system comprising: a medical device, wherein the
medical device has a flexible body that conforms to a shape of a
lumen of an anatomical structure when the flexible body is
positioned in the lumen, and wherein the medical device has a
plurality of sensors disposed along a length of the flexible body;
and a processor programmed to receive information concurrently read
from the plurality of sensors of the medical device, and to
computationally determine the shape of the lumen by using the
received information.
45. The medical system of claim 44, wherein the medical device has
a fiber optic cable extending along the length of the flexible
body, and wherein the plurality of sensors comprise a plurality of
strain sensors configured on the fiber optic cable.
46. The medical system of claim 45, wherein the plurality of strain
sensors employ Rayleigh scattering.
47. The medical system of claim 45, wherein the plurality of strain
sensors comprise a plurality of Fiber Bragg Gratings.
48. The medical system of claim 44, further comprising: an input
device; a controller; and an electromechanical interface having an
actuator whose actuation is controlled by the controller in
response to input received from the input device so that the
medical device is electromechanically positionable in the
lumen.
49. The medical system of claim 44, wherein the anatomical
structure is part of one of the heart, brain, digestive system,
circulatory system, respiratory system, and urinary system.
50. The medical system of claim 49, wherein the anatomical
structure is a lung and the lumen is an air passageway in the lung.
Description
FIELD OF THE INVENTION
[0001] The present invention generally relates to medical systems
and in particular, to a medical system providing dynamic
registration of a model of an anatomical structure for image-guided
surgery.
BACKGROUND
[0002] Image guided surgery helps surgeons navigate medical devices
to targets in patients so that therapeutic and/or diagnostic
medical procedures may be performed on the targets. For guidance,
the pose (i.e., position and orientation) of a working end of a
medical device may be tracked and its image displayed along with or
superimposed on a model of an anatomical structure associated with
the target. The model may be computer generated from pre-operative
and/or intra-operative patient anatomy scan data such as x-ray,
ultrasound, fluoroscopy, computed tomography (CT), magnetic
resonance imaging (MRI), and other imaging technologies. The
medical device may be an endoscope, catheter, or medical instrument
that has a steerable tip and flexible body capable of conforming to
body passages leading to the target in an anatomical structure of
the patient.
[0003] Displaying the target upon which the therapeutic and/or
diagnostic medical procedure is to be performed, the model of the
anatomical structure in which the target resides or is adjacent to,
and an image of the working end of the medical device superimposed
on the model of the anatomical structure may be particularly useful
to the surgeon to provide assistance in guiding the medical device
through natural and/or artificial body passages to and through the
anatomical structure to the target. Proper registration of the
model to the medical device, however, may be very difficult when
the anatomical structure is neither immobile nor rigid, but
instead, moves and/or changes shape according to periodic or
non-periodic movement of the anatomical structure such as the case
with a patient's lung or beating heart.
OBJECTS AND SUMMARY
[0004] Accordingly, one object of one or more aspects of the
present invention is a medical system and method implemented
therein for providing dynamic registration of a model of an
anatomical structure with intra-operative anatomical information
for image-guided surgery.
[0005] Another object of one or more aspects of the present
invention is a medical system and method implemented therein for
providing dynamic registration of a model of an anatomical
structure with intra-operative anatomical information for
image-guided surgery that are simple to implement and do not
require an expensive tracking system.
[0006] Another object of one or more aspects of the present
invention is a medical system and method implemented therein for
providing dynamic registration of a model of an anatomical
structure during image-guided surgery that are computationally
efficient and suitable for real-time applications.
[0007] Another object of one or more aspects of the present
invention is a medical system and method implemented therein for
providing dynamic registration of a model of an anatomical
structure during image-guided surgery that are accurate.
[0008] These and additional objects are accomplished by the various
aspects of the present invention, wherein briefly stated, one
aspect is a medical system comprising: a memory storing information
of a computer model of an anatomical structure; a medical device
having a flexible body and a plurality of sensors distributed along
the length of the flexible body; and a processor programmed to
determine the pose and shape of the flexible body while disposed in
a passage of the anatomical structure using information provided by
the plurality of sensors at the same point in time and register the
computer model to the medical device by matching at least the
determined shape of the flexible body to a best fitting one of the
shapes of one or more potential passages in the computer model of
the anatomical structure.
[0009] Another aspect is a method for registering a computer model
of an anatomical structure with a flexible medical device disposed
within a passage in the anatomical structure, wherein the flexible
medical device has a plurality of sensors distributed along the
length of the flexible medical device, the method comprising:
determining a current pose and shape of the flexible medical device
using information provided by the plurality of sensors at the same
point in time; registering the computer model of the anatomical
structure to the flexible medical device by matching at least the
determined shape of the flexible medical device to a best fitting
one of the shapes of potential passages in the computer model.
[0010] Another aspect is a medical system comprising: a memory
storing information of a computer model of an anatomical structure;
a medical device; an image capturing device for capturing images
from a perspective of a distal end of the medical device; and a
processor programmed to periodically perform a global registration
of the computer model to the medical device by determining the pose
and shape of the medical device while disposed in a passage of the
anatomical structure and matching at least the determined shape of
the medical device to a best fitting one of the shapes of one or
more potential passages in the computer model of the anatomical
structure, followed by performing a local registration of the
computer model to the medical device by comparing an image captured
by the image capturing device with a plurality of virtual views of
the computer model of the anatomical structure, wherein the
plurality of virtual views is generated from the perspective of a
virtual camera whose pose is initially set at the pose of the
distal end of the medical device and then perturbed about the
initial pose.
[0011] Still another aspect is a method for registering a computer
model of anatomical structure to a medical device, the method
comprising: periodically performing a global registration of the
computer model to the medical device by determining the pose and
shape of the medical device while disposed in a passage of the
anatomical structure and matching at least the determined shape of
the medical device to a best fitting one of the shapes of one or
more potential passages in the computer model of the anatomical
structure, followed by performing a local registration of the
computer model to the medical device by comparing an image captured
by the image capturing device with a plurality of virtual views of
the computer model of the anatomical structure, wherein the
plurality of virtual views is generated from the perspective of a
virtual camera whose pose is initially set at the pose of the
distal end of the medical device and then perturbed about the
initial pose.
[0012] Additional objects, features and advantages of the various
aspects of the present invention will become apparent from the
following description which should be taken in conjunction with the
accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] FIG. 1 illustrates a medical system, utilizing aspects of
the present invention, which includes a hand-operated medical
device.
[0014] FIG. 2 illustrates an alternative medical system, utilizing
aspects of the present invention, which includes a teleoperated
medical device.
[0015] FIG. 3 illustrates a diagram of a medical device inserted
into an anatomical structure of a patient.
[0016] FIG. 4 illustrates a flow diagram of preoperative tasks
conducted prior to performing a medical procedure on a patient.
[0017] FIG. 5 illustrates movement of a lung during a respiratory
cycle.
[0018] FIG. 6 illustrates a view of a primary screen during
navigation of a medical device to a target area in an anatomical
structure before registration of a computer model of the anatomical
structure to the medical device.
[0019] FIG. 7 illustrates a view of an auxiliary screen during
navigation of a medical device to a target area in an anatomical
structure.
[0020] FIG. 8 illustrates a flow diagram of a method for performing
a medical procedure including one of a first and second method,
utilizing aspects of the present invention, for registering a
computer model of an anatomical structure with a medical
device.
[0021] FIG. 9 illustrates a flow diagram of a first method,
utilizing aspects of the present invention, for registering a
computer model of an anatomical structure with a medical
device.
[0022] FIG. 10 illustrates a flow diagram of a second method,
utilizing aspects of the present invention, for registering a
computer model of an anatomical structure with a medical
device.
[0023] FIG. 11 illustrates a flow diagram of a method for
performing a medical procedure including both a first and second
method, utilizing aspects of the present invention, for registering
a computer model of an anatomical structure with a medical
device.
[0024] FIGS. 12A-C illustrate schematic drawings of a medical
device having a single end sensor respectively at three different
points in time as the medical device moves through a passage of an
anatomical structure in a patient.
[0025] FIG. 13 illustrates a schematic drawing of a medical device
having a plurality of distributed sensors at a single point in time
while the medical device is disposed in a passage of an anatomical
structure in a patient.
[0026] FIG. 14 illustrates a view of a primary screen during
navigation of a medical device to a target area in an anatomical
structure after registration of a computer model of the anatomical
structure to the medical device.
[0027] FIG. 15 illustrates a virtual reality system to be
optionally used in a medical system utilizing aspects of the
present invention.
DETAILED DESCRIPTION
[0028] FIG. 1 illustrates, as an example, a medical system 100
including a steerable medical device 110, one or more fiber optic
cables 120 inserted in the medical device 110, a pose/shape
processor 130, an image processor 140, an image capturing element
141, a display processor 150, a primary display screen 151, an
auxiliary display screen 152, a navigation processor 160, and
memory 161. Although shown as separate units, the pose/shape
processor 130, image processor 140, display processor 150, and
navigation processor 160 may each be implemented as hardware,
firmware, software or a combination thereof, which interact with or
are otherwise executed by one or more computer processors. The
primary and auxiliary display screens, 151 and 152, are preferably
computer monitors capable of displaying three-dimensional images to
an operator of the system 100. However, for cost considerations,
either or both of the primary and auxiliary display screens, 151
and 152, may be a standard computer monitor capable of only
displaying two-dimensional images.
[0029] The medical device 110 has a flexible body 114, a steerable
tip 112 at its distal end 111, and a hand-operable handle 116 at
its proximal end 115. Control cables (not shown) or other control
means typically extend from the handle 116 to the steerable tip 112
so that the tip 112 may be controllably bent or turned as shown for
example by dotted line versions of the bent tip 112. The medical
device 110 may be an endoscope, catheter or other medical
instrument having a flexible body and steerable tip.
[0030] The image capturing element 141 may be a stereoscopic or
monoscopic camera disposed at the distal end 111 for capturing
images that are transmitted to and processed by the image processor
140 and/or display processor 150 and displayed on the primary
display screen 151, auxiliary display screen 152, and/or other
display means according to the various aspects of the invention as
described herein. Alternatively, the image capturing element 141
may be a coherent fiber-optic bundle that couples to an imaging and
processing system on the proximal end of the medical device 110,
such as a fiberscope. The image capturing element 141 may also be
single or multi-spectral that captures image data in the visible or
infrared/ultraviolet spectrum. Thus, any image capturing element,
device, or system referred to herein may be any one or a
combination of these and other imaging technologies. One of a
plurality of fiber optic cables 120 may be coupled at its proximal
end to a light source (not shown) for illumination purposes at the
distal end 111. Others of the fiber optic cables 120 may be
configured with position and bend or shape sensors such as Fiber
Bragg Gratings (or other strain sensors such as those employing
Rayleigh scattering) distributed along the length of the medical
device 110 so that light passing through the fiber optic cable is
processed by the pose/shape processor 130 to determine a current
pose and shape of the medical device 110.
[0031] FIG. 2 illustrates, as an example, an alternative embodiment
of the medical system 100 in which the handle 116 is replaced by an
electromechanical interface 170, controller 180, and input device
190 for teleoperating the medical device 110. The interface 170
includes actuators for actuating cables in the medical device 110
to steer its tip 112 as well as an actuator for moving the entire
medical device 110 forward and backward so that it may be inserted
into and retracted out of a patient through an entry port such as a
natural body orifice or a surgeon created one. The controller 180
is preferably implemented as hardware, firmware or software (or a
combination thereof) in the same one or more computer processors as
the processors 130, 140, 150, and 160, or a different computer
processor. The flexible body 114 may be passively or actively
bendable in this embodiment.
[0032] Examples of such steerable medical devices are described in
U.S. 2010/0249506 A1 entitled "Method and System for Assisting an
Operator in Endoscopic Navigation" and WO 2009/097461 A1 entitled
"Apparatus and Methods for Automatically Controlling an Endoscope,
which are each incorporated herein by reference. Details on the
determination of the endoscope's position and bending using Fiber
Bragg Gratings may be found, for examples, in U.S. 2007/0156019 A1
entitled "Robotic Surgery System Including Position Sensors Using
Fiber Bragg Gratings", U.S. 2008/0212082 A1 entitled "Fiber Optic
Position and/or Shape Sensing Based on Rayleigh Scatter", U.S.
2008/0218770 A1 entitled "Robotic Surgical Instrument and Methods
using Bragg Fiber Sensors", and U.S. 2009/0324161 A1 entitled
"Fiber Optic Shape Sensor", which are each incorporated herein by
reference.
[0033] FIG. 3 illustrates, as an example, a diagram of a medical
device 110 inserted through an entry port 310 and extending into an
anatomical structure 330 of a patient 300. In this example, the
anatomical structure 330 is a pair of lungs having a plurality of
natural body passages including a trachea, bronchi, and
bronchioles; the entry port 310 is the patient's mouth; and the
medical device 110 is a bronchoscope. Due to the nature of the
lung, the medical device 110 may be guided through a number of
linked passages of the bronchial tree. In doing so, the flexible
body 114 of the medical device 110 conforms to the passages through
which it travels. Although a pair of lungs is shown in the present
example, it is to be appreciated that the various aspects of the
present invention are also applicable and useful for other
anatomical structures such as the heart, brain, digestive system,
circulatory system, and urinary system, in addition to the
respiratory system. Further, although only natural body passages
are shown, the methods described herein are also applicable to
artificial or surgeon created passages that may be formed during or
prior to a medical procedure and superimposed on the computer model
of the patient anatomy.
[0034] FIG. 4 illustrates, as an example, a flow diagram of
preoperative tasks that are performed in preparation for a medical
procedure on a patient. In the following example, the anatomical
structure is presumed to be one that moves during a medical
procedure in an identifiable way such as periodic motion of the air
and blood circulatory systems or a non-periodic motion such as a
body response to a stimulus. Although aspects of the invention may
still be applicable and useful when the anatomical structure does
not move during a medical procedure, the full advantages of the
present invention are best experienced in an environment in which
the anatomical structure moves in an identifiable or otherwise
known manner during the medical procedure.
[0035] In block 401, one or more sets of images of a patient is
acquired using an appropriate imaging technology from which a set
of three-dimensional (3-D) computer models of the anatomical
structure may be generated, wherein each 3-D computer model is
associated with a different point in time over a period of time so
that time represents a fourth dimension and the images are referred
to herein as four-dimensional (4-D) images. Additional dimensions
may also be defined and used in the methods described herein.
Examples of such an imaging technology include, but are not limited
to, fluoroscopy, Magnetic Resonance Imaging, thermography,
tomography, ultrasound, Optical Coherence Tomography, Thermal
Imaging, Impedance Imaging, Laser Imaging, nano-tube X-ray imaging,
etc.
[0036] The period of time over which images are captured depends
upon the anatomical structure and the motion of interest. For
example, when the anatomical structure is the lungs, one set of
images may be for a periodic motion such as a respiratory cycle
shown in FIG. 5 where the lung expands from a maximum exhalation
state 501 (solid lines) to a maximum inhalation state 502 (dotted
lines). Another set of images may be for a non-periodic motion such
as a cough or other body reaction to a stimulus resulting in
movement of the lungs. As another example, when the anatomical
structure is the heart, one set of images may be for a periodic
motion such as a blood circulatory cycle. The sampling rate which
determines the number of such 3-D computer models is chosen so that
the movement of the anatomical structure during such period of
motion is adequately described for accurate registration and
navigation purposes.
[0037] In block 402, 4-D shape information is extracted from the
acquired images of the anatomical structure. When the acquired
images are sets of two-dimensional (2-D) slices of the anatomical
structure sampled at incremental points in time (e.g., according to
a sampling rate) over the period of motion, 3-D shape information
for the anatomical structure is generated for each set of 2-D
slices corresponding to the same point in time. Thus, for n-points
in time, "n" sets of 3-D shape information are extracted, where "n"
is the number of sampling points in time over the period of
motion.
[0038] In block 403, one or more targets are identified in the
anatomical structure. The targets are locations or objects in or
adjacent to the anatomical structure where or upon which a medical
procedure is to be performed. For example, the target may be a
tumor in or adjacent to the anatomical structure. The target(s) may
be identified by a surgeon in a conventional manner by analysis of
the acquired images of the anatomical structure or the extracted
4-D shape information, whichever is more convenient and/or reliable
for such identification.
[0039] In block 404, a navigational path is determined to and
through the anatomical structure for the working end of the medical
device 110 to travel to each target. In this case, the working end
is assumed to be the distal end 111 of the medical device 110. The
surgeon may determine a suitable navigational path to a target by
analyzing the acquired images of the anatomical structure or the
extracted 4-D shape information so as to take into account any
damage to the patient that the medical device 110 may cause as it
moves towards the target as well as the shortest time and/or
shortest path. Alternatively, a computer program may cause a
processor to perform such analysis to determine the navigational
path using artificial intelligence techniques.
[0040] FIG. 6 illustrates, as an example, a view of the primary
screen 151 during navigation of the medical device 110 to a target
area in an anatomical structure before registration of a computer
model of the anatomical structure to the medical device. A left
image 610 is the image captured by the image capturing element 141
while viewing a bifurcation in a lung, wherein the bifurcation
indicates a left passage 611 and a right passage 612 through which
one or the other the medical device 110 may pass through as it is
inserted further into the lung. Also shown is a right image 620
which is a virtual image generated by a virtual camera viewing a
corresponding location in a 4-D computer model of the anatomical
structure which has been generated from the 4-D shape information
extracted in block 402 of FIG. 4 before the 4-D computer model is
registered in some fashion to the medical device 110. In
particular, although left 621 and right 622 passages corresponding
to the passages 611 and 612 are shown, their sizes and alignments
differ due to translational and rotational errors in the
registration transformation relating the 4-D computer model of the
anatomical structure to the medical device 110.
[0041] FIG. 7 illustrates, as an example, a view of the auxiliary
screen 152 during navigation of the medical device 110 to a target
area in an anatomical structure. The view may be either a 2-D or
3-D view of a computer model 720 of the anatomical structure 330
and a computer model 710 of the medical device 110, which is
updated in real-time as the medical device 110 moves through the
anatomical structure 330. Also shown is an indication 721 of the
target. Thus, the auxiliary screen 152 assists the surgeon to steer
the medical device 110 through the anatomical structure 330 to the
target.
[0042] FIG. 8 illustrates, as an example, a flow diagram of a
method for performing a medical procedure on a patient. In block
801, a number of pre-operative tasks are performed in preparation
of performing the medical procedure. First, the medical device 110
is localized to a fixed reference frame in a conventional manner
by, for example, touching the distal end 111 of the medical device
110 to a known and stationary point in the fixed reference frame.
Second, the patient may be registered to the fixed reference frame
in a conventional manner by touching and holding the distal end 111
of the medical device 110 to one or more points on the patient,
which points correspond to identifiable points on the acquired
images of the patient as described in block 401 of FIG. 4, during
the period of motion associated with the 4-D computer model. Thus,
by applying known relationships between the one or more points on
the patient to the anatomical structure 330, the computer model of
the anatomical structure may be registered to the anatomical
structure of the patient, the fixed reference frame, and the
medical device 110.
[0043] Navigation of the medical device 110 through the linked
passages of the anatomical structure 330 to the target is performed
from START to END in FIG. 8. In block 802, the medical device 110
is moved through the linked passages in either the insertion or
retraction direction by the surgeon either manipulating the handle
116 or the input device 190, depending upon the embodiment of the
medical system 100 being used by the surgeon. In block 803, the
navigation processor 160 receives pose and shape information for
the medical device 110 from the pose/shape processor 130 and image
data from the image processor 140. Thus, the navigation processor
160 has information on the current position and orientation (i.e.,
pose) of the distal end 111 of the medical device 110 and the shape
of the flexible body 114 of the medical device 110 along with an
image that has been captured by the image capturing element 141 at
that time.
[0044] In block 804, the navigation processor 160 performs a
correction to the registration of the 4-D computer model of the
anatomical structure 330 to the medical device 110. One method for
performing this registration is described in reference to FIG. 9
and another method is described in reference to FIG. 10.
Alternatively, rather than performing one or the other of the two
methods, both methods may be performed as shown and described in
reference to FIG. 11. In performing block 804, it is assumed that
the shape of the medical device 110 conforms to the shape of the
passage of the anatomical structure in which the medical device 110
is disposed at the time. Therefore, registration of the computer
model to the medical device 110 effectively registers the computer
model of the anatomical structure to the actual anatomical
structure of the patient.
[0045] In block 805, the captured image and virtual image are
displayed in a similar manner as shown and described in reference
to FIG. 6 except that the virtual image 620 is now adjusted to
resemble that of the captured image 610, such as shown in FIG. 14,
due to the proper registration of the 4-D computer model of the
anatomy 330 to the medical device 110. In particular, the size and
orientations of the left and right passages, 621 and 622, of the
virtual image 620 match those of the left and right passages, 611
and 612, of the captured image 610. In block 806, a navigational
path indication such as the arrow 623 in the virtual image 620 is
provided so that the surgeon knows that the medical device 110 is
to be steered into the indicated passage.
[0046] In block 807, a determination is made whether the working
end 111 of the medical device 110 has come within a threshold
distance to the target. The threshold distance in this case is a
distance that is sufficient so that the working end 111 of the
medical device 110 can be manipulated by the surgeon to perform its
intended purpose without requiring further insertion of the medical
device 110 into the anatomical structure 330. If the determination
in 807 is YES, then the guided navigation to the target is
completed and the method ends. On the other hand, if the medical
device 110 has not reached the threshold distance to the target,
then the method jumps back to 802 so that the medical device 110 is
moved further through the linked passages by the surgeon either
manipulating the handle 116 or the input device 190, depending upon
the embodiment of the medical system 100 being used by the
surgeon.
[0047] FIG. 9 illustrates, as an example, a flow diagram of a first
method (referred to as "shape registration") performable by the
navigation processor 160 for registering a computer model of an
anatomical structure with a medical device. This method is
particularly useful when real-time images are unavailable from the
perspective of the distal end 111 of the medical device 110, such
as when the image capturing element 141 is either removed or its
view is obstructed.
[0048] As previously explained, since the flexible body 114
conforms to the shape of the passage of the anatomical structure
through which the medical device 110 is passing through at the
time, the shape of the medical device 110 resembles that of the
passage. Thus, by registering the computer model of the anatomical
structure to the medical device 110, this is effectively the same
as registering the computer model of the anatomical structure to
the actual anatomical structure. Alternatively, the shape of the
passage might be determined using an approach as described in
reference to FIGS. 12A-C, where the pose of a distal end sensor
1210 is recorded at different points in time as the medical device
110 moves through the passage 1202 of an anatomical structure 1200.
One problem with this approach, however, is that when the
anatomical structure 1200 is moving, the different position
measurements which are made at different points in time (and
possibly different points in the dynamic movement of the anatomical
structure), can lead to errors or complicated correctional
adjustments. Therefore, a preferred embodiment of the present
invention is shown in FIG. 13, where a plurality of sensors
1310a-1310k are employed that are sufficient in number and properly
distributed along the length of the medical device 110 so that all
pose and shape measurements may be accurately made at the same
point in time.
[0049] In block 901, a 3-D computer model corresponding to the
current pose and shape of the medical device 110 is generated using
the pose and shape information received from the pose/shape
processor 130. Since the pose and shape information is readily
generated from position and shape sensors disposed in the medical
device 110, a computationally fast determination of the medical
device's pose and shape is made.
[0050] In block 902, the shape of the medical device 110 is
compared against shapes of the linked passages in the 3-D computer
model for each sampled point in time to find a closest match of
linked passages. A number of well-known matching techniques may be
used to perform this function such as an Iterative Closest Point
(ICP) algorithm or a Singular Value Decomposition (SVD) algorithm
as described, for example, in U.S. 2005/0182319 A1, which is
incorporated herein by reference. Thus, for each sample time in a
dynamic motion cycle, a closest match of the current shape of the
medical device 110 (and consequently the passage in which it is
disposed at the time) and one of the linked passages in a computer
model of the anatomical structure is determined.
[0051] In block 903, deviations are determined between each closest
match of linked passages determined in 902 and the shape of the
medical device 110. The closest match of linked passages having the
smallest deviation with the current shape of the medical device 110
is then determined to be the "best fit" among the matches. Thus,
whereas block 902 determines for each 3-D computer model, the
closest match between one or more of its passages with the current
shape of the medical device, block 903 determines the 3-D computer
model whose closest match of linked passages is the "best fit"
(i.e., closest match) of the closest matches of all the 3-D
computer models. In 904, the "best fit" of linked passages in the
4-D computer model of the anatomical structure is then localized to
the portion of the medical device 110 which it has been determined
to be the "best fit" so that the 4-D computer model is registered
to the medical device 110 (and consequently, the anatomical
structure of the patient).
[0052] FIG. 10 illustrates, as an example, a flow diagram of a
second method (referred to as "virtual camera registration")
performable by the navigation processor 160 for correcting the
registration of a computer model of an anatomical structure with a
medical device. In performing the method, it is assumed that a
prior registration of the 4-D computer model and the medical device
110 has been performed (such as initially in block 801 of FIG.
8).
[0053] In block 1001, a virtual camera is initially assumed to be
disposed at the current pose of the distal end of the medical
device 110. In block 1002, one or more virtual images of the 4-D
computer model of the anatomic structure are generated as though
being captured by the virtual camera by perturbing the current pose
of the virtual camera translationally and/or orientationally. In
block 1003, the one or more virtual images are compared with the
current image of the anatomical structure captured by the image
capturing element 141. In block 1004, the virtual camera pose is
adjusted according to the comparisons performed in block 1003 so
that a virtual image captured by the virtual camera at the adjusted
pose will better match the current image of the anatomical
structure captured by the image capturing element 141. In block
1005, a virtual image of the 4-D computer model is generated as
though being captured by the virtual camera at the adjusted pose.
In block 1006, the virtual image captured by the virtual camera at
the adjusted pose is compared to the current image of the
anatomical structure captured by the image capturing element 141.
In block 1007, a determination is made whether the deviation
between the virtual image and the real captured image is within a
tolerance range. The tolerance range may be pre-set to limit values
previously determined in some fashion to result in acceptable
matches within a reasonable time period. Alternatively, an
algorithm may be used to incrementally change an initial tolerance
range as a function of the results of the processing through the
loop of blocks 1002-1007.
[0054] If the determination is YES, then in 1908, the adjusted pose
of the virtual camera is used to generate a registration transform
to register the 4-D computer model of the anatomical structure to
the medical device 110 and the registration transform is used to
localize the 4-D computer model to the medical device 110. On the
other hand, if the determination is NO, then the method jumps back
to block 1002 to generate one or more virtual images of the 4-D
computer model of the anatomic structure from the perspective of
the virtual camera by perturbing the adjusted pose of the virtual
camera. The method then continues to loop through blocks 1002-1007
until the determination in block 1007 is YES.
[0055] FIG. 11 illustrates, as an example, a flow diagram of a
method for performing a medical procedure including both a first
and second method for registering a computer model of an anatomical
structure with a medical device. In this method, blocks 1101-1103
are performed identically to blocks 801-803 of FIG. 8 and blocks
1106-1108 are performed identically to blocks 805-807 of FIG. 8.
Block 1104 is performed identically as the method described in
reference to FIG. 9 and may be thought of as a global or coarse
registration that is relatively fast to execute. Block 1105 is
performed identically to the method described in reference to FIG.
10 and may be thought of as a local or fine registration that
corrects for any "residual errors" that may remain after
performance of block 1104. Thus, in this example, periodically
performing the combination of the methods described in reference to
FIGS. 9 and 10 may provide a more accurate registration of the 4-D
computer model of the anatomical structure to the medical device
110. Further, periodically performing the global registration of
block 1104 may serve to prevent any "drift" errors that may result
by only periodically performing block 1105 after an initial
registration such as block 801 of FIG. 8.
[0056] After performing any of the registration methods described
herein, if the resulting virtual image 620 is still visibly
misaligned with the captured image 610 (such as viewed on the
primary display screen 151), manual registration means may be
provided whereby the computer model may be translated and/or
oriented according to operator manipulation of an input device
until the virtual and captured images appear aligned.
[0057] FIG. 15 illustrates, as an example, a virtual reality system
1500 to be optionally used in the medical system 100 for providing
navigation guidance to a surgeon in a virtual reality environment
to a target in or adjacent to an anatomical structure in a patient.
In the virtual reality system 1500, stereo goggles or glasses 1501,
worn by the surgeon, displays either virtual images generated by
the virtual camera or real-time images captured by the image
capturing element 141 in 3-D as the surgeon moves the medical
device 110 through the anatomical structure. As the surgeon
approaches each bifurcation in the linked passages of the
anatomical structure, an indication of the navigational path to be
taken may be provided in one or more of the sense modalities. For
example, the navigation processor 160 may perform the steps 801-805
as described in reference to FIG. 8, but in lieu of displaying an
arrow in the virtual image 620 on the primary display screen 151,
it may provide the navigation indication as an arrow indicating the
correct passage to be taken in the stereo glasses 1501 (through the
display processor 150) so that the surgeon receives a visual
indication of the correct navigational path.
[0058] Alternatively or additionally, a navigational path
indication may be provided through a sound system 1502 when the
medical device 110 approaches a bifurcation by a warning sound
being heard if the surgeon directs the distal end 111 of the
medical device 110 to enter the wrong passage and/or an assuring
sound being heard if the surgeon directs the distal end 111 of the
medical device 110 to enter the correct passage. Alternatively or
additionally, a navigational path indication may be provided
through a smell system 1503 when the medical device 110 approaches
a bifurcation by a foul odor being smelt if the surgeon directs the
distal end 111 of the medical device 110 to enter the wrong passage
and/or pleasing odor being smelt if the surgeon directs the distal
end 111 of the medical device 110 to enter the correct passage.
Alternatively or additionally, a navigational path indication may
be provided through a taste system 1504 when the medical device 110
approaches a bifurcation by a bitter taste being sensed on a
mouthpiece 1515 inserted in the surgeon's mouth if the surgeon
directs the distal end 111 of the medical device 110 to enter the
wrong passage and/or sweet taste being sensed on the mouthpiece
1515 if the surgeon directs the distal end 111 of the medical
device 110 to enter the correct passage. Alternatively or
additionally, a navigational path indication may be provided
through a touch system 1505 when the medical device 110 approaches
a bifurcation by a resistive force being felt on the input device
190 if the surgeon directs the distal end 111 of the medical device
110 to enter the wrong passage and/or a forward nudging force being
felt on the input device 190 if the surgeon directs the distal end
111 of the medical device 110 to enter the correct passage.
[0059] Although the various aspects of the present invention have
been described with respect to one or more embodiments, it will be
understood that the invention is entitled to full protection within
the full scope of the appended claims.
* * * * *