U.S. patent application number 14/190725 was filed with the patent office on 2014-06-26 for multiple medical device guidance.
This patent application is currently assigned to InnerOptic Technology, Inc.. The applicant listed for this patent is InnerOptic Technology, Inc.. Invention is credited to Caroline Green, Brian Heaney, Sharif Razzaque, Andrei State.
Application Number | 20140180074 14/190725 |
Document ID | / |
Family ID | 48870823 |
Filed Date | 2014-06-26 |
United States Patent
Application |
20140180074 |
Kind Code |
A1 |
Green; Caroline ; et
al. |
June 26, 2014 |
MULTIPLE MEDICAL DEVICE GUIDANCE
Abstract
A system and method for providing image guidance for placement
of one or more medical devices at a target location. The system
receives emplacement information of medical devices within a
predetermined area. The system calculates a viewing angle in a
virtual 3D space of a plurality of virtual medical devices
corresponding to the plurality of medical devices. The system also
causes a display device to display the plurality of virtual medical
devices based at least on the calculated viewing angle.
Inventors: |
Green; Caroline; (Carrborro,
NC) ; Heaney; Brian; (Durham, NC) ; Razzaque;
Sharif; (Chapel Hill, NC) ; State; Andrei;
(Chapel Hill, NC) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
InnerOptic Technology, Inc. |
Hillsborough |
NC |
US |
|
|
Assignee: |
InnerOptic Technology, Inc.
Hillsborough
NC
|
Family ID: |
48870823 |
Appl. No.: |
14/190725 |
Filed: |
February 26, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13753274 |
Jan 29, 2013 |
8670816 |
|
|
14190725 |
|
|
|
|
61592531 |
Jan 30, 2012 |
|
|
|
61736789 |
Dec 13, 2012 |
|
|
|
Current U.S.
Class: |
600/424 |
Current CPC
Class: |
A61B 5/742 20130101;
A61B 90/361 20160201; A61B 8/523 20130101; A61B 2034/107 20160201;
A61B 8/4483 20130101; A61B 34/20 20160201; A61B 6/463 20130101;
A61B 8/466 20130101; A61B 5/055 20130101; A61B 18/1815 20130101;
A61B 8/0841 20130101; A61B 6/032 20130101; A61B 5/7475 20130101;
A61B 8/468 20130101; A61B 6/12 20130101; A61B 2018/1861 20130101;
A61B 5/04 20130101; A61B 2018/00577 20130101; A61B 17/3403
20130101; A61B 18/20 20130101; A61B 1/303 20130101; A61B 6/5223
20130101; A61B 5/743 20130101; A61B 5/0066 20130101; A61B 2090/378
20160201; A61B 6/468 20130101; A61B 8/483 20130101; A61B 6/037
20130101; A61B 5/7425 20130101; A61B 2018/00559 20130101; A61B
2018/1425 20130101; A61B 5/062 20130101; A61B 1/3132 20130101; A61B
8/12 20130101; A61B 17/221 20130101; A61B 18/14 20130101; A61B
6/466 20130101; A61B 5/72 20130101; A61B 5/6848 20130101; A61B
34/10 20160201; A61B 1/04 20130101; A61B 17/320016 20130101; A61B
2018/143 20130101 |
Class at
Publication: |
600/424 |
International
Class: |
A61B 19/00 20060101
A61B019/00 |
Claims
1. A method, comprising receiving emplacement information of a
plurality of needles; receiving emplacement information of a
medical imaging device; receiving at least one image from the
medical imaging device; calculating a first viewing angle in a
virtual 3D space of the at least one image based at least on the
emplacement information of the medical imaging device with respect
to a perspective view; calculating a second viewing angle in the
virtual 3D space of a virtual medical imaging device corresponding
to the medical imaging device based at least on the emplacement
information of the medical imaging device with respect to the
perspective view; calculating a plurality of viewing angles in the
virtual 3D space of a plurality of virtual needles corresponding to
the plurality of needles based at least on the emplacement
information of the plurality of needles with respect to the
perspective view; causing the display device to display the at
least one image based at least on the first calculated viewing
angle in the virtual 3D space; causing the display device to
display the virtual medical imaging device based at least on the
second calculated viewing angle in the virtual 3D space; and
causing a display device to display the plurality of virtual
needles based at least on the plurality of calculated viewing
angles in the virtual 3D space.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] The present application is a continuation of U.S.
application Ser. No. 13/753,274 filed Jan. 29, 2013, entitled
MULTIPLE MEDICAL DEVICE GUIDANCE, which claims priority benefit to
U.S. Provisional Application Nos. 61/592,531, filed Jan. 30, 2012
and 61/736,789 filed Dec. 13, 2012, each of which is hereby
incorporated herein by reference in its entirety.
TECHNICAL FIELD
[0002] The systems and methods disclosed herein relate generally to
computer systems facilitating medical device guidance through
tissue by a medical practitioner.
BACKGROUND
[0003] Various existing medical device systems, such as for
ablation procedures, require a healthcare provider, such as a
physician, to place controlled formations of medical devices into
patient tissue. Examples of these systems include the RFA Medical
InCircle.TM. System and the AngioDynamics NanoKnife.TM. system.
Similarly, the Covidien Evident.TM. MWA System does not require,
but supports, multi-medical device configurations. Many of these
systems include a plastic guide piece to help the healthcare
provider hold the medical devices in an acceptable spatial
configuration (e.g. three medical devices held parallel, with
shafts 2 cm apart).
[0004] Unfortunately, controlled placement of the medical devices
often comes at the expense of operator flexibility and
responsiveness. Accordingly, there is a need for a medical device
guidance system which assists operators in placing medical devices,
while permitting the operator to maintain an appropriate degree of
hand flexibility.
BRIEF DESCRIPTION OF THE DRAWINGS
[0005] FIG. 1A illustrates a first exemplary system for
image-guided medical procedures.
[0006] FIG. 1B illustrates a second exemplary system for
image-guided medical procedures.
[0007] FIG. 2 illustrates a virtual rendering of an exemplary
surgical instrument being displayed on a screen.
[0008] FIG. 3 illustrates a virtual rendering of an ablation volume
of a surgical instrument being displayed on a screen.
[0009] FIG. 4 illustrates a virtual rendering of a trajectory and
ablation volume of a surgical instrument being displayed on a
screen.
[0010] FIG. 5 illustrates a virtual rendering of a surgical
instrument and medical images displayed on a screen.
[0011] FIG. 6 illustrates a virtual rendering of a surgical
instrument and guidance cues displayed on a screen.
[0012] FIG. 7 illustrates a virtual rendering of a surgical
instrument and guidance cues displayed on a screen.
[0013] FIG. 8 illustrates a virtual rendering of a surgical
instrument and guidance cues displayed on a screen.
[0014] FIG. 9 illustrates a virtual rendering of a surgical
instrument and guidance cues displayed on a screen.
[0015] FIG. 10A-10D illustrates a virtual rendering of a surgical
instrument and annotations displayed on a screen.
[0016] FIG. 11 is a diagram illustrating various features of
multiple medical devices that can be tracked by the system.
[0017] FIGS. 12A-12D illustrate embodiments of displaying image
guidance data for multiple medical devices.
[0018] FIG. 13 is a flow diagram illustrative of an embodiment of a
routine implemented by the system to display a plurality of virtual
medical devices.
[0019] FIG. 14A is a perspective view of an embodiment of a display
for medical device placement guidance.
[0020] FIG. 14B illustrates the content on a display 1500 in
certain embodiments depicting manipulation of a second medical
device with respect to a target region.
[0021] FIG. 15 is a flow diagram illustrative of an embodiment of a
routine implemented by the system to provide medical device
placement guidance.
[0022] FIG. 16 is a flow diagram illustrative of an embodiment of a
routine implemented by the system to display an altered image of a
virtual medical device after a medical device has been removed from
a predetermined area.
[0023] FIG. 17 is a diagram illustrative of an embodiment of a
needle tip map.
[0024] FIGS. 18A and 18B are diagrams illustrating embodiments of a
rotated view.
DETAILED DESCRIPTION
System Overview
[0025] Implementations disclosed herein provide systems, methods
and apparatus for generating images facilitating medical device
insertion into tissue by an operator. Certain embodiments pertain
to a free-hand medical device guidance system. The system can
provide the healthcare provider full manual control over the
medical device, while making the spatial relationships between the
target, medical device and U/S image more intuitive via a visual
display. Using this visual feedback, the operator can adjust the
medical device's position, orientation, or trajectory.
Particularly, the system can be used to facilitate multiple-medical
device configurations. Certain of the contemplated embodiments can
be used in conjunction with previous systems, such as U.S. patent
application Ser. No. 13/014,587 and U.S. patent application Ser.
No. 11/137,156, each of which is hereby incorporated by reference
in its entirety.
[0026] In some embodiments, a user desires that the medical devices
that are used together (e.g., used simultaneously for treating one
tumor) are parallel to each other. In certain embodiments, the
medical devices are arranged to be equally spaced from the center
of the tumor. In some embodiments, a user desires that the tips of
the medical devices be in the same plane, and all medical device
shafts be perpendicular to that plane. In some embodiments, each
medical device includes a single shaft-like electrode surrounded by
a tube of electrically insulating material. The medical provider
can expose some length of the electrode by sliding the insulating
tube partially into the medical device's handle. In some
embodiments, the length of the exposed electrode can be specified
by the user (e.g. 0.5 cm-2 cm). In some embodiments, the length of
the exposed electrodes can be the same for all medical devices. In
certain embodiments, the healthcare provider exposes the electrode
(by withdrawing the insulating sleeve) before placing the medical
device into the patient. In some embodiments, the power generator
is provided the distances between the medical devices, and the
length of the exposed electrodes. In some embodiments, the medical
devices can move continuously, even after the healthcare provider
has placed them into the proper position in the patient's tissue
because of patient motion from breathing, manual manipulation, etc.
In some embodiments, the medical devices are not parallel to an
image plane.
[0027] The system can aid the healthcare provider in placing the
medical devices. In some embodiments, the system improves the
healthcare provider's ability to place the medical devices level
with each other (e.g., tips are co-planar) and parallel with each
other. The system can also aid the healthcare provider in
determining the number of medical devices to be used and what their
optimal positions are, including: distance from tumor's center;
distance from tumor's extent or boundary; medical device depth;
spacing between medical devices; angle between medical devices;
exposure of the deployable electrodes (or optimal retraction of the
electrode's insulation), etc.
[0028] The system can also help the healthcare provider determine
what healthy tissues are in an ablation zone volume (by displaying
the predicted ablation zone of the multi-medical device
configuration). The system can help the healthcare provider place
the medical devices in the above determined (planned)
configuration. The system can help the healthcare provider
understand how the current configuration (i.e. the way the medical
devices are currently placed) differs from the optimal, acceptable
or pre-planned configurations. The system can output the distance
between medical devices to the power generator, so that the
ablation time, the ablation power and other ablation parameters can
be automatically computed by the ablation generator. The system can
be used for treatment of tumors, fibroids or cysts, with bipolar
radiofrequency medical device ablation, multiple microwave medical
devices, electroporation, and/or electrochemotherapy systems. It
can also be used for nerve or muscle stimulation or sensing
(electrodes in the spine, brain). The system can be used during
open surgery, laparoscopic surgery, endoscopic procedures,
biopsies, and/or interventional radiology procedures.
[0029] The system can be used in conjunction with live
intraoperative ultrasound (U/S), pre-operative CT, or any
cross-sectional medical imaging modality (e.g. MRI, OCT, etc.). In
addition, the system can use a variety of techniques to determine
each medical device's position and orientation. For example, the
system can use the NDI Aurora magnetic system, NDI Polaris optical
system, etc. In some embodiments, a position sensor can be embedded
inside, or affixed to the outside of each medical device, at the
tip, along the shaft, or on the handle. Sensors can be built into
the medical devices or attached after manufacturing, before use.
Each medical device can have its own sensor, which continually
reports position and orientation, or a single sensor can be used
for all the medical devices. In embodiments where one sensor is
used, the healthcare provider can attach the sensor to the
particular medical device that she is intentionally repositioning,
and then, once she has placed that medical device, she would remove
the sensor and attach it to the next medical device she is
repositioning. In some embodiments, the medical devices, U/S probe
and/or laparoscope can be manipulated by the healthcare provider.
In certain embodiments, the system can be used with a robotic
manipulator, where the robot controls the medical devices, U/S
probe and/or laparoscope.
[0030] In some embodiments, the handles of medical devices can have
push-button switches, to allow the user to select a medical device,
indicate a tissue target, etc. The handle can also have an
indicator light to indicate to the users which medical device is
selected. Finally, the handle can have an encoder to detect how
much length of electrode has been exposed by the user, and report
this information to the guidance system and therapeutic
generator
Image Guidance Systems
[0031] FIG. 1A illustrates a first exemplary system for image
management in image-guided medical procedures. FIG. 1B illustrates
a second exemplary system for image management in image-guided
medical procedures. In many respects the embodiments illustrated by
FIGS. 1A and 1B are similar and use similar numbering. Where the
two are different, those differences are noted. The differences
between the two figures can include that, in FIG. 1A, two position
sensing units 110 and 140 are shown, whereas in FIG. 1B, only a
single position sensing unit 110 is shown.
[0032] In some embodiments, position sensing units 110 and 140 can
track surgical instruments, also referred to herein as medical
devices, within a tracking area and provide data to the image
guidance unit. The medical devices can include invasive medical
devices, biopsy needles, ablation needles, surgical needles,
nerve-block needles, or other needles, electrocautery device,
catheters, stents, laparoscopic cameras, or other instruments that
enter a part of the body, and non-invasive medical devices that do
not enter the body, such as ultrasound transducers. The medical
devices can also include medical imaging devices that provide or
aid in the selection of medical images for display. In some
embodiments, the medical imaging device can be any device that is
used to select a particular medical image for display. The medical
imaging devices can include invasive medical devices, such as
laparoscopic cameras, and non-invasive medical devices, such as
ultrasound transducers.
[0033] Although only two surgical instruments 145 and 155 are shown
in FIGS. 1A and 1B, it will be understood that additional surgical
instruments can be tracked and associated data can be provided to
the image guidance unit 130. The image guidance unit 130 can
process or combine the data and show image guidance data on display
120. This image guidance data can be used by a healthcare provider
to guide a procedure and improve care. There are numerous other
possible embodiments of system 100. For example, many of the
depicted modules can be joined together to form a single module and
can be implemented in a single computer or machine. Further,
position sensing units 110 and 140 can be combined and track all
relevant surgical instruments 145 and 155, as discussed in more
detail below and exemplified in FIG. 1B. Additional imaging units
150 can be included, and combined imaging data from the multiple
imaging units 150 can be processed by image guidance unit 130 and
shown on display unit 120. Additionally, two or more surgical
systems 149 can also be included.
[0034] Information about and from multiple surgical systems 149 and
attached surgical instruments 145 (and additional surgical
instruments not shown) can be processed by image guidance unit 130
and shown on display 120. These and other possible embodiments are
discussed in more detail below. Imaging unit 150 can be coupled to
image guidance unit 130. In some embodiments, imaging unit 150 can
be coupled to a second display unit (not shown). The second display
unit can display imaging data from imaging unit 150. The imaging
data displayed on display unit 120 and displayed on second display
unit can be the same or different. In some embodiments, the imaging
unit 150 is an ultrasound machine 150, the movable imaging device
155 is an ultrasound transducer 155 or ultrasound probe 155, and
the second display unit is a display associated with the ultrasound
machine 150 that displays the ultrasound images from the ultrasound
machine 150. In some embodiments, a movable imaging unit 155 can be
connected to image guidance unit 130. The movable imaging unit 155
can be useful for allowing a user to indicate what portions of a
first set of imaging data are to be displayed. For example, the
movable imaging unit 155 can be an ultrasound transducer 155, a
needle or other medical device, for example, and can be used by a
user to indicate what portions of imaging data, such as a
pre-operative CT scan, to show on a display unit 120 as image 125.
Further, in some embodiments, there can be a third set of
pre-operative imaging data that can be displayed with the first set
of imaging data.
[0035] In some embodiments, system 100 comprises a first position
sensing unit 110, a display unit 120, and second position sensing
unit 140 (if it is included) all coupled to image guidance unit
130. In some embodiments, first position sensing unit 110, display
unit 120, and image guidance unit 130 are all physically connected
to stand 170. Image guidance unit 130 can be used to produce images
125 that are displayed on display unit 120. The images 125 produced
on display unit 120 by the image guidance unit 130 can be
determined based on ultrasound or other visual images from the
first surgical instrument 145 and second surgical instrument
155.
[0036] For example, if the first surgical instrument 145 is an
ablation needle 145 and the second surgical instrument 155 is an
ultrasound probe 155, then images 125 produced on display 120 can
include the images, or video, from the ultrasound probe 155
combined with graphics, such as projected medical device drive or
projected ablation volume, determined based on the emplacement of
ablation needle 145. If the first surgical instrument 145 is an
ultrasound probe 145 and the second surgical instrument 155 is a
laparoscopic camera 155, then images 125 produced on display 120
can include the video from the laparoscopic camera 155 combined
with ultrasound data superimposed on the laparoscopic image. More
surgical instruments can be added to the system. For example, the
system can include an ultrasound probe, ablation needle,
laparoscopic camera, cauterizer, scalpel and/or any other surgical
instrument or medical device. The system can also process and/or
display collected data, such as preoperative CT scans, X-Rays,
MRIs, laser scanned 3D surfaces etc.
[0037] The term "emplacement" and the term "pose" as used herein
are broad terms encompassing their plain and ordinary meanings and
may refer to, without limitation, position, orientation, the
combination of position and orientation, or any other appropriate
location information. In some embodiments, the imaging data
obtained from one or both of surgical instruments 145 and 155 can
include other modalities such as a CT scan, MRI, open-magnet MRI,
optical coherence tomography ("OCT"), positron emission tomography
("PET") scans, fluoroscopy, ultrasound, or other preoperative, or
intraoperative 2D or 3D anatomical imaging data. In some
embodiments, surgical instruments 145 and 155 can also be scalpels,
implantable hardware, or any other device used in surgery. Any
appropriate surgical system 149 or imaging unit 150 can be attached
to the corresponding medical instruments 145 and 155.
[0038] As noted above, images 125 produced can also be generated
based on live, intraoperative, or real-time data obtained using the
second surgical instrument 155, which is coupled to second imaging
unit 150. The term "real time" as used herein is a broad term and
has its ordinary and customary meaning, including without
limitation instantaneously or nearly instantaneously. The use of
the term real time can also mean that actions are performed or data
is obtained with the intention to be used immediately, upon the
next cycle of a system or control loop, or any other appropriate
meaning. Additionally, as used herein, real-time data can be data
that is obtained at a frequency that would allow a healthcare
provider to meaningfully interact with the data during surgery. For
example, in some embodiments, real-time data can be a medical image
of a patient that is updated one time per second. In some
embodiments, real-time data can be ultrasound data that is updated
multiple times per second.
[0039] Second surgical instrument 155 can be coupled to second
position sensing unit 140. Second position sensing unit 140 can be
part of imaging unit 150 or it can be separate. Second position
sensing unit 140 can be used to determine the emplacement of second
surgical instrument 155. In some embodiments, first and/or second
position sensing units 110 and/or 140 can be magnetic trackers and
magnetic can be coils coupled to surgical instruments 145 and/or
155. In some embodiments, first and/or second position sensing
units 110 and/or 140 can be optical trackers and
visually-detectable fiducials can be coupled to surgical
instruments 145 and/or 155.
[0040] Images 125 can be produced based on intraoperative or
real-time data obtained using first surgical instrument 145, which
is coupled to first surgical system 149. In FIGS. 1A and 1B, first
surgical system 149 is shown as coupled to image guidance unit 130.
The coupling between the first surgical system 149 and image
guidance unit 130 may not be present in all embodiments. In some
embodiments, the coupling between first surgical system 149 and
image guidance unit 130 can be included where information about
first surgical instrument 145 available to first surgical system
149 is useful for the processing performed by image guidance unit
130. For example, in some embodiments, the first surgical
instrument 145 is an ablation needle 145 and first surgical system
149 is an ablation system 149. In some embodiments, it can be
useful to send a signal about the relative strength of planned
ablation from ablation system 149 to image guidance unit 130 in
order that image guidance unit 130 can show a predicted ablation
volume. In other embodiments, the first surgical system 149 is not
coupled to image guidance unit 130. Example embodiments including
images and graphics that can be displayed are included below.
[0041] In some embodiments, the first position sensing unit 110
tracks the emplacement of first surgical device 145. First position
sensing unit 110 can be an optical tracker 110 and first surgical
device 145 can have optical fiducials attached thereto. The
emplacement of optical fiducials can be detected by first position
sensing unit 110, and, therefrom, the emplacement of first surgical
device 145 can be determined.
[0042] In various embodiments, as depicted in FIG. 1B, a single
position sensing unit 110 can track both first medical device 145
and second medical device 155. In FIG. 1B, in some embodiments,
position sensing unit 110 is a magnetic tracker and is mounted
below a surgical table 180. Such an arrangement can be useful when
the tracking volume of the position sensing unit 110 is dependent
on the location of the position sensing unit, as with many magnetic
trackers. Magnetic tracking coils can be mounted in or on the
medical devices 145 and 155.
[0043] In some embodiments, either or both of the first position
sensing unit 110 and the second position sensing unit 140 can be an
Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE,
trakSTAR, miniBIRD, MotionSTAR, pciBIRD, or Calypso 2D Localization
System and tracking units attached to the first and/or second
medical devices 145 and 155 can be magnetic tracking coils. The
term "tracking unit," as used herein, is a broad term encompassing
its plain and ordinary meaning and includes without limitation all
types of magnetic coils or other magnetic field sensing devices for
use with magnetic trackers, fiducials or other optically detectable
markers for use with optical trackers, such as those discussed
above and below. In some embodiments, the tracking units can be
implemented using optical position sensing devices, such as the
HiBall tracking system and the first and second position sensing
units 110 and 140 can form part of the HiBall tracking system.
Tracking units can also include a GPS device or signal emitting
device that allows for tracking of the position and, optionally,
orientation of the tracking unit. In some embodiments, a signal
emitting device might include a radio-frequency identifier (RFID).
In such embodiments, the first and/or second position sensing unit
110 and 140 can use the GPS coordinates of the tracking units or
can, for example, triangulate the radio frequency signal being
emitted by the RFID associated with tracking units. The tracking
systems can also include one or more 3D mice.
[0044] In some embodiments, either or both of the first position
sensing unit 110 and the second position sensing unit 140 can be an
electromagnetic measurement system (e.g., NDI Aurora system) using
sensor coils for tracking units attached to the first and/or second
surgical devices 145 and 155. In some embodiments, either or both
of the first position sensing unit 110 and the second position
sensing unit 140 can be an optical 3D tracking system using
fiducials. Such optical 3D tracking systems can include the NDI
Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX,
InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak,
IsoTrak, or Claron MicronTracker2. In some embodiments, either or
both of position sensing units 110 and 140 can each be an inertial
3D tracking system comprising a compass, accelerometer, tilt sensor
and/or gyro, such as the InterSense InertiaCube or the Nintendo Wii
controller. In some embodiments, either or both of position sensing
units 110 and 140 can be attached to or affixed on the
corresponding surgical device 145 and 155. In some embodiments, the
position sensing units, 110 and 140, can include sensing devices
such as the HiBall tracking system, a GPS device, or signal
emitting device that would allow for tracking of the position and,
optionally, orientation of the tracking unit. In some embodiments,
a position sensing unit 110 or 140 can be affixed to either or both
of the surgical devices 145 and 155. The surgical devices 145 or
155 can be tracked by the position sensing units 110 or 140. A room
coordinate system reference, such as the display 120 can also be
tracked by the position sensing unit 110 or 140 in order to
determine the emplacements of the surgical devices 145 and 155 with
respect to the room coordinate system. Devices 145 and 155 can also
include or have coupled thereto one or more accelerometers, which
can be used to estimate movement, position, and location of the
devices.
[0045] In some embodiments, the display unit 120 displays 3D images
to a user, such as a healthcare provider. Stereoscopic 3D displays
separate the imagery shown to each of the user's eyes. This can be
accomplished by a stereoscopic display, a lenticular
auto-stereoscopic display, or any other appropriate type of
display. The display 120 can be an alternating row or alternating
column display. Example alternating row displays include the
Miracube G240S, as well as Zalman Trimon Monitors. Alternating
column displays include devices manufactured by Sharp, as well as
many "auto-stereoscopic" displays (e.g., Philips). Display 120 can
also be a cathode ray tube. Cathode Ray Tube (CRT) based devices,
can use temporal sequencing, showing imagery for the left and right
eye in temporal sequential alternation. This method can also be
used projection-based devices, as well as by liquid crystal display
(LCD) devices, light emitting diode (LED) devices, and/or organic
LED (OLED) devices.
[0046] In certain embodiments, a user can wear a head mounted
display in order to receive 3D images from the image guidance unit
130. In such embodiments, a separate display, such as the pictured
display unit 120, can be omitted. The 3D graphics can be produced
using underlying data models, stored in the image guidance unit 130
and projected onto one or more 2D planes in order to create left
and right eye images for a head mount, lenticular, or other 3D
display. The underlying 3D model can be updated based on the
relative emplacements of the various devices 145 and 155, as
determined by the position sensing unit(s), and/or based on new
data associated with the devices 145 and 155. For example, if the
second medical device 155 is an ultrasound probe, then the
underlying data model can be updated to reflect the most recent
ultrasound image. If the first medical device 145 is an ablation
needle, then the underlying model can be updated to reflect any
changes related to the needle, such as power or duration
information. Any appropriate 3D graphics processing can be used for
rendering including processing based on OpenGL, Direct3D, Java 3D,
etc. Whole, partial, or modified 3D graphics packages can also be
used, such packages including 3DS Max, SolidWorks, Maya, Form Z,
Cybermotion 3D, VTK, Slicer, or any others. In some embodiments,
various parts of the needed rendering can occur on traditional or
specialized graphics hardware. The rendering can also occur on the
general CPU, on programmable hardware, on a separate processor, be
distributed over multiple processors, over multiple dedicated
graphics cards, or using any other appropriate combination of
hardware or technique.
[0047] One or more modules, units, devices, or elements of various
embodiments can be packaged and/or distributed as part of a kit.
For example, in one embodiment, an ablation needle, tracking
elements, 3D viewing glasses, and/or a portion of an ultrasound
wand can form a kit. Other embodiments can have different elements
or combinations of elements grouped and/or packaged together. Kits
can be sold or distributed separately from or with the other
portions of the system.
[0048] One will readily recognize that there are numerous other
examples of image guidance systems which can use, incorporate,
support, or provide for the techniques, methods, processes, and
systems described herein.
Depicting Surgical Instruments
[0049] Previous systems do not provide satisfactory image guidance
data. It can often be difficult to discern the content of a 3D
scene from a 2D depiction of it, or even from a 3D depiction of it.
Therefore, various embodiments herein provide image guidance that
can help the doctor better understand the scene, relative
emplacements or poses of object in the scene and thereby provide
improved image guidance.
[0050] FIG. 2 illustrates a virtual rendering 201 of an exemplary
surgical instrument 245 being displayed on a screen 220. In this
case, the surgical instrument displayed is an ablation needle 245.
The wire 246 connecting the ablation needle 245 to an ablation
system is also depicted. Although only one surgical instrument 245
is displayed, it will be understood that multiple surgical devices
can be tracked and displayed simultaneously on screen 220, as
described in greater detail below with reference to FIG. 11-17.
[0051] The virtual surgical instrument 201 can be displayed in a
virtual 3D space with the screen 220 acting as a window into the
virtual 3D space, which can also be referred to as the perspective
view. Thus, as the surgical instrument 245 is moved to the right,
the virtual surgical instrument 201 also moves to the right.
Similarly, if the surgical instrument 245 is rotated 90 degrees so
that the tip of the surgical instrument is pointing towards the
screen 220, the virtual surgical instrument 201 will likewise show
the change in orientation, and show the tip of the virtual surgical
instrument 201 in the background and the other end of the image 201
in the foreground.
[0052] Some models of medical devices have markings such as bands
around the shaft (to indicate distance along the shaft), and a
colored region near the tip to indicate where the radio frequency
or microwave energy is emitted from in the case of an ablation
probe. Healthcare providers performing medical device procedures
are often familiar with these markings and can use them to help
understand the spatial relationship between the medical device and
anatomy. In some embodiments, the make and model of the medical
device 245 is known to the image guidance system and the virtual
medical device displayed (201) in display 220 can resemble medical
device 245. The features of medical devices that can be rendered in
the scene include the overall shape (diameter, cross sectional
shape, curvature, etc.), color, distance markers, visuals or
echogenic fiduciary markers, the state of deployable elements such
as tines, paddles, anchors, resection loops, stiffening or
steerable sleeves, temperature, radiation, light or magnetic field
sensors, lens, waveguides, fluid transfer channels, and the
like.
[0053] The type of medical device being used can be input into the
image guidance system, can be a system default, can be detected by
a camera or other device, can be received as data from an attached
medical device, such as surgical system 149 in FIGS. 1A and 1B, or
the information can be received in any other appropriate manner.
Making the surgical instrument displayed on display 220 resemble
the surgical instrument 245 can help healthcare providers associate
the image guidance data with the real world and can provide more
familiar guidance information to a healthcare provider, thereby
further aiding the healthcare provider in the guidance task. For
example, the healthcare provider can see the familiar markings on
the medical device being displayed on the display 220 and therefore
be familiar with the distance and relative placement of the
displayed medical device with respect to other data, such as a
tumor seen in an ultrasound (not depicted in FIG. 2). This
knowledge of relative placement of items being displayed can help
the healthcare provider move the medical device into place.
[0054] Consider an embodiment in which the virtual surgical
instrument 201 in the display 220 is an ablation needle depicting
the portion of the needle that will perform the ablation, for
example, the portion that emits the radio or microwave energy. If
the display 220 also includes ultrasound data, then the doctor can
be able to find the tumor she wishes to ablate by moving the
ultrasound probe around until she spots the tumor. In various
embodiments, she will be able to see the displayed ultrasound data
and its location relative to the displayed medical device with the
markings. She can then drive the medical device until she sees, on
display 220, that the emitter-portion of the medical device
encompasses the tumor in the ultrasound, also seen on display 220.
When she activates the ablation, she can then be much more certain
that she has ablated the correct portion of the tissue. Various
embodiments of this are discussed more below.
[0055] As another example, consider the physical markings that can
be on the instruments themselves. These markings can help orient a
healthcare provider during use of the instrument. In some
embodiments, the image guidance unit can represent these markings
in the images displayed in the display. For example, certain
ultrasound transducers are built with an orientation mark (e.g., a
small bump) on one side of the transducing array. That mark can
also be shown in the ultrasound image on the scanner's display, to
help the healthcare provider understand where the scanned
anatomical structures shown on screen are located under the
transducer, inside the patient. In some embodiments, the image
guidance system can display a symbolic 3D representation of the
orientation mark both next to the motion-tracked ultrasound slice
(e.g., moving with the displayed ultrasound slice) and next to the
2D ultrasound slice also displayed by the system. An example of
this is displayed in FIG. 5, where a small rectilinear volume
corresponding to a feature on an ultrasound probe is shown both in
proximity to the ultrasound slice displayed in 3D and the
ultrasound slice displayed as a 2D image.
[0056] Other embodiments can track and display other types of
instruments and their features. For example, a healthcare provider
may want to track one or more of a scalpel, a biopsy, a cauterizer
(including an electrocauterizer and Bovies), forceps, cutting loops
on hysteroscopes, harmonic sheers, lasers (including CO.sub.2
lasers), etc. For example, in various embodiments, the following
devices can be tracked and various aspects of their design
displayed on display 220: Olympus.TM. OES Pro Hystero-Resectoscope,
SonoSurg Ultrasonic Surgical System Olympus.TM. GF-UC 160 Endoscope
Wallus.TM. Embryo Transfer Catheter AngioDynamics.RTM.
NanoKnife.TM., VenaCure.TM. laser, StarBurst, Uniblade, Habib.RTM.
Resector Bovie.TM. Electrodes, Covidien Evident.TM., Cool-Tip.TM.
Ablation Antennas, Opti4.TM. Electrodes Microsulis MEA (microwave
endometrial ablation), Acculis Halt.TM. Medical System Optimed
BigLumen Aspiration Catheter Optimed Optipure Stent Central venous
catheterization introducer medical device (such as those made by
Bard and Arrow).
[0057] Once tracked, a healthcare provider is able to see image
guidance data on display 220 that will allow her to know the
relative pose, location, or emplacement of the tracked
instrument(s) with respect to one another or with respect to
imaging data and will be able to see, on display 220, the features
of the instrument rendered in the scene.
Depicting Ablation Volume and Other Instrument Information
[0058] Various embodiments of the systems herein depict as part of
the image guidance data information related to the surgical
instruments. For example, in some embodiments, an image guidance
system such as the systems of FIG. 1A or 1B can illustrate an
expected spherical ablation volume. For example, FIG. 3 shows an
ablation needle 345 which has a darkened portion that indicates
where the radio frequency or microwave energy for ablation will be
emitted. In some embodiments, an image guidance system can display
on display 320 the expected ablation volume 302. The ablation
volume 302 can be shown as a transparent volume, a wireframe volume
(depicted in FIG. 3), as a point cloud of various densities, as an
outline, as a volume, or in any other appropriate manner. Although
only one ablation volume 302 is displayed, it will be understood
that multiple ablation volumes can be displayed for each medical
device 345 that is displayed on the screen 320.
[0059] For some ablation needles, the expected volume of ablated
tissue is neither spherical nor centered at the tip of the medical
device. For example: a Covidien surgical microwave medical device
has an ellipsoidal ablation volume; a Covidien Evident
transcutaneous microwave medical device has a teardrop-like
ablation volume; RFA Medical's bipolar ablation system uses two
medical devices simultaneously, where each medical device has
paddles that deploy after the medical device is inserted inside the
tissue (which one can equate to a canoe's oar). In some
embodiments, the ablation volume for such a medical device is, to a
first approximation, a volume that lies directly between the
paddles of the two medical devices.
[0060] The position and orientation of the volume can be specified
by the placement of a tracked medical device, such as medical
device 345 in FIG. 3. In some embodiments, with single medical
device ablation systems, the volume's approximate size (e.g., girth
and length, if ellipsoidal) can be either specified by the
healthcare provider, or automatically computed by the guidance
system. The ablation volume can be based on numerous parameters
such as the medical device make and model, power and duration
settings of the microwave or radio frequency generator, measured or
estimated temperature and impedance of the target tissue or other
tissue information, a formula, a look-up-table, fixed or default
values, or based on any other appropriate available
information.
[0061] Other instrument information can also be depicted. For
example, if a cauterizer is tracked as part of an image guidance
system, then the cauterization volume can be determined or
estimated and that volume can be displayed. If a laser is tracked
as part of the image guidance system, then the projected laser path
can be determined or estimated and displayed. In embodiments where
multiple medical devices are used, the combined volume can be
shown, as described in greater detail below with reference to FIGS.
12A and 12B.
Depicting Medical Device Placement Trajectory, and Other Prediction
Information
[0062] In certain procedures, the system can provide prediction
information related to the surgical instruments. In the context of
scalpel movement, this can be the location that the scalpel will
hit if a healthcare provider continues to move the scalpel in a
particular direction. In the context of ablation or biopsies, this
can be the projected medical device placement if it is driven along
its central axis, which is also referred to herein as a
longitudinal axis.
[0063] FIG. 4 illustrates both an ablation volume 404 for an
ablation needle 445 and a projected needle drive 403. If a
healthcare provider is driving an ablation needle 445 into tissue
(not pictured), then she can know where the medical device will be
driven. In some embodiments, the projected drive of a medical
device can be depicted on the display 420 and can show the
healthcare provider the projected path 403 that the medical device
will take if it is driven along its central axis. Although the
trajectory of only one medical device is displayed, it will be
understood that the trajectory of multiple medical devices can be
determined and displayed simultaneously on screen 420, as described
in greater detail below with reference to FIGS. 12A, 12B, 13A, and
13B.
[0064] In some embodiments, in order to aid the healthcare provider
in placing or orienting a medical device 445, an image guidance
system, such as that depicted in FIG. 1A or FIG. 1B, can draw a
number of rings about the axis of the medical device shaft,
extrapolated beyond its tip, as depicted in FIG. 4. A healthcare
provider can view and manipulate the position and orientation of
the medical device 445 and its expected drive projection (via its
displayed projected trajectory) before it enters the patient's
tissue. In some embodiments, this is accomplished by the doctor
positioning the virtual rings in the drive projection such that
they are co-incident (or pass through) the ultrasound
representation of a target, such as a tumor that the doctor has
spotted in the ultrasound. This can allow the healthcare provider
to verify that the medical device 445 is properly aimed at the
target and can drive the medical device 445 forward into the tissue
such that it reaches its desired target or destination. For
example, if the doctor identifies a tumor in the ultrasound image
(not pictured in FIG. 4), she can align the ablation needle 445
such that the drive projection rings on display 420 intersect or
otherwise indicate that the medical device, if driven straight,
will reach the tumor.
[0065] The rings can be spaced at regular (e.g., 0.5, 1, or 2 cm)
intervals to provide the healthcare provider with visual or
guidance cues regarding the distance from the medical device tip to
the targeted anatomy. In some embodiments, the spacing of the rings
can indicate other aspects of the data, such as the drive speed of
the medical device, the density of the tissue, the distance to a
landmark, such as the ultrasound data, or any other appropriate
guidance data or property. In some embodiments, the rings or other
trajectory indicators can extend beyond the medical device tip, by
a distance equal to the length of the medical device-shaft. This
way, the user knows if the medical device is long enough to reach
the target--even before the tip enters the patient. That is, in
some embodiments, if the rings do not reach the target with the tip
still outside the body, then the tip won't reach the target even
when the entire length shaft is inserted into the body.
[0066] Other display markers can be used to show trajectory, such
as a dashed, dotted, or solid line, transparent medical device
shaft, point cloud, wire frame, etc. In some embodiments,
three-dimensional rings can be used and provide depth cues and
obscure little of the ultrasound image. Virtual rings or other
virtual markers can be displayed semi-transparently, so that they
obscure less of the ultrasound image than an opaque marker
would.
[0067] Other prediction information can also be displayed. For
example, if a scalpel is being tracked by the image guidance
system, then a cutting plane corresponding to the scalpel can be
displayed (not pictured). Such a cutting plan can be coplanar with
the blade of the scalpel and can project from the blade of the
scalpel. For example, the projected cutting plane can show where
the scalpel would cut if the doctor were to advance the scalpel.
Similar prediction information can be estimable or determinable for
cauterizers, lasers, and numerous other surgical instruments.
Depicting Combinations of Graphics
[0068] As discussed herein, when there are multiple instruments or
devices being used in a procedure, images, graphics, and data
associated with the multiple instruments can be displayed to the
healthcare provider. In some embodiments, as depicted in FIG. 5,
when there are two medical devices 545 and 555 being used and
tracked in a procedure, data, images, and graphics associated with
those two images can be combined and displayed on the same
display.
[0069] FIG. 5 depicts an ablation needle 545 and an ultrasound
probe 555 being used during a procedure. Data associated with each
of the devices 545 and 555 are displayed on the display 520. As
described previously with reference to FIG. 2, the devices 545 and
555 can be displayed in a virtual 3D space with the screen 520
acting as a window into the virtual 3D space and providing a
perspective view. Thus, as the surgical instrument 545 are moved to
the right, the virtual surgical instrument 501 also moves to the
right. Similarly, if the ultrasound probe 555 is moved to the
right, the ultrasound image 504 is also moved to the right. If the
surgical instrument 545 is rotated 90 degrees so that the tip of
the surgical instrument is pointing towards the screen 520, the
system 100 will likewise adjust the virtual surgical instrument 501
to show the change in orientation, and show the tip of the virtual
surgical instrument 501 in the background and the other end of the
virtual surgical instrument 201 in the foreground. Similarly a
rotation of the ultrasound probe 555 will result in a rotation of
the ultrasound image 504.
[0070] The data from two or more devices can be combined and
displayed based on their relative emplacements or poses. For
example, the system 100 can determine an image plane based on the
emplacement information of the ultrasound probe 555. Further, the
ultrasound image 504 can be displayed on the image plane with
respect to a virtual ablation needle 502 on a display 520 in a
manner that estimates the relative emplacements or poses of an
ultrasound probe 555 and ablation needle 545. As illustrated in
FIG. 5, the graphics associated with the virtual ablation needle
502, including the ablation volume 506 and projected drive location
508, are shown spatially located with the oriented planar
ultrasound image 504 on display 520.
[0071] In addition, the display 520 includes an intersection
indicator 510 that indicates the where the virtual ablation medical
device 502 intersects the ultrasound image 504. In some
embodiments, the intersection indicator 510 can be displayed before
the medical device is inserted, thereby allowing the healthcare
provider to see where the medical device will intersect the
image.
[0072] In this image, a tumor 512 appears in the ultrasound image
504 and the virtual ablation needle 502 is shown driven through the
tumor 512. The ablation volume 506 estimates where ablation would
occur if the tissue were ablated at that time. The healthcare
provider can see that the ablation volume 506 appears to cover the
tumor displayed in the ultrasound image.
[0073] Various embodiments can include any combinations of the
graphics described above with reference to FIGS. 2-7 and/or other
graphics. For example, in some embodiments, data related to a
single surgical instrument (such as an ablation needle, ultrasound
probe, etc.) can be presented in more than one manner on a single
display. Consider an embodiment in which device 545 is an ablation
needle and device 555 is an ultrasound transducer. As mentioned
previously, as the medical devices are displayed in a virtual 3D
space, with the screen 520 acting as a window into the virtual 3D
space, if a healthcare provider orients ultrasound transducer 555
such that it is perpendicular to the monitor, the 3D view of the
ultrasound image would show only the edge and the ultrasound image
would not be visible. In some embodiments, the image guidance
system can track the healthcare provider's head using a position
sensor, such as first and/or second position sensing units 110
and/or 140 of FIG. 1A or FIG. 1B. The healthcare provider can then
move her head to the side, so that she sees the ultrasound image
from a different perspective.
[0074] In some embodiments, the image guidance system can
constantly display an additional 2D view of the ultrasound image
505 (in screen space), simultaneous to the 3D depiction of the
procedure, so that the ultrasound image is always visible,
regardless of the orientation in which the healthcare provider
holds the transducer. This is illustrated in FIG. 5. This display
of the ultrasound data can be similar to what a healthcare provider
is accustomed to seeing with traditional ultrasound displays. This
can be useful to provide the healthcare provider with imaging to
which she is accustomed and allows a healthcare provider to see the
ultrasound data regardless of the then-current orientation of the
ultrasound probe with respect to the user.
[0075] In some embodiments, the 2D view 505 of an ultrasound image
is depicted in the upper right corner of the monitor (though it can
be placed in any location). In some embodiments, the guidance
system can automatically (and continually) choose a corner in which
to render the 2D view of the ultrasound image, based on the 3D
position of the surgical instruments in the rendered scene. For
example, in FIG. 5, ablation needle 545 can be held in the
healthcare provider's left hand and the medical device shaft is to
the left of the 3D ultrasound image slice, so that the 2D
ultrasound image 505 in the upper right corner of display 520 does
not cover any of the 3D features of the medical device (or
vice-versa). If the medical device were held in the healthcare
provider's right hand, the virtual medical device shaft would
appear on the right side. To prevent the 2D ultrasound image in the
corner of display 520 from covering the medical device shaft, the
system can automatically move it to a corner that would not
otherwise be occupied by graphics or data.
[0076] In some embodiments, the system attempts to avoid having the
2D ultrasound image quickly moving among corners of the display in
order to avoid overlapping with graphics and data in the display.
For example, a function f can be used to determine which corner is
most suitable for the 2D ultrasound image to be drawn in. The
inputs to f can include the locations, in the screen coordinate
system, of the displayed medical device tip, the corners of the 3D
ultrasound image, etc. In some embodiments, fs output for any given
point in time is independent of fs output in the previous frames,
which can cause the ultrasound image to move among corners of the
display rapidly. In some embodiments, the image guidance system
will filter fs output over time. For example, the output of a
filter g, for any given frame, could be the corner which has been
output by f the most number of times over the last n frames,
possibly weighting the most recent values for f most heavily. The
output of the filter g can be used to determine in which corner of
display 520 to display the 2D ultrasound image and the temporal
filtering provided by g can allow the 2D ultrasound image display
to move more smoothly among the corners of the display 520.
[0077] In some embodiments, other appropriate virtual information
can be overlaid on the 2D ultrasound image as well. Examples
include: an indication of the distance between the medical device's
tip and the point in the plane of the ultrasound image that is
closest to the medical device tip; the cross section or outline of
the ablation volume that intersects with the ultrasound slice;
and/or the intersection point, box, outline, etc. between the
medical device's axis and the ultrasound image plane.
Representing Spatial Relationships
[0078] At times, when three dimensional relationships are depicted
in 2D, or even in 3D, it can be difficult to gauge the relative
positions, orientations, and distances among various objects.
Consider FIG. 5, in which an ablation needle is shown intersecting
an ultrasound image. Depending on the embodiment, it can be
difficult to determine the relative angle of the ablation needle
and the ultrasound image as well as the distances of various
portions of the image plane to the ablation needle.
[0079] In some embodiments, the image guidance system can indicate
spatial relationships with graphical indicators. For example, in
FIG. 6, graphical indicators help indicate the spatial relationship
between a medical device and an ultrasound image plane. These also
provide an indication of the relative angle of the medical device
and the image plane.
[0080] In some unpictured embodiments, the image guidance system
can draw "guidance graphics" in the form of projective lines
between the medical device and the ultrasound slice. These lines
can be perpendicular to the plane of the slice and serve to
indicate the most likely location in the slice where the medical
device will become visible if it is moved to become coplanar with
the slice. Together with stereoscopic head-tracked visualization,
the projective lines help a healthcare provider determine a more
accurate assessment of the location of the medical device with
respect to the ultrasound slice.
[0081] Returning to FIG. 6, in some embodiments, uniform-thickness
lines 602 between virtual medical device and the image plane can be
displayed on display 620. The lines can represent the spatial
relationship with three-dimensional rectangular (or any shape)
medical device projection bars, lines (dashed or solid), etc. In
various embodiments, the projection bars can be drawn perpendicular
to the image, and in such a way that their small edges are aligned
with (or parallel to) either the vertical (FIG. 6) or the
horizontal margins of the ultrasound slice. In some embodiments,
the screen-space size of the projection bars can be variable (e.g.,
distance-dependent) due to perspective. Thus, they can provide
depth cues for the healthcare provider. Further, the staircase
appearance of the bars' end edges at the plane of the slice can be
a further visual cue for the orientation of the medical device with
respect to the image plane.
Representing Non-Intersecting Objects or Images
[0082] When data related to two devices or surgical instruments are
displayed with relative emplacement, it can be difficult to orient
their relative locations if they do not intersect. In some
embodiments, an image guidance system will render relative location
information. The relative location information can be shown with
color (e.g., objects can be rendered in brighter colors if they are
closer), with rendering techniques (e.g., objects can be rendered
with transparency so that one object behind another can be visible,
but visually appear behind the closer object), with geometry (e.g.,
a geometric connector can be shown that will allow the viewer to
discern the relative relationships), or with any other appropriate
technique. FIG. 7 illustrates example geometry and transparency
being used to show relative locations of two objects.
[0083] For example, in some embodiments, if the intersection point
of a medical device 702 is outside of the area of the ultrasound
slice 704, the image guidance system can draw geometry, such as a
line (or rectangle) in the image plane to indicate the relative
positions of the medical device(s) and ultrasound image. This is
depicted in FIG. 7. In some embodiments, the relative locations can
also be represented using vertical and horizontal elements coplanar
with the ultrasound or other image. In some embodiments, using
geometry that is coplanar with the ultrasound image slice can
provide an intuitive understanding of the relative locations of an
image slice and an ablation needle.
Marking Points of Interest
[0084] In certain procedures, healthcare providers desire to keep
track of multiple spots within the volume of the patient or keep
track of a single point or feature while looking at other parts of
the volume. For example, when a healthcare provider is going to
perform an ablation, before inserting any medical devices, the
healthcare provider will often scan the tissues at the procedures
site to find all targets (e.g., tumors) and note other features of
the tissues. Then, later in the procedure, the healthcare provider
can return to the previously identified points-of-interest. For
example, a healthcare provider might first scan the liver and find
seven lesions that she will attempt to ablate. After ablating the
first lesion, she might be required to find the second lesion
again, and so forth. Before finishing the procedure, she might be
required to verify that she has ablated all seven of the lesions
that she identified at the beginning of the procedure. This
constant scanning and rescanning can be time consuming and error
prone. Further, in a procedure where the healthcare provider is
attempting to locate, for example, fluid-filled cysts, once a
medical device pierces the cyst, the fluid can drain out, making
the target difficult or impossible to locate again with
ultrasound.
[0085] In some embodiments, the image guidance system allows the
healthcare provider to mark or keep track of points or features of
interest. In various embodiments, the healthcare provider can mark
the points or features of interest in various ways. For example,
consider a procedure where the doctor is using the image guidance
system with an ablation needle and an ultrasound probe. The doctor
can be able to mark the point by pressing a button on a keyboard or
medical device, by gesturing or issuing a verbal command, or with
any other appropriate method. The point of interest can be marked
at the point where the medical device intersects with the
ultrasound image plane, where the medical device's projection
intersects with the ultrasound image plane, or any other
appropriate relationship (such as at the location of the tip of the
medical device). For example, when the healthcare provider
identifies a point-of-interest within the ultrasound image, she can
point to it using the medical device even if the medical device is
outside the body of the patient. This is depicted in FIG. 8. The
healthcare provider (or assistant) can then press, for example, a
button or foot pedal, which informs the image guidance system to
store the 3D position of this point-of-interest 801. FIG. 8
illustrates an X being displayed where a point of interest 801 has
been marked. In some embodiments, the system can then display the
position of this point-of-interest 801 relative to the ultrasound
plane and the medical device. For example, an X-shaped marker 902
can be displayed on display 920 to show the relative position of
the marked position and the surgical instruments, as depicted in
FIG. 9. In some embodiments, the system can also display a bar that
connects the X marker 902 of the point-of-interest to the nearest
point (or the point to which a normal vector of the image plane
would reach the X), as depicted in FIG. 9. This visually indicates
to the healthcare provider the distance between the ultrasound
image and this point-of-interest. Should the healthcare provider
want to see the point of interest again in the live ultrasound
image, the graphics indicate to where she should move the
ultrasound transducer to view that point in the ultrasound image.
In some embodiments, the image guidance system can also display the
numerical distance (e.g., in mm) between the ultrasound image and
the point-of-interest (not shown).
[0086] Healthcare providers, during some liver ablation procedures,
can manage fifteen points-of-interest, or even more. As depicted in
FIG. 9, in some embodiments, there can also be multiple markers 902
of point of interest simultaneously displayed. The image guidance
system can be able to store and display any number of points of
interest simultaneously. If there is more than one
point-of-interest in view, the image guidance system can display a
number next to each one (not pictured). In some embodiments, in
order to reduce visual clutter if there are many points of
interest, those points which are closer to the ultrasound image
plane are drawn more saliently or vividly (with more bold color and
thicker lines) while the points that are far away are drawn less
saliently (more transparent, blurred, muted colors, etc.).
Additionally, in various embodiments, other representations other
than an X (such as a point, point cloud, sphere, box, etc.) can be
used and multiple markers or locations can be represented with
different markings.
[0087] In some embodiments, the image guidance system stores the
points-of-interests' positions in the position sensing system's
coordinate system. If the position sensing system is fixed to the
image guidance system, then, if the patient or image guidance
system are moved, stored points-of-interest can become incorrectly
located. In some embodiments, this can be remedied via a fiducial
or other detectable feature or item, the pose of which relative to
the tracking system can be continually, continuously, periodically,
or occasionally measured. The fiducial can be attached to the
operating table, the patient's skin, or even embedded into the
tissue itself (e.g., as a magnetic tracking coil), and the
points-of-interest' positions, relative to it, can be stored and
displayed. For example, in a system where magnetic tracking is
used, a magnetic tracking coil can be affixed to the operating
table or patient. In some embodiments, the healthcare provider can
draw the annotations.
[0088] FIGS. 10A-10D illustrate examples of image annotation in
image-guided medical procedures. FIGS. 10A-10D show a
representation on a computer screen 1020 of an annotation being
made with a medical device (represented on the display 1020 as
medical device 85). The medical device can be used to annotate an
image 1056. FIG. 10A illustrates the manipulation of a medical
device 85 pointing at an image 1056 (e.g., an ultrasound image
1056). The operator can make an annotation by moving the medical
device 85 through space in order to draw curve 1071 on image 1056.
In certain embodiments, arrow 1074 can indicate the direction that
the operator plans to or will draw in the future. In some
embodiments, arrow 1074 is not displayed. Location indicator 1080
can represent the place on image 1056 currently pointed to by
medical device 85. Location indicator 1080 can be any appropriate
indicator such as an "X," an arrow, a differently-colored area,
etc. In FIG. 10B the operator has further moved medical device 85
in order to complete the annotation 1071 on image 1056. As is
depicted in FIG. 10B, the indicator 1080 of the intersection
between the axis of the medical device 85 and the image 1056 has
now reached the lower-right quadrant of the image 1056.
[0089] The image 1056 can be associated with a medical device, such
as an ultrasound transducer (not pictured in FIGS. 10A-10D). The
image 1056 can be an ultrasound image 1056, or the image 1056 can
be a slice or image from other 3D visualizable medical data such as
is described above.
[0090] The annotation 1071, although it has been drawn on an image
1056, can be located in the virtual 3D space--defined by the
placement of the image 1056 and the annotation 1071. FIG. 10C
depicts image 1056, associated with the ultrasound transducer
turned or rotated about its vertical axis (axis not depicted in
FIG. 10C). Therefore, part of the annotation 1071 is depicted in
front of the image 1056, and part of the annotation 1073 is behind
the image 1056, thus illustrating the existence in the virtual 3D
space of the annotation 1071/1073. The location and display of
annotations in the virtual 3D space allow an operator to make an
annotation for a feature (e.g., a tumor, cyst, or vein), and allow
her to locate that feature again later.
[0091] FIG. 10D illustrates that an operator can make a second
annotation 1072 on the image 1056. Part of the first annotation
1071 is in front of the image 1056 and part 1073 is behind. By
manipulating the pose of the image 1056 (e.g. by manipulating the
ultrasound transducer), the operator can choose new locations
within 3D space for annotations. As noted above, the annotations
can be for a blood vessel, tumor, or any other object or location
of interest for the operator. There need not even be a particular
object in the medical image that the operator is annotating. The
operator can, for example, sign her name or write a note. For
example, an operator can circle or make marks near multiple tumors,
trace a line such as annotation 1071 along a vein or artery, etc.
In some embodiments, if the operator moves image 1056 during
annotation, the operator can make non-planar annotation (see, e.g.,
FIGS. 2 and 3). As such, the operator can make a sphere or other
non-planar annotation in order to annotate the volumetric aspects
of a feature of interest. For example, the operator can draw the
outline of a sphere around a tumor or cyst.
Multiple Medical Device Tracking
[0092] FIG. 11 is a diagram illustrating various features of
multiple medical devices 1102, 1104 that can be tracked by the
system. Although two medical devices are illustrated, it will be
understood that the system can track and/or display fewer or more
medical devices as desired. In the illustrated embodiment, each
medical device has a tip 1106A, 1106B, an exposed electrode 1108A,
1108B, an insulating tube 1110A, 1110B, and are associated with one
or more tracking units (not shown), each of which are described in
greater detail above.
[0093] Using the emplacement information received from the tracking
units, the system can calculate the emplacement of each medical
device within a predefined area as well as the relative emplacement
with respect to each other. The system can also determine the
longitudinal axis 1112A, 1112B (also referred to as the central
axis) of the medical devices 1102, 1104 and use the longitudinal
axes to calculate the trajectories, of each medical device, angle
differences between the medical devices, etc. This data can be used
to generate useful information for display for the healthcare
provider.
[0094] As mentioned previously, in some medical procedures that use
multiple medical devices, it is desirable to have the medical
devices on the same plane (e.g., the tips on the same plane) and
parallel with each other. The system described herein can aid a
healthcare provider in placing the medical devices so that they are
level and parallel with each other.
[0095] During, or prior to, a medical procedure, one of the medical
devices can be selected as the foundational medical device. In the
illustrated embodiment of FIG. 11, medical device 1102 is selected
as the foundational medical device. In some embodiments, the
foundational medical device is selected as the first medical device
placed in the tissue, and secondary medical devices (e.g., medical
device 1104 in the illustrated embodiment) are placed thereafter.
In certain embodiments, the foundational medical device is 1102
selected by the user and the remaining medical devices are the
secondary medical device 1104.
[0096] Once a foundational medical device is selected, the system
can calculate one or more foundational planes, and can generate
foundational plane indicators for the different foundational
planes. As used herein, a foundational plane is a plane that is
perpendicular to a trajectory of the foundational medical device
and intersects at least one point of the foundational medical
device and/or intersects at least one point in the trajectory of
the foundational medical device. In some embodiments, the
trajectory of the foundational medical device is determined by the
longitudinal axis of the foundational medical device.
[0097] Accordingly, a variety of foundational planes can be
calculated and used as described herein. For example, the
foundational tip plane 1114 is a foundational plane that intersects
a tip of the foundational medical device 1102. As another example,
the foundational electrode plane 1116 is a foundational plane that
intersects the electrode of the foundational medical device 1102.
In some embodiments, the foundational electrode plane intersects
the electrode at a location where the exposed electrode 1108A ends,
such as where the exposed electrode meets the insulated tube 1110A
or handle. In certain embodiments the foundational electrode plane
1116 intersects the exposed electrode 1108A at any location.
[0098] The emplacement information can also be used to calculate
various distances between the foundational medical device and other
medical devices and between the foundational planes and the medical
devices. In some embodiments, the emplacement information can be
used to calculate the relative distance 1118 between the tips
1106A, 1106B of the medical devices 1102, 1104.
[0099] In certain embodiments, the emplacement information can be
used to calculate the horizontal distance 1120 between the tips
1106A, 1106B of the medical devices 1102, 1104. The horizontal
distance 1120 can represent the distance between the tips 1106A,
1106B if the tips were (or are) level (e.g., on the same plane,
such as the foundational tip plane). In some embodiments, the
horizontal distance can be calculated by determining the distance
between the tip 1106A of the foundational medical device 1102 and
the location of the tip 1106B of the secondary medical device 1104
if the secondary medical device 1104 were on the foundational tip
plane 1114.
[0100] The emplacement information can also be used to calculate
the vertical distance 1122 between the tips 1106A, 1106B of the
medical devices 1102, 1104. The vertical distance can 1122
represent the distance one tip is in front of or behind the other
tip (e.g., how far the tip of one medical device is from the
foundational tip plane). In some embodiments, the vertical distance
can be calculated by determining the distance between the tip 1106B
of the secondary medical device 1104 and the foundational plane
1114. This information can be used to determine whether the medical
devices 1102, 1104 are level (e.g., whether the tips are on the
same plane, such as the foundational tip plane).
[0101] The emplacement information can also be used to calculate
the relative difference in degrees between the medical devices
1102, 1104 (e.g., how many degrees from being parallel). For
example, the system can compare the longitudinal axis 1112A of the
foundational medical 1102 with the longitudinal axis 1112B of the
secondary medical device 1104 to determine how many degrees
difference there is between the two. The difference in degrees can
be displayed as a number or graphical indicator. For example, the
system can provide projective lines as discussed previously and
also in greater detail below with reference to FIGS. 12C and 12D to
indicate the difference in degrees between medical devices. The
number or graphical indicator can aid a user in placing the needles
parallel to each other. This can be used in conjunction with the
foundational plane information to place the needles level (e.g.,
the tips on the same plane) and parallel with each other.
[0102] The emplacement information can also be used to determine a
target axis 1124 for the secondary medical device 1104. In some
embodiments, the target axis 1124 can be the axis at which the
secondary medical device 1104 is to be placed. The target axis 1124
location can be based on user input entered prior to or during the
procedure, such as where the user wants the secondary medical
device to be placed for a medical procedure and/or dynamically
determined based on the emplacement of the foundational medical
device 1102. In certain embodiments, the target axis 1124 is a
predetermined distance from and parallel to the longitudinal axis
1112A of the foundational medical device 1102. The predetermined
distance can be selected by a user or dynamically determined by the
system 100 based on the length of the exposed electrodes of the
medical devices, a model of the ablation or biopsy parameters of
the medical devices (e.g. a lookup table), 4) tumor size, etc.
[0103] In some embodiments, the system can determine relative
spatial indicators that indicate the relative distance between
portions of the secondary medical device 1104 and the target plane
or target axis (e.g., the longitudinal axis 1112A of the
foundational medical device 1102, location of the preferred
placement of the secondary medical device 1104, etc.). In certain
embodiments, the relative spatial indicators can indicate the
distance from portions of the secondary medical device 1104 to
corresponding portions of the foundational medical device 1102.
[0104] Once the system determines one or more of the parameters
described above, it can cause a display device to display those
parameters. In some embodiments, one or more of the determined
parameters are displayed for each medical device that is being
tracked. In certain embodiments, the system displays one or more
parameters of a selected medical device (or multiple selected
devices) that is being tracked. In some embodiments, the one or
more parameters of the selected medical device are displayed in
conjunction with the foundational medical device.
[0105] A user can select the selected medical device using any
number of different inputs (e.g., touchscreen, keyboard, mouse,
foot pedal, button, etc.). Once the selected medical device is
selected, the system can cause the display device to display one or
more of the associated parameters. For example, when a user is
attempting to place a medical device during a procedure, she can
press a button that causes the medical device to be the selected
medical device. In response, the system can display one or more
parameters associated with the selected medical device, such as,
but not limited to, trajectory, intersection indicators, relative
spatial indicators, etc., as will be described in greater detail
below with reference to FIGS. 12A-12D.
[0106] In addition, the system can use the emplacement information
to determine a perspective view of the virtual 3D space. For
example, the system can use any one or a combination of the
following points as the center of perspective for the perspective
view: an image (e.g., center of an ultrasound image); the center of
the foundational needle; the center of the exposed electrode 1108A
of the foundational medical device 1102, the center of the exposed
electrode 1108B of the selected secondary medical device 1104 (or
other secondary medical device); the location between the
foundational medical device 1102 and the selected secondary medical
device 1104; the center of the exposed electrode of a non-selected
secondary medical device; the center of all medical devices
selected thus far in the procedure; the center of all medical
devices within a distance from the foundational medical device; the
center of all medical devices.
[0107] Using the emplacement information and calculated data, the
system can provide a user various indicators to aid in the
placement of the medical devices, as will be described in greater
detail below with reference to FIGS. 12A-17. The system can
determine, calculate and/or display any of the emplacement
information, or any combination of the emplacement information
described herein in an embodiment.
Rendering with Multiple Medical Devices
[0108] FIGS. 12A-12D illustrate embodiments of displaying image
guidance data for multiple medical devices. In the illustrated
embodiments of FIGS. 12A and 12B, the display 1200 includes a 2D
image display area 1202, as described in greater detail above with
reference to FIG. 5, a 3D image display area 1204, and a relative
emplacement measurement display area 1206. However, it will be
understood that in some embodiments, fewer or more of the areas
described can be included. For example, in certain embodiments any
one or any combination of the areas mentioned above can be included
in the display 1200.
2D Image Display Area
[0109] The 2D image display area 1202, described in greater detail
above with reference to FIG. 5, can include an image 1223 and image
guidance cues. The image 1123 can include any one of, or any
combination of, an ultrasound image or images, ultrasound video,
MRI, CT scan images, and/or any other image. In some embodiments,
the image 1223 in the 2D image display area 1202 is the same as, or
identical to, the image data provided in the 3D image display area
1204. In addition, the 2D image display area 1202 can include some
or all of the image guidance cues and virtual medical devices
described in greater detail below with reference to the 3D image
display area 1204. In the illustrated embodiment of FIG. 12A, the
2D image display area 1202 includes trajectory indicators 1218,
1222 and an image plane intersection indicator 1226 overlaid onto
an ultrasound image.
3D Image Display Area
[0110] The 3D image display area 1204 can represent a virtual 3D
space that corresponds to an actual 3D space that is being tracked.
In the illustrated embodiment, the 3D image display area 1204
includes a virtual medical imaging device 1208, an image 1210, as
described in greater detail above with reference to FIGS. 2 and
5.
[0111] Furthermore, the 3D image display area 1204 can include
multiple virtual medical devices 1212, 1214, 1216, image guidance
cues (e.g., trajectory indicators 1218, 1220, 1222, image plane
intersection indicators 1224, 1226, foundational plane indicators,
1228, 1330, and foundational plane intersection indicator 1232), a
patient orientation indicator 1234, and medical provider indicator
1236.
[0112] Although three medical devices 1212, 1214, 1216 are
displayed, it will be understood that fewer or more medical devices
can be used and displayed as desired. In addition, it will be
understood that in some embodiments not all of the features shown
in FIG. 12A are present in the 3D image display area 1204. For
example, in some embodiments, any one or any combination of the
features described above can be included in the 3D image display
area 1204.
[0113] The multiple virtual medical devices 1212, 1214, 1216, can
be implemented by tracking emplacement information of multiple real
medical devices located within a predetermined area. The
predetermined area can correspond to a location of a medical
procedure, the location of a patient, the location of tracking
units, the range of position sensing units, a surgical table, an
area corresponding to a virtual 3D area displayed on a display etc.
Tracking emplacement information for a medical device is described
in greater detail above with reference to FIGS. 1A and 1B. For
example, a tracking unit can be associated with each medical device
and provide emplacement information for the medical device. The
system can use the emplacement information of the medical devices
to generate the display illustrated in FIG. 12A. Furthermore, as
described previously with reference to FIGS. 2-10, any one or any
combination of cues can be displayed for each of the medical
devices 1212, 1214, 1216.
[0114] In some embodiments, each medical device and its associated
image guidance cues (e.g. trajectory rings, intersection square,
text, etc.) can be associated with a color. In some embodiments,
each medical device with its associated image guidance cues is
associated with a different color. For example, the first medical
device 1212 and image guidance cues related to it can be drawn in
pink, a second medical device 1214 and its associated image
guidance cues can be drawn in green, and a third medical device
1216 and its associated image guidance cues can be drawn in blue.
It will be understood that any combination of colors can be used as
desired. Furthermore, the medical devices and associated image
guidance cues can be distinguished based on different patterns or
markings. For example, the system can cause the medical devices to
have zigzags, lines, be bolded, flicker, etc.
Foundational Plane Indicators
[0115] In some embodiments, the image guidance cues can include
foundational plane indicators, such as foundational plane
indicators 1228, 1230 and foundational plane intersection
indicators, such as foundational plane intersection indicator 1232.
The foundational plane indicators 1228, 1230 can indicate a
location of a foundational plane and/or indicate a location where
secondary medical devices are to be placed in order to be level
with the foundational medical device. In some embodiments, the
foundational plane indicators 1228, 1230 can also indicate how the
medical devices are to be placed so they are parallel to each
other. The foundational plane intersection indicators (e.g.,
foundational plane intersection indicator 1232) can indicate a
location where the trajectory of a medical device intersects a
foundational plane.
[0116] In the illustrated embodiment of FIG. 12A, the first medical
device 1212 is selected as the foundational medical device. As
mentioned previously, the foundational medical device can be the
first medical device placed in the tissue and/or selected by the
user. Once the foundational medical device is selected, the system
can calculate one or more foundational planes, as described in
greater detail above with reference to FIG. 11, and can generate
foundational plane indicators. In the illustrated example of FIG.
12A, the 3D image display area 1204 includes a foundational tip
plane indicator 1228 and a foundational electrode plane indicator
1230. However, it will be understood that in some embodiments only
one foundational plane indicator is displayed. Furthermore, in
certain embodiments, three or more foundational plane indicators
can be displayed.
[0117] In the illustrated embodiment, the foundational tip plane
indicator 1228 extends from the tip of the foundational medical
device 1212 to locations on the foundational tip plane where the
tips of the secondary medical devices 1214, 1216, are to be placed.
Accordingly, a user can use the foundational tip plane indicator
1228 to identify the location where the secondary medical devices
1214, 1216 are to be located to be level with the foundational
medical device 1212. In certain embodiments, the foundational
electrode plane indicators can indicate the location where the
trajectories of the secondary medical devices intersect the
foundational plane.
[0118] In some embodiments, the desired location for the tips of
the secondary medical devices on the foundational tip plane is
determined based on user input. For example, the user can indicate
at what distances and locations the secondary medical devices are
to be placed with respect to the foundational medical device. Based
on the input, the system can determine the appropriate shape and
size of the foundational tip plane indicator 1228. In certain
embodiments, the system dynamically calculates the location for the
tips of the secondary medical devices on the foundational tip plane
based on an identified object, such as a tumor, fibroid, etc., the
size of the identified object, the number of medical devices to be
used, electrical power specifications of the medical devices,
etc.
[0119] In the illustrated embodiment of FIG. 12A, the foundational
tip plane indicator 1230 creates a triangular shape because three
medical devices are used. However, it will be understood that the
shape created by the foundational tip plane indicator 1228 can be
any shape or line, and in some embodiments, is based at least in
part on the number of medical devices used. For example, the
foundational tip plane indicator 1228 can create a line when two
medical devices are used (or when one medical device is selected as
described in greater detail below in FIGS. 12B and 12C), a
quadrilateral (e.g., square, rectangle, parallelogram, kite, etc.)
when four medical devices are used, a pentagon when five medical
devices are used, a hexagon when six medical devices are used,
etc.
[0120] The foundational electrode plane indicator 1230 extends from
the electrode of the foundational medical device 1212 to locations
on the foundational electrode plane where the electrodes of
secondary medical devices are to be placed. In the illustrated
embodiment of FIG. 12A, the foundational electrode plane indicator
1230 extends from a location on the foundational medical device
1212 where the electrode ends. However, it will be understood that
the foundational electrode plane indicator 1230 can extend from any
location on the foundational medical device. As described above,
with reference to the foundational tip plane indicator 1230, the
foundational electrode plane indicator 1230 location and shape of
the foundational electrode plane indicator 1230 can be determined
based on user input, the number of medical devices used, and/or
dynamically.
[0121] The foundational plane intersection indicator 1232 can be
used to indicate where the trajectory of a medical device
intersects with a foundational plane, or where a medical device
will be if it is moved forward. In the illustrated embodiment of
FIG. 12A, the display 1200 includes foundational plane intersection
indicator 1232 indicating the intersection of the foundational tip
plane and the trajectory of the medical device 1216. However, it
will be understood that a foundational plane intersection indicator
can be used to indicate the intersection of any foundational plane
with a trajectory of any medical device.
[0122] In some embodiments, the guidance cues (e.g., trajectory
indicators, intersection indicators, etc.) associated with one or
more medical devices are displayed simultaneously. In certain
embodiments, the guidance cues associated with each medical device
are displayed only when the associated medical device is selected.
For example, in the illustrated embodiment of FIG. 12A, the medical
device 1216 is selected. Accordingly, the foundational plane
intersection indicator 1232 indicates the intersection of the
trajectory of the medical device 1216 and the foundational tip
plane, and the image plane intersection indicator 1226 indicates
the intersection of the trajectory of the medical device 1216 and
the image plane.
Patient and Provider Indicators
[0123] The patient orientation indicator 1234 can indicate the
orientation of the patient with respect to the perspective view of
the 3D image display area 1204. In the illustrated embodiment of
FIG. 12A, the patient orientation indicator 1234 indicates that the
head of the patient is on the right side with respect to the image
1210. In addition, in the illustrated embodiment of FIG. 12A, the
patient orientation indicator 1234 is implemented as a human
figure, however, it will be understood that the patient orientation
indicator 1234 can be implemented as any number of graphical
indicators, such as a line, an arrow, a shape, etc. For example, an
arrow can point in the direction of the head of a patient, or a
shape or letter can be used to indicate the location of the head,
feet, or other part of the patient. Furthermore, in some
embodiments, the 3D image display area 1204 does not include the
patient orientation indicator 1234.
[0124] In some embodiments, the display 1200 can include a provider
indicator 1236 that indicates the location of a medical provider
with respect to the patient. The provider indicator can be
implemented as a dot, shape, color, arrow, line, or any other
graphical indicator to indicate the location of the medical
provider with respect to the patient. The system can determine the
location of the provider based on user input and/or dynamically
based on one or more tracking units associated with the medical
provider.
[0125] In certain embodiments, the location of the medical provider
coincides with the perspective view of the virtual 3D space. In the
illustrated embodiment of FIG. 12A, the provider indicator
indicates that the medical provider is located in front of the
patient with respect to the perspective view. With reference to
FIG. 12C, the provider indicator 1236 indicates that the medical
provider is located behind the patient with respect to the
perspective view.
Emplacement Measurement Display Area
[0126] The emplacement measurement display area 1206 can provide a
user with information regarding the emplacement of the medical
devices. In some embodiments, the emplacement measurement display
area 1206 provides relative emplacement information of the medical
devices. For example, the emplacement measurement display area 1206
can indicate relative distances between the medical devices,
relative degrees between the medical devices, etc. The relative
distances between the medical devices can include the distance
between the ends of medical devices, the distance between other
corresponding portions of the medical devices, the horizontal
distance between corresponding portions of medical devices (e.g.,
the distance between a tip of the foundational medical device and
the tip of the second medical device if the tip of the second
medical device was on the foundational tip plane), the vertical
distance between corresponding portions of medical devices (e.g.,
the distance between the foundational tip plane and the tip of a
second medical device), etc. The relative degrees between the
medical devices can include the difference in degrees between the
longitudinal axis of one medical device and the longitudinal axis
of another medical device or some other axis of the medical
devices. The emplacement measurement display area 1206 can include
any one of or any combination of the distances and/or degrees
described herein.
[0127] In the illustrated embodiment of FIG. 12A, the emplacement
measurement display area 1206 includes the relative emplacement
information of the medical devices 1212, 1214, 1216, with respect
to one another. For example, the emplacement measurement display
area 1206 indicates that the tips of the medical devices 1214
(referenced by the `2`) and 1216 (referenced by the `3`) are 1.8 cm
apart and are parallel. The emplacement measurement display area
1206 also indicates that the tips of the foundational medical
device 1212 (referenced by the `1`) and the medical device 1216
(referenced by the `3`) are 2.1 cm apart and 1 degrees off from one
another (i.e., there is a 1 degree difference between the
longitudinal axis of the foundational medical device 1212 and the
medical device 1216). The emplacement measurement display area 1206
further indicates that the tips of the foundational medical device
1212 (referenced by the `1`) and the medical device 1214
(referenced by the `2`) are 2.8 cm apart and 2 degrees off from one
another. As mentioned previously, any one of, or any combination
of, the emplacement information described previously can be
included in the emplacement measurement display area 1206. For
example, the emplacement measurement display area 1206 can include
the relative horizontal difference between portions of the medical
devices and/or the relative vertical distances between portions of
the medical devices, etc.
[0128] In some embodiments, the text indicating the distance
between the tips of a pair of medical devices can be drawn in a
first color (e.g., white) if it satisfies a threshold distance
(e.g., 2 cm), and a second color (e.g., red) it does not.
Similarly, the angles can be drawn in the first color if they meet
a threshold angle (e.g., 10 degrees), and the second color if they
do not. It will be understood that other color schemes can be used
as well. In some embodiments, the medical device numbers can be
included next to the distances and angles, and in certain
embodiments, the number of the medical device can be color-coded
similar to the image guidance cues discussed previously.
Relative Location Indicators
[0129] As illustrated in FIG. 12B, in certain embodiments, relative
location indicators, such as relative location indicators 1238,
1240, can be used in place of, or in any combination with, one or
more foundational plane indicators, such as foundational plane
indicators 1228, 1230 of FIG. 12A. The relative location indicators
1238, 1240 can indicate the relative locations of corresponding
locations of the medical devices. For example, the relative
location indicators 1238, 1240 can indicate the relative locations
between the ends of the medical devices (e.g., location indicators
1238), the exposed electrodes of the medical devices (e.g.,
location indicators 1240) or other locations of the medical
devices. Furthermore, the relative location indicators can be
implemented as any graphical indicator between the corresponding
locations of the medical devices. For example, the relative
location indicators can be implemented as lines, bars, etc. between
the corresponding locations of the medical devices. Furthermore,
the shape of the location indicators can be any shape, as described
in greater detail above with reference to FIG. 12A.
[0130] In the illustrated embodiment of FIG. 12B, the relative
location indicators include a relative tip location indicator 1238
and a relative electrode location indicator 1240, and are
implemented as bars between the corresponding locations of the
medical devices. The relative tip location indicator 1238 indicates
the relative location of the tips of each of the medical devices
1212, 1214, 1216 with respect to one another and the relative
electrode location indicator 1240 indicates the relative location
of the ends of the exposed electrodes of each of the medical
devices 1212, 1214, 1216 with respect to one another. It will be
understood that in some embodiments, any one of, or any combination
of, the relative tip location indicator 1238 and a relative
electrode location indicator 1240 can be included in the display
1200. Furthermore, it will be understood that other relative
location indicators can be used in place of, or in any combination
with, the relative tip location indicator 1238 and/or the relative
electrode location indicator 1240.
Multiple Medical Device Guidance--Relative Spatial
Relationships
[0131] When multiple medical devices are depicted in 2D or 3D, it
can be difficult to determine when the medical devices are
parallel. Similarly, it can be difficult to determine the relative
angle between a longitudinal axis of the foundational medical
device and the longitudinal axis of another medical device. It can
also be difficult to determine the distance between various
portions of the foundational medical device and corresponding
portions of a secondary medical device.
[0132] FIGS. 12C and 12D are diagrams illustrating the display 1200
with graphical indicators that indicate spatial relationships. For
simplicity, the graphical cues described above with reference to
FIGS. 12A and 12B are not shown. However, it will be understood,
any one or any combination of the features described with reference
to
[0133] FIGS. 12A and 12B can be included as part of the embodiments
described herein with reference to FIGS. 12C and 12D. Furthermore,
as mentioned previously, in some embodiments only some of the
features described herein are present. For example, in some
embodiments, the display 1202 only includes the 3D image display
area 1204.
[0134] Furthermore, in the illustrated embodiment of FIG. 12C, the
medical device 1216 is selected for placement. Based on this
selection, only the guidance cues associated with the medical
device 1216 are displayed. However, it will be understood that
similar guidance cues can be displayed for the other secondary
medical device 1214, as illustrated in FIG. 12C. Furthermore,
although illustrated as being displayed separately, it will be
understood that the guidance cues for more than one secondary
medical device and/or the foundational medical device can be
displayed simultaneously.
[0135] In the illustrated embodiment, the guidance cues include
relative spatial indicators 1250 and foundational plane indicators
1252, 1254. The relative spatial indicators 1250 indicate the
distance between portions of a longitudinal axis of the secondary
medical device 1216 and portions of a target axis or target plane.
In some embodiments, the target axis can be the axis at which the
secondary medical device 1216 is to be placed. The target axis can
be based on user input, such as where the user wants the secondary
medical device 1216 to be placed and/or dynamically based on the
emplacement of the foundational medical device 1212. In certain
embodiments, the target axis or target plane is parallel to the
longitudinal axis of the foundational medical device 1212. In some
embodiments the target plane is the image plane, described
previously.
[0136] In some embodiments, the relative spatial indicators 1250
indicate the relative distance between portions of the secondary
medical device 1216 and the longitudinal axis of the foundational
medical device 1212. In certain embodiments, the relative spatial
indicators 1250 can indicate the distance from portions of the
secondary medical device 1216 to corresponding portions of the
foundational medical device 1212.
[0137] As the secondary medical device 1216 is moved closer to, or
farther away from, the target axis or target plane, the spatial
indicators 1250 can shorten or lengthen, respectively. If the
secondary medical device 1212 is placed at the target axis or
target plane, the spatial indicators 1250 can disappear. In some
embodiments, if the secondary medical device 1212 is parallel to
the target axis or target plane, the spatial indicators 1250 can be
equal in length. However, if the secondary medical device 1216 is
angled with respect to the target axis, the spatial indicators 1250
can have different lengths.
[0138] In the illustrated embodiment of FIGS. 12C and 12D, the
spatial indicators 1250 are shown as rectangular bars, however, it
will be understood that the spatial indicators 1250 can be
implemented as any one of, or a combination of, bars, dashed lines,
solid lines, symbols, etc.
[0139] In the illustrated embodiment of FIG. 12C, the foundational
plane indicators 1252, 1254 include a foundational tip plane
indicator 1252 and a foundational electrode plane indicator 1254.
It will be understood that in some embodiments, only one
foundational plane indicator is displayed and in certain
embodiments more than two foundational plane indicators are
displayed.
[0140] The foundational plane indicators 1252, 1254 are similar to
the foundational plane indicators 1228, 1230 described above with
reference to FIGS. 12A and 12B. However, as illustrated, the
foundational plane indicators 1252, 1254 are graphical bars instead
of shapes and indicate the location of the foundational plane with
respect to the selected medical device (e.g., the medical device
1216) and do not indicate the location of the foundational plane
with respect to the other secondary medical device (medical device
1214). However, as illustrated in FIG. 12D, when the secondary
medical device 1214 is selected, corresponding foundational plane
indicators 1256, 1258 can be displayed. As mentioned previously,
the graphical cues associated with the different medical devices
can be displayed simultaneously.
[0141] In some embodiments, the foundational plane indicators 1252,
1254 can indicate when the secondary medical device 1216 is
parallel with the foundational medical device 1212 and/or indicate
when the secondary medical device 1216 is level with the
foundational medical device 1212. For example, the end of each
foundational plane indicator 1252, 1254 can include a line that
indicates at what angle the medical device is to be placed in order
to be parallel to the target plane or target axis. In certain
embodiments, a number indicating the relative difference in degrees
can be provided as part of the foundational plane indicators 1252,
1254.
[0142] It will be understood that any one, or any combination, of
the embodiments described above with respect to the foundational
plane indicators of FIGS. 12A and 12B can be used in place of, or
in conjunction with, any of the embodiments of the foundational
plane indicators 1252, 1254 described above with reference to FIGS.
12C and 12D.
Multiple Medical Device Guidance Routine
[0143] FIG. 13 is a flow diagram illustrative of an embodiment of a
routine 1300 implemented by the system 100 to display a plurality
of virtual medical devices. One skilled in the relevant art will
appreciate that the elements outlined for routine 1300 can be
implemented by one or more computing devices/components that are
associated with the system 100, such as the position sensing units
110, 140, the image guidance unit 130, surgical system 149, and/or
imaging unit 150. Accordingly, routine 1300 has been logically
associated as being generally performed by the system 100. However,
the following illustrative embodiment should not be construed as
limiting.
[0144] At block 1302, the system 100 receives emplacement
information of a plurality of medical devices within a
predetermined area. In some embodiments, the medical devices are
invasive medical devices, such as ablation or biopsy needles,
catheters, etc. As described previously, the medical devices, such
as needles, can include tips, electrodes, and handles. In certain
embodiments, the medical devices are non-invasive medical devices.
In some embodiments, the medical devices are medical imaging
devices, such as ultrasound transducers and/or laparoscopic
cameras.
[0145] As described in greater detail above with reference to FIGS.
1A and 1B, each medical device can be associated with a tracking
unit that provides emplacement information, such as position and
orientation information. As described previously, the tracking
units can be affixed to or implanted into the medical devices.
Using the emplacement information of the tracking unit and known
characteristics of the medical devices, the emplacement information
can be determined. Accordingly, by receiving emplacement
information from a tracking unit, the system 100 can also receive
and/or determine emplacement information of the medical device.
[0146] At block 1304, the system 100 calculates a viewing angle in
a virtual 3D space of a plurality of virtual medical devices. The
virtual medical devices can correspond to the medical devices that
are being tracked. Further, the viewing angle can be based at least
on the emplacement information of the plurality of medical devices.
In some embodiments, the system calculates a viewing angle for each
of the medical devices. In certain embodiments, to calculate the
viewing angle, the system determines the emplacement of the medical
devices with respect to a perspective view.
[0147] As mentioned previously with reference to FIG. 2, the system
100 can use the emplacement information of the plurality of medical
devices with respect to the perspective view to determine how the
virtual medical devices are to be displayed on a display. The
perspective view can be determined based on user input and/or
dynamically based on a position of the healthcare provider with
respect to the display device and/or the medical devices. In some
embodiments, the perspective view corresponds to the position of
the healthcare provider with respect to the medical devices and/or
or with respect to the display. In addition, as discussed in
greater detail with reference to FIG. 11, the perspective view can
be based on the number and location of the virtual medical devices
as well.
[0148] At block 1306, the system 100 causes a display device to
display the plurality of virtual medical devices based at least on
the calculated viewing angle(s) in the virtual 3D space. Based on
the calculated viewing angle(s) and the perspective view, the
system 100 can cause the display to display the medical devices
with respect to one another. As the system 100 can calculate the
viewing angle for each virtual medical device separately, each
virtual medical device can be displayed at a different angle with
respect to the perspective view. As mentioned previously, the
perspective view can be based on the location of the healthcare
provider, the location of an image in the virtual 3D space, the
location and number of medical devices, etc.
[0149] As the position and orientation of the medical devices
change, the system 100 can display the change with respect to the
perspective view. For example, if the longitudinal axis of a
virtual medical device is displayed as being left to right on the
display and the medical device is rotated 90 degrees around the
z-axis, the system 100 will change the display to show the
longitudinal axis of the virtual medical device as being from the
foreground to the background (or front to back).
[0150] Additional, fewer, or different blocks can be used to
implement the routine 1300 without departing from the spirit and
scope of the description. For example, any one or a combination of
blocks 1308-1322 can be used as part of routine 1300.
[0151] At block 1308, the system 100 receives emplacement
information of a medical imaging device within the predetermined
area. As described previously, the medical imaging device can be an
ultrasound transducer, laparoscopic camera, etc. Similar to the
receipt of emplacement information described above with reference
to block 1302, the system 100 can receive emplacement information
of the medical imaging device.
[0152] At block 1310, the system 100 receives image data based at
least on the emplacement information of the medical imaging device.
In some embodiments, system receives one or more images from the
medical imaging device. For example, the medical imaging device can
be an ultrasound transducer and can provide one or more ultrasound
images to the system 100. As described in greater detail above with
reference to FIGS. 1A and 1B, in some embodiments, the medical
imaging device is used to select image data from a set of image
data stored previously. For example, based on the emplacement of
the medical imaging device, the system can receive images
corresponding to a CT scan or MRI. In such embodiments, any device
can be used as the medical imaging device.
[0153] At block 1312, the system 100 calculates a viewing angle in
the virtual 3D space of the image data (e.g., one or more images)
based at least on the emplacement information of the medical
imaging device. As described previously with reference to block
1304, the system can calculate viewing angles in the virtual 3D
space based on the emplacement information of the medical imaging
device with respect to the perspective view. Using this
information, the system 100 can calculate a viewing angle for image
data in the virtual 3D space.
[0154] At block 1314, the system 100 causes the display device to
display the image data based at least on the calculated viewing
angle in the virtual 3D space. As described previously with
reference to block 1306 the system can cause a display device to
display images based on the calculated viewing angle. Similarly,
the system 100 can cause the display device to display the image
data (e.g., one or more images).
[0155] At block 1316, the system 100 identifies a foundational
medical device. As described in greater detail above with reference
to FIGS. 12A and 12B, the system 100 can identify the foundational
medical device from among the medical devices being tracked. In
some embodiments, the foundational medical device is the first
medical device that is placed in the patient.
[0156] At block 1318, the system 100 determines one or more
foundational planes and displays one or more foundational plane
indicators, as described in greater detail above with reference to
FIGS. 12A and 12B. As further described previously, the
foundational planes can include a foundational tip planes,
foundational electrode planes, etc. Once the foundational planes
are determined, the system 100 can cause the display device to
display foundational plane indicators corresponding to the
foundational planes. The foundational plane indicators can indicate
relationships between secondary medical devices and the
foundational plane and between the foundational medical device and
the foundational plane. Such information can be useful to a
healthcare provider when placing the secondary medical devices. For
example, the foundational plane indicators can indicate the
location at which a secondary medical device is to be placed to be
level with the foundational medical device (e.g., corresponding
portions, such as the tip or handle, of the two medical devices are
parallel).
[0157] At block 1320, the system 100 determines one or more
foundational plane intersections and displays one or more
foundational plane intersection indicators. As described in greater
detail above with reference to FIGS. 12A and 12B, the system 100
can determine the points of intersection between the axis of the
virtual medical devices and the foundational plane based on the
emplacement information of the medical devices. Once determined,
the system 100 can display indicators for those intersections.
Using the indicators, the healthcare provider can identify where a
medical device will intersect the foundational plane if pushed
forward. Based on the desired placement of the medical device, the
healthcare provider can change the position and orientation of the
medical device as desired. Thus, the indicators can aid a
healthcare provider during the placement of a medical device.
Furthermore, the system can calculate distances between the
foundational planes and the tips of the medical devices or other
portions of the medical devices. The calculated distances can be
displayed and/or used to display the medical devices.
[0158] At block 1322, the system 100 determines and displays one or
more distances between the medical devices. As described in greater
detail above with reference to FIGS. 12A and 12B, the system 100
can use the emplacement information to determine various distances
between the medical devices. For example, the system 100 can
determine the distance between the tips of two or more medical
devices, the horizontal distance between two or more medical
devices, and/or the vertical distance between two or more medical
devices. The system 100 can display the distances as numbers and/or
as graphical indicators as described above.
[0159] Furthermore, additional blocks can be used as part of the
routine 1300. As described in greater detail above with reference
to FIGS. 12A and 12B, the system 100 can determine and display many
different parameters using the emplacement information of the
multiple medical devices, such as angular differences between
medical devices, trajectory intersections with an image plane,
distances between the foundational plane and secondary medical
devices, relative location differences of different portions of the
medical devices with respect to target axis or target planes, and
relative size of portions of the medical devices with respect to a
target region. The angular differences between the axes of the
medical devices can be displayed as a number or graphically as bars
or lines between medical devices. The angular differences can be
used to determine when two medical devices are parallel. The
intersections of the trajectory of medical devices with an image
plane can be indicated on the display to aid a healthcare provider
in the placement of the medical device. The distances between the
foundational plane and secondary medical devices can be displayed
numerically or graphically and can aid the healthcare provider in
placing the secondary medical devices with respect to the
foundational medical device. The relative locations between
different portions of the medical devices with respect to a target
axis or target plane can be displayed graphically as described in
greater detail above, with reference to FIGS. 12C and 12D, and used
by a healthcare provider to aid in the placement of the medical
devices. The relative size of portions of the medical device with
respect to a target region can be displayed graphically, as
described in greater detail below with reference to FIG. 14A and
can help a healthcare provider determine whether the medical device
is large enough to treat a target, such as a tumor.
[0160] In addition, as described above with reference to FIGS.
12A-12D, the system 100 can cause the display device to display a
second set of the image data (e.g., in the
Medical Device Guidance--Planning Mode
[0161] FIG. 14A is a perspective view of an embodiment of a display
in which the system provides medical device placement guidance to a
user. In the illustrated embodiment, the display 1400 includes an
image 1402, medical device placement suggestions 1404A and 1404B, a
virtual medical device 1406, and guidance cues 1408, 1410, 1412.
Furthermore, any one or any combination of the guidance cues
described above with reference to FIGS. 2-11 and 12A-12D can be
used in conjunction with any of the embodiments described herein
with reference to FIGS. 14A and 14B. For example, the system can
identify one of the medical devices or one of the suggested
placements as the foundational medical device and can calculate and
display the various guidance cues associated with the foundational
plane, etc. Furthermore, it will be understood that although
described in the context of providing guidance for the placement of
multiple medical devices, the embodiments described herein can
provide guidance for the placement of a single medical device with
respect to a target region.
[0162] The guidance cues in the illustrated embodiment include the
center of the tumor marker 1412, which can be manually placed by a
user, and tumor boundary markers 1410 which indicates the extents
or boundary of the tumor in two perpendicular planes. As described
in greater detail above with reference to FIGS. 8, 9, and 10A-10D,
a user can mark and/or annotate images. The system 100 can store
the emplacement information of the markers and annotations in the
virtual 3D space.
[0163] The guidance cues in the illustrated embodiment also include
draws bars 1408A-1408D, which can be generated by the system after
the user has marked the center of the tumor (or other point in the
tumor) and the boundary of the tumor. The system can calculate the
location of the draw bars based on the suggested placement of a
medical device, the length of the exposed electrode of the medical
device and/or different locations within the exposed electrode of
the medical device (e.g., tip, center, and end).
[0164] In the illustrated embodiment, the draw bars include draw
bar 1408A between the end of the exposed electrode of the virtual
medical device 1406 (when placed at the placement suggestion 1404A)
and the image 1402, draw bar 1408B between the center of the
exposed electrode of the medical device 1406 (when placed at the
placement suggestion 1404A) and the center of the tumor marker
1412, draw bar 1408C between the tip of the virtual medical device
1406 (when placed at the placement suggestion 1404A) and the image
1402, and draw bar 1408D parallel to the exposed electrode of the
virtual medical device (when placed at the placement suggestion
1404A), but running through the previously marked center of the
tumor.
[0165] The draws bars 1408A-1408D can show the user 1) if the
exposed electrode is long enough to cover the extent/boundary of
the tumor, 2) if the exposed electrode is centered with respect to
the tumor, and 3) the distance between the tumor-center and the
electrode. This distance can also be displayed numerically on the
screen (not shown).
[0166] In addition, the system can display placement suggestions
1404A, 1404B for the medical devices. The system can generate the
placement suggestions 1404A, 1404B and number of placement
suggestions based on: 1) the distance between the first medical
device (after placement) and the tumor-center 2) the length of the
exposed electrodes of the medical devices, 3) a model of the
ablation parameters of the medical device (e.g. a lookup table), 4)
tumor size, etc. In illustrated example, the system suggests a
configuration of two medical devices.
[0167] In the illustrated embodiment, the placement suggestions
1404A, 1404B are illustrated as faded virtual medical devices,
however, it will be understood that other method can be used to
provide the placement suggestions. For example, the placement
suggestion 1404A, 1404B can be illustrated as lines, bars,
cylinders, letters (e.g., X's), etc.
[0168] Furthermore, when the medical device 1406 is close to, or
enters the predetermined area, the system can generate the virtual
medical device 1406 and the user can guide the virtual medical
device 1406 to the placement suggestion 1404A. The system's
proposed configuration can be repeatedly updated as the user
manipulates the first medical device (still outside the patient),
until she accepts the proposed position.
[0169] FIG. 14B illustrates the content on a display 1400 in
certain embodiments depicting manipulation of the second medical
device in context with the target region. Here, the first medical
device has already been placed as indicated by the location of the
first virtual medical device 1406, and the system provides the user
with the placement suggestion 1404B for the second medical
device.
[0170] The second virtual medical device 1414 is in a vertical
orientation, and the user manipulates the second medical device
corresponding to the second virtual medical device 1414 (e.g.,
while it is outside the patient) such that the intersection
indicator 1416 (described in FIGS. 12A and 12B) is co-located with
the suggested intersection indicator 1420 that the system displays
at the tip of the placement suggestion 1404B. Once this is
achieved, the second medical device will be lined up with its
trajectory directly toward the placement suggestion 1404B. The user
can then push the second medical device into the patient tissue
until the system displays the tip of the second virtual medical
device 1414 at the suggested intersection indicator 1420. To aid a
user in the placement of the second virtual medical device, the
system includes relative spatial indicators 1418 and trajectory
indicators 1422, as described in greater detail above with
reference to FIGS. 12A-12D.
Medical Device Guidance--Planning Routine
[0171] FIG. 15 is a flow diagram illustrative of an embodiment of a
routine 1500 implemented by the system 100 to provide medical
device placement guidance. The medical devices can include invasive
medical devices, non-invasive medical devices, and/or medical
imaging devices. In some embodiments, the medical devices include
ablation and/or biopsy needles. One skilled in the relevant art
will appreciate that the elements outlined for routine 1500 can be
implemented by one or more computing devices/components that are
associated with the system 100, such as the position sensing units
110, 140, the image guidance unit 130, surgical system 149, and/or
imaging unit 140. Accordingly, routine 1500 has been logically
associated as being generally performed by the system 100. However,
the following illustrative embodiment should not be construed as
limiting.
[0172] At block 1502, the system 100 receives emplacement
information associated with a target region. The emplacement
information associated with the target region can be based on a
marker or annotation placed by a healthcare provider as described
earlier with reference to FIGS. 8, 9, and 10A-10D. The system can
use the emplacement of the marker and/or annotation to determine
the emplacement of the target region. In some embodiments, the
emplacement information is received during a medical procedure. In
certain embodiments, the emplacement information is received prior
to a medical procedure.
[0173] At block 1504, the system 100 determines a first position
relative to the target region for a first medical device based at
least on the emplacement information of the target region. As
described in greater detail above with reference to FIGS. 14A and
14B, the system 100 can determine a suggested placement of a first
medical device with respect to the target region. As described
previously, the suggested placement can be based on the length of
the exposed electrodes of the medical device a model of ablation
parameters, tumor size, number of medical devices to be used,
etc.
[0174] At block 1506, the system 100 determines a second position
relative to the target region for a second medical device. As
mentioned previously the second position can be based on the
emplacement information of the target region, the emplacement
information of the first position, and/or any one or any
combination of the parameters described above with reference to
FIGS. 14A and 14B.
[0175] At block 1508, the system 100 causes a display device to
display the target region, as described in greater detail above
with reference to FIGS. 14A and 14B. The display can be based on a
calculated viewing angle in a virtual 3D space, described in
greater detail above with reference to FIGS. 2, 5, and 12A-12D, of
the target region.
[0176] At block 1510, the system 100 causes the display device to
display a first position indicator. The system can cause the
display device to display the first position indicator at the first
position. As discussed previously with reference to FIGS. 14A and
14B, the first position indicator can be in the form of the first
virtual medical device or some other graphical indicator. A user
can use the first position indicator to place the first medical
device.
[0177] As described in greater detail above with reference to FIGS.
13, 14A, and 14B, the system 100 can receive emplacement
information of the first medical device, as illustrated at block
1512. At block 1514, the system 100 causes the display device to
display a first virtual medical device corresponding to the first
medical device based at least on the emplacement information of the
first medical device, as also described in greater detail above
with reference to FIGS. 13, 14A, and 14B.
[0178] Additional, fewer, or different blocks can be used to
implement the routine 1500 without departing from the spirit and
scope of the description. For example, any one or a combination of
blocks 1516-1520 can be used as part of routine 1500.
[0179] Similar to blocks 1510-1514 described previously, the system
can display a second position indicator, receive emplacement
information for the second medical device and display a virtual
second medical device, as illustrated at blocks 1516-1520,
respectively.
[0180] Furthermore, in some embodiments and as described in greater
detail above with reference to FIGS. 14A and 14B, the system 100
can determine and display draw bars indicating the relative
emplacement of the virtual medical devices and/or suggested
placements of the virtual medical devices with the target region.
In addition, as described in greater detail above, with reference
to FIGS. 12A-12D, the system can cause the display device to
display various guidance cues (e.g., trajectory indicators,
intersection indicators, patient orientation indicators, medical
provider location indicators, etc.). As also described in greater
detail above, with reference to FIGS. 12A-12D, the system can
identify one of the virtual medical devices or position indicators
as the foundational medical device and calculate and display
various distances between the first and second medical devices, the
foundational plane and the medical devices, a target plane and a
medical device, etc.
Previous Emplacement of a Medical Device
[0181] Referring back to FIG. 14A, FIG. 14A also provides a
perspective view of an embodiment in which the location of a
medical device that has been removed is shown. In this embodiment,
the system receives emplacement information of a medical device
within a predetermined area at a first time. In some embodiments
the first time is during a medical procedure. Based on the
emplacement information, the system can display the medical device,
as described previously. The system can also use emplacement
information to determine when the medical device has been removed
(or when a tracking unit associated with the medical device has
been removed).
[0182] Typically, when a medical device is removed (or the
associated tracking unit is removed), the corresponding virtual
medical device is not shown on the display. However, in some
embodiments, when a medical device is removed (or when a tracking
unit associated with the medical device is removed), the system can
display an altered image 1404B (e.g., a faded image) of the virtual
medical device.
[0183] The system can use the emplacement information received at
the first time to determine the emplacement of the altered image
1404B. In certain embodiments, the system displays the altered
image 1404B at the location of the virtual medical device at the
first time. Over time, as the location of the previously removed
(or no longer tracked) medical device becomes less reliable (e.g.,
due to normal organ movement, etc.), the system can continue to
alter the altered image 1404B, until it is removed. For example,
the system can make the altered image 1404B more transparent over
time (e.g., more faded).
[0184] In the interim, a user can use the altered image 1404B to
place a second medical device 1406. For example, this can be done
when a second biopsy is taken. The system can provide the user with
the guidance cues described above with reference to FIGS. 2-12D to
aid in the placement of the second medical device 1406. For
example, the system can identify the altered image 1404B as a
foundational medical device and the second medical device 1406 as a
secondary medical device. Using this information, the system can
calculate and display one or more guidance cues associated with the
altered image 1404B, such as foundational plane indicators,
foundational plane intersection indicators, relative spatial
indicators, relative distances between portions the second medical
device 1406 and the altered image 1404B, etc.
Previous Emplacement of a Medical Device Routine
[0185] FIG. 16 is a flow diagram illustrative of an embodiment of a
routine 1600 implemented by the system 100 to display an altered
image of a virtual medical device after a medical device has been
removed from a predetermined area. One skilled in the relevant art
will appreciate that the elements outlined for routine 1600 can be
implemented by one or more computing devices/components that are
associated with the system 100, such as the position sensing units
110, 140, the image guidance unit 130, surgical system 149, and/or
imaging unit 150. Accordingly, routine 1600 has been logically
associated as being generally performed by the system 100. However,
the following illustrative embodiment should not be construed as
limiting.
[0186] At block 1602, the system 100 receives emplacement
information of a tracking unit associated with a medical device at
a first time. As described in greater detail above with reference
to block 1302 of FIG. 13, the system 100 can receive emplacement
information of medical devices. In some embodiments, the first time
is during a medical procedure, such as during an ablation or biopsy
procedure. In certain embodiments, the first time is based on a
user input. For example, a healthcare provider can indicate a
certain time at which the system 100 is to receive and/or collect
the emplacement information. To indicate the predetermined time,
the healthcare provider can touch a screen, click a button, or
enter information into the system 100, etc.
[0187] At block 1604, the system 100 calculates a viewing angle of
a virtual medical device in a virtual 3D space based at least on
the emplacement information. As described in greater detail above
with reference to block 1302 of FIG. 13, the system 100 can
calculate the viewing angle of virtual medical devices using
received emplacement information.
[0188] At block 1606, the system 100 causes a display device to
display the virtual medical device based at least on the calculated
viewing angle. As described in greater detail above with reference
to block 1302 of FIG. 13, the system 100 can cause a display device
to display a virtual medical device based on emplacement
information of the corresponding medical device within a
predetermined area. As mentioned previously, the predetermined area
can correspond to a location of a medical procedure, the location
of a patient, the location of tracking units, the range of position
sensing units, a surgical table, an area corresponding to a virtual
3D area displayed on a display etc.
[0189] At block 1608, the system 100 determines that the tracking
unit associated with the medical device has been removed from the
predetermined area. As the system 100 receives emplacement
information from the tracking unit, it can determine the location
of the tracking unit and associated medical device. Further, it can
determine when the tracking unit has left the predetermined area,
such as the area corresponding to the virtual 3D space displayed by
the display. To determine that the tracking unit has left the
predetermined area, the system 100 can compare the emplacement
information of the tracking unit with emplacement information of
the predetermined area. When the tracking unit is outside the
predetermined area, the system 100 can determine that the tracking
unit has been removed from the predetermined area.
[0190] In some embodiments, the removal of the tracking unit from
the predetermined area corresponds to the removal of the medical
device from the predetermined area. In certain embodiments, the
removal of the tracking unit from the predetermined area does not
correspond to the removal of the medical device from the
predetermined area. As described in greater detail above with
reference to FIGS. 1A and 1B, in some embodiments, the tracking
units can be removed from the medical device. For example, in some
cases the tracking unit can be snapped on and/or off of a medical
device. In certain embodiments, the tracking unit cannot be easily
removed from the medical device. For example, in some cases, the
tracking unit is embedded into the medical device or located at or
near the tip of the medical device and is inserted into the
patient. Accordingly, in some cases a tracking unit can be used
with multiple medical devices. Thus, removal of the tracking unit
can mean that the associated medical device has been removed or
that it is no longer associated with the tracking unit.
[0191] At block 1610, the system 100 causes the display device to
display an altered image of the virtual medical device based at
least on the emplacement information received at the first time.
Upon determining that the tracking unit has left the predetermined
area, the system 100 can cause the display device to display an
altered the image of the virtual medical device.
[0192] Typically, when a medical device is removed from the
predetermined area, the corresponding virtual medical device is no
longer displayed by the system 100. However, in some cases it can
be useful to identify where medical devices were located at a first
time, such as during previous medical procedures (e.g., ablations,
biopsies, etc.). Accordingly, in some embodiments, the system 100
can use the emplacement information received at the first time to
display an altered image of the medical device on the display. In
some embodiments, the virtual medical device is displayed at its
previous location at the predetermined time. In certain
embodiments, the system 100 can cause the virtual medical device to
be faded or grayed out to indicate that it is no longer
present.
[0193] Similarly, in cases where a tracking unit is removed from
one medical device (or stops working) it can be useful to retain an
image of the virtual medical device on the display. Accordingly, in
some embodiments, the system 100 can use the emplacement
information received during at the predetermined time to display an
altered image of the medical device on the display. In some
embodiments, the altered image of the virtual medical device is
displayed at its last known location (based on the first time). In
certain embodiments, the system 100 can cause the virtual medical
device to be faded or grayed out to indicate that its location is
no longer being tracked and/or may be unreliable.
[0194] Additional, fewer, or different blocks can be used to
implement the routine 1600 without departing from the spirit and
scope of the description. For example, the system can omit blocks
1604 and 1606 determine a previous placement of a medical device
based on emplacement information received previously. Using that
information, the system can cause the display device to display an
altered image of the virtual medical device or can cause the
display device to display the virtual medical device. Further, any
one or a combination of blocks 1612-1020 can be used as part of
routine 1600.
[0195] At block 1612, the system 100 further alters the image over
time. Over time the reliability of the location of the altered
virtual medical device will decrease. This can be due to normal
organ movement of the patient or movement due to the healthcare
provider. Accordingly, based at least on an amount of time that has
passed since the first time, the system can further alter (e.g.,
fade or gray out) the altered virtual medical device. Thus, the
altered virtual medical device can continue to fade until it is
removed from the display (e.g., the system ceases to display the
altered image), as illustrated at block 1614.
[0196] At block 1616, the system 100 can receive emplacement
information of a tracking unit associated with a second medical
device after causing the display device to display the altered
image. In some embodiments, the second medical device is the same
as the first medical device. For example, after the medical device
has been removed from the predetermined area, the healthcare
provider may re-introduce it at another location for another
medical procedure. Accordingly, the system 100 can receive the new
emplacement information of the tracking unit associated with the
medical device. Based on the new emplacement information, the
system 100 can calculate the new viewing angle of the virtual
medical device in the virtual 3D space and cause the display to
display it. However, as mentioned previously, the system 100 can
retain the altered image as a reference for the healthcare
provider.
[0197] Similarly, the system can receive emplacement information of
a tracking unit (the same or different tracking unit from the one
mentioned above with reference to block 1616) associated with a
second medical device after causing the display device to display
the altered image, as illustrated at block 1618. The second medical
device can be a different medical device with its own tracking unit
or a different medical device using the same tracking unit as the
first medical device. In either instance, the system 100 can
receive the emplacement information of the second medical device
(via the emplacement information of the tracking unit), calculate
the viewing angle of the second virtual medical device in the
virtual 3D space, and cause the display to display it. As mentioned
previously, the system 100 can retain the altered image of the
first virtual medical device on the display as a reference.
[0198] At block 1620, the system 100 determines and displays one or
more guidance cues associated with the altered image of the first
virtual medical device, the first virtual medical device (based on
new emplacement information), and/or the second virtual medical
device. The guidance cues can be any one or more of the guidance
cues described above with reference to FIGS. 2-12D, 14A, and 14B
(e.g., foundational plane indicators, intersection indicators,
trajectories, ablation volumes, annotations, etc.). For example,
the system can use the altered image of the first virtual medical
device as the foundational medical device or as a secondary medical
device in determining and displaying the guidance cues. As another
example, the system can use the axis of the altered image as a
target axis. In some embodiments, the system 100 can display the
ablation volume of the altered image based on the volume that was
ablated previously, etc.
Foundational Plane Plot
[0199] FIG. 17 is a diagram illustrative of an embodiment of a
foundational plane plot 1702, or needle tip map. In the illustrated
embodiment, the distances between the tips of the needles, as well
as the angles between their shafts can be displayed in the relative
emplacement measurement display area 1206. Furthermore, as
described previously, the text can be white if it meets the
requirement, and red if not. The needles can be represented by
their number and color.
[0200] FIG. 17 further illustrates a foundational plane plot 1702
in the bottom left-hand corner. The foundational plane plot 1702
provides a visual indication of the distances between the tips of
the needles from the point of view of the user looking "down" onto
the foundation-tip-plane. In some embodiments, the location of the
foundational needle tip can be the center of the tip map 1702.
[0201] In the illustrated embodiment, dots 1712, 1714, 1716 (other
marks can be used) are drawn at the real-time location of the tip
of the foundational needle 1212, as well as the relative location
of the tips of the other needles 1214, 1216. In FIG. 17, the
foundational needle's dot 1712 is drawn in the third color
(approximate center of the tip map and bottom-most of the three
tips), the dot 1714 for the tip of the second needle is drawn in
the second color (middle tip of the three tips), and the dot 1716
for the third needle's tip is drawn in the first color (top tip of
the three tips). Looking at the needles' locations in the 2D or 3D
view, a user can confirm that the tips are in the correct
arrangement in the foundational plane plot 1702. Additionally, a
grid can be drawn to give scale. In the illustrated embodiment, the
grid marks 1704 are at 1 cm intervals, providing a visual cue to
how far apart the needle tips are in reality. It will be understood
that other intervals can be used as desired.
[0202] The needle-tip distances can be measured and displayed in
different ways: For example, the needle-tip distances can be
displayed as the actual 3D distances, and/or the distances in the
foundational-tip-plane (which is to say, the lengths of the legs of
the foundational plane indicators 1228, 1230). In some embodiments,
the foundational-tip-plane is used when the secondary needles are
parallel and level to the foundational needle.
[0203] Additionally, the foundational plane plot 1702 can be drawn
such that if the user moves a needle closer to himself, the
needle's dot in the foundational plane plot 1702 will be seen
moving lower on the screen, providing the user with intuitive
feedback on how to place needles relative to each other.
Rotated View
[0204] FIGS. 18A and 18B are diagrams illustrating embodiments of
rotated views. In some instances when displaying 3D information on
a 2D display, objects, such as needles in this example, can look
parallel from a perspective 3D view in the 3D image display area
1204, as illustrated in FIGS. 18A and 18B. However, looking from a
different perspective would reveal that they are not parallel.
[0205] With reference to FIG. 18A, from the point of view in the 3D
image display area 1204, the needles 1212, 1216 appear parallel. To
alleviate this issue, an alternate 2D view "from the side" can be
provided in a second 2D image display area 1802. The alternate 2D
view can be drawn by rotating the typical 2D view in the 2D image
display area 1202 by 90 degrees (or -90 degrees). It will be
understood that other degrees can be used for the rotation as well.
In some embodiments, this alternate 2D view in the second 2D image
display area 1802 can be included in addition to the 2D view in the
2D image display area 1202.
[0206] By rotating the point of view by 90 degrees (without moving
the needles or transducer), it becomes obvious that the needle 1216
is not parallel to the foundational needle 1212, as is indicated by
the angle of 22 degrees in the emplacement measurement display area
1206. Using this rotated point of view, the user can identify when
the needles are not parallel, despite the image in the 3D image
display area 1204 indicating that the needles are parallel.
[0207] With reference to FIG. 18B, in some embodiments, an
alternate 3D view in a second 3D image display area 1804 can be
generated by rotating the typical 3D view in the 3D image display
area 1204 by 90 degrees (or -90 degrees). It will be understood
that other degrees can be used as well for the rotation. The
alternate 3D view can be used to identify needles that are not in
parallel despite the typical 3D view indicating that they are in
parallel. The multiple perspectives can provide the user with
increased confidence that the needles are parallel.
Example Embodiments
[0208] Various example embodiments of the disclosure can be
described in view of the following clauses: [0209] Clause 1. A
system comprising: [0210] a display; and [0211] a computer system
comprising a computer processor and memory and in communication
with the display and a plurality of tracking units associated with
a plurality of medical devices, wherein the computer system is
configured to: [0212] receive emplacement information of at least a
portion of the plurality of medical devices within a predetermined
area; [0213] calculate a viewing angle in a virtual 3D space of a
plurality of virtual medical devices corresponding to the plurality
of medical devices based at least on the emplacement information of
the plurality of medical devices; and [0214] cause the display to
display the plurality of virtual medical devices based at least on
the calculated viewing angle in the virtual 3D space. Clause 2. The
system of Clause 1, wherein the plurality of medical devices are
invasive medical devices. [0215] Clause 3. The system of any of
Clauses 1 and 2, wherein the plurality of medical devices comprises
a plurality of ablation needles. [0216] Clause 4. The system of any
of Clauses 1-3, wherein the computer system is further configured
to: [0217] calculate an ablation volume of each of the plurality of
medical devices; and [0218] cause the display device to display the
calculated ablation volume of each of the plurality of medical
devices. [0219] Clause 5. The system of any of Clauses 1-4, wherein
the computer system is further configured to: [0220] calculate a
trajectory of each of the plurality of medical devices; and [0221]
cause the display device to display the calculated trajectory of
each of the plurality of medical devices. [0222] Clause 6. The
system of any of Clauses 1-5, wherein the computer system is
further configured to indicate an orientation of a patient with
respect to a viewing reference of the image data in the virtual 3D
space. [0223] Clause 7. The system of any of Clauses 6, wherein the
computer system is further configured to indicate a location of a
medical provider with respect to the patient in the virtual 3D
space. [0224] Clause 8. The system of any of Clauses 1-7, wherein
the computer system is further configured to: [0225] receive
emplacement information of a medical imaging device within the
predetermined area; [0226] receive image data based at least on the
emplacement information of the medical imaging device; [0227]
calculate a viewing angle in the virtual 3D space of the image data
based at least on the emplacement information of the medical
imaging device; and [0228] cause the display device to display the
image data based at least on the calculated viewing angle in the
virtual 3D space. [0229] Clause 9. The system of any of Clauses
1-8, wherein the image data is received from the medical imaging
device. [0230] Clause 10. The system of Clause 9, wherein the
medical imaging device comprises an ultrasound transducer and the
image data comprises an ultrasound image. [0231] Clause 11. The
system of any of Clauses 9 and 10, wherein the computer system is
further configured to cause the display device to display a second
set of the image data at a second location on the display device.
[0232] Clause 12. The system of any of Clauses 9-11, wherein the
computer system is further configured to: [0233] calculate an
intersection between an image plane of the image data and a
calculated trajectory of at least one medical device of the
plurality of medical devices; and [0234] cause the display device
to display an indication of the intersection of the image plane and
the calculated trajectory of the at least one medical device.
[0235] Clause 13. The system of any of Clauses 9-12, wherein the
computer system is further configured to: [0236] calculate a
viewing angle in the virtual 3D space of a virtual medical imaging
device space corresponding to the medical imaging device based at
least on the emplacement information of the medical imaging device;
and [0237] cause the display device to display the virtual medical
imaging device space based at least on the calculated viewing angle
in the virtual 3D space. [0238] Clause 14. The system of any of
Clauses 1-13, further wherein the computer system is further
configured to identify a foundational medical device from the
plurality of medical devices, wherein each of the plurality of
medical devices comprises a first portion and a second portion.
[0239] Clause 15. The system of Clause 14, wherein the first
portion of each of the plurality of medical devices is a tip of the
medical device and the second portion of each of the plurality of
medical devices is an end of an electrode of the medical device
distal to the tip of the medical device. [0240] Clause 16. The
system of any of Clauses 14 and 15, wherein the computer system is
further configured to: [0241] determine a first foundational plane
based at least on a location of the first portion of the
foundational medical device, wherein the first foundational plane
is orthogonal to a trajectory of the foundational medical device
and intersects with the first portion of the foundational medical
device; and [0242] cause the display device to indicate the first
foundational plane in the virtual 3D space. [0243] Clause 17. The
system of Clause 16, wherein the computer system is further
configured to: [0244] calculate a distance between the first
foundational plane and the first portion of a medical device of the
plurality of medical devices; and [0245] cause the display device
to display the plurality of medical devices based at least in part
on the calculated distance. [0246] Clause 18. The system of any of
Clauses 16 and 17, wherein the computer system is further
configured to: [0247] determine an intersection between the first
foundational plane and a trajectory of one or more medical devices
of the plurality of medical devices; and [0248] cause the display
device to display an indication of the intersection of the first
foundational plane and the trajectory of the one or more medical
devices. [0249] Clause 19. The system of any of Clauses 16-18,
wherein the computer system is further configured to: [0250]
determine a second foundational plane based at least on a location
of the second portion of the foundational medical device, wherein
the second foundational plane is orthogonal to a trajectory of the
foundational medical device and intersects with the second portion
of the foundational medical device; and [0251] cause the display
device to indicate the second foundational plane in the virtual 3D
space. [0252] Clause 20. The system of any of Clauses 16-19,
wherein the computer system is further configured to cause the
display device to display a graphical indicator on the foundational
plane from the first portion of the foundational medical device to
a location on the foundational plane where the first portion of at
least one other medical device of the plurality of medical devices
is to be located. [0253] Clause 21. The system of any of Clauses
14-20, wherein the computer system is further configured to: [0254]
calculate an angle difference between the foundational medical
device and at least one other medical device of the plurality of
medical devices; and [0255] cause the display device to display an
indication of the calculated angle difference. [0256] Clause 22.
The system of any of Clauses 14-21, wherein the computer system is
further configured to cause the display device to display a
plurality of graphical indicators indicating a distance between a
plurality of portions of the foundational medical device and a
plurality of portions of at least one other medical device of the
plurality of medical devices. [0257] Clause 23. The system of any
of Clauses 14-22, wherein the computer system is further configured
to cause the display device to display a plurality of graphical
indicators indicating a distance between at least one of a target
axis and a target plane and a plurality of portions of at least one
medical device of the plurality of medical devices. [0258] Clause
24. The system of any of Clauses 14-23, wherein the computer system
is further configured to: [0259] calculate a distance between the
first portion of the foundational medical device and the first
portion of at least one other medical device of the plurality of
medical devices based at least on the received emplacement
information of at least a portion of the plurality of medical
devices; and [0260] cause the display device to display an
indication of the calculated distance. [0261] Clause 25. The system
of any of Clauses 24, wherein the indication comprises at least one
of a graphical indicator between the first portion of the
foundational medical device and the first portion of the at least
one other medical device and a number. [0262] Clause 26. The system
of any of Clauses 1-25, further comprising the tracking units.
[0263] Clause 27. The system of any of Clauses 1-26, further
comprising the medical devices. [0264] Clause 28. A method,
comprising: [0265] receiving emplacement information of a plurality
of needles during a medical procedure of a patient; [0266]
receiving emplacement information of a medical imaging device
during the medical procedure; [0267] receiving at least one image
from the medical imaging device; calculating a first viewing angle
in a virtual 3D space of the at least one image based at least on
the emplacement information of the medical imaging device with
respect to a perspective view; [0268] calculating a second viewing
angle in the virtual 3D space of a virtual medical imaging device
corresponding to the medical imaging device based at least on the
emplacement information of the medical imaging device with respect
to the perspective view; [0269] calculating a plurality of viewing
angles in the virtual 3D space of a plurality of virtual needles
corresponding to the plurality of needles based at least on the
emplacement information of the plurality of needles with respect to
the perspective view; [0270] causing the display device to display
the at least one image based at least on the first calculated
viewing angle in the virtual 3D space; [0271] causing the display
device to display the virtual medical imaging device based at least
on the second calculated viewing angle in the virtual 3D space; and
[0272] causing a display device to display the plurality of virtual
needles based at least on the plurality of calculated viewing
angles in the virtual 3D space. [0273] Clause 29. A method,
comprising [0274] receiving emplacement information of a plurality
of medical devices within a predetermined area; [0275] calculating
a viewing angle in a virtual 3D space of a plurality of virtual
medical devices corresponding to the plurality of medical devices
based at least on the emplacement information of the plurality of
medical devices; and [0276] causing a display device to display the
plurality of virtual medical devices based at least on the
calculated viewing angle in the virtual 3D space. [0277] Clause 30.
The method of Clause 29, wherein the plurality of medical devices
are invasive medical devices. [0278] Clause 31. The method of any
of Clauses 29 and 30, wherein the plurality of medical devices
comprises a plurality of ablation needles. [0279] Clause 32. The
method of any of Clauses 29-31, further comprising: [0280]
calculating an ablation volume of each of the plurality of medical
devices; and [0281] causing the display device to display the
calculated ablation volume of each of the plurality of medical
devices. [0282] Clause 33. The method of any of Clauses 29-32,
further comprising: [0283] calculating a trajectory of each of the
plurality of medical devices; and [0284] causing the display device
to display the calculated trajectory of each of the plurality of
medical devices. [0285] Clause 34. The method of any of Clauses
29-33, further comprising indicating an orientation of a patient
with respect to a viewing reference of the image data in the
virtual 3D space. [0286] Clause 35. The method of Clause 34,
further comprising indicating a location of a medical provider with
respect to the patient in the virtual 3D space. [0287] Clause 36.
The method of any of Clauses 29-35, further comprising: [0288]
receiving emplacement information of a medical imaging device
within the predetermined area; [0289] receiving image data based at
least on the emplacement information of the medical imaging device;
[0290] calculating a viewing angle in the virtual 3D space of the
image data based at least on the emplacement information of the
medical imaging device; and [0291] causing the display device to
display the image data based at least on the calculated viewing
angle in the virtual 3D space. [0292] Clause 37. The method of
Clause 36, wherein the image data is received from the medical
imaging device. [0293] Clause 38. The method of any of Clauses 35
or 37, wherein the medical imaging device comprises an ultrasound
transducer and the image data comprises an ultrasound image. [0294]
Clause 39. The method of any of Clauses 35-38, further comprising
causing the display device to display a second set of the image
data at a second location on the display device. [0295] Clause 40.
The method of any of Clauses 35-39, further comprising: [0296]
calculating an intersection between an image plane of the image
data and a calculated trajectory of at least one medical device of
the plurality of medical devices; and [0297] causing the display
device to display an indication of the intersection of the image
plane and the calculated trajectory of the at least one medical
device. [0298] Clause 41. The method of any of Clauses 35-40,
further comprising: [0299] calculating a viewing angle in the
virtual 3D space of a virtual medical imaging device space
corresponding to the medical imaging device based at least on the
emplacement information of the medical imaging device; and [0300]
causing the display device to display the virtual medical imaging
device space based at least on the calculated viewing angle in the
virtual 3D space. [0301] Clause 42. The method of any of Clauses
29-41, further comprising identifying a foundational medical device
from the plurality of medical devices, wherein each of the
plurality of medical devices comprises a first portion and a second
portion. [0302] Clause 43. The method of Clause 42, wherein the
first portion of each of the plurality of medical devices is a tip
of the medical device and the second portion of each of the
plurality of medical devices is an end of an electrode of the
medical device distal to the tip of the medical device. [0303]
Clause 44. The method of any of Clauses 42 and 43, further
comprising: [0304] determining a first foundational plane based at
least on a location of the first portion of the foundational
medical device, wherein the first foundational plane is orthogonal
to a trajectory of the foundational medical device and intersects
with the first portion of the foundational medical device; and
[0305] causing the display device to indicate the first
foundational plane in the virtual 3D space. [0306] Clause 45. The
method of Clause 44, further comprising: [0307] calculating a
distance between the first foundational plane and the first portion
of a medical device of the plurality of medical devices; and [0308]
causing the display device to display the plurality of medical
devices based at least in part on the calculated distance. [0309]
Clause 46. The method of any of Clauses 44 and 45, further
comprising: [0310] determining an intersection between the first
foundational plane and a trajectory of one or more medical devices
of the plurality of medical devices; and [0311] causing the display
device to display an indication of the intersection of the first
foundational plane and the trajectory of the one or more medical
devices. [0312] Clause 47. The method of any of Clauses 44-46,
further comprising: [0313] determining a second foundational plane
based at least on a location of the second portion of the
foundational medical device, wherein the second foundational plane
is orthogonal to a trajectory of the foundational medical device
and intersects with the second portion of the foundational medical
device; and [0314] causing the display device to indicate the
second foundational plane in the virtual 3D space. [0315] Clause
48. The method of any of Clauses 44-47, further comprising causing
the display device to display a graphical indicator on the
foundational plane from the first portion of the foundational
medical device to a location on the foundational plane where the
first portion of at least one other medical device of the plurality
of medical devices. [0316] Clause 49. The method of any of Clauses
42-48, further comprising: [0317] calculating an angle difference
between the foundational medical device and at least one other
medical device of the plurality of medical devices; and [0318]
causing the display device to display an indication of the
calculated angle difference. [0319] Clause 50. The method of any of
Clauses 42-49, further comprising causing the display device to
display a plurality of graphical indicators indicating a distance
between a plurality of portions of the foundational medical device
and a plurality of portions of at least one other medical device of
the plurality of medical devices. [0320] Clause 51. The method of
any of Clauses 42-50, further comprising causing the display device
to display a plurality of graphical indicators indicating a
distance between at least one of a target axis and a target plane
and a plurality of portions of at least one medical device of the
plurality of medical devices. [0321] Clause 52. The method of any
of Clauses 42-51, further comprising: [0322] calculating a distance
between the first portion of the foundational medical device and
the first portion of at least one other medical device of the
plurality of medical devices based at least on the received
emplacement information of at least a portion of the plurality of
medical devices; and [0323] causing the display device to display
an indication of the calculated distance. [0324] Clause 53. The
method of any of Clauses 52, wherein the indication comprises at
least one of a graphical indicator between the first portion of the
foundational medical device and the first portion of the at least
one other medical device and a number. [0325] Clause 54. A
computer-readable, non-transitory storage medium having one or more
computer-executable modules, the one or more computer-executable
modules comprising: [0326] a first module in communication with a
display and a plurality of tracking units associated with a
plurality of medical devices, wherein the first module is
configured to: [0327] receive emplacement information of at least a
portion of the plurality of medical devices within a predetermined
area; [0328] calculate a viewing angle in a virtual 3D space of a
plurality of virtual medical devices corresponding to the plurality
of medical devices based at least on the emplacement information of
the plurality of medical devices; and [0329] cause the display to
display the plurality of virtual medical devices based at least on
the calculated viewing angle in the virtual 3D space. [0330] Clause
54. The computer-readably medium of Clause 54, wherein the one or
more computer-executable modules are configured to perform any of,
or any combination of, the steps recited in Clauses 29-53. [0331]
Clause 55. A system comprising: [0332] a display; and [0333] a
computer system comprising a computer processor and memory and in
communication with the display and a tracking unit associated with
a first medical device and, wherein the computer system is
configured to: [0334] receive emplacement information of a target
region; [0335] determine a first position relative to the target
region for a first medical device based at least on the emplacement
information of the target region; [0336] determine a second
position relative to the target region for a second medical device
based at least on the emplacement information of the target region;
[0337] cause a display device to display the target region; [0338]
cause the display device to display a first position indicator to
indicate a suggested placement of the first medical device; [0339]
receive emplacement information of the first medical device; and
[0340] cause the display device to display a first virtual medical
device corresponding to the first medical device based at least on
the emplacement information of the first medical device. [0341]
Clause 56. The system of Clause 55, further comprising the tracking
unit. [0342] Clause 57. The system of any of Clauses 55 and 56,
further comprising the first medical device and the second medical
device. [0343] Clause 58. The system of any of Clauses 55-57,
wherein the plurality of medical devices are invasive medical
devices. [0344] Clause 59. The system of any of Clauses 55-58,
wherein the plurality of medical devices comprises at least one of
a plurality of ablation needles and a plurality of biopsy needles.
[0345] Clause 60. The system of any of Clauses 55-59, wherein the
computer system is further configured to: [0346] calculate a
trajectory of each of the plurality of medical devices; and [0347]
cause the display device to display the calculated trajectory of
each of the plurality of medical devices. [0348] Clause 61. The
system of any of Clauses 55-60, wherein the computer system is
further configured to indicate an orientation of a patient with
respect to a viewing reference of the image data in the virtual 3D
space. [0349] Clause 62. The system of any of Clauses 61, wherein
the computer system is further configured to indicate a location of
a medical provider with respect to the patient in the virtual 3D
space. [0350] Clause 63. The system of any of Clauses 55-62,
wherein the computer system is further configured to: [0351] cause
the display device to display a second position indicator to
indicate a suggested placement of the second medical device; [0352]
receive emplacement information of the second medical device; and
[0353] cause the display device to display a second virtual medical
device corresponding to the second medical device based at least on
the emplacement information of the second medical device. [0354]
Clause 64. The system of any of Clauses 55-63, wherein the computer
system is further configured to: [0355] determine a distance
between a portion of the target region and a portion of the first
medical device; [0356] cause the display device to display a
graphical indicator indicating the distance between the portion of
the target region and the portion of the first medical device
[0357] Clause 65. The system of any of Clauses 55-64 wherein the
computer system is further configured to cause the display device
to display a plurality of graphical indicators indicating a
distance between at least one of a target axis and a target plane
and a plurality of portions of at least one of the first medical
device and the second medical device. [0358] Clause 66. A method
for multi-needle image guided placement, comprising: [0359]
receiving emplacement information of a target region during a
medical procedure; [0360] determining a first suggested position
relative to the target region for a first needle based at least on
the emplacement information of the target region; [0361]
determining a second suggested position relative to the target
region for a second needle based at least on the emplacement
information of the target region; [0362] causing a display device
to display the target region; [0363] causing the display device to
display a first position indicator at the first suggested position
to indicate a suggested placement for the first needle; [0364]
causing the display device to display a second position indicator
at the second suggested position to indicate a suggested placement
for second needle; [0365] receiving emplacement information of the
first needle; [0366] causing the display device to display a first
virtual needle corresponding to the first needle based at least on
the emplacement information of the first needle; [0367] receiving
emplacement information of the second needle; and [0368] causing
the display device to display a second virtual needle corresponding
to the second needle based at least on the emplacement information
of the second needle. [0369] Clause 67. A method for multi-medical
device image guided placement, comprising: [0370] receiving
emplacement information of a target region; [0371] determining a
first position relative to the target region for a first medical
device based at least on the emplacement information of the target
region; [0372] determining a second position relative to the target
region for a second medical device based at least on the
emplacement information of the target region; [0373] causing a
display device to display the target region; causing the display
device to display a first position indicator to indicate a
suggested placement of the first medical device; [0374] receiving
emplacement information of the first medical device; and [0375]
causing the display device to display a first virtual medical
device corresponding to the first medical device based at least on
the emplacement information of the first medical device. [0376]
Clause 68. The method of Clause 67, wherein the plurality of
medical devices are invasive medical devices. [0377] Clause 69. The
method of any of Clauses 67 and 68, wherein the plurality of
medical devices comprises at least one of a plurality of ablation
needles and a plurality of biopsy needles. [0378] Clause 70. The
method of Clause 67-69, further comprising: [0379] calculating a
trajectory of each of the plurality of medical devices; and [0380]
causing the display device to display the calculated trajectory of
each of the plurality of medical devices. [0381] Clause 71. The
method of Clause 67-70, further comprising indicating an
orientation of a patient with respect to a viewing reference of the
image data in the virtual 3D space. [0382] Clause 72. The method of
Clause 71, further comprising indicating a location of a medical
provider with respect to the patient in the virtual 3D space.
[0383] Clause 73. The method of any of Clauses 67-72, further
comprising: [0384] causing the display device to display a second
position indicator to indicate a suggested placement of the second
medical device; [0385] receiving emplacement information of the
second medical device; and [0386] causing the display device to
display a second virtual medical device corresponding to the second
medical device based at least on the emplacement information of the
second medical device.
[0387] The method of any of Clauses 1, further comprising: [0388]
determining a distance between a portion of the target region and a
portion of the first medical device; [0389] causing the display
device to display a graphical indicator indicating the distance
between the portion of the target region and the portion of the
first medical device [0390] Clause 74. The method of any of Clauses
67-74, further comprising causing the display device to display a
plurality of graphical indicators indicating a distance between at
least one of a target axis and a target plane and a plurality of
portions of at least one of the first medical device and the second
medical device. [0391] Clause 75. A computer-readable,
non-transitory storage medium having one or more
computer-executable modules, the one or more computer-executable
modules comprising: [0392] a first module in communication with a
display and a tracking unit associated with a first medical device,
wherein the first module is configured to: [0393] receive
emplacement information of a target region; [0394] determine a
first position relative to the target region for a first medical
device based at least on the emplacement information of the target
region; [0395] determine a second position relative to the target
region for a second medical device based at least on the
emplacement information of the target region; [0396] cause a
display device to display the target region; [0397] cause the
display device to display a first position indicator to indicate a
suggested placement of the first medical device; [0398] receive
emplacement information of the first medical device; and [0399]
cause the display device to display a first virtual medical device
corresponding to the first medical device based at least on the
emplacement information of the first medical device. [0400] Clause
76. The computer-readably medium of Clause 75, wherein the one or
more computer-executable modules are configured to perform any of,
or any combination of, the steps recited in Clauses 67-73. [0401]
Clause 77. A system comprising: [0402] a display; [0403] a computer
system comprising a computer processor and memory and in
communication with the display and a tracking unit associated with
a medical device, wherein the computer system is configured to:
[0404] receive emplacement information of a tracking unit within a
predetermined area and associated with a medical device at a first
time; [0405] determine the tracking unit associated with the
medical device has been removed from the predetermined area at a
second time after the first time; and [0406] cause the display
device to display an altered image of a virtual medical device
based at least on the emplacement information received at the first
time. [0407] Clause 78. The system of Clause 77, wherein the
computer system is further configured to: [0408] calculate a
viewing angle in a virtual 3D space of a virtual medical device
corresponding to the medical device based at least on the
emplacement information; [0409] cause a display device to display
the virtual medical device based at least on the calculated viewing
angle in the virtual 3D space; [0410] Clause 79. The system of any
of Clauses 77 and 78, further comprising the tracking unit. [0411]
Clause 80. The system of any of Clauses 77-79, further comprising
the first medical device and the second medical device. [0412]
Clause 81. The system of any of Clauses 77, wherein the computer
system is further configured to further alter the altered image of
the first virtual medical device based at least on an amount of
time that has passed since the first time. [0413] Clause 82. The
system of any of Clauses 77-81, wherein the computer system is
further configured to cease display of the altered image once a
threshold time is satisfied. [0414] Clause 83. The system of any of
Clauses 77-82, wherein the virtual medical device is a first
virtual medical device, the method wherein the computer system is
further configured to: [0415] receive emplacement information of a
tracking unit associated with a second medical device at a third
time after the second time; and [0416] calculate a second viewing
angle in the virtual 3D space of a second virtual medical device
based at least on the emplacement information received at the third
time; and [0417] cause the display device to display the second
virtual medical device based at least on the second calculated
viewing angle in the virtual 3D space, wherein the altered image of
the virtual medical device is also displayed. [0418] Clause 84. The
system of Clause 83, wherein the tracking unit associated with the
first medical device and the tracking unit associated with the
second medical device is the same. [0419] Clause 85. The system of
any of Clauses 83 and 84, wherein the second virtual medical device
corresponds to a second medical device that is different from the
first medical device. [0420] Clause 86. The system of any of
Clauses 83-85, wherein the second virtual medical device
corresponds to the first medical device. [0421] Clause 87. The
system of any of Clauses 83-86, wherein the computer system is
further configured to cause the display device to display a
plurality of graphical indicators indicating a distance between at
least one of a target axis and a target plane and a plurality of
portions of the second virtual medical device. [0422] Clause 88.
The system of any of Clauses 87, wherein the target axis is the
axis of the altered image of the first virtual medical device.
[0423] Clause 89. The system of any of Clauses 83-88, wherein the
computer system is further configured to cause the display device
to display a plurality of graphical indicators indicating a
distance between a plurality of portions of the altered image of
the first virtual medical device and a plurality of portions of the
second virtual medical device. [0424] Clause 90. The system of any
of Clauses 83-89, wherein the computer system is further configured
to: [0425] calculate a distance between a first portion of the
altered image of the first medical device and a first portion of
the second virtual medical device based at least on the received
emplacement information at the first time and the received
emplacement information at the third time; and [0426] cause the
display device to display an indication of the calculated distance.
[0427] Clause 91. A method, comprising [0428] receiving emplacement
information of a first needle within a predetermined area during a
first medical procedure; [0429] calculating a first viewing angle
in a virtual 3D space of a first virtual needle corresponding to
the first needle based at least on the emplacement information of
the first needle; [0430] causing a display device to display the
first virtual needle based at least on the first calculated viewing
angle in the virtual 3D space; [0431] determining the first needle
is removed from the predetermined area after the first medical
procedure; [0432] causing the display device to display an altered
image of the first virtual needle based at least on the emplacement
information received during the first medical procedure and on an
amount of time that has passed since the first medical procedure;
[0433] receiving emplacement information of a second needle after
the first medical procedure and prior to a second medical
procedure; [0434] calculating a viewing angle in the virtual 3D
space of a second virtual needle corresponding to the second needle
based at least on the emplacement information of the second needle;
and [0435] causing the display device to display the second virtual
needle based at least on the second calculated viewing angle in the
virtual 3D space, wherein at least for a time period the altered
image of the first virtual needle is displayed simultaneously with
the second virtual needle. [0436] Clause 92. A method, comprising
[0437] receiving emplacement information of a tracking unit
associated with a medical device at a first time; [0438]
calculating a viewing angle in a virtual 3D space of a virtual
medical device corresponding to the medical device based at least
on the emplacement information; [0439] causing a display device to
display the virtual medical device based at least on the calculated
viewing angle in the virtual 3D space; [0440] determining the
tracking unit associated with the medical device has been removed
from the predetermined area at a second time after the first time;
[0441] causing the display device to display an altered image of
the virtual medical device based at least on the emplacement
information received at the first time. [0442] Clause 93. The
method of Clause 92, further comprising further altering the
altered image of the first virtual medical device based at least on
an amount of time that has passed since the first time. [0443]
Clause 94. The method of any of Clauses 92 and 93, further
comprising ceasing display of the altered image once a threshold
time is satisfied. [0444] Clause 95. The method of any of Clauses
92-94, wherein the virtual medical device is a first virtual
medical device, the method further comprising: [0445] receiving
emplacement information of a tracking unit associated with a second
medical device at a third time after the second time; and [0446]
calculating a second viewing angle in the virtual 3D space of a
second virtual medical device based at least on the emplacement
information received at the third time; and [0447] causing the
display device to display the second virtual medical device based
at least on the second calculated viewing angle in the virtual 3D
space, wherein the altered image of the virtual medical device is
also displayed. [0448] Clause 96. The method of Clause 95, wherein
the tracking unit associated with the first medical device and the
tracking unit associated with the second medical device is the
same. [0449] Clause 97. The method of any of Clauses 95 and 96,
wherein the second virtual medical device corresponds to a second
medical device that is different from the first medical device.
[0450] Clause 98. The method of any of Clauses 95-97, wherein the
second virtual medical device corresponds to the first medical
device. [0451] Clause 99. The method of any of Clauses 95-98,
further comprising causing the display device to display a
plurality of graphical indicators indicating a distance between at
least one of a target axis and a target plane and a plurality of
portions of the second virtual medical device. [0452] Clause 100.
The method of Clause 99, wherein the target axis is the axis of the
altered image of the first virtual medical device. [0453] Clause
101. The method of any of Clauses 95-100, further comprising
causing the display device to display a plurality of graphical
indicators indicating a distance between a plurality of portions of
the altered image of the first virtual medical device and a
plurality of portions of the second virtual medical device. [0454]
Clause 102. The method of any of Clauses 95-101, further
comprising: [0455] calculating a distance between a first portion
of the altered image of the first medical device and a first
portion of the second virtual medical device based at least on the
received emplacement information at the first time and the received
emplacement information at the third time; and [0456] causing the
display device to display an indication of the calculated distance.
[0457] Clause 103. A computer-readable, non-transitory storage
medium having one or more computer-executable modules, the one or
more computer-executable modules comprising: [0458] a first module
in communication with a display and a tracking unit associated with
a medical device, wherein the first module is configured to: [0459]
receive emplacement information of a tracking unit associated with
a medical device at a first time; [0460] calculate a viewing angle
in a virtual 3D space of a virtual medical device corresponding to
the medical device based at least on the emplacement information;
[0461] cause a display device to display the virtual medical device
based at least on the calculated viewing angle in the virtual 3D
space; [0462] determine the tracking unit associated with the
medical device has been removed from the predetermined area at a
second time after the first time; [0463] cause the display device
to display an altered image of the virtual medical device based at
least on the emplacement information received at the first time.
[0464] Clause 104. The computer-readable medium of Clause 103,
wherein the one or more computer-executable modules are configured
to perform any of, or any combination of, the steps recited in
Clauses 93-102. [0465] Clause 105. A method, comprising [0466]
receiving emplacement information of a tracking unit associated
with a medical device at a first time; [0467] determining the
tracking unit associated with the medical device has been removed
from the predetermined area at a second time after the first time;
and causing the display device to display an altered image of a
virtual medical device based at least on the emplacement
information received at the first time.
TERMINOLOGY
[0468] Those having skill in the art will further appreciate that
the various illustrative logical blocks, modules, circuits, and
process steps described in connection with the implementations
disclosed herein can be implemented as electronic hardware,
computer software, or combinations of both. To clearly illustrate
this interchangeability of hardware and software, various
illustrative components, blocks, modules, circuits, and steps have
been described above generally in terms of their functionality.
Whether such functionality is implemented as hardware or software
depends upon the particular application and design constraints
imposed on the overall system. Skilled artisans can implement the
described functionality in varying ways for each particular
application, but such implementation decisions should not be
interpreted as causing a departure from the scope of the present
invention. One skilled in the art will recognize that a portion, or
a part, can comprise something less than, or equal to, a whole. For
example, a portion of a collection of pixels can refer to a
sub-collection of those pixels.
[0469] The various illustrative logical blocks, modules, and
circuits described in connection with the implementations disclosed
herein can be implemented or performed with a general purpose
processor, a digital signal processor (DSP), an application
specific integrated circuit (ASIC), a field programmable gate array
(FPGA) or other programmable logic device, discrete gate or
transistor logic, discrete hardware components, or any combination
thereof designed to perform the functions described herein. A
general purpose processor can be a microprocessor, but in the
alternative, the processor can be any conventional processor,
controller, microcontroller, or state machine. A processor can also
be implemented as a combination of computing devices, e.g., a
combination of a DSP and a microprocessor, a plurality of
microprocessors, one or more microprocessors in conjunction with a
DSP core, or any other such configuration.
[0470] The steps of a method or process described in connection
with the implementations disclosed herein can be embodied directly
in hardware, in a software module executed by a processor, or in a
combination of the two. A software module can reside in RAM memory,
flash memory, ROM memory, EPROM memory, EEPROM memory, registers,
hard disk, a removable disk, a CD-ROM, or any other form of
non-transitory storage medium known in the art. An exemplary
computer-readable storage medium is coupled to the processor such
the processor can read information from, and write information to,
the computer-readable storage medium. In the alternative, the
storage medium can be integral to the processor. The processor and
the storage medium can reside in an ASIC. The ASIC can reside in a
user terminal, camera, or other device. In the alternative, the
processor and the storage medium can reside as discrete components
in a user terminal, camera, or other device.
[0471] Headings are included herein for reference and to aid in
locating various sections. These headings are not intended to limit
the scope of the concepts described with respect thereto. Such
concepts can have applicability throughout the entire
specification.
[0472] The previous description of the disclosed implementations is
provided to enable any person skilled in the art to make or use the
present invention. Various modifications to these implementations
will be readily apparent to those skilled in the art, and the
generic principles defined herein can be applied to other
implementations without departing from the spirit or scope of the
invention. Thus, the present invention is not intended to be
limited to the implementations shown herein but is to be accorded
the widest scope consistent with the principles and novel features
disclosed herein.
* * * * *