U.S. patent application number 15/294275 was filed with the patent office on 2017-04-20 for ultrasound simulation system and tool.
This patent application is currently assigned to VIRTAMED AG. The applicant listed for this patent is VIRTAMED AG. Invention is credited to Erika BEUDEKER, Carolyn O'BRIEN.
Application Number | 20170110032 15/294275 |
Document ID | / |
Family ID | 57178401 |
Filed Date | 2017-04-20 |
United States Patent
Application |
20170110032 |
Kind Code |
A1 |
O'BRIEN; Carolyn ; et
al. |
April 20, 2017 |
ULTRASOUND SIMULATION SYSTEM AND TOOL
Abstract
Embodiments of flexible ultrasound simulation devices, methods,
and systems described herein may facilitate various medical
procedure training by rendering realistic ultrasound images in
accordance with the end user interaction with an anatomy model, an
ultrasound probe replica and a medical tool. An ultrasound probe
replica may be adapted with a low friction tip, such as a roller
ball. A data processing unit may compute and display an accurate
Virtual Reality/Augmented Reality (VR/AR) model and an ultrasound
image in accordance with the position and orientation tracking of
the VR/AR ultrasound simulator elements. The VR/AR ultrasound
simulator may be adapted to realistically render the VR/AR model
and the ultrasound image as a function of the user interaction with
the VR/AR simulator elements.
Inventors: |
O'BRIEN; Carolyn;
(Schlieren, CH) ; BEUDEKER; Erika; (Schlieren,
CH) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
VIRTAMED AG |
Schlieren |
|
CH |
|
|
Assignee: |
VIRTAMED AG
Schlieren
CH
|
Family ID: |
57178401 |
Appl. No.: |
15/294275 |
Filed: |
October 14, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62242322 |
Oct 16, 2015 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A61B 2090/3925 20160201;
A61B 8/4444 20130101; G06T 2207/10136 20130101; A61B 8/4254
20130101; A61B 8/461 20130101; G09B 23/286 20130101; A61B 8/483
20130101; G09B 23/281 20130101; G06T 19/006 20130101; G09B 23/30
20130101; G06T 3/4007 20130101; A61B 2090/3925 20160201; G06T
19/006 20130101; G06T 2207/10136 20130101; A61B 8/4254 20130101;
A61B 8/483 20130101; G09B 9/00 20130101; A61B 2090/378 20160201;
A61B 8/4444 20130101; G09B 9/00 20130101; A61B 8/461 20130101; A61B
8/145 20130101; G06T 3/4007 20130101; A61B 8/4254 20130101; G09B
23/286 20130101; A61B 8/145 20130101; A61B 2090/378 20160201; G09B
23/281 20130101; G09B 23/30 20130101 |
International
Class: |
G09B 23/28 20060101
G09B023/28; G09B 23/30 20060101 G09B023/30; G09B 9/00 20060101
G09B009/00 |
Claims
1. A simulation system comprising: a data processing unit; a
display in communication with the data processing unit; an anatomy
model; and a tool adapted to interact with low friction with a
surface of the anatomy model.
2. The simulation system of claim 1, wherein the tool is adapted to
roll over a surface of the anatomy model.
4. The simulation system of claim 1, wherein the tool is a replica
of an ultrasound probe.
5. The simulation system of claim 4, wherein the ultrasound probe
is an abdominal probe.
6. The simulation system of claim 4, wherein the ultrasound probe
is a vaginal probe.
7. The simulation system of claim 1, wherein the surface of the
anatomy model is deformable.
8. The simulation system of claim 1, further comprising a position
and orientation sensor configured and positioned to sense a
position and/or orientation of the anatomy model.
9. The simulation system of claim 1, further comprising a position
and orientation sensor configured and positioned to sense a
position and/or orientation of the tool.
10. A tool for a medical simulation system comprising: a body; and
a tip connected to the body and adapted to interact with low
friction with a surface of an anatomy model.
11. The tool of claim 10, wherein the tool tip is adapted with a
roller ball.
12. The tool of claim 10, wherein the tool tip is adapted with a
low friction material.
13. The tool of claim 12, wherein the surface of the anatomy model
is deformable.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to 62/242,322 filed on Oct.
16, 2015, which is herein incorporated by reference in its
entirety.
FIELD
[0002] The present invention relates to computerized medical
simulation in general, and more specifically to virtual reality
and/or augmented reality simulation devices and systems for medical
training purposes.
BACKGROUND
Virtual Reality/Augmented Reality Medical Simulation
[0003] Medical imaging has become more and more used for both
diagnostic/examination and therapeutic purposes in a number of
medical applications, such as endoscopy for surgery, or ultrasound
imaging for various gynecology and/or obstetrics applications, for
instance in the embryo transfer procedure for In Vitro
Fertilization (IVF). These new techniques may require dedicated
training for physicians and surgeons to master the indirect
hand-eye coordination required by the imaging system as well as the
manipulation of the imaging tools in addition to the conventional
medical instruments and procedures for a diversity of patient
anatomies as may be encountered in medical practice. Computerized
medical procedure training simulators may enable the physicians and
trainees to develop and improve their practice in a virtual reality
environment before actually practicing in the operation room.
[0004] Advanced medical procedure simulators may be based on a
virtual reality ("VR") and/or a mixed or augmented reality ("AR")
simulation apparatus by which the physician can experiment a
medical procedure scenario. The VR/AR system may compute and
display a visual VR/AR model of anatomical structures in accordance
with physician gestures and actions to provide various feedback,
such as visual feedback. In a VR system, an entire image may be
simulated for display to a user, and in an AR system, a simulated
image may be overlaid or otherwise incorporated with an actual
image for display to a user. Various patient models with different
pathologies can be selected. Therefore, natural variations as
encountered over the years by practicing doctors can be simulated
for a user over a compressed period of time for training purposes.
The medical simulation procedure can be recorded and rehearsed for
evaluation purpose. The VR/AR simulation system can also compute
and provide various metrics and statistics.
[0005] VR/AR simulation systems such as the one described in U.S.
Pat. No. 8,992,230 include a human anatomy model of a joint of
organ in real size. The VR/AR simulation system may further
comprise a medical instrument to more realistically simulate the
medical procedure. The model is further adapted with sensors for
tracking the position and/or orientation of both the anatomy model
and the medical instrument. As described in U.S. Pat. No.
8,992,230, calibration units may be further used to automatically
setup and align the VR/AR simulation system to a diversity of
anatomy models and medical procedure training scenarios without
requiring a cumbersome, manual calibration procedure each time a
new model is adapted to the system.
[0006] A passive feedback VR/AR simulation system such as for
instance the one described in U.S. Pat. No. 8,992,230 may also be
used with a diversity of medical procedure training scenarios, some
of which may possibly result in a mismatch between an anatomy model
surface as touched by the trainee and a virtual environment surface
as computed by the VR/AR simulation system and rendered on the
screen. In order to further improve the passive haptic experience
and increase the realism in such medical training scenarios, the
VR/AR simulation system may be further adapted with space warping
methods and systems as described in US patent application
US20140071165.
Ultrasound Imaging Simulation
[0007] Most prior art ultrasound simulation solutions have been
developed based on interpolative ultrasound simulation, as used for
instance by a number of commercial ultrasound training simulators
such as Medsim (http://www.medsim.com/), EchoCom
(http://www.echocom.de) and MedCom/Sonofit (http://www.sonofit.de).
These prior art simulators use a set of collected 2D images for
each case, which are reconstructed into 3D volumes per case in an
offline "case generation" stage with minor differences between
individual methods in how the 3D volumes and their corresponding
graphical models are generated. For instance, Dror Aiger and Daniel
Cohen-Or described the use of deformable registration techniques
for generating large 3D volumes from smaller swept scans in
"Real-time ultrasound imaging simulation", Real-Time Imaging,
4(4):263-274, 1998. In "Augmented reality simulator for training in
two-dimensional echocardiography", Computers and Biomedical
Research, 33:11-22, 2000, M. Weidenbach, C. Wick, S. Pieper, K. J.
Quast, T. Fox, G. Grunst, and D. A. Redel proposed to register the
recorded patient-specific volumes to a generic 3D anatomical heart
model, which they thus call augmented training. All the above
methods present the user a non-deformable model of the anatomy. In
other words, although the images may change with the transducer
position/orientation in some simulators, they do not change
according to the trainee's interaction with the mannequin or the
virtual model, which negatively impacts the simulation realism.
Deformable Interactive Ultrasound Simulation
[0008] Compared to non deformable solutions, deformable,
interactive ultrasound simulation may generate a better sense of
simulator realism and consequent trainee immersion. In "B-mode
ultrasound image simulation in deformable 3-D medium", IEEE Trans
Medical Imaging, 28(11):1657-69, November 2009, O. Goksel and S. E.
Salcudean introduced the first interpolative simulator that allows
for deformation by using a fast mapping technique for image pixels
to be simulated, from the deformed simulation tissue coordinates to
a nominal recorded volumetric coordinate frame. This method
combines an input (reconstructed) 3D volume and interactive,
volumetric tissue deformation models such as the finite element
method and has been further applied to prostate brachytherapy
simulation as published in Orcun Goksel, Kirill Sapchuk, and
Septimiu E. Salcudean, "Haptic simulator for prostate brachytherapy
with simulated needle and probe interaction", IEEE Trans Haptics,
4(3):188-198, May 2011. It has also been applied to a transrectal
ultrasound training application with fluoroscopy imaging as
published in Orcun Goksel, Kirill Sapchuk, William James Morris,
and Septimiu E. Salcudean, "Prostate brachytherapy training with
simulated ultrasound and fluoroscopy images", IEEE Trans Biomedical
Engineering, 60(4):n2002-12, April 2013.
[0009] However, the latter methods by Goksel et al. only used a
virtual patient model for simplicity. In a passive haptic VR/AR
medical simulator, a physical anatomy model (mannequin) is further
used with medical tools or instruments to simulate the medical
procedure training as realistically as possible. For example in a
state of the art embryo transfer IVF medical procedure as
represented by FIG. 1, tools such as an ultrasound probe 130, a
speculum 120, and an embryo transfer catheter 112 may be used. As
described for instance by
http://www.advancedfertility.com/ivf-embryo-transfer-catheter.htm,
a stiffer outer steath (catheter guide) is first used to guide the
soft and flexible inner embryo transfer catheter 112 through the
cervical canal to the proper location in the uterine cavity 100.
Once at the right location, the transfer catheter 112 is loaded
with the embryos 115 which are propulsed with the embryo transfer
syringe 110 into the uterine cavity 100. The whole procedure
requires very careful gesture as any misposition or wound to the
uterine may result in the failure of the IVF, which is a very
costly and time-consuming procedure for the patients. The IVF
embryo transfer procedure is conducted under ultrasound imaging
supervision, by means of an abdominal ultrasound probe 130. The
user manipulates the abdominal ultrasound transducer 130 by
pressing it on the belly skin and positioning and orienting it to
optimize the ultrasound imaging capture. A jelly is used to
facilitate the ultrasound propagation as well as the manipulation
of the ultrasound transducer in contact with the patient skin. In
general, a moderately full bladder 140 is advisable for better
ultrasound imaging quality, which will more or less deform with the
probe compression on the above patient skin. The speculum tool 120
is usually made of metal and thus acts as a shield to the
ultrasound waves, which also results in specific artefacts into the
ultrasound images.
[0010] As known to those skilled in the art, the simulation of such
a complex ultrasound procedure may raise specific issues to ensure
a realistic virtual-physical dualism, e.g. the image should appear
only when the ultrasound probe physically touches the anatomy model
also emanating only from the curved surface of contact. Deformation
of the ultrasound images and its components such as the skin (high
compression), the bladder (moderate compression) and the uterine,
the speculum and the catheter (no compression) needs to be
synchronized with the user manipulation of the ultrasound probe, so
the latter interaction remains as realistic as possible.
[0011] In "Patient-Specific Interactive Ultrasound Image Simulation
with Soft-Tissue Deformation", PhD thesis, University of California
Los Angeles, 2013, K. Petrinec proposed to adapt a commercially
available ultrasound simulator (www.sonosim.com) to further
simulate the deformation of soft tissues when in contact with the
ultrasound probe, based on the Goksel method. In this simulator,
only 3-DOF motion trackers are used to track the ultrasound probe
orientation--that is, the probe is assumed to be positioned at a
given 3D static position over a mannequin, and the system does not
track its translational movement.
[0012] More recently, as described in US20150154890, Sonosim
further introduced a 5-DOF tracking solution for this simulator by
adding a translation sensor as a patch that can be applied over a
live body or an anatomy model, in association with an electronic
tag, so that the probe translation over a curved surface may be
further tracked. From a passive haptic perspective, one major
drawback of this solution is the lack of realism when manipulating
the probe compared to a real ultrasound procedure, as no ultrasound
gel can be used with this probe according to its instructions of
use (http://sonosim.com/support/). In particular, friction of the
rigid plastic virtual probe tip over the anatomy model surface
significantly increases when the user has to push the probe to
compress the skin and get a better ultrasound image. This friction
is much lower in a real procedure where ultrasound gel or jelly is
used to improve the ultrasound capture, but in general, it is not
desirable to use any liquid or gel in a training setup due to the
overhead required to clean it.
[0013] There is therefore a need for better ultrasound simulation
devices, methods and systems that facilitate a diversity of
ultrasound training procedures without requiring an expensive
hardware setup and cumbersome registration and calibration
procedures.
BRIEF SUMMARY
[0014] It is an object of the invention to provide a VR/AR
ultrasound simulation system comprising a data processing unit, a
display in communication with the data processing unit, an anatomy
model and a tool, wherein the tool is adapted to interact with low
friction with a surface of the anatomy model. The tool may be
adapted to roll over a surface of the anatomy model. The tool may
be a replicate of an ultrasound probe, such as an abdominal probe
or a vaginal probe. The surface of the anatomy model may be
deformable. The VR/AR ultrasound simulation system may comprise a
position and orientation sensor configured and positioned to sense
a position and/or orientation of the anatomy model, and/or a
position and orientation sensor configured and positioned to sense
a position and/or orientation of the tool.
[0015] It is a further object of the invention to provide a medical
simulation tool, wherein the tool is adapted to interact with low
friction with a surface of an anatomy model. The tool tip or end,
may be adapted with a roller ball or with a low friction
material.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] FIG. 1 illustrates an IVF embryo transfer procedure as known
in the prior art.
[0017] FIG. 2 illustrates an ultrasound simulator adapted to
simulate an IVF embryo transfer procedure.
[0018] FIG. 3 shows four cervix and uterine anatomy models
corresponding to some anatomy shapes that may be encountered in IVF
embryo transfer medical practice.
[0019] FIG. 4 shows engineering views of an exemplary ultrasound
simulation probe adapted to provide realistic passive haptic
feedback without requiring the use of ultrasound gel over the
anatomy model.
[0020] FIG. 5 shows a first screen capture of an ultrasound
simulation rendering for the IVF embryo transfer procedure
training.
[0021] FIG. 6 shows a second screen capture of an ultrasound
simulation rendering for the IVF embryo transfer procedure
training.
[0022] FIG. 7 illustrates a flow chart of an ultrasound simulation
method in accordance with different possible embodiments.
DETAILED DESCRIPTION
[0023] Embodiments of flexible ultrasound simulation devices,
methods, and systems will now be described in more detail with
reference to an exemplary medical simulation of an IVF embryo
transfer procedure, as illustrated on FIG. 1. FIG. 2 shows a
partial view of an ultrasound simulator system comprising a data
processing unit 200, a display screen 210, and an anatomy model
220, according to an embodiment of the invention. For the purpose
of illustration, in FIG. 2, a pelvic model 220 of the human female
anatomy is shown, but other models can be used as well. Examples of
models can be found in the catalogues of specialized anatomic model
suppliers such as Limbs&Things, Bristol, UK. Some models may be
representative of the human anatomy. Other models may be
representative of an animal anatomy, e.g. for veterinary training
purpose.
[0024] The anatomy model 220 may be made of plastic or any other
suitable material. Preferably, the anatomy model 220 is made of a
flexible material, such as flexible plastic, so that it can deform
under pressure. In some embodiments, the anatomy model 220, or
parts of the anatomy model, may be interchanged with another
similar anatomy model corresponding to different simulated patient
characteristics or pathologies. For instance in an IVF embryo
transfer training simulator, various combinations of cervix canal
models (straight, tortuous, with a trap . . . ) and uterus types
(axial, anteverted, retroverted . . . ) may be used to simulate
more or less challenging variations of the patient pelvic organs
anatomy. FIG. 3 shows four different cervix models: a) base cervix
case with axial uterus, b) cervix entrance deflected down 40
degrees in an anteverted uterus, c) tortuous S-shaped cervix with
axial uterus and d) trap cervix with 20 degrees anteverted
uterus.
[0025] In some embodiments, the anatomy model 220, or parts of the
anatomy model, such as for instance the cervix canal and/or the
uterine in an OB/GYN training application, may be manufactured in
the training room by using a manufacturing unit. Examples of such
manufacturing units are well known to those skilled in the art of
rapid prototyping; they may be based on additive manufacturing,
such as 3D printing, or substractive manufacturing, such as CNC
milling.
[0026] In some embodiments, the anatomy model may also be
manufactured according to a specific patient anatomy. Examples of
manufacturing units as used in emerging medical applications are
described for instance in Popescu, A. T.; Stan, O.; Miclea, L., "3D
printing bone models extracted from medical imaging data", 2014
IEEE International Conference on Automation, Quality and Testing,
Robotics, vol., no., pp. 1,5, 22-24 May 2014. Patient specific
anatomy models may also comprise specific patient pathologies.
Patient specific anatomy models may be manufactured and
interchanged with the anatomy model 220. Patient specific anatomy
models may enable the physicians and trainees to develop and
improve their practice in a virtual reality environment before
actually performing the medical procedure. Furthermore, in order to
support other medical procedures with ultrasound imaging, patient
specific anatomy models may be interchanged with the anatomy model
220, such as for instance a bladder, a womb, an upper torso, a
lower torso, or a joint model. In some embodiments the patient
specific model may comprise parts of the anatomy model 220. The
patient specific model may be manufactured by the manufacturing
unit. In some embodiments the patient specific model may be created
with additive manufacturing, such as 3D printing, or substractive
manufacturing, such as CNC milling.
[0027] The data processing unit 200 may comprise one or more
central processing units ("CPU"), memory modules, controlling
modules, and/or communication modules, for example. Other
embodiments may include data processing units 200 with other
configurations and combinations of hardware and software elements.
A distributed data processing unit may be used. In some
embodiments, a communication module of the data processing 200 may
be connected to a manufacturing unit. In some embodiments, the data
processing unit and the manufacturing unit may be combined in a
single unit. Some or all of the data processing unit 200 components
may be used to compute and display onto a display screen 210 a
VR/AR simulation model that may correspond to a chosen medical
procedure training scenario. Multiple display screens may also be
used. The display screen 210 may comprise a touch interface to
provide an interface for a physician during a simulation exercise.
In other embodiments (not illustrated) the simulator cart may
further comprise a camera.
[0028] In FIG. 2, the anatomy model 220 may comprise at least one
position and orientation sensor (not represented). As described in
U.S. Pat. No. 8,992,230, said position and orientation sensor may
be associated with at least one calibration unit. For example, six
degree of freedom ("6DOF") miniaturized magnetic tracking sensors
may be used. Other embodiments are also possible, e.g. optical
tracking or motion tracking may be used. The anatomy model 220 may
be connected to the data processing unit 200 through a connection
such as a USB link, other standard wired or wireless connection, or
other communication link. The data processing unit 200 may receive
the sensor information from the anatomy model 220 and may calculate
the virtual anatomy model 220 position and orientation in
accordance with the sensor measurement. The data processing unit
200 may use the calculated model 220 position and orientation to
generate a visual model and display the visual model onto the
display screen 210. As described in U.S. Pat. No. 8,992,230, the
data processing unit 200 may compute the VR/AR model position and
orientation by combining the absolute sensor measurement received
from at least one anatomy model sensor with the pre-computed
calibration data from the calibration unit associated with said
sensor for the simulated VR/AR model.
[0029] FIG. 2 further shows a replica of an ultrasound probe 230
which may be used to simulate an ultrasound imaging capture. For
the purpose of illustration, in the case of IVF embryo transfer
simulation, the ultrasound probe replica 230 of FIG. 2 replicates
the shape of a convex abdominal ultrasound transducer. Other probes
may be replicated in other ultrasound training procedures, for
instance a transvaginal or a transrectal probe. The replica may not
need to be an exact replica of an ultrasound probe, but it may give
a realistic haptic feedback similar to a real probe. The ultrasound
probe replica 230 may be handled and put in contact with the
anatomy model skin surface by the trainee. The trainee may also
push the anatomy model 220 skin surface with the ultrasound probe
replica 230 as in a real imaging procedure, and the anatomy model
220 skin surface may deform accordingly. The ultrasound probe
replica 230 may be adapted to comprise at least one position and
orientation sensor, possibly in association with a calibration
unit, for instance as described in U.S. Pat. No. 8,992,230. The
data processing unit 200 may receive the sensor information from
the ultrasound probe replica 230 and may calculate the VR/AR model
position and orientation in accordance with the model sensor
measurement and the probe replica sensor measurement
respectively.
[0030] FIG. 2 further shows a medical tool 240 appended to the
anatomy model 220. For the purpose of illustration, in the case of
IVF embryo transfer simulation, the medical tool 240 of FIG. 2 is a
speculum that can be inserted into the pelvic model 220 through the
vulva portal of anatomy model. Other tools may be used, for
instance an embryo transfer catheter, an embryo transfer syringe, a
tenaculum or, a Hegra dilator. The medical tool 240 may be a
standard medical tool suitable for various medical procedures, for
instance a tenaculum tool that may be used in gynecology or in
surgery. In some embodiments, the tool may be adapted to comprise
at least one position and orientation sensor and one calibration
unit associated with the position and orientation sensor, for
instance as described in U.S. Pat. No. 8,992,230. In other
embodiments, the position and orientation of the tool may be
derived from the position and orientation of another element in the
simulator system, such as an internal element to which the tool is
mechanically bound without any remaining degree of freedom; for
instance the tenaculum for steadying the cervix and uterus during
the insertion of an intrauterine device is typically bound to the
cervix position and orientation once steadying it. Thus, in some
embodiments of the simulator system, a part of the anatomy model,
for instance the cervix, rather than the tool, for instance the
tenaculum, may be adapted to comprise at least one position and
orientation sensor and one calibration unit associated with the
position and orientation sensor, for instance as described for the
femur or tibia parts in U.S. Pat. No. 8,992,230. Other embodiments
are also possible to adapt the VR/AR ultrasound simulator to
various medical practices, anatomy models, and tools.
[0031] The data processing unit 200 may receive the sensor
information from the anatomy model 220 and may calculate the VR/AR
model position and orientation in accordance with the anatomy model
sensor measurement, the probe replica sensor measurement, and the
tool sensor measurement respectively.
[0032] In a preferred embodiment, the tip, or end, attached to the
tool body of the ultrasound probe replica 230 is further
mechanically adapted to comprise a rolling part 235, such as a
roller ball, which may simulate a more realistic low friction
motion, when the ultrasound probe replica 230 tip interacts with
the anatomy model 220 surface. FIG. 4 shows the engineering views
of a possible mechanical embodiment of a roller ball 235 which
decreases the friction for different translational and rotational
movements of the ultrasound probe replica 230 over the anatomy
model 220 surface. Other embodiments than a roller ball are also
possible, for instance a low friction material may be used at the
tip 235 of the ultrasound probe replica 230, or on the anatomy
model 220 surface.
[0033] The data processing unit 200 may use the calculated VR/AR
model position and orientation to generate a visual model and
display the visual model onto the display screen 210. As known to
those skilled in the art, initial alignment of the VR/AR model with
the position and orientation of the anatomy model 220, the
ultrasound probe replica 230 or the medical tool 240 may also
require calibration. The methods and systems as described in U.S.
Pat. No. 8,992,230 for an endoscopy simulation system may be used
to calibrate the VR/AR ultrasound simulation system.
[0034] In an ultrasound simulation system, the ultrasound image
also needs to be rendered as realistically as possible. The data
processing unit 200 may use the calculated VR/AR model position and
orientation to identify a slice to be rendered from a reconstructed
3D ultrasound volume or a 3D ultrasound generation model and to
render the simulated ultrasound image in real time onto the display
screen 210.
[0035] FIG. 5 shows a screen capture of the ultrasound image and
the VR/AR model as rendered on display screen 210 by the data
processing unit 200 in the embryo transfer procedure. The embryo
transfer catheter tool 540 is rendered into the uterine cavity 500
on the ultrasound image 530 on the left as well as on the VR/AR
model computer graphics representation on the right. A moderately
filled bladder 520 and abdominal skin layers 510 are also visible
in the ultrasound image as in a real ultrasound capture. FIG. 6
shows a screen capture of the ultrasound image and the VR/AR model
as rendered on display screen 210 by the data processing unit 200
at the next step in the embryo transfer procedure, where the
trainee transfers the embryos 600 from the embryo transfer catheter
into the uterine cavity.
[0036] In a possible embodiment, the data processing unit 200 may
be adapted to implement the interpolative ultrasound simulation
methods as proposed by Goksel et al. to render realistic ultrasound
images by interpolating a 2D slice in a 3D ultrasound volume
reconstructed from formerly registered ultrasound image records,
such as MR or CT data records, in combination with real-time
induced deformations due to the user interaction with the
ultrasound simulator system. Other methods may be used, for
instance the ultrasound simulation may use a ray-based generated
ultrasound simulation with segmentation rather that interpolative
ultrasound simulation methods, as known to those skilled in the art
of ultrasound simulation. A diversity of different patient VR/AR
models may also be used as the basis for a diversity of ultrasound
practice training scenarios, for instance to represent different
pathologies. In order to further improve the passive haptic
experience and increase the realism in such diversity of medical
training scenarios without requiring too many different setups of
the underlying anatomy model, the VR/AR ultrasound simulation
system and methods may be further adapted with space warping
methods and systems as described in US patent application
US20140071165.
[0037] FIG. 7 shows a general flowchart of the proposed VR/AR
ultrasound simulation method, comprising the steps of: [0038]
Acquiring the position and orientation of the VR/AR simulator
elements, such as the anatomy model parts, the ultrasound probe
replica and any medical tools relevant to the ultrasound procedure
training; [0039] Aligning the VR/AR model to the tracked position
and orientation of the VR/AR simulator elements; [0040] Generating
an ultrasound image matching the VR/AR model; [0041] Rendering the
ultrasound image and a computer graphics representation of the
VR/AR model onto a display screen.
[0042] While a VR/AR ultrasound simulator for the IVF embryo
transfer procedure has been detailed herein as an example for a
better illustration of the disclosure, the proposed methods and
systems can be generalized to any type of ultrasound
simulations.
[0043] While a passive haptic VR/AR ultrasound simulator has been
detailed herein as an example for a better illustration of the
disclosure, the proposed methods and systems can be generalized to
any type of hardware simulator setups. In particular, various
position and orientation tracking means may be used, such as
magnetic or optical tracking systems, or a mix of such systems.
Various calibration procedures may also be used. Active haptic
hardware may also be integrated into the VR/AR ultrasound
simulator.
[0044] While a VR/AR ultrasound simulation method using
interpolation from a reconstructed 3D ultrasound volume has been
detailed herein as an example for a better illustration of the
disclosure, the proposed methods and systems can be generalized to
any type of ultrasound imaging simulations. In particular,
generative ray-based simulation systems and methods may also be
used for ultrasound image modeling.
[0045] While various embodiments of an IVF embryo transfer
simulator have been described above, it should be understood that
they have been presented by way of example and not limitation. It
will be apparent to persons skilled in the relevant art(s) that the
proposed methods and systems can be generalized to any type of
ultrasound simulations. Various changes in form and detail can be
made therein without departing from the spirit and scope. In fact,
after reading the above description, it will be apparent to one
skilled in the relevant art(s) how to implement alternative
embodiments. Thus, the present embodiments should not be limited by
any of the above-described embodiments.
[0046] In addition, it should be understood that any figures which
highlight the functionality and advantages are presented for
example purposes only. The disclosed methodology and system are
each sufficiently flexible and configurable such that they may be
utilized in ways other than that shown.
[0047] Although the term "at least one" may often be used in the
specification, claims and drawings, the terms "a", "an", "the",
"said", etc. also signify "at least one" or "the at least one" in
the specification, claims and drawings.
[0048] Finally, it is the applicant's intent that only claims that
include the express language "means for" or "step for" be
interpreted under 35 U.S.C. 112, paragraph 6. Claims that do not
expressly include the phrase "means for" or "step for" are not to
be interpreted under 35 U.S.C. 112, paragraph 6.
* * * * *
References