U.S. patent application number 17/608715 was filed with the patent office on 2022-07-07 for patient-matched orthopedic implant.
The applicant listed for this patent is Howmedica Osteonics Corp.. Invention is credited to Jean Chaoui, Pierric Deransart, Florence Delphine Muriel Maille, Sergii Poltaretskyi, Vincent Abel Maurice Simoes.
Application Number | 20220211507 17/608715 |
Document ID | / |
Family ID | 1000006285390 |
Filed Date | 2022-07-07 |
United States Patent
Application |
20220211507 |
Kind Code |
A1 |
Simoes; Vincent Abel Maurice ;
et al. |
July 7, 2022 |
PATIENT-MATCHED ORTHOPEDIC IMPLANT
Abstract
An example system for designing a patient matched implant for an
orthopedic joint repair surgical procedure includes a memory
configured to store a model of a bone of a patient; and processing
circuitry. The processing circuitry may be configured to: obtain
the model of the bone of the patient; obtain a template model of an
implant; determine a shape of a surface of the implant; determine a
volume between the shape of the surface of the implant and a
surface of the bone defined by the model of the bone; generate,
based on the determined volume and the template model, a patient
matched implant model; and output a file representing the patient
matched implant model.
Inventors: |
Simoes; Vincent Abel Maurice;
(Locmaria Plouzane, FR) ; Deransart; Pierric;
(Saint Martin d'Uriage, FR) ; Poltaretskyi; Sergii;
(Plougonvelin, FR) ; Chaoui; Jean; (Locmaria
Plouzane, FR) ; Maille; Florence Delphine Muriel;
(Locmaria Plouzane, FR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Howmedica Osteonics Corp. |
Mahwah |
NJ |
US |
|
|
Family ID: |
1000006285390 |
Appl. No.: |
17/608715 |
Filed: |
May 1, 2020 |
PCT Filed: |
May 1, 2020 |
PCT NO: |
PCT/US2020/031116 |
371 Date: |
November 3, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62847100 |
May 13, 2019 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A61B 5/0013 20130101;
A61B 34/20 20160201; A61B 5/7425 20130101; A61B 5/1124 20130101;
A61B 5/1079 20130101; A61B 5/0035 20130101; A61B 5/1075 20130101;
B33Y 80/00 20141201; A61B 5/0077 20130101; B22F 12/00 20210101;
A61B 34/10 20160201; A61F 2/30942 20130101; B33Y 10/00 20141201;
A61B 5/1114 20130101; A61B 5/4851 20130101; B33Y 50/00 20141201;
A61F 2/4081 20130101; A61B 5/4528 20130101; A61B 5/1077
20130101 |
International
Class: |
A61F 2/30 20060101
A61F002/30; A61B 34/10 20060101 A61B034/10; A61B 34/20 20060101
A61B034/20; A61B 5/00 20060101 A61B005/00; A61B 5/107 20060101
A61B005/107; A61B 5/11 20060101 A61B005/11; A61F 2/40 20060101
A61F002/40; B22F 12/00 20060101 B22F012/00 |
Claims
1-39. (canceled)
40. A system for designing a patient matched implant for an
orthopedic joint repair surgical procedure, the system comprising:
a memory configured to store a model of a bone of a patient; and
processing circuitry configured to: obtain the model of the bone of
the patient; obtain a template model of an implant; determine a
shape of a surface of the implant; determine a volume between the
shape of the surface of the implant and a surface of the bone
defined by the model of the bone; generate, based on the determined
volume and the template model, a patient matched implant model; and
output a file representing the patient matched implant model.
41. The system of claim 40, wherein, to generate the patient
matched implant model, the processing circuitry is configured to:
add the determined volume to the template model to generate the
patient matched implant model.
42. The system of claim 40, wherein the template model of the
implant comprises a pre-defined porous model, and wherein, to
generate the patient matched implant model, the processing
circuitry is configured to: add the determined volume to the
pre-defined porous model to generate a patient matched porous
model.
43. The system of claim 42, wherein the processing circuitry is
further configured to: populate the patient matched porous model
with a porous structure.
44. The system of claim 43, wherein the porous structure is
generic.
45. The system of claim 43, wherein the porous structure is patient
matched.
46. The system of claim 43, wherein the template model of the
implant further comprises a pre-defined solid model, and wherein
the processing circuitry is further configured to: generate the
patient matched implant model based on the patient matched porous
model and the pre-defined solid model.
47. The system of claim 40, wherein, to obtain the model of the
bone, the processing circuitry is configured to obtain a
three-dimensional model of the bone as the bone exists before an
operation to implant the patient matched implant.
48. The system of claim 40, wherein, to obtain the model of the
bone, the processing circuitry is configured to obtain a
three-dimensional model of the bone as the bone will exist after
one or more work steps are performed during an operation to implant
the patient matched implant.
49. The system of claim 40, wherein the processing circuitry is
further configured to generate a model of an area of interest on
the bone based on the model of the bone, and wherein, to generate
the patient matched implant model, the processing circuitry is
configured to generate the patient matched implant model based on
the model of the area of interest.
50. The system of claim 49, wherein the bone comprises a scapula of
the patient, wherein the area of interest comprises a glenoid of
the scapula, and wherein the surface of the implant comprises a
backside of a baseplate of a glenoid implant.
51. The system of claim 40, further comprising an additive
manufacturing device configured to fabricate a physical patient
matched implant based on the file representing the patient matched
implant model.
52. The system of claim 51, wherein the additive manufacturing
device comprises a direct metal laser sintering (DMLS) device.
53. A computer-implemented method for designing a patient matched
implant for an orthopedic joint repair surgical procedure, the
method comprising: obtaining a model of the bone of the patient;
obtaining a template model of an implant; determining a shape of a
surface of the implant; determining a volume between the shape of
the surface of the implant and a surface of the bone defined by the
model of the bone; generating, based on the determined volume and
the template model, a patient matched implant model; and outputting
a file representing the patient matched implant model.
54. The method of claim 53, wherein generating the patient matched
implant model comprises: combining the determined volume and the
template model to generate the patient matched implant model.
55. The method of claim 53, wherein the template model of the
implant comprises a pre-defined porous model, and wherein
generating the patient matched implant model comprises: combining
the determined volume and the pre-defined porous model to generate
a patient matched porous model.
56. The method of claim 55, further comprising populating the
patient matched porous model with a porous structure.
57. The method of claim 56, wherein the porous structure is
generic.
58. The method of claim 56, wherein the porous structure is patient
matched.
59. The method of claim 55, wherein the template model of the
implant further comprises a pre-defined solid model, and wherein
the method further comprises: generating the patient matched
implant model based on the patient matched porous model and the
pre-defined solid model.
60. The method of claim 53, wherein obtaining the model of the bone
comprises obtaining a three-dimensional model of the bone as the
bone exists before an operation to implant the patient matched
implant.
61. The method of claim 53, wherein obtaining the model of the bone
comprises obtaining a three-dimensional model of the bone as the
bone will exist after one or more work steps are performed during
an operation to implant the patient matched implant.
62. The method of claim 53, further comprising generating a model
of an area of interest on the bone based on the model of the bone,
and wherein generating the patient matched implant model comprises
generating the patient matched implant model based on the model of
the area of interest.
63. The method of claim 62, wherein the bone comprises a scapula of
the patient, wherein the area of interest comprises a glenoid of
the scapula, and wherein the surface of the implant comprises a
backside of a baseplate of a glenoid implant.
64. The method of claim 53, further comprising: displaying, via a
visualization device and overlaid on a portion of the bone of the
patient viewable via the visualization device, a virtual model of
the portion of the bone obtained from a virtual surgical plan for
the orthopedic joint repair surgical procedure; and displaying, via
the visualization device and overlaid on the portion of the bone, a
virtual guide that guides attachment of the patient matched implant
to the bone.
65. The method of claim 53, further comprising fabricating a
physical patient matched implant based on the file representing the
patient matched implant model.
66. The method of claim 65, wherein fabricating the physical
patient matched implant comprises additively manufacturing the
physical patient matched implant.
67. The method of claim 66, wherein additively manufacturing the
physical patient matched implant comprises additively manufacturing
the physical patient matched implant using direct metal laser
sintering (DMLS).
68. A computer-readable storage medium storing instructions that,
when executed, cause one or more processors to design a patient
matched implant for an orthopedic joint repair surgical procedure,
wherein the instructions that cause the one or more processors to
design the patient matched implant comprise instructions that cause
the one or more processors to: obtain a model of the bone of the
patient; obtain a template model of an implant; determine a shape
of a surface of the implant; determine a volume between the shape
of the surface of the implant and a surface of the bone defined by
the model of the bone; generate, based on the determined volume and
the template model, a patient matched implant model; and output a
file representing the patient matched implant model.
69. The computer-readable storage medium of claim 68, wherein the
template model of the implant comprises a pre-defined porous model
and a pre-defined solid model, and wherein the instructions that
cause the one or more processors to generate the patient matched
implant model comprise instructions that cause the one or more
processors to: combine the determined volume and the pre-defined
porous model to generate a patient matched porous model; populate
the patient matched porous model with a porous structure; and
generate the patient matched implant model based on the populated
patient matched porous model and the pre-defined solid model.
70. The computer-readable storage medium of claim 69, wherein the
bone comprises a scapula of the patient, and wherein the surface of
the implant comprises a backside of a baseplate of a glenoid
implant.
Description
BACKGROUND
[0001] Surgical joint repair procedures involve repair and/or
replacement of a damaged or diseased joint. Many times, a surgical
joint repair procedure, such as joint arthroplasty as an example,
involves replacing the damaged joint with a prosthetic, or set of
prosthetics, that is implanted into the patient's bone. Proper
selection of a prosthetic that is appropriately sized and shaped
and proper positioning of that prosthetic to ensure an optimal
surgical outcome can be challenging. To assist with positioning,
the surgical procedure often involves the use of surgical
instruments to control the shaping of the surface of the damaged
bone and cutting or drilling of bone to accept the prosthetic.
SUMMARY
[0002] This disclosure describes a variety of techniques for
designing, manufacturing, and using patient specific implants for
surgical joint repair procedures. The techniques may be used
independently or in various combinations to support particular
phases or settings for surgical joint repair procedures or to
provide a multi-faceted ecosystem to support surgical joint repair
procedures. In various examples, this disclosure describes
techniques for preoperative surgical planning including implant
design, implant manufacture, intra-operative surgical planning,
intra-operative surgical guidance, intra-operative surgical
tracking and post-operative analysis using mixed reality (MR)-based
visualization. In some examples, the disclosure also describes
surgical items and/or methods for performing surgical joint repair
procedures.
[0003] The details of various examples of the disclosure are set
forth in the accompanying drawings and the description below.
Various features, objects, and advantages will be apparent from the
description, drawings, and claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 is a block diagram of an orthopedic surgical system
according to an example of this disclosure.
[0005] FIG. 2 is a block diagram of an orthopedic surgical system
that includes a mixed reality (MR) system, according to an example
of this disclosure.
[0006] FIG. 3 is a flowchart illustrating example phases of a
surgical lifecycle.
[0007] FIG. 4 is a flowchart illustrating preoperative,
intraoperative and postoperative workflows in support of an
orthopedic surgical procedure.
[0008] FIG. 5 is a schematic representation of a visualization
device for use in a mixed reality (MR) system, according to an
example of this disclosure.
[0009] FIG. 6 is a block diagram illustrating example components of
a visualization device for use in a mixed reality (MR) system,
according to an example of this disclosure.
[0010] FIG. 7 is a flowchart illustrating example steps in the
preoperative phase of the surgical lifecycle.
[0011] FIG. 8 is a flowchart illustrating example steps for
tailoring a surgical plan to a patient.
[0012] FIG. 9 is a flowchart illustrating example steps for
obtaining a model of a bone of a patient.
[0013] FIGS. 10A-10D are conceptual diagrams illustrating example
phases in a mask generation process.
[0014] FIG. 11 is a flowchart illustrating example steps for
generating a patient matched implant model.
[0015] FIGS. 12A-12I are conceptual diagrams illustrating example
phases in a patient matched implant design process.
[0016] FIGS. 13A-13C are conceptual diagrams illustrating example
views of a virtual extrusion for a patient matched implant design
process.
[0017] FIGS. 14A and 14B are conceptual diagrams illustrating a
virtual extrusion and corresponding projected points for a patient
matched implant design process.
[0018] FIGS. 15A and 15B are conceptual diagrams illustrating
examples of patient matched implants.
[0019] FIGS. 16A and 16B are conceptual diagrams illustrating
examples of patient matched implants.
[0020] FIGS. 17A and 17B are conceptual diagrams illustrating
examples of patient matched implants.
[0021] FIG. 18 illustrates an example of a page of a user interface
of a mixed reality (MR) system, according to an example of this
disclosure.
[0022] FIG. 19 is an example of an install guide page of the user
interface of FIG. 18, according to an example of this
disclosure.
[0023] FIG. 20 is an example of an install implant page of the user
interface of FIG. 18, according to an example of this
disclosure.
[0024] FIG. 21 is a flowchart illustrating example stages of a
shoulder joint repair surgery.
[0025] FIG. 22 illustrates an image perceptible to a user when in
an augment surgery mode of a mixed reality (MR) system, according
to an example of this disclosure.
[0026] FIG. 23 is a conceptual diagram illustrating an MR system
providing virtual guidance to a user for installation of a guide in
a glenoid of a scapula, in accordance with one or more techniques
of this disclosure.
[0027] FIG. 24 is a conceptual diagram illustrating an example
guide as installed in a glenoid in a shoulder arthroplasty
procedure.
[0028] FIG. 25 is a conceptual diagram illustrating reaming of a
glenoid in a shoulder arthroplasty procedure, in accordance with
one or more techniques of this disclosure.
[0029] FIGS. 26 and 27 are conceptual diagrams illustrating
creation of a central hole in a glenoid in a shoulder arthroplasty
procedure, in accordance with one or more techniques of this
disclosure.
[0030] FIG. 28 is a conceptual diagram illustrating a glenoid
prosthesis with keel type anchorage.
[0031] FIGS. 29-31 are conceptual diagrams illustrating creation of
keel type anchorage positions in a glenoid in a shoulder
arthroplasty procedure, in accordance with one or more techniques
of this disclosure.
[0032] FIG. 32 is a conceptual diagram illustrating a glenoid
prosthesis with pegged type anchorage.
[0033] FIGS. 33 and 34 are conceptual diagrams illustrating
creation of pegged type anchorage positions in a glenoid in a
shoulder arthroplasty procedure, in accordance with one or more
techniques of this disclosure.
[0034] FIG. 35 is a conceptual diagram illustrating attachment of
an implant to a glenoid in a shoulder arthroplasty procedure, in
accordance with one or more techniques of this disclosure.
[0035] FIGS. 36 and 37 illustrate screws and a central stem that
may be used to attach a prothesis to a glenoid in a shoulder
arthroplasty procedure.
DETAILED DESCRIPTION
[0036] In some orthopedic surgical procedures, a surgeon may
implant one or more implant devices in a patient. The implant
devices may be available in several different standard shapes,
styles, and sizes. The surgeon may select a particular prosthetic
device (e.g., a particular shape, style, and/or size) to implant
based on various characteristic of the patient. The surgeon may
perform various steps to prepare the patient's bone to receive the
implant device. These steps may include removal of portions of the
bone (e.g., via reaming) in order to create a surface of the bone
that matches a surface of the implant device. Matching surfaces
between the bone and the implant device may provide for better
patient outcomes (e.g., as the implant device may have a better fit
with the bone and be more solidly affixed to the bone). However, in
some examples, it may be desirable to minimize, or eliminate, the
need to remove portions of a bone to prepare the bone to receive an
implant device. For instance, patients who undergo an orthopedic
surgical procedure may have limited healthy bone available.
[0037] In accordance with one or more techniques of this
disclosure, a system (e.g., a surgical planning system) may
facilitate the designing of patient specific implant devices. For
instance, the system may obtain a three-dimensional (3D) model of a
bone of the patient (e.g., generated based on images of the bone,
such as x-ray or magnetic resonance imaging (MRI) images), and a
template model of an implant device (e.g., a computer-aided design
(CAD) model of the implant device). The system may generate a model
of a patient specific implant device based on the 3D model of the
bone and the template model of the implant device. For instance,
the system may generate the model of a patient specific implant
device such that a surface of the patient specific implant device
matches a surface of the bone.
[0038] The system may output the generated model for manufacturing.
For instance, the system may output the model to be manufactured
into a physical patient specific implant device that a surgeon may
subsequently implant into the patient. In this way, the system may
facilitate the design of patient specific implant devices.
[0039] Orthopedic surgery can involve implanting one or more
prosthetic devices to repair or replace a patient's damaged or
diseased joint. Virtual surgical planning tools that use image data
of the diseased or damaged joint may be used to generate an
accurate three-dimensional bone model that can be viewed and
manipulated preoperatively by the surgeon. These tools can enhance
surgical outcomes by allowing the surgeon to simulate the surgery,
select or design an implant that more closely matches the contours
of the patient's actual bone, and select or design surgical
instruments and guide tools that are adapted specifically for
repairing the bone of a particular patient. Use of these planning
tools typically results in generation of a preoperative surgical
plan, complete with an implant and surgical instruments that are
selected or manufactured for the individual patient. Oftentimes,
once in the actual operating environment, the surgeon may desire to
verify the preoperative surgical plan intraoperatively relative to
the patient's actual bone.
[0040] This verification may result in a determination that an
adjustment to the preoperative surgical plan is needed, such as a
different implant, a different positioning or orientation of the
implant, and/or a different surgical guide for carrying out the
surgical plan. In addition, a surgeon may want to view details of
the preoperative surgical plan relative to the patient's real bone
during the actual procedure in order to more efficiently and
accurately position and orient the implant components. For example,
the surgeon may want to obtain intraoperative visualization that
provides guidance for positioning and orientation of implant
components, guidance for preparation of bone or tissue to receive
the implant components, guidance for reviewing the details of a
procedure or procedural step, and/or guidance for selection of
tools or implants and tracking of surgical procedure workflow.
[0041] Accordingly, this disclosure describes systems and methods
for using a mixed reality (MR) visualization system to assist with
creation, implementation, verification, and/or modification of a
surgical plan before and during a surgical procedure. Because MR,
or in some instances VR, may be used to interact with the surgical
plan, this disclosure may also refer to the surgical plan as a
"virtual" surgical plan. Visualization tools other than or in
addition to mixed reality visualization systems may be used in
accordance with techniques of this disclosure. A surgical plan,
e.g., as generated by the BLUEPRINT.TM. system, available from
Wright Medical Group, N.V., or another surgical planning platform,
may include information defining a variety of features of a
surgical procedure, such as features of particular surgical
procedure steps to be performed on a patient by a surgeon according
to the surgical plan including, for example, bone or tissue
preparation steps and/or steps for selection, modification and/or
placement of implant components. Such information may include, in
various examples, dimensions, shapes, angles, surface contours,
and/or orientations of implant components to be selected or
modified by surgeons, dimensions, shapes, angles, surface contours
and/or orientations to be defined in bone or tissue by the surgeon
in bone or tissue preparation steps, and/or positions, axes,
planes, angle and/or entry points defining placement of implant
components by the surgeon relative to patient bone or tissue.
Information such as dimensions, shapes, angles, surface contours,
and/or orientations of anatomical features of the patient may be
derived from imaging (e.g., x-ray, CT, MRI, ultrasound or other
images), direct observation, or other techniques.
[0042] In this disclosure, the term "mixed reality" (MR) refers to
the presentation of virtual objects such that a user sees images
that include both real, physical objects and virtual objects.
Virtual objects may include text, 2-dimensional surfaces,
3-dimensional models, or other user-perceptible elements that are
not actually present in the physical, real-world environment in
which they are presented as coexisting. In addition, virtual
objects described in various examples of this disclosure may
include graphics, images, animations or videos, e.g., presented as
3D virtual objects or 2D virtual objects. Virtual objects may also
be referred to as virtual elements. Such elements may or may not be
analogs of real-world objects. In some examples, in mixed reality,
a camera may capture images of the real world and modify the images
to present virtual objects in the context of the real world. In
such examples, the modified images may be displayed on a screen,
which may be head-mounted, handheld, or otherwise viewable by a
user. This type of mixed reality is increasingly common on
smartphones, such as where a user can point a smartphone's camera
at a sign written in a foreign language and see in the smartphone's
screen a translation in the user's own language of the sign
superimposed on the sign along with the rest of the scene captured
by the camera. In some examples, in mixed reality, see-through
(e.g., transparent) holographic lenses, which may be referred to as
waveguides, may permit the user to view real-world objects, i.e.,
actual objects in a real-world environment, such as real anatomy,
through the holographic lenses and also concurrently view virtual
objects.
[0043] The Microsoft HOLOLENS.TM. headset, available from Microsoft
Corporation of Redmond, Washington, is an example of a MR device
that includes see-through holographic lenses, sometimes referred to
as waveguides, that permit a user to view real-world objects
through the lens and concurrently view projected 3D holographic
objects. The Microsoft HOLOLENS.TM. headset, or similar
waveguide-based visualization devices, are examples of an MR
visualization device that may be used in accordance with some
examples of this disclosure. Some holographic lenses may present
holographic objects with some degree of transparency through
see-through holographic lenses so that the user views real-world
objects and virtual, holographic objects. In some examples, some
holographic lenses may, at times, completely prevent the user from
viewing real-world objects and instead may allow the user to view
entirely virtual environments. The term mixed reality may also
encompass scenarios where one or more users are able to perceive
one or more virtual objects generated by holographic projection. In
other words, "mixed reality" may encompass the case where a
holographic projector generates holograms of elements that appear
to a user to be present in the user's actual physical
environment.
[0044] In some examples, in mixed reality, the positions of some or
all presented virtual objects are related to positions of physical
objects in the real world. For example, a virtual object may be
tethered to a table in the real world, such that the user can see
the virtual object when the user looks in the direction of the
table but does not see the virtual object when the table is not in
the user's field of view. In some examples, in mixed reality, the
positions of some or all presented virtual objects are unrelated to
positions of physical objects in the real world. For instance, a
virtual item may always appear in the top right of the user's field
of vision, regardless of where the user is looking.
[0045] Augmented reality (AR) is similar to MR in the presentation
of both real-world and virtual elements, but AR generally refers to
presentations that are mostly real, with a few virtual additions to
"augment" the real-world presentation. For purposes of this
disclosure, MR is considered to include AR. For example, in AR,
parts of the user's physical environment that are in shadow can be
selectively brightened without brightening other areas of the
user's physical environment. This example is also an instance of MR
in that the selectively-brightened areas may be considered virtual
objects superimposed on the parts of the user's physical
environment that are in shadow.
[0046] Furthermore, in this disclosure, the term "virtual reality"
(VR) refers to an immersive artificial environment that a user
experiences through sensory stimuli (such as sights and sounds)
provided by a computer. Thus, in virtual reality, the user may not
see any physical objects as they exist in the real world. Video
games set in imaginary worlds are a common example of VR. The term
"VR" also encompasses scenarios where the user is presented with a
fully artificial environment in which some virtual object's
locations are based on the locations of corresponding physical
objects as they relate to the user. Walk-through VR attractions are
examples of this type of VR.
[0047] The term "extended reality" (XR) is a term that encompasses
a spectrum of user experiences that includes virtual reality, mixed
reality, augmented reality, and other user experiences that involve
the presentation of at least some perceptible elements as existing
in the user's environment that are not present in the user's
real-world environment. Thus, the term "extended reality" may be
considered a genus for MR and VR. XR visualizations may be
presented in any of the techniques for presenting mixed reality
discussed elsewhere in this disclosure or presented using
techniques for presenting VR, such as VR goggles.
[0048] These mixed reality systems and methods can be part of an
intelligent surgical planning system that includes multiple
subsystems that can be used to enhance surgical outcomes. In
addition to the preoperative and intraoperative applications
discussed above, an intelligent surgical planning system can
include postoperative tools to assist with patient recovery and
which can provide information that can be used to assist with and
plan future surgical revisions or surgical cases for other
patients.
[0049] Accordingly, systems and methods are also described herein
that can be incorporated into an intelligent surgical planning
system, such as artificial intelligence systems to assist with
planning, implants with embedded sensors (e.g., smart implants) to
provide postoperative feedback for use by the healthcare provider
and the artificial intelligence system, and mobile applications to
monitor and provide information to the patient and the healthcare
provider in real-time or near real-time.
[0050] Visualization tools may utilize patient image data to
generate three-dimensional models of bone contours to facilitate
preoperative planning for joint repairs and replacements. These
tools allow surgeons to design and/or select surgical guides and
implant components that closely match the patient's anatomy. These
tools can improve surgical outcomes by customizing a surgical plan
for each patient. An example of such a visualization tool for
shoulder repairs is the BLUEPRINT.TM. system available from Wright
Medical Group, N.V. The BLUEPRINT.TM. system provides the surgeon
with two-dimensional planar views of the bone repair region as well
as a three-dimensional virtual model of the repair region. The
surgeon can use the BLUEPRINT.TM. system to select, design or
modify appropriate implant components, determine how best to
position and orient the implant components and how to shape the
surface of the bone to receive the components, and design, select
or modify surgical guide tool(s) or instruments to carry out the
surgical plan. The information generated by the BLUEPRINT.TM.
system is compiled in a preoperative surgical plan for the patient
that is stored in a database at an appropriate location (e.g., on a
server in a wide area network, a local area network, or a global
network) where it can be accessed by the surgeon or other care
provider, including before and during the actual surgery.
[0051] FIG. 1 is a block diagram of an orthopedic surgical system
100 according to an example of this disclosure. Orthopedic surgical
system 100 includes a set of subsystems. In the example of FIG. 1,
the subsystems include a virtual planning system 102, a planning
support system 104, a manufacturing and delivery system 106, an
intraoperative guidance system 108, a medical education system 110,
a monitoring system 112, a predictive analytics system 114, and a
communications network 116. In other examples, orthopedic surgical
system 100 may include more, fewer, or different subsystems. For
example, orthopedic surgical system 100 may omit medical education
system 110, monitoring system 112, predictive analytics system 114,
and/or other subsystems. In some examples, orthopedic surgical
system 100 may be used for surgical tracking, in which case
orthopedic surgical system 100 may be referred to as a surgical
tracking system. In other cases, orthopedic surgical system 100 may
be generally referred to as a medical device system.
[0052] Users of orthopedic surgical system 100 may use virtual
planning system 102 to plan orthopedic surgeries. Users of
orthopedic surgical system 100 may use planning support system 104
to review surgical plans generated using orthopedic surgical system
100. Manufacturing and delivery system 106 may assist with the
manufacture and delivery of items needed to perform orthopedic
surgeries. Intraoperative guidance system 108 provides guidance to
assist users of orthopedic surgical system 100 in performing
orthopedic surgeries. Medical education system 110 may assist with
the education of users, such as healthcare professionals, patients,
and other types of individuals. Pre- and postoperative monitoring
system 112 may assist with monitoring patients before and after the
patients undergo surgery. Predictive analytics system 114 may
assist healthcare professionals with various types of predictions.
For example, predictive analytics system 114 may apply artificial
intelligence techniques to determine a classification of a
condition of an orthopedic joint, e.g., a diagnosis, determine
which type of surgery to perform on a patient and/or which type of
implant to be used in the procedure, determine types of items that
may be needed during the surgery, and so on.
[0053] The subsystems of orthopedic surgical system 100 (i.e.,
virtual planning system 102, planning support system 104,
manufacturing and delivery system 106, intraoperative guidance
system 108, medical education system 110, pre- and postoperative
monitoring system 112, and predictive analytics system 114) may
include various systems. The systems in the subsystems of
orthopedic surgical system 100 may include various types of
computing systems, computing devices, including server computers,
personal computers, tablet computers, smartphones, display devices,
Internet of Things (IoT) devices, visualization devices (e.g.,
mixed reality (MR) visualization devices, virtual reality (VR)
visualization devices, holographic projectors, or other devices for
presenting extended reality (XR) visualizations), surgical tools,
and so on. A holographic projector, in some examples, may project a
hologram for general viewing by multiple users or a single user
without a headset, rather than viewing only by a user wearing a
headset. For example, virtual planning system 102 may include a MR
visualization device and one or more server devices, planning
support system 104 may include one or more personal computers and
one or more server devices, and so on. A computing system is a set
of one or more computing systems configured to operate as a system.
In some examples, one or more devices may be shared between two or
more of the subsystems of orthopedic surgical system 100. For
instance, in the previous examples, virtual planning system 102 and
planning support system 104 may include the same server
devices.
[0054] In the example of FIG. 1, the devices included in the
subsystems of orthopedic surgical system 100 may communicate using
communications network 116. Communications network 116 may include
various types of communication networks including one or more
wide-area networks, such as the Internet, local area networks, and
so on. In some examples, communications network 116 may include
wired and/or wireless communication links.
[0055] Many variations of orthopedic surgical system 100 are
possible in accordance with techniques of this disclosure. Such
variations may include more or fewer subsystems than the version of
orthopedic surgical system 100 shown in FIG. 1. For example, FIG. 2
is a block diagram of an orthopedic surgical system 200 that
includes one or more mixed reality (MR) systems, according to an
example of this disclosure. Orthopedic surgical system 200 may be
used for creating, verifying, updating, modifying and/or
implementing a surgical plan. In some examples, the surgical plan
can be created preoperatively, such as by using a virtual surgical
planning system (e.g., the BLUEPRINT.TM. system), and then
verified, modified, updated, and viewed intraoperatively, e.g.,
using MR visualization of the surgical plan. In other examples,
orthopedic surgical system 200 can be used to create the surgical
plan immediately prior to surgery or intraoperatively, as needed.
In some examples, orthopedic surgical system 200 may be used for
surgical tracking, in which case orthopedic surgical system 200 may
be referred to as a surgical tracking system. In other cases,
orthopedic surgical system 200 may be generally referred to as a
medical device system.
[0056] In the example of FIG. 2, orthopedic surgical system 200
includes a preoperative surgical planning system 202, a healthcare
facility 204 (e.g., a surgical center or hospital), a storage
system 206, and a network 208 that allows a user at healthcare
facility 204 to access stored patient information, such as medical
history, image data corresponding to the damaged joint or bone and
various parameters corresponding to a surgical plan that has been
created preoperatively (as examples). Preoperative surgical
planning system 202 may be equivalent to virtual planning system
102 of FIG. 1 and, in some examples, may generally correspond to a
virtual planning system similar or identical to the BLUEPRINT.TM.
system.
[0057] In the example of FIG. 2, healthcare facility 204 includes a
mixed reality (MR) system 212. In some examples of this disclosure,
MR system 212 includes one or more processing device(s) (P) 210 to
provide functionalities that will be described in further detail
below. Processing device(s) 210 may also be referred to as
processor(s). In addition, one or more users of MR system 212
(e.g., a surgeon, nurse, or other care provider) can use processing
device(s) (P) 210 to generate a request for a particular surgical
plan or other patient information that is transmitted to storage
system 206 via network 208. In response, storage system 206 returns
the requested patient information to MR system 212. In some
examples, the users can use other processing device(s) to request
and receive information, such as one or more processing devices
that are part of MR system 212, but not part of any visualization
device, or one or more processing devices that are part of a
visualization device (e.g., visualization device 213) of MR system
212, or a combination of one or more processing devices that are
part of MR system 212, but not part of any visualization device,
and one or more processing devices that are part of a visualization
device (e.g., visualization device 213) that is part of MR system
212.
[0058] In some examples, multiple users can simultaneously use MR
system 212. For example, MR system 212 can be used in a spectator
mode in which multiple users each use their own visualization
devices so that the users can view the same information at the same
time and from the same point of view. In some examples, MR system
212 may be used in a mode in which multiple users each use their
own visualization devices so that the users can view the same
information from different points of view.
[0059] In some examples, processing device(s) 210 can provide a
user interface to display data and receive input from users at
healthcare facility 204. Processing device(s) 210 may be configured
to control visualization device 213 to present a user interface.
Furthermore, processing device(s) 210 may be configured to control
visualization device 213 to present virtual images, such as 3D
virtual models, 2D images, and so on. Processing device(s) 210 can
include a variety of different processing or computing devices,
such as servers, desktop computers, laptop computers, tablets,
mobile phones and other electronic computing devices, or processors
within such devices. In some examples, one or more of processing
device(s) 210 can be located remote from healthcare facility 204.
In some examples, processing device(s) 210 reside within
visualization device 213. In some examples, at least one of
processing device(s) 210 is external to visualization device 213.
In some examples, one or more processing device(s) 210 reside
within visualization device 213 and one or more of processing
device(s) 210 are external to visualization device 213.
[0060] In the example of FIG. 2, MR system 212 also includes one or
more memory or storage device(s) (M) 215 for storing data and
instructions of software that can be executed by processing
device(s) 210. The instructions of software can correspond to the
functionality of MR system 212 described herein. In some examples,
the functionalities of a virtual surgical planning application,
such as the BLUEPRINT.TM. system, can also be stored and executed
by processing device(s) 210 in conjunction with memory storage
device(s) (M) 215. For instance, memory or storage system 215 may
be configured to store data corresponding to at least a portion of
a virtual surgical plan. In some examples, storage system 206 may
be configured to store data corresponding to at least a portion of
a virtual surgical plan. In some examples, memory or storage
device(s) (M) 215 reside within visualization device 213. In some
examples, memory or storage device(s) (M) 215 are external to
visualization device 213. In some examples, memory or storage
device(s) (M) 215 include a combination of one or more memory or
storage devices within visualization device 213 and one or more
memory or storage devices external to the visualization device.
[0061] Network 208 may be equivalent to network 116. Network 208
can include one or more wide area networks, local area networks,
and/or global networks (e.g., the Internet) that connect
preoperative surgical planning system 202 and MR system 212 to
storage system 206. Storage system 206 can include one or more
databases that can contain patient information, medical
information, patient image data, and parameters that define the
surgical plans. For example, medical images of the patient's
diseased or damaged bone typically are generated preoperatively in
preparation for an orthopedic surgical procedure. The medical
images can include images of the relevant bone(s) taken along the
sagittal plane and the coronal plane of the patient's body. The
medical images can include X-ray images, magnetic resonance imaging
(MRI) images, computerized tomography (CT) images, ultrasound
images, and/or any other type of 2D or 3D image that provides
information about the relevant surgical area. Storage system 206
also can include data identifying the implant components selected
for a particular patient (e.g., type, size, etc.), surgical guides
selected for a particular patient, and details of the surgical
procedure, such as entry points, cutting planes, drilling axes,
reaming depths, etc. Storage system 206 can be a cloud-based
storage system (as shown) or can be located at healthcare facility
204 or at the location of preoperative surgical planning system 202
or can be part of MR system 212 or visualization device (VD) 213,
as examples.
[0062] MR system 212 can be used by a surgeon before (e.g.,
preoperatively) or during the surgical procedure (e.g.,
intraoperatively) to create, review, verify, update, modify and/or
implement a surgical plan. In some examples, MR system 212 may also
be used after the surgical procedure (e.g., postoperatively) to
review the results of the surgical procedure, assess whether
revisions are required, or perform other postoperative tasks. To
that end, MR system 212 may include a visualization device 213 that
may be worn by the surgeon and (as will be explained in further
detail below) is operable to display a variety of types of
information, including a 3D virtual image of the patient's
diseased, damaged, or postsurgical joint and details of the
surgical plan, such as a 3D virtual image of the prosthetic implant
components selected for the surgical plan, 3D virtual images of
entry points for positioning the prosthetic components, alignment
axes and cutting planes for aligning cutting or reaming tools to
shape the bone surfaces, or drilling tools to define one or more
holes in the bone surfaces, in the surgical procedure to properly
orient and position the prosthetic components, surgical guides and
instruments and their placement on the damaged joint, and any other
information that may be useful to the surgeon to implement the
surgical plan. MR system 212 can generate images of this
information that are perceptible to the user of the visualization
device 213 before and/or during the surgical procedure.
[0063] In some examples, MR system 212 includes multiple
visualization devices (e.g., multiple instances of visualization
device 213) so that multiple users can simultaneously see the same
images and share the same 3D scene. In some such examples, one of
the visualization devices can be designated as the master device
and the other visualization devices can be designated as observers
or spectators. Any observer device can be re-designated as the
master device at any time, as may be desired by the users of MR
system 212.
[0064] In this way, FIG. 2 illustrates a surgical planning system
that includes a preoperative surgical planning system 202 to
generate a virtual surgical plan customized to repair an anatomy of
interest of a particular patient. For example, the virtual surgical
plan may include a plan for an orthopedic joint repair surgical
procedure, such as one of a standard total shoulder arthroplasty or
a reverse shoulder arthroplasty. In this example, details of the
virtual surgical plan may include details relating to at least one
of preparation of glenoid bone or preparation of humeral bone. In
some examples, the orthopedic joint repair surgical procedure is
one of a stemless standard total shoulder arthroplasty, a stemmed
standard total shoulder arthroplasty, a stemless reverse shoulder
arthroplasty, a stemmed reverse shoulder arthroplasty, an augmented
glenoid standard total shoulder arthroplasty, and an augmented
glenoid reverse shoulder arthroplasty.
[0065] The virtual surgical plan may include a 3D virtual model
corresponding to the anatomy of interest of the particular patient
and a 3D model of a prosthetic component matched to the particular
patient to repair the anatomy of interest or selected to repair the
anatomy of interest. Furthermore, in the example of FIG. 2, the
surgical planning system includes a storage system 206 to store
data corresponding to the virtual surgical plan. The surgical
planning system of FIG. 2 also includes MR system 212, which may
comprise visualization device 213. In some examples, visualization
device 213 is wearable by a user. In some examples, visualization
device 213 is held by a user, or rests on a surface in a place
accessible to the user. MR system 212 may be configured to present
a user interface via visualization device 213. The user interface
is visually perceptible to the user using visualization device 213.
For instance, in one example, a screen of visualization device 213
may display real-world images and the user interface on a screen.
In some examples, visualization device 213 may project virtual,
holographic images onto see-through holographic lenses and also
permit a user to see real-world objects of a real-world environment
through the lenses. In other words, visualization device 213 may
comprise one or more see-through holographic lenses and one or more
display devices that present imagery to the user via the
holographic lenses to present the user interface to the user.
[0066] In some examples, visualization device 213 is configured
such that the user can manipulate the user interface (which is
visually perceptible to the user when the user is wearing or
otherwise using visualization device 213) to request and view
details of the virtual surgical plan for the particular patient,
including a 3D virtual model of the anatomy of interest (e.g., a 3D
virtual bone of the anatomy of interest) and a 3D model of the
prosthetic component selected to repair an anatomy of interest. In
some such examples, visualization device 213 is configured such
that the user can manipulate the user interface so that the user
can view the virtual surgical plan intraoperatively, including (at
least in some examples) the 3D virtual model of the anatomy of
interest (e.g., a 3D virtual bone of the anatomy of interest). In
some examples, MR system 212 can be operated in an augmented
surgery mode in which the user can manipulate the user interface
intraoperatively so that the user can visually perceive details of
the virtual surgical plan projected in a real environment, e.g., on
a real anatomy of interest of the particular patient. In this
disclosure, the terms real and real world may be used in a similar
manner. For example, MR system 212 may present one or more virtual
objects that provide guidance for preparation of a bone surface and
placement of a prosthetic implant on the bone surface.
Visualization device 213 may present one or more virtual objects in
a manner in which the virtual objects appear to be overlaid on an
actual, real anatomical object of the patient, within a real-world
environment, e.g., by displaying the virtual object(s) with actual,
real-world patient anatomy viewed by the user through holographic
lenses. For example, the virtual objects may be 3D virtual objects
that appear to reside within the real-world environment with the
actual, real anatomical object.
[0067] FIG. 3 is a flowchart illustrating example phases of a
surgical lifecycle 300. In the example of FIG. 3, surgical
lifecycle 300 begins with a preoperative phase (302). During the
preoperative phase, a surgical plan is developed. The preoperative
phase is followed by a manufacturing and delivery phase (304).
During the manufacturing and delivery phase, patient-specific
items, such as parts and equipment, needed for executing the
surgical plan are manufactured and delivered to a surgical site.
For instance, a patient specific implant may be manufactured based
on a design generated during the preoperative phase. An
intraoperative phase follows the manufacturing and delivery phase
(306). The surgical plan is executed during the intraoperative
phase. In other words, one or more persons perform the surgery on
the patient during the intraoperative phase. The intraoperative
phase is followed by the postoperative phase (308). The
postoperative phase includes activities occurring after the
surgical plan is complete. For example, the patient may be
monitored during the postoperative phase for complications.
[0068] As described in this disclosure, orthopedic surgical system
100 (FIG. 1) may be used in one or more of preoperative phase 302,
the manufacturing and delivery phase 304, the intraoperative phase
306, and the postoperative phase 308. For example, virtual planning
system 102 and planning support system 104 may be used in
preoperative phase 302. Manufacturing and delivery system 106 may
be used in the manufacturing and delivery phase 304. Intraoperative
guidance system 108 may be used in intraoperative phase 306. Some
of the systems of FIG. 1 may be used in multiple phases of FIG. 3.
For example, medical education system 110 may be used in one or
more of preoperative phase 302, intraoperative phase 306, and
postoperative phase 308; pre- and postoperative monitoring system
112 may be used in preoperative phase 302 and postoperative phase
308. Predictive analytics system 114 may be used in preoperative
phase 302 and postoperative phase 308.
[0069] Various workflows may exist within the surgical process of
FIG. 3. For example, different workflows within the surgical
process of FIG. 3 may be appropriate for different types of
surgeries. FIG. 4 is a flowchart illustrating preoperative,
intraoperative and postoperative workflows in support of an
orthopedic surgical procedure. In the example of FIG. 4, the
surgical process begins with a medical consultation (400). During
the medical consultation (400), a healthcare professional evaluates
a medical condition of a patient. For instance, the healthcare
professional may consult the patient with respect to the patient's
symptoms. During the medical consultation (400), the healthcare
professional may also discuss various treatment options with the
patient. For instance, the healthcare professional may describe one
or more different surgeries to address the patient's symptoms.
[0070] Furthermore, the example of FIG. 4 includes a case creation
step (402). In other examples, the case creation step occurs before
the medical consultation step. During the case creation step, the
medical professional or other user establishes an electronic case
file for the patient. The electronic case file for the patient may
include information related to the patient, such as data regarding
the patient's symptoms, patient range of motion observations, data
regarding a surgical plan for the patient, medical images of the
patients, notes regarding the patient, billing information
regarding the patient, and so on.
[0071] The example of FIG. 4 includes a preoperative patient
monitoring phase (404). During the preoperative patient monitoring
phase, the patient's symptoms may be monitored. For example, the
patient may be suffering from pain associated with arthritis in the
patient's shoulder. In this example, the patient's symptoms may not
yet rise to the level of requiring an arthroplasty to replace the
patient's shoulder. However, arthritis typically worsens over time.
Accordingly, the patient's symptoms may be monitored to determine
whether the time has come to perform a surgery on the patient's
shoulder. Observations from the preoperative patient monitoring
phase may be stored in the electronic case file for the patient. In
some examples, predictive analytics system 114 may be used to
predict when the patient may need surgery, to predict a course of
treatment to delay or avoid surgery or make other predictions with
respect to the patient's health.
[0072] Additionally, in the example of FIG. 4, a medical image
acquisition step occurs during the preoperative phase (406). During
the image acquisition step, medical images of the patient are
generated. The medical images may be generated in a variety of
ways. For instance, the images may be generated using a Computed
Tomography (CT) process, a Magnetic Resonance Imaging (MRI)
process, an ultrasound process, or another imaging process. The
medical images generated during the image acquisition step include
images of an anatomy of interest of the patient. For instance, if
the patient's symptoms involve the patient's shoulder, medical
images of the patient's shoulder may be generated. The medical
images may be added to the patient's electronic case file.
Healthcare professionals may be able to use the medical images in
one or more of the preoperative, intraoperative, and postoperative
phases.
[0073] Furthermore, in the example of FIG. 4, an automatic
processing step may occur (408). During the automatic processing
step, virtual planning system 102 (FIG. 1) may automatically
develop a preliminary surgical plan for the patient. In some
examples of this disclosure, virtual planning system 102 may use
machine learning techniques to develop the preliminary surgical
plan based on information in the patient's virtual case file.
[0074] The example of FIG. 4 also includes a manual correction step
(410). During the manual correction step, one or more human users
may check and correct the determinations made during the automatic
processing step. In some examples of this disclosure, one or more
users may use mixed reality or virtual reality visualization
devices during the manual correction step. In some examples,
changes made during the manual correction step may be used as
training data to refine the machine learning techniques applied by
virtual planning system 102 during the automatic processing
step.
[0075] A virtual planning step (412) may follow the manual
correction step in FIG. 4. During the virtual planning step, a
healthcare professional may develop a surgical plan for the
patient. In some examples of this disclosure, one or more users may
use mixed reality or virtual reality visualization devices during
development of the surgical plan for the patient. As discussed in
further detail below, during the virtual planning step, virtual
planning system 102 may design a patient matched implant.
[0076] Furthermore, in the example of FIG. 4, intraoperative
guidance may be generated (414). The intraoperative guidance may
include guidance to a surgeon on how to execute the surgical plan.
In some examples of this disclosure, virtual planning system 102
may generate at least part of the intraoperative guidance. In some
examples, the surgeon or other user may contribute to the
intraoperative guidance.
[0077] Additionally, in the example of FIG. 4, a step of selecting
and manufacturing surgical items is performed (416). During the
step of selecting and manufacturing surgical items, manufacturing
and delivery system 106 (FIG. 1) may manufacture surgical items for
use during the surgery described by the surgical plan. For example,
the surgical items may include surgical implants (e.g., generic
and/or patient specific), surgical tools, and other items required
to perform the surgery described by the surgical plan.
[0078] In the example of FIG. 4, a surgical procedure may be
performed with guidance from intraoperative system 108 (FIG. 1)
(418). For example, a surgeon may perform the surgery while wearing
a head-mounted MR visualization device of intraoperative system 108
that presents guidance information to the surgeon. The guidance
information may help guide the surgeon through the surgery,
providing guidance for various steps in a surgical workflow,
including sequence of steps, details of individual steps, and tool
or implant selection, implant placement and position, and bone
surface preparation for various steps in the surgical procedure
workflow.
[0079] Postoperative patient monitoring may occur after completion
of the surgical procedure (420). During the postoperative patient
monitoring step, healthcare outcomes of the patient may be
monitored. Healthcare outcomes may include relief from symptoms,
ranges of motion, complications, performance of implanted surgical
items, and so on. Pre- and postoperative monitoring system 112
(FIG. 1) may assist in the postoperative patient monitoring
step.
[0080] The medical consultation, case creation, preoperative
patient monitoring, image acquisition, automatic processing, manual
correction, and virtual planning steps of FIG. 4 are part of
preoperative phase 302 of FIG. 3. The surgical procedures with
guidance steps of FIG. 4 is part of intraoperative phase 306 of
FIG. 3. The postoperative patient monitoring step of FIG. 4 is part
of postoperative phase 308 of FIG. 3.
[0081] As mentioned above, one or more of the subsystems of
orthopedic surgical system 100 may include one or more mixed
reality (MR) systems, such as MR system 212 (FIG. 2). Each MR
system may include a visualization device. For instance, in the
example of FIG. 2, MR system 212 includes visualization device 213.
In some examples, in addition to including a visualization device,
an MR system may include external computing resources that support
the operations of the visualization device. For instance, the
visualization device of an MR system may be communicatively coupled
to a computing device (e.g., a personal computer, backpack
computer, smartphone, etc.) that provides the external computing
resources. Alternatively, adequate computing resources may be
provided on or within visualization device 213 to perform necessary
functions of the visualization device.
[0082] FIG. 5 is a schematic representation of visualization device
213 for use in an MR system, such as MR system 212 of FIG. 2,
according to an example of this disclosure. As shown in the example
of FIG. 5, visualization device 213 can include a variety of
electronic components found in a computing system, including one or
more processor(s) 514 (e.g., microprocessors or other types of
processing units) and memory 516 that may be mounted on or within a
frame 518. Furthermore, in the example of FIG. 5, visualization
device 213 may include a transparent screen 520 that is positioned
at eye level when visualization device 213 is worn by a user. In
some examples, screen 520 can include one or more liquid crystal
displays (LCDs) or other types of display screens on which images
are perceptible to a surgeon who is wearing or otherwise using
visualization device 213 via screen 520. Other display examples
include organic light emitting diode (OLED) displays. In some
examples, visualization device 213 can operate to project 3D images
onto the user's retinas using techniques known in the art.
[0083] In some examples, screen 520 may include see-through
holographic lenses. sometimes referred to as waveguides, that
permit a user to see real-world objects through (e.g., beyond) the
lenses and also see holographic imagery projected into the lenses
and onto the user's retinas by displays, such as liquid crystal on
silicon (LCoS) display devices, which are sometimes referred to as
light engines or projectors, operating as an example of a
holographic projection system 538 within visualization device 213.
In other words, visualization device 213 may include one or more
see-through holographic lenses to present virtual images to a user.
Hence, in some examples, visualization device 213 can operate to
project 3D images onto the user's retinas via screen 520, e.g.,
formed by holographic lenses. In this manner, visualization device
213 may be configured to present a 3D virtual image to a user
within a real-world view observed through screen 520, e.g., such
that the virtual image appears to form part of the real-world
environment. In some examples, visualization device 213 may be a
Microsoft HOLOLENS.TM. headset, available from Microsoft
Corporation, of Redmond, Wash., USA, or a similar device, such as,
for example, a similar MR visualization device that includes
waveguides. The HOLOLENS.TM. device can be used to present 3D
virtual objects via holographic lenses, or waveguides, while
permitting a user to view actual objects in a real-world scene,
i.e., in a real-world environment, through the holographic
lenses.
[0084] Although the example of FIG. 5 illustrates visualization
device 213 as a head-wearable device, visualization device 213 may
have other forms and form factors. For instance, in some examples,
visualization device 213 may be a handheld smartphone or
tablet.
[0085] Visualization device 213 can also generate a user interface
(UI) 522 that is visible to the user, e.g., as holographic imagery
projected into see-through holographic lenses as described above.
For example, UI 522 can include a variety of selectable widgets 524
that allow the user to interact with a mixed reality (MR) system,
such as MR system 212 of FIG. 2. Imagery presented by visualization
device 213 may include, for example, one or more 3D virtual
objects. Details of an example of UI 522 are described elsewhere in
this disclosure. Visualization device 213 also can include a
speaker or other sensory devices 526 that may be positioned
adjacent the user's ears. Sensory devices 526 can convey audible
information or other perceptible information (e.g., vibrations) to
assist the user of visualization device 213.
[0086] Visualization device 213 can also include a transceiver 528
to connect visualization device 213 to a processing device 510
and/or to network 208 and/or to a computing cloud, such as via a
wired communication protocol or a wireless protocol, e.g., Wi-Fi,
Bluetooth, etc. Visualization device 213 also includes a variety of
sensors to collect sensor data, such as one or more optical
camera(s) 530 (or other optical sensors) and one or more depth
camera(s) 532 (or other depth sensors), mounted to, on or within
frame 518. In some examples, the optical sensor(s) 530 are operable
to scan the geometry of the physical environment in which a user of
MR system 212 is located (e.g., an operating room) and collect
two-dimensional (2D) optical image data (either monochrome or
color). Depth sensor(s) 532 are operable to provide 3D image data,
such as by employing time of flight, stereo or other known or
future-developed techniques for determining depth and thereby
generating image data in three dimensions. Other sensors can
include motion sensors 533 (e.g., Inertial Mass Unit (IMU) sensors,
accelerometers, etc.) to assist with tracking movement.
[0087] MR system 212 processes the sensor data so that geometric,
environmental, textural, or other types of landmarks (e.g.,
corners, edges or other lines, walls, floors, objects) in the
user's environment or "scene" can be defined and movements within
the scene can be detected. As an example, the various types of
sensor data can be combined or fused so that the user of
visualization device 213 can perceive 3D images that can be
positioned, or fixed and/or moved within the scene. When a 3D image
is fixed in the scene, the user can walk around the 3D image, view
the 3D image from different perspectives, and manipulate the 3D
image within the scene using hand gestures, voice commands, gaze
line (or direction) and/or other control inputs. As another
example, the sensor data can be processed so that the user can
position a 3D virtual object (e.g., a bone model) on an observed
physical object in the scene (e.g., a surface, the patient's real
bone, etc.) and/or orient the 3D virtual object with other virtual
images displayed in the scene. In some examples, the sensor data
can be processed so that the user can position and fix a virtual
representation of the surgical plan (or other widget, image or
information) onto a surface, such as a wall of the operating room.
Yet further, in some examples, the sensor data can be used to
recognize surgical instruments and the position and/or location of
those instruments.
[0088] Visualization device 213 may include one or more processors
514 and memory 516, e.g., within frame 518 of the visualization
device. In some examples, one or more external computing resources
536 process and store information, such as sensor data, instead of
or in addition to in-frame processor(s) 514 and memory 516. In this
way, data processing and storage may be performed by one or more
processors 514 and memory 516 within visualization device 213
and/or some of the processing and storage requirements may be
offloaded from visualization device 213. Hence, in some examples,
one or more processors that control the operation of visualization
device 213 may be within visualization device 213, e.g., as
processor(s) 514. Alternatively, in some examples, at least one of
the processors that controls the operation of visualization device
213 may be external to visualization device 213, e.g., as
processor(s) 210. Likewise, operation of visualization device 213
may, in some examples, be controlled in part by a combination one
or more processors 514 within the visualization device and one or
more processors 210 external to visualization device 213.
[0089] For instance, in some examples, when visualization device
213 is in the context of FIG. 2, processing of the sensor data can
be performed by processing device(s) 210 in conjunction with memory
or storage device(s) (M) 215. In some examples, processor(s) 514
and memory 516 mounted to frame 518 may provide sufficient
computing resources to process the sensor data collected by cameras
530, 532 and motion sensors 533. In some examples, the sensor data
can be processed using a Simultaneous Localization and Mapping
(SLAM) algorithm, or other known or future-developed algorithms for
processing and mapping 2D and 3D image data and tracking the
position of visualization device 213 in the 3D scene. In some
examples, image tracking may be performed using sensor processing
and tracking functionality provided by the Microsoft HOLOLENS.TM.
system, e.g., by one or more sensors and processors 514 within a
visualization device 213 substantially conforming to the Microsoft
HOLOLENS.TM. device or a similar mixed reality (MR) visualization
device.
[0090] In some examples, MR system 212 can also include
user-operated control device(s) 534 that allow the user to operate
MR system 212, use MR system 212 in spectator mode (either as
master or observer), interact with UI 522 and/or otherwise provide
commands or requests to processing device(s) 210 or other systems
connected to network 208. As examples, control device(s) 534 can
include a microphone, a touch pad, a control panel, a motion sensor
or other types of control input devices with which the user can
interact.
[0091] FIG. 6 is a block diagram illustrating example components of
visualization device 213 for use in a MR system. In the example of
FIG. 6, visualization device 213 includes processors 514, a power
supply 600, display device(s) 602, speakers 604, microphone(s) 606,
input device(s) 608, output device(s) 610, storage device(s) 612,
sensor(s) 614, and communication devices 616. In the example of
FIG. 6, sensor(s) 616 may include depth sensor(s) 532, optical
sensor(s) 530, motion sensor(s) 533, and orientation sensor(s) 618.
Optical sensor(s) 530 may include cameras, such as Red-Green-Blue
(RGB) video cameras, infrared cameras, or other types of sensors
that form images from light. Display device(s) 602 may display
imagery to present a user interface to the user.
[0092] Speakers 604, in some examples, may form part of sensory
devices 526 shown in FIG. 5. In some examples, display devices 602
may include screen 520 shown in FIG. 5. For example, as discussed
with reference to FIG. 5, display device(s) 602 may include
see-through holographic lenses, in combination with projectors,
that permit a user to see real-world objects, in a real-world
environment, through the lenses, and also see virtual 3D
holographic imagery projected into the lenses and onto the user's
retinas, e.g., by a holographic projection system. In this example,
virtual 3D holographic objects may appear to be placed within the
real-world environment. In some examples, display devices 602
include one or more display screens, such as LCD display screens,
OLED display screens, and so on. The user interface may present
virtual images of details of the virtual surgical plan for a
particular patient.
[0093] In some examples, a user may interact with and control
visualization device 213 in a variety of ways. For example,
microphones 606, and associated speech recognition processing
circuitry or software, may recognize voice commands spoken by the
user and, in response, perform any of a variety of operations, such
as selection, activation, or deactivation of various functions
associated with surgical planning, intra-operative guidance, or the
like. As another example, one or more cameras or other optical
sensors 530 of sensors 614 may detect and interpret gestures to
perform operations as described above. As a further example,
sensors 614 may sense gaze direction and perform various operations
as described elsewhere in this disclosure. In some examples, input
devices 608 may receive manual input from a user, e.g., via a
handheld controller including one or more buttons, a keypad, a
touchscreen, joystick, trackball, and/or other manual input media,
and perform, in response to the manual user input, various
operations as described above.
[0094] As discussed above, surgical lifecycle 300 may include a
preoperative phase 302 (FIG. 3). One or more users may use
orthopedic surgical system 100 in preoperative phase 302. For
instance, orthopedic surgical system 100 may include virtual
planning system 102 to help the one or more users generate a
virtual surgical plan that may be customized to an anatomy of
interest of a particular patient. As described herein, the virtual
surgical plan may include a 3-dimensional virtual model that
corresponds to the anatomy of interest of the particular patient
and a 3-dimensional model of one or more prosthetic components
matched to the particular patient to repair the anatomy of interest
or selected to repair the anatomy of interest. The virtual surgical
plan also may include a 3-dimensional virtual model of guidance
information to guide a surgeon in performing the surgical
procedure, e.g., in preparing bone surfaces or tissue and placing
implantable prosthetic hardware relative to such bone surfaces or
tissue.
[0095] FIG. 7 is a flowchart illustrating example steps in
preoperative phase 302 of surgical lifecycle 300. In other
examples, preoperative phase 302 may include more, fewer, or
different steps. Moreover, in other examples, one or more of the
steps of FIG. 7 may be performed in different orders. In some
examples, one or more of the steps may be performed automatically
within a surgical planning system such as virtual planning system
102 (FIG. 1) or 202 (FIG. 2).
[0096] In the example of FIG. 7, a model of the area of interest is
generated (700). For example, a scan (e.g., a CT scan, MRI scan, or
other type of scan) of the area of interest may be performed. For
example, if the area of interest is the patient's shoulder, a scan
of the patient's shoulder may be performed. Furthermore, a
pathology in the area of interest may be classified (702). In some
examples, the pathology of the area of interest may be classified
based on the scan of the area of interest. For example, if the area
of interest is the user's shoulder, a surgeon may determine what is
wrong with the patient's shoulder based on the scan of the
patient's shoulder and provide a shoulder classification indicating
the diagnosis, e.g., such as primary glenoid humeral osteoarthritis
(PGHOA), rotator cuff tear arthropathy (RCTA) instability, massive
rotator cuff tear (MRCT), rheumatoid arthritis, post-traumatic
arthritis, and osteoarthritis.
[0097] Additionally, a surgical plan may be selected based on the
pathology (704). The surgical plan is a plan to address the
pathology. For instance, in the example where the area of interest
is the patient's shoulder, the surgical plan may be selected from
an anatomical shoulder arthroplasty, a reverse shoulder
arthroplasty, a post-trauma shoulder arthroplasty, or a revision to
a previous shoulder arthroplasty. The surgical plan may then be
tailored and/or matched to the patient (706). For instance,
tailoring the surgical plan may involve designing, selecting and/or
sizing surgical items needed to perform the selected surgical plan.
Additionally, the surgical plan may be tailored to the patient in
order to address issues specific to the patient, such as the
presence of osteophytes. As described in detail elsewhere in this
disclosure, one or more users may use mixed reality systems of
orthopedic surgical system 100 to tailor the surgical plan to the
patient.
[0098] The surgical plan may then be reviewed (708). For instance,
a consulting surgeon may review the surgical plan before the
surgical plan is executed. As described in detail elsewhere in this
disclosure, one or more users may use mixed reality (MR) systems of
orthopedic surgical system 100 to review the surgical plan. In some
examples, a surgeon may modify the surgical plan using an MR system
by interacting with a UI and displayed elements, e.g., to select a
different procedure, change the sizing, shape or positioning of
implants, or change the angle, depth or amount of cutting or
reaming of the bone surface to accommodate an implant.
[0099] Additionally, in the example of FIG. 7, surgical items
needed to execute the surgical plan may be requested (710). For
instance, one or more files representing patient matched implants
may be transmitted to a manufacturing system, such as manufacturing
and delivery system 106 of FIG. 1
[0100] As described in the following sections of this disclosure,
orthopedic surgical system 100 may assist various users in
performing one or more of the preoperative steps of FIG. 7.
[0101] As discussed above, in some examples, it may be desirable
for a surgeon to utilize a patient matched (e.g., patient specific,
custom, etc.) implant when performing an orthopedic surgical
procedure. For instance, using an implant that is custom designed
and manufactured for a particular patient (i.e., a patient matched
implant) may enable the surgeon to minimize, or eliminate, the need
to remove portions of a bone to prepare the bone to receive an
implant device. Additionally, using a patient matched implant may
improve fixation of an implant to bone, which may yield better
patient outcomes.
[0102] FIG. 8 is a flowchart illustrating example steps for
tailoring a surgical plan to a patient. The steps of FIG. 8 may be
considered one example of step 706 of FIG. 7 and/or one example of
step 412 of FIG. 4. In other examples, the technique of FIG. 8 may
include more, fewer, or different steps. Moreover, in other
examples, one or more of the steps of FIG. 8 may be performed in
different orders. In some examples, one or more of the steps may be
performed automatically within a surgical planning system such as
virtual planning system 102 (FIG. 1) or 202 (FIG. 2).
[0103] A surgical planning system may obtain a 3D model of a bone
of a patient (802). For instance, virtual planning system 102 may
obtain the 3D model of the bone generated from medical images of
the bone. As discussed above, the medical images may be acquired
during the pre-operative phase (e.g., during step 406 of FIG. 4).
Virtual planning system 102 may generate the 3D model based on
various features of the bone in the image. For instance, as
discussed below with reference to FIG. 9, where the bone is a
scapula, virtual planning system 102 may generate a 3D model of a
glenoid of the scapula.
[0104] In some examples, the surgical planning system may
facilitate the design of a patient matched implant to conform to a
patient's bone as it exists pre-operation. In such examples,
virtual planning system 102 may use an unmodified version of the 3D
model of the bone. In other examples, the surgical planning system
may facilitate the design of a patient matched implant to conform
to a patient's bone as it will exist after one or more work steps
are performed during an operation (e.g., reaming). In such
examples, virtual planning system 102 may use a modified version of
the 3D model of the bone that represents a shape of the bone after
the planned work steps are performed.
[0105] The surgical planning system may identify an implant type
(804). For instance, virtual planning system 102 may determine the
type of implant selected during step 704 of FIG. 7. The determined
implant type may indicate one or more of: a style (e.g.,
stemmed/stemless, anatomic/reversed, etc.), a manufacturer, a
model, a part number, or any other identifying characteristic of
the selected implant.
[0106] In some examples, identifying the implant type may include
identifying one or more features of the identified implant. Some
example features include, but are not limited to, articular surface
shape, articular surface location, peripheral shape, anchorage
type, anchorage location, modified vs. unmodified bone (e.g.,
reamed vs. un-reamed bone), etc. The surgical planning system may
automatically identify, suggest, or recommend any of the features.
Similarly, the surgeon may provide user input to the surgical
planning system to manually select and of the features. One of more
the features may be selected from a pre-defined library. For
instance, the peripheral shape and/or anchorage type may be
selected from a pre-defined library. Additionally or alternatively,
one of more the features may be selected from a parametric shape
library. For instance, the peripheral shape and/or anchorage type
may be selected from a parametric shape library.
[0107] The surgical planning system may obtain a template model
corresponding to the identified implant type (806). The template
model may be a model of an implant that is used as a starting point
for the generation of a patient matched implant. For instance,
virtual planning system 102 may obtain, from a storage system
(e.g., storage system 206 of FIG. 2), a 3D model (e.g., a CAD
model) of at least a portion of the identified implant type. As one
specific example, where the identified implant type is a glenoid
implant, virtual planning system 102 may obtain a 3D model of a
baseplate of the glenoid implant.
[0108] The surgical planning system may generate, based on the 3D
model and the template model, a patient matched implant model
(808). For instance, to determine the patient matched implant
model, virtual planning system 102 may determine a 3D shape bounded
on one side by a surface of the 3D model of the bone and bounded on
another side by a surface of the obtained template model. As one
specific example, virtual planning system 102 may virtually extrude
a boss from a surface of the template model (e.g., a lower
surface), and remove portions of the extruded boss that overlap
with the 3D model of the glenoid (e.g., perform a Boolean
intersection). The combination of the determined 3D shape and the
template model may represent the patient matched implant model. In
some examples, as discussed in further detail below, virtual
planning system 102 may generate the patient matched implant model
as including one or more porous sections and one or more solid
sections.
[0109] The surgical planning system may output the generated
patient matched implant model for manufacturing (810). For
instance, virtual planning system 102 may output a file containing
the generated patient matched implant model to manufacturing and
delivery system 106, which may manufacture a physical patient
matched implant corresponding to the patient matched implant model.
As one example, manufacturing and delivery system 106 may use
additive manufacturing (e.g., 3D printing) techniques (e.g., direct
metal laser sintering (DMLS)) to manufacture the physical patient
matched implant. Other example additive manufacturing techniques
include, but are not limited to, fused deposition modeling (FDM),
fused filament fabrication (FFF), and electron beam melting
(EBM).
[0110] FIG. 9 is a flowchart illustrating example steps for
obtaining a model of a bone of a patient. The steps of FIG. 9 may
be considered one example of step 802 of FIG. 8. In other examples,
the technique of FIG. 9 may include more, fewer, or different
steps. Moreover, in other examples, one or more of the steps of
FIG. 9 may be performed in different orders. In some examples, one
or more of the steps may be performed automatically within a
surgical planning system such as virtual planning system 102 (FIG.
1) or 202 (FIG. 2).
[0111] A surgical planning system may obtain a 3D model of the bone
generated from medical images of the bone (902). As discussed
above, the medical images may be acquired during the pre-operative
phase (e.g., during step 406 of FIG. 4). In the example of FIG.
10A, virtual planning system 102 may obtain 3D model 903 of a
scapula of a patient, including glenoid 905.
[0112] The surgical planning system may generate a mask defining an
outline of an area of interest in the 3D model. For instance,
virtual planning system 102 may identify anterior, posterior,
superior, and inferior points of the area of interest in the 3D
model (904). Virtual planning system 102 may identify the points
automatically, with manual input, or a combination of automatic and
manual input. In the example of FIG. 10A, where the area of
interest is a glenoid of a scapula, virtual planning system 102 may
identify anterior points 952, posterior points 954, superior points
956, and inferior points 958 of glenoid 905 on 3D model 903.
[0113] Virtual surgical system 102 may generate anterior,
posterior, superior, and inferior masks based on the identified
anterior, posterior, superior, and inferior points (906). For
instance, in the example of FIG. 10B, virtual surgical system 102
may generate anterior mask 953, posterior mask 955, superior mask
957, and inferior mask 959. Collectively, the generated masks may
define the outline of the area of interest in the 3D model. For
instance, in the example of FIG. 10C, generate anterior mask 953,
posterior mask 955, superior mask 957, and inferior mask 959 may be
combined to form glenoid mask 960 that defines an outline of
glenoid 905.
[0114] The surgical planning system may utilize the generated mask
to identify the area of interest in the 3D model (908). For
instance, in the example of FIG. 10D, virtual planning system 102
may use glenoid mask 960 of FIG. 10C to "mask out" (e.g., cover up,
remove, etc.) portions of 3D model 903 other than glenoid 905
(i.e., the area of interest). In this way, the techniques of this
disclosure enable a system to obtain a 3D model of the area of
interest.
[0115] FIG. 11 is a flowchart illustrating example steps for
generating a patient matched implant model. The steps of FIG. 11
may be considered one example of step 808 of FIG. 8. In other
examples, the technique of FIG. 11 may include more, fewer, or
different steps. Moreover, in other examples, one or more of the
steps of FIG. 11 may be performed in different orders. In some
examples, one or more of the steps may be performed automatically
within a surgical planning system such as virtual planning system
102 (FIG. 1) or 202 (FIG. 2).
[0116] The surgical planning system may obtain a baseplate final
state model (1102). For instance, virtual planning system 102 may
obtain, from a storage system (e.g., storage system 206 of FIG. 2),
a 3D model (e.g., a CAD model) of a version of a baseplate of the
identified implant type. In one specific example where the
identified implant type is a glenoid implant, virtual planning
system 102 may obtain baseplate final state model 1103A of FIG.
12A. The baseplate final state model may include a surface defined
as a backside. For instance, in the example of FIG. 12A, baseplate
final state model 1103A may include backside 809. The backside may
be considered to be a surface of an implant that faces away from an
articular surface of the implant. The baseplate final state model
may include various additional features. For instance, in the
example of FIG. 12A, baseplate final state model 1103A may include
holes 812A-812F (collectively, "holes 812") (hole 812F is not shown
in FIG. 12A as it is obstructed by another portion of baseplate
final state model 1103A).
[0117] The surgical planning system may generate a patient matched
augment model based on the baseplate final state model and the 3D
model of the area of interest (1104). In general, a patient matched
augment model may define a volume that is matched to the patient.
For instance, virtual planning system 102 may determine a shape of
a backside (e.g., bottom) of the baseplate final state model, and
determine a volume between the shape of the backside and a surface
of a bone defined by the model of the bone. The determined shape
may include an outline of the backside and/or may include various
features (e.g., holes 812). For instance, in the example of FIG.
12B, virtual planning system 102 may determine that shape 1103B of
backside 809 of baseplate final state model 1103A is a circle with
a particular diameter (e.g., 25 mm, 29 mm, etc.) including several
holes.
[0118] Virtual planning system 102 may determine a virtual
extrusion (e.g., a boss) of the determined shape. In other words,
virtual planning system 102 may extend the 2-dimensional determined
shape of the backside of baseplate final state model 1103A into the
3.sup.rd dimension. FIGS. 13A-13C are conceptual diagrams
illustrating example views of a virtual extrusion for a patient
matched implant design process. FIG. 13A illustrates a first view,
FIG. 13B illustrates a second view that is 90 degrees offset from
the first view in a first direction, and FIG. 13C illustrates a
third view that is 90 degrees offset from the first view in a
second direction that is opposite the first direction. As shown in
the example of FIGS. 13A-13C, virtual planning system 102 may
determine virtual extrusion 907 (e.g., shown as a cylinder as, in
this example, the outline of shape 1103B of the backside of
baseplate final state model 1103A is a circle, however other shapes
are possible). Virtual extrusion 907 may include a first face 909
and a second face 910. In examples where the patient matched
implant is a glenoid implant, first face 909 may be referred to as
a medial face and second face 910 may be referred to as a lateral
face. Virtual planning system 102 may create virtual extrusion 907
based on a uniform repartition of points (e.g., an even
distribution of points) on the determined shape of the backside
(shown in FIG. 14A as points 909). In some examples, such as where
the determined shape of the backside includes one or more voids
(e.g., holes for fasteners) virtual planning system 102 may
generate virtual extrusion 907 to include the holes. In the example
of FIGS. 13A-13C, even though shape 1103B includes various holes,
virtual extrusion 907 is illustrated as a cylinder for
simplicity.
[0119] Virtual planning system 102 may determine the patient
matched augment model based on the virtual extrusion and the 3D
model of the area of interest. For instance, to determine the
patient-matched implant model, virtual planning system 102 may
modify a face of virtual extrusion 907 to conform to a surface of
the area of interest. As shown in the example of FIGS. 13A-13C,
virtual planning system 102 may conform first face 909 (e.g., the
medial face) of virtual extrusion 907 to a surface of the 3D model
of glenoid 905 (e.g., as masked out from 3D model 903 as discussed
above). As one example, virtual planning system 102 may perform a
Boolean intersection of points on virtual extrusion 907 and points
on the 3D model of glenoid 905. In other words, virtual planning
system 102 may identify points that are within virtual extrusion
907 that are also within the 3D model of glenoid 905. Virtual
planning system 102 may remove the portion of virtual extrusion 907
that intersects the 3D model of glenoid 905 from virtual extrusion
907, resulting in a patient-matched augment model.
[0120] As another example, virtual planning system 102 may compute
projections of the points of the surface of the extrusion on the 3D
model of the area of interest. For instance, virtual planning
system 102 may determine a projection of the points on the surface
of extrusion 907 and the surface of the 3D model of glenoid 905. As
shown in the example of FIGS. 14A and 14B, virtual planning system
102 may project points 909 of virtual extrusion 907 onto the
surface of glenoid 905 to obtain projected points 911. As discussed
below, the surface defined by the obtained projected points may be
used to generate the patient-matched augment model.
[0121] FIG. 12C is a conceptual diagram of a patient-matched
augment model 1105 that may be generated based on virtual extrusion
907. As shown in FIG. 12C, patient-matched augment model 1105
includes surface 980 that is matched to a corresponding surface of
a bone of the patient. For example, virtual planning system 102 may
utilize projected points 911 to define the shape of surface 980.
Where the implant is a glenoid implant, surface 980 may be a medial
surface that conforms to a glenoid of the patient. In other words,
surface 980 may be complimentary to a surface of the glenoid of the
patient. As discussed above, in some examples, virtual planning
system 102 may generate the virtual extrusion to include one or
more holes. In such examples, the determined patient matched
augment model may include the one or more holes. For instance, as
shown in FIG. 12C, patient matched augment model 1105 includes
holes corresponding to the holes in shape 1103B of FIG. 12B.
[0122] As discussed above, in some examples, virtual planning
system 102 may generate the patient matched implant model as
including one or more porous sections and one or more solid
sections. When the patient matched implant model is manufactured
into a physical patient matched implant, the sections defined as
porous may be manufactured to be porous and the sections defined as
solid may be manufactured to be solid. Including one or more porous
sections in an implant may provide one or more advantages. As one
example, including one or more porous sections in an implant may
facilitate bony ingrowth into the implant, which may improve
implant fixation. In some examples, there may be a sharp transition
between solid and porous sections. In other examples, there may be
a transition region between solid and porous sections with
different porosity than the porous section. For instance, pores of
the transition region may be smaller than pores of the porous
section. Including a transition region may provide various benefits
such as reduced manufacturing complexity.
[0123] The surgical planning system may obtain a pre-defined porous
model (1106). For instance, virtual planning system 102 may obtain,
from a storage system (e.g., storage system 206 of FIG. 2), a 3D
model (e.g., a CAD model) of a portion of the identified implant
type that is to be formed of a porous structure. As one specific
example, where the identified implant type is a glenoid implant,
virtual planning system model 102 may obtain pre-defined porous
model 1107 of FIG. 12D.
[0124] The surgical planning system may generate a porous patient
matched model based on the pre-defined porous model and the patient
matched augment model (1108). For instance, virtual planning system
102 may add/merge (e.g., Boolean add the volumes) the patient
matched augment model (e.g., the volume determined between backside
809 and the glenoid represented in the 3D model) to the pre-defined
porous model to generate the porous patient matched model. In other
words, virtual planning system 102 may identify points that are
within the patient matched augment model and points that are within
the pre-defined porous model. Virtual planning system 102 may
combine the points identified within the patient matched augment
model and the points identified within pre-defined porous model,
resulting in a porous patient matched model (e.g., a patient
matched porous model). As one specific example, virtual planning
system 102 may add patient matched augment 1105 of FIG. 12C to
pre-defined porous model 1107 of FIG. 12D to obtain porous patient
matched model 1109A of FIG. 12E.
[0125] The surgical planning system may populate (e.g., fill) the
obtained porous patient matched model with a porous structure. For
instance, virtual planning system 102 may modify one or more
parameters of the porous patient matched model to indicate that the
volume defined by the porous patient matched model is porous. As
one specific example, virtual planning system 102 may populate
porous patient matched model 1109A with a porous structure to
obtain porous patient matched model 1109B of FIG. 12D. In some
examples, the porous structure may be predefined such that virtual
planning system 102 uses the same porosity for all patients (i.e.,
the porous structure may be generic). In some examples, the porous
structure may be patient specific. For instance, virtual planning
system 102 may select a particular combination of pore size and
pore density based on one or more parameters of the patient (e.g.,
bone density, age, etc.).
[0126] The surgical planning system may obtain a pre-defined solid
model (1110). For instance, virtual planning system 102 may obtain,
from a storage system (e.g., storage system 206 of FIG. 2), a 3D
model (e.g., a CAD model) of a portion of the identified implant
type that is to be formed of a solid structure. The pre-defined
solid model may define a generic structure that is to be included
in all patient matched implants of the identified implant type. As
one specific example, where the identified implant type is a
glenoid implant, virtual planning system model 102 may obtain
pre-defined solid model 1111 of FIG. 12G.
[0127] The surgical planning system may generate a mixed patient
matched implant model based on the pre-defined solid model and the
porous patient matched model (1112). For instance, virtual planning
system 102 may add (e.g., Boolean add the volumes) the pre-defined
solid model and the porous patient matched model to generate the
mixed patient matched implant model. As one specific example,
virtual planning system 102 may add porous patient matched model
1109B of FIG. 12D to pre-defined solid model 1111 of FIG. 12G to
obtain mixed patient matched model 1113A of FIG. 12H.
[0128] As discussed above, in some cases, the surgical planning
system may generate a patient matched implant model without any
porous portions. In such examples, the surgical planning system may
generate the patient matched implant model by adding the patient
matched augment to a pre-defined solid model.
[0129] The surgical planning system may generate a file that
includes the mixed patient matched implant model. For instance,
virtual planning system 102 may generate a ".stl" file, a CAD file,
or any other type of file capable of representing the mixed patient
matched implant model. Virtual planning system 102 may output the
generated file for manufacturing into a physical patient matched
implant. For instance, virtual planning system 102 may output the
generated file to an additive manufacturing device (e.g., a 3D
printer) to fabricate physical patient matched implant model 1115
of FIG. 12I.
[0130] The physical patient matched implant may be manufactured
based on the patient matched mixed model (1114). For instance,
manufacturing and delivery system 106 may use additive
manufacturing (e.g., 3D printing) techniques (e.g., direct metal
laser sintering (DMLS)) to manufacture the physical patient matched
implant. In some examples, manufacturing and delivery system 106
may manufacture one or more other components in addition to the
physical patient matched implant. For instance, manufacturing and
delivery system 106 may manufacture one or more patient matched
guides (e.g., patient-matched guide 1600 of FIG. 19) and/or one or
more patient matched models (e.g., models of the patient's anatomy
on-which a surgeon can practice before an actual implantation
procedure). Where manufacturing and delivery system 106
manufactures the other components, the other components may be
packaged and shipped to the surgical center along with the physical
patient matched implant.
[0131] FIGS. 12I and 12J are conceptual diagrams illustrating an
example patient matched implant 1115. FIG. 12I illustrates a side
view of patient matched implant 1115 and FIG. 12J illustrates a top
view of patient matched implant 1115. As shown in the example of
FIG. 12I, patient matched implant 1115 may include porous portions
972 and solid portions 974. Additionally, as shown in the example
of FIG. 12I, surface 970 (e.g., a medial surface in the context of
a glenoid implant) of patient matched implant 1115 may be contoured
to match a shape of a glenoid of the patient for which implant 1115
is matched.
[0132] In some examples, the mixed patient matched model may
include components that will be removed during the manufacturing
process. For instance, as shown in FIG. 12H, mixed patient matched
model 1113 may include flange 971 which may be fabricated as part
of the physical patient matched implant. As shown in FIG. 12G,
flange 971 may be included in mixed patient matched model 1113 from
pre-defined solid model 1111. However, during the manufacturing
process, flange 971 may be removed. For instance, the physical
patient matched implant may be turned (e.g., on a lathe) to remove
flange 971.
[0133] The physical patient matched implant may be processed in one
or more ways during or post fabrication. As one example, the
physical patient matched implant may be heat treated after 3D
printing, before removal of components (e.g., before removal of
flange 971). As another example, the physical patient matched
implant may be cleaned, packaged, labeled, sterilized, etc. prior
to shipment to a surgical center (e.g., at which the physical
patient matched implant is to be implanted into the patient).
[0134] In some examples, the steps of the technique of FIG. 11 may
be performed by a single device or system. For instance, the steps
of the technique of FIG. 11 may be performed by virtual planning
system 102 (e.g., running the BLUEPRINT.TM. system available from
Wright Medical Group, N.V.). In other examples, the steps of the
technique of FIG. 11 may be performed by multiple devices or
systems. For instance, a first set of the steps of the technique of
FIG. 11 (e.g., steps 1102 and 1104) may be performed by a first
device (e.g., a computer directly used by a surgeon) and a second
set of the steps of the technique of FIG. 11 (e.g., steps
1106-1112) may be performed by one or more servers (e.g., a cloud
computing system). Similarly, the manufacturing process (e.g., step
1114 of FIG. 11) may be performed at a manufacturing facility.
[0135] FIGS. 15A and 15B are conceptual diagrams illustrating
examples of patient matched implants. As shown in the example of
FIG. 15A, patient matched implant 1115A may be a glenoid implant
for a reverse shoulder arthroplasty. Patient matched implant 1115A
may include post 982, or other anchorage, configured to be inserted
into a hole made in glenoid 905 (e.g. using the techniques
discussed below with reference to FIGS. 26-37), and glenoid sphere
984 configured to engage a corresponding element attached to a
humerus of the patient. As shown in FIG. 15A, surface 980 of
patient matched implant 1115A may be configured to match a surface
of glenoid 905. For instance, patient matched implant 1115A,
including surface 980, may be designed and fabricated using the
techniques discussed above with reference to FIGS. 8-14B.
[0136] As shown in the example of FIG. 15B, patient matched implant
1115A may be another example of a glenoid implant for a reverse
shoulder arthroplasty. Similar to patient matched implant 1115A,
patient matched implant 1115B includes surface 980 configured to
match a surface of glenoid 905. Patient matched implant 1115A and
patient matched implant 1115B may be considered examples of full
augment patient matched implants in that the entire contact area
between the implants and the bone is "matched" to the bone. For
instance, patient matched implant 1115A and patient matched implant
1115B may be considered examples of full augment patient matched
implants because the entire area of surface 980 is matched to the
contour of glenoid 905.
[0137] FIGS. 16A and 16B are conceptual diagrams illustrating
examples of patient matched implants. As shown in the examples of
FIGS. 16A and 16B, patient matched implant 1115C and patient
matched implant 1115D may be other examples of glenoid implants for
a reverse shoulder arthroplasty. Similar to patient matched
implants 1115A and 1115B, patient matched implants 1115C and 1115D
each include surface 980 configured to match a surface of glenoid
905. However, in contrast to patient matched implants 1115A and
1115B, surface 980 of patient matched implants 1115C and 1115D does
not span the entire contact area between the implants and the bone.
In particular, patient matched implants 1115C and 1115D both
include a portion of surface 980 (i.e., portion 981) where surface
980 is not matched to the contour of glenoid 905. As such, of
patient matched implants 1115C and 1115D may be considered to be
examples of partial augment patient matched implants.
[0138] FIGS. 17A and 17B are conceptual diagrams illustrating
examples of patient matched implants. As shown in the examples of
FIGS. 17A and 17B, implant 1117 and patient matched implant 1115E
may be other examples of glenoid implants for a reverse shoulder
arthroplasty. As discussed above, in some examples, a patient
matched implant may be designed and manufactured to conform to a
patient bone as it exists pre-operation. Similarly, in other
examples, a patient matched implant may be designed and
manufactured to conform to a patient bone as it will exist after
one or more work steps are performed during an operation (e.g.,
reaming). In the example of FIG. 17B, patient matched implant
1115
[0139] FIG. 18 illustrates an example of a page of a user interface
of a mixed reality system, according to an example of this
disclosure, e.g. as produced for a particular patient's surgical
plan. Using visualization device 213, a user can perceive and
interact with UI 522. In the example shown in FIG. 18, UI 522
includes a workflow bar 1000 with selectable buttons 1002 that
represent a surgical workflow, spanning various surgical procedure
steps for operations on the humerus and glenoid in a shoulder
arthroplasty procedure. Selection of a button 1002 can lead to
display of various selectable widgets with which the user can
interact, such as by using hand gestures, voice commands, gaze
direction, connected lens and/or other control inputs. Selection of
widgets can launch various modes of operation of MR system 212,
display information or images generated by MR system 212, allow the
user to further control and/or manipulate the information and
images, lead to further selectable menus or widgets, etc.
[0140] The user can also organize or customize UI 522 by
manipulating, moving and orienting any of the displayed widgets
according to the user's preferences, such as by visualization
device 213 or other device detecting gaze direction, hand gestures
and/or voice commands. Further, the location of widgets that are
displayed to the user can be fixed relative to the scene. Thus, as
the user's gaze (i.e., eye direction) moves to view other features
of the user interface 522, other virtual images, and/or real
objects physically present in the scene (e.g., the patient, an
instrument set, etc.), the widgets may remain stationary and do not
interfere with the user's view of the other features and objects.
As yet another example, the user can control the opacity or
transparency of the widgets or any other displayed images or
information. The user also can navigate in any direction between
the buttons 1002 on the workflow bar 1000 and can select any button
1002 at any time during use of MR system 212. Selection and
manipulation of widgets, information, images or other displayed
features can be implemented based on visualization device 213 or
other device detecting user gaze direction, hand motions, voice
commands or any combinations thereof.
[0141] In the example of FIG. 18, UI 522 is configured for use in
shoulder repair procedures and includes, as examples, buttons 1002
on workflow bar 1000 that correspond to a "Welcome" page, a
"Planning" page, a "Graft" page, a "Humerus Cut" page, an "Install
Guide" page, a "Glenoid Reaming" page, and a "Glenoid Implant"
page. The presentation of the "Install Guide" page may be optional
as, in some examples, glenoid reaming may be accomplished using
virtual guidance and without the application of a glenoid
guide.
[0142] As shown FIG. 18, the "Planning" page in this example of UI
522 displays various information and images corresponding to the
selected surgical plan, including an image 1006 of a surgical plan
file (e.g., a pdf file or other appropriate media format) that
corresponds to the selected plan (including preoperative and
postoperative information); a 3D virtual bone model 1008 and a 3D
virtual implant model 1010 along with a 3D image navigation bar
1012 for manipulating the 3D virtual models 1008, 1010 (which may
be referred to as 3D images); a viewer 1014 and a viewer navigation
bar 1016 for viewing a multi-planar view associated with the
selected surgical plan. MR system 212 may present the "Planning"
page as a virtual MR object to the user during preoperative phase
302 (FIG. 3). For instance, MR system 212 may present the
"Planning" page to the user to help the user classify a pathology,
select a surgical plan, tailor the surgical plan to the patient,
revise the surgical plan, and review the surgical plan, as
described in steps 702, 704, 706, and 708 of FIG. 7.
[0143] The surgical plan image 1006 may be a compilation of
preoperative (and, optionally, postoperative) patient information
and the surgical plan for the patient that are stored in a database
in storage system 206. In some examples, surgical plan image 1006
can correspond to a multi-page document through which the user can
browse. For example, further images of pages can display patient
information, information regarding the anatomy of interest,
postoperative measurements, and various 2D images of the anatomy of
interest. Yet further page images can include, as examples,
planning information associated with an implant selected for the
patient, such as anatomy measurements and implant size, type and
dimensions; planar images of the anatomy of interest; images of a
3D model showing the positioning and orientation of a surgical
guide selected for the patient to assist with execution of the
surgical plan; etc.
[0144] It should be understood that the surgical plan image 1006
can be displayed in any suitable format and arrangement and that
other implementations of the systems and techniques described
herein can include different information depending upon the needs
of the application in which the plan image 1006 is used.
[0145] Referring again FIG. 18, the Planning page of UI 522 also
may provide images of the 3D virtual bone model 1008 and the 3D
model of the implant components 1010 along with navigation bar 1012
for manipulating 3D virtual models 1008, 1010. For example,
selection or de-selection of the icons on navigation bar 1012 allow
the user to selectively view different portions of 3D virtual bone
model 1008 with or without the various implant components 1010. For
example, the scapula of virtual bone model 1008 and the glenoid
implant of implant model 1010 have been de-selected, leaving only
the humerus bone and the humeral implant components visible. Other
icons can allow the user to zoom in or out, and the user also can
rotate and re-orient 3D virtual models 1008, 1010, e.g., using gaze
detection, hand gestures and/or voice commands.
[0146] The Planning page of UI 522 also provides images of 3D
virtual bone model 1008 and the 3D model of the implant components
1010 along with navigation bar 1012 for manipulating 3D virtual
models 1008, 1010. The Planning page presented by visualization
device 213 also includes multi-planar image viewer 1014 (e.g., a
DICOM viewer) and navigation bar 1016 that allow the user to view
patient image data and to switch between displayed slices and
orientations. For example, the user can select 2D Planes icons 1026
on navigation bar 1016 so that the user can view the 2D sagittal
and coronal planes of the patient's body in multi-planar image
viewer 1014.
[0147] Workflow bar 1000 in FIG. 18 includes further pages that
correspond to steps in the surgical workflow for a particular
orthopedic procedure (here, a shoulder repair procedure). In the
example of FIG. 18, workflow bar 1000 includes elements labeled
"Graft," "Humerus Cut," "Install Guide," "Glenoid Reaming," and
"Glenoid Implant" that correspond to workflow pages for steps in
the surgical workflow for a shoulder repair procedure. In general,
these workflow pages include information that can be useful for a
health care professional during planning of or during performance
of the surgical procedure, and the information presented upon
selection of these pages is selected and organized in a manner that
is intended to minimize disturbances or distractions to the surgeon
during a procedure. Thus, the amount of displayed information is
optimized and the utility of the displayed information is
maximized. These workflow pages may be used as part of
intraoperative phase 306 (FIG. 3) to guide a surgeon, nurse or
other medical technician through the steps in a surgical procedure.
In some examples, these workflow pages may be used as part of
preoperative phase 302 (FIG. 3) to enable a user to visualize
3-dimensional models of objects involved in various steps of a
surgical workflow.
[0148] With reference to FIG. 19, the Install Guide page allows the
user to visualize a physical position of a patient-specific or
patient-matched guide 1600, e.g., for guidance of a drill to place
a reaming guide pin in the glenoid bone, on the patient's glenoid
1602 in order to assist with the efficient and correct placement of
the guide 1600 during the actual surgical procedure. Selection of
items on menu 1604 can remove features from the 3D images or add
other parameters of the surgical plan, such as a reaming axis 1606,
e.g., by voice commands, gaze direction and/or hand gesture
selection. Placement of guide 1600 may be unnecessary for
procedures in which visualization device 213 presents a virtual
reaming axis or other virtual guidance, instead of a physical
guide, to guide a drill for placement of a reaming guide pin in the
glenoid bone. The virtual guidance or other virtual objects
presented by visualization device 213 may include, for example, one
or more 3D virtual objects. In some examples, the virtual guidance
may include 2D virtual objects. In some examples, the virtual
guidance may include a combination of 3D and 2D virtual
objects.
[0149] With reference to FIG. 20, the Glenoid Implant page allows
the user to visualize the orientation and placement of a glenoid
implant 1700 and bone graft 1402 on glenoid 1602.
[0150] It should be understood that the workflow pages illustrated
and described herein are examples and that UI 522 can include
fewer, more, or different pages. For example, in applications of MR
system 212 for procedures involving other patient anatomies, such
as the ankle, foot, knee, hip or elbow, UI 522 can include pages
corresponding to the particular steps specific to the surgical
workflow for those procedures.
[0151] The images displayed on UI 522 of MR system 212 can be
viewed outside or within the surgical operating environment and, in
spectator mode, can be viewed by multiple users outside and within
the operating environment at the same time. In some circumstances,
such as in the operating environment, the surgeon may find it
useful to use a control device 534 to direct visualization device
213 such that certain information should be locked into position on
a wall or other surface of the operating room, as an example, so
that the information does not impede the surgeon's view during the
procedure. For example, relevant surgical steps of the surgical
plan can be selectively displayed and used by the surgeon or other
care providers to guide the surgical procedure.
[0152] In various some examples, the display of surgical steps can
be automatically controlled so that only the relevant steps are
displayed at the appropriate times during the surgical
procedure.
[0153] As discussed above, surgical lifecycle 300 may include an
intraoperative phase 306 during which a surgical operation is
performed. One or more users may use orthopedic surgical system 100
in intraoperative phase 306.
[0154] In some examples, one or more users, including at least one
surgeon, may use orthopedic surgical system 100 in an
intraoperative setting to perform shoulder surgery. FIG. 21 is a
flowchart illustrating example stages of a shoulder joint repair
surgery. As discussed above, FIG. 21 describes an example surgical
process for a shoulder surgery. The surgeon may wear or otherwise
use visualization device 213 during each step of the surgical
process of FIG. 18. In other examples, a shoulder surgery may
include more, fewer, or different steps. For example, a shoulder
surgery may include step for adding a bone graft, adding cement,
and/or other steps. In some examples, visualization device 213 may
present virtual guidance to guide the surgeon, nurse, or other
users, through the steps in the surgical workflow.
[0155] In the example of FIG. 21, a surgeon performs an incision
process (1900). During the incision process, the surgeon makes a
series of incisions to expose a patient's shoulder joint. In some
examples, an MR system (e.g., MR system 212, MR system 1800A, etc.)
may help the surgeon perform the incision process, e.g., by
displaying virtual guidance imagery illustrating how to where to
make the incision.
[0156] Furthermore, in the example of FIG. 21, the surgeon may
perform a humerus cut process (1902). During the humerus cut
process, the surgeon may remove a portion of the humeral head of
the patient's humerus. Removing the portion of the humeral head may
allow the surgeon to access the patient's glenoid. Additionally,
removing the portion of the humeral head may allow the surgeon to
subsequently replace the portion of the humeral head with a humeral
implant compatible with a glenoid implant that the surgeon plans to
implant in the patient's glenoid.
[0157] As discussed above, the humerus preparation process may
enable the surgeon to access the patient's glenoid. In the example
of FIG. 21, after performing the humerus preparation process, the
surgeon may perform a registration process that registers a virtual
glenoid object with the patient's actual glenoid bone (1904) in the
field of view presented to the surgeon by visualization device
213.
[0158] In general terms, registration can be viewed as determining
a first local reference coordinate system with respect to the 3D
virtual model and determining a second local reference coordinate
system with respect to the observed real anatomy. In some examples,
MR system 212 also can use the optical image data collected from
optical cameras 530 and/or depth cameras 532 and/or motion sensors
533 (or any other acquisition sensor) to determine a global
reference coordinate system with respect to the environment (e.g.,
operating room) in which the user is located. In other examples,
the global reference coordinate system can be defined in other
manners. In some examples, depth cameras 532 are externally coupled
to visualization device 213, which may be a mixed reality headset,
such as the Microsoft HOLOLENS.TM. headset or a similar MR
visualization device. For instance, depth cameras 532 may be
removable from visualization device 213. In some examples, depth
cameras 532 are part of visualization device 213, which again may
be a mixed reality headset. For instance, depth cameras 532 may be
contained within an outer housing of visualization device 213.
[0159] The registration process may result in generation of a
transformation matrix that then allows for translation along the x,
y, and z axes of the 3D virtual bone model and rotation about the
x, y and z axes in order to achieve and maintain alignment between
the virtual and observed bones. In some examples, after
registration is complete, MR system 212 utilize the results of the
registration to perform simultaneous localization and mapping
(SLAM) to maintain alignment of the virtual model to the
corresponding observed object.
[0160] Once registration is complete the surgical plan can be
executed using the Augment Surgery mode of MR system 212. For
example, FIG. 22 illustrates an image perceptible to a user when in
the augment surgery mode of a mixed reality system, according to an
example of this disclosure. As shown in the example of FIG. 22, the
surgeon can visualize a virtually planned entry point 2700 and
drilling axis 2702 on observed bone structure 2200 and use those
virtual images to assist with positions and alignment of surgical
tools. Drilling axis 2702 may also be referred to as a reaming axis
and provides a virtual guide for drilling a hole in the glenoid for
placement of a guide pin that will guide a reaming process.
[0161] The registration process may be used in conjunction with the
virtual planning processes and/or intra-operative guidance
described elsewhere in this disclosure. Thus, in one example, a
virtual surgical plan is generated or otherwise obtained to repair
an anatomy of interest of a particular patient (e.g., the shoulder
joint of the particular patient). In instances where the virtual
surgical plan is obtained, another computing system may generate
the virtual surgical plan and an MR system (e.g., MR system 212) or
other computing system obtains the virtual surgical plan from a
computer readable medium, such as a communication medium or a
non-transitory storage medium. In this example, the virtual
surgical plan may include a 3D virtual model of the anatomy of
interest generated based on preoperative image data and a
prosthetic component selected for the particular patient to repair
the anatomy of interest. Furthermore, in this example, a user may
use a MR system (e.g., MR system 212) to implement the virtual
surgical plan. In this example, as part of using the MR system, the
user may request the virtual surgical plan for the particular
patient.
[0162] Additionally, the user may view virtual images of the
surgical plan projected within a real environment. For example, MR
system 212 may present 3D virtual objects such that the objects
appear to reside within a real environment, e.g., with real anatomy
of a patient, as described in various examples of this disclosure.
In this example, the virtual images of the surgical plan may
include one or more of the 3D virtual model of the anatomy of
interest, a 3D model of the prosthetic component, and virtual
images of a surgical workflow to repair the anatomy of interest.
Furthermore, in this example, the user may register the 3D virtual
model with a real anatomy of interest of the particular patient.
The user may then implement the virtually generated surgical plan
to repair the real anatomy of interest based on the registration.
In other words, in the augmented surgery mode, the user can use the
visualization device to align the 3D virtual model of the anatomy
of interest with the real anatomy of interest.
[0163] In such examples, the MR system implements a registration
process whereby the 3D virtual model is aligned (e.g., optimally
aligned) with the real anatomy of interest. In this example, the
user may register the 3D virtual model with the real anatomy of
interest without using virtual or physical markers. In other words,
the 3D virtual model may be aligned (e.g., optimally aligned) with
the real anatomy of interest without the use of virtual or physical
markers. The MR system may use the registration to track movement
of the real anatomy of interest during implementation of the
virtual surgical plan on the real anatomy of interest. In some
examples, the MR system may track the movement of the real anatomy
of interest without the use of tracking markers.
[0164] In some examples, as part of registering the 3D virtual
model with the real anatomy of interest, the 3D virtual model can
be aligned (e.g., by the user) with the real anatomy of interest
and generate a transformation matrix between the 3D virtual model
and the real anatomy of interest based on the alignment. The
transformation matrix provides a coordinate system for translating
the virtually generated surgical plan to the real anatomy of
interest. For instance, the registration process may allow the user
to view steps of the virtual surgical plan projected on the real
anatomy of interest. For instance, the alignment of the 3D virtual
model with the real anatomy of interest may generate a
transformation matrix that may allow the user to view steps of the
virtual surgical plan (e.g., identification of an entry point for
positioning a prosthetic implant to repair the real anatomy of
interest) projected on the real anatomy of interest.
[0165] In some examples, the registration process (e.g., the
transformation matrix generated using the registration process) may
allow the user to implement the virtual surgical plan on the real
anatomy of interest without use of tracking markers. In some
examples, aligning the 3D virtual model with the real anatomy of
interest including positioning a point of interest on a surface of
the 3D virtual model at a location of a corresponding point of
interest on a surface of the real anatomy of interest and adjusting
an orientation of the 3D virtual model so that a virtual surface
normal at the point of interest is aligned with a real surface
normal at the corresponding point of interest. In some such
examples, the point of interest is a center point of a glenoid.
[0166] With continued reference to FIG. 21, after performing the
registration process, the surgeon may perform a reaming axis
drilling process (1906). During the reaming axis drilling process,
the surgeon may drill a reaming axis guide pin hole in the
patient's glenoid to receive a reaming guide pin. At a later stage
of the shoulder surgery, the surgeon may insert a reaming axis pin
into the reaming axis guide pin hole. In some examples, an MR
system (e.g., MR system 212, MR system 1800A, etc.) may present a
virtual reaming axis to help the surgeon perform the drilling in
alignment with the reaming axis and thereby place the reaming guide
pin in the correct location and with the correct orientation.
[0167] The surgeon may perform the reaming axis drilling process in
one of various ways. For example, the surgeon may perform a
guide-based process to drill the reaming axis pin hole. In the
case, a physical guide is placed on the glenoid to guide drilling
of the reaming axis pin hole. In other examples, the surgeon may
perform a guide-free process, e.g., with presentation of a virtual
reaming axis that guides the surgeon to drill the reaming axis pin
hole with proper alignment. An MR system (e.g., MR system 212, MR
system 1800A, etc.) may help the surgeon perform either of these
processes to drill the reaming axis pin hole.
[0168] Furthermore, in the surgical process of FIG. 21, the surgeon
may perform a reaming axis pin insertion process (1908). During the
reaming axis pin insertion process, the surgeon inserts a reaming
axis pin into the reaming axis pin hole drilled into the patient's
scapula. In some examples, an MR system (e.g., MR system 212, MR
system 1800A, etc.) may present virtual guidance information to
help the surgeon perform the reaming axis pin insertion
process.
[0169] After performing the reaming axis insertion process, the
surgeon may perform a glenoid reaming process (1910). During the
glenoid reaming process, the surgeon reams the patient's glenoid.
Reaming the patient's glenoid may result in an appropriate surface
for installation of a glenoid implant. In some examples, to ream
the patient's glenoid, the surgeon may affix a reaming bit to a
surgical drill. The reaming bit defines an axial cavity along an
axis of rotation of the reaming bit. The axial cavity has an inner
diameter corresponding to an outer diameter of the reaming axis
pin. After affixing the reaming bit to the surgical drill, the
surgeon may position the reaming bit so that the reaming axis pin
is in the axial cavity of the reaming bit. Thus, during the glenoid
reaming process, the reaming bit may spin around the reaming axis
pin. In this way, the reaming axis pin may prevent the reaming bit
from wandering during the glenoid reaming process. In some
examples, multiple tools may be used to ream the patient's glenoid.
An MR system (e.g., MR system 212, MR system 1800A, etc.) may
present virtual guidance to help the surgeon or other users to
perform the glenoid reaming process. For example, the MR system may
help a user, such as the surgeon, select a reaming bit to use in
the glenoid reaming process. In some examples, the MR system
present virtual guidance to help the surgeon control the depth to
which the surgeon reams the user's glenoid. In some examples, the
glenoid reaming process includes a paleo reaming step and a neo
reaming step to ream different parts of the patient's glenoid.
[0170] As discussed above, in some examples, the use of a
patient-matched (e.g., patient-specific) implant may reduce or
eliminate the need to perform the glenoid reaming process. For
instance, by using a patient-matched implant designed in accordance
with the technique discussed above with reference to FIGS. 8-17B,
the surgeon can reduce or eliminate the need to perform the glenoid
reaming process.
[0171] Additionally, in the surgical process of FIG. 21, the
surgeon may perform a glenoid implant installation process (1912).
During the glenoid implant installation process, the surgeon
installs a glenoid implant in the patient's glenoid. In some
instances, when the surgeon is performing an anatomical shoulder
arthroplasty, the glenoid implant has a concave surface that acts
as a replacement for the user's natural glenoid. In other
instances, when the surgeon is performing a reverse shoulder
arthroplasty, the glenoid implant has a convex surface that acts as
a replacement for the user's natural humeral head. In this reverse
shoulder arthroplasty, the surgeon may install a humeral implant
that has a concave surface that slides over the convex surface of
the glenoid implant. As in the other steps of the shoulder surgery
of FIG. 21, an MR system (e.g., MR system 212, MR system 1800A,
etc.) may present virtual guidance to help the surgeon perform the
glenoid installation process.
[0172] In some examples, the glenoid implantation process includes
a process to fix the glenoid implant (e.g., a patient-matched
glenoid implant) to the patient's scapula (1914). In some examples,
the process to fix the glenoid implant to the patient's scapula
includes drilling one or more anchor holes or one or more screw
holes into the patient's scapula and positioning an anchor such as
one or more pegs or a keel of the implant in the anchor hole(s)
and/or inserting screws through the glenoid implant and the screw
holes, possibly with the use of cement or other adhesive. An MR
system (e.g., MR system 212, MR system 1800A, etc.) may present
virtual guidance to help the surgeon with the process of fixing the
glenoid implant the glenoid bone, e.g., including virtual guidance
indicating anchor or screw holes to be drilled or otherwise formed
in the glenoid, and the placement of anchors or screws in the
holes.
[0173] Furthermore, in the example of FIG. 21, the surgeon may
perform a humerus preparation process (1916). During the humerus
preparation process, the surgeon prepares the humerus for the
installation of a humerus implant. In instances where the surgeon
is performing an anatomical shoulder arthroplasty, the humerus
implant may have a convex surface that acts as a replacement for
the patient's natural humeral head. The convex surface of the
humerus implant slides within the concave surface of the glenoid
implant. In instances where the surgeon is performing a reverse
shoulder arthroplasty, the humerus implant may have a concave
surface and the glenoid implant has a corresponding convex surface.
As described elsewhere in this disclosure, an MR system (e.g., MR
system 212, MR system 1800A, etc.) may present virtual guidance
information to help the surgeon perform the humerus preparation
process.
[0174] Furthermore, in the example surgical process of FIG. 21, the
surgeon may perform a humerus implant installation process (1918).
During the humerus implant installation process, the surgeon
installs a humerus implant on the patient's humerus. As described
elsewhere in this disclosure, an MR system (e.g., MR system 212, MR
system 1800A, etc.) may present virtual guidance to help the
surgeon perform the humerus preparation process.
[0175] After performing the humerus implant installation process,
the surgeon may perform an implant alignment process that aligns
the installed glenoid implant and the installed humerus implant
(1920). For example, in instances where the surgeon is performing
an anatomical shoulder arthroplasty, the surgeon may nest the
convex surface of the humerus implant into the concave surface of
the glenoid implant. In instances where the surgeon is performing a
reverse shoulder arthroplasty, the surgeon may nest the convex
surface of the glenoid implant into the concave surface of the
humerus implant. Subsequently, the surgeon may perform a wound
closure process (1922). During the wound closure process, the
surgeon may reconnect tissues severed during the incision process
in order to close the wound in the patient's shoulder.
[0176] As discussed above with regard to step 1904, the surgeon may
perform a registration process. For a shoulder arthroplasty
application, the registration process may start by virtualization
device 213 presenting the user with 3D virtual bone model 1008 of
the patient's scapula and glenoid that was generated from
preoperative images of the patient's anatomy, e.g., by surgical
planning system 102. The user can then manipulate 3D virtual bone
model 1008 in a manner that aligns and orients 3D virtual bone
model 1008 with the patient's real scapula and glenoid that the
user is observing in the operating environment. As such, in some
examples, the MR system may receive user input to aid in the
initialization and/or registration. However, discussed above, in
some examples, the MR system may perform the initialization and/or
registration process automatically (e.g., without receiving user
input to position the 3D bone model). For other types of
arthroplasty procedures, such as for the knee, hip, foot, ankle or
elbow, different relevant bone structures can be displayed as
virtual 3D images and aligned and oriented in a similar manner with
the patient's actual, real anatomy.
[0177] Regardless of the particular type of joint or anatomical
structure involved, selection of the augment surgery mode initiates
a procedure where 3D virtual bone model 1008 is registered with an
observed bone structure. In general, the registration procedure can
be considered as a classical optimization problem (e.g., either
minimization or maximization). For a shoulder arthroplasty
procedure, known inputs to the optimization (e.g., minimization)
analysis are the 3D geometry of the observed patient's bone
(derived from sensor data from the visualization device 213,
including depth data from the depth camera(s) 532) and the geometry
of the 3D virtual bone derived during the virtual surgical planning
state (such as by using the BLUEPRINT.TM. system). Other inputs
include details of the surgical plan (also derived during the
virtual surgical planning stage, such as by using the BLUEPRINT.TM.
system), such as the position and orientation of entry points,
cutting planes, reaming axes and/or drilling axes, as well as
reaming or drilling depths for shaping the bone structure, the
type, size and shape of the prosthetic components, and the position
and orientation at which the prosthetic components will be placed
or, in the case of a fracture, the manner in which the bone
structure will be rebuilt.
[0178] Upon selection of a particular patient from the welcome page
of UI 522 of MR system 212 (FIG. 5), the surgical planning
parameters associated with that patient are connected with the
patient's 3D virtual bone model 1008, e.g., by one or more
processors of visualization device 213. In the Augment Surgery
mode, registration of 3D virtual bone model 1008 (with the
connected preplanning parameters) with the observed bone by
visualization device 213 allows the surgeon to visualize virtual
representations of the surgical planning parameters on the
patient.
[0179] The optimization (e.g., minimization) analysis that is
implemented to achieve registration of the 3D virtual bone model
1008 with the real bone generally is performed in two stages: an
initialization stage and an optimization (e.g., minimization)
stage. During the initialization stage, the user approximately
aligns the 3D virtual bone model 1008 with the patient's real bone,
such as by using gaze direction, hand gestures and/or voice
commands to position and orient, or otherwise adjust, the alignment
of the virtual bone with the observed real bone. The initialization
stage will be described in further detail below. During the
optimization (e.g., minimization) stage, which also will be
described in detail below, an optimization (e.g., minimization)
algorithm is executed that uses information from the optical
camera(s) 530 and/or depth camera(s) 532 and/or any other
acquisition sensor (e.g., motion sensors 533) to further improve
the alignment of the 3D model with the observed anatomy of
interest. In some examples, the optimization (e.g., minimization)
algorithm can be a minimization algorithm, including any known or
future-developed minimization algorithm, such as an Iterative
Closest Point algorithm or a genetic algorithm as examples.
[0180] In this way, in one example, a mixed reality surgical
planning method includes generating a virtual surgical plan to
repair an anatomy of interest of a particular patient. The virtual
surgical plan including a 3D virtual model of the anatomy of
interest is generated based on preoperative image data and a
prosthetic component selected for the particular patient to repair
the anatomy of interest. Furthermore, in this example, the method
includes using a MR visualization system to implement the virtual
surgical plan. In this example, using the MR system may comprise
requesting the virtual surgical plan for the particular patient.
Using the MR system also comprises viewing virtual images of the
surgical plan projected within a real environment. For example,
visualization device 213 may be configured to present one or more
3D virtual images of details of the surgical plan that are
projected within a real environment, e.g., such that the virtual
image(s) appear to form part of the real environment. The virtual
images of the surgical plan may include the 3D virtual model of the
anatomy of interest, a 3D model of the prosthetic component, and
virtual images of a surgical workflow to repair the anatomy of
interest. Using the MR system may also include registering the 3D
virtual model with a real anatomy of interest of the particular
patient. Additionally, in this example, using the MR system may
include implementing the virtually generated surgical plan to
repair the real anatomy of interest based on the registration.
[0181] Furthermore, in some examples, the method comprises
registering the 3D virtual model with the real anatomy of interest
without using virtual or physical markers. The method may also
comprise using the registration to track movement of the real
anatomy of interest during implementation of the virtual surgical
plan on the real anatomy of interest. The movement of the real
anatomy of interest may be tracked without the use of tracking
markers. In some instances, registering the 3D virtual model with
the real anatomy of interest may comprise aligning the 3D virtual
model with the real anatomy of interest and generating a
transformation matrix between the 3D virtual model and the real
anatomy of interest based on the alignment. The transformation
matrix provides a coordinate system for translating the virtually
generated surgical plan to the real anatomy of interest. In some
examples, aligning may comprise virtually positioning a point of
interest on a surface of the 3D virtual model within a
corresponding region of interest on a surface of the real anatomy
of interest; and adjusting an orientation of the 3D virtual model
so that a virtual surface shape associated with the point of
interest is aligned with a real surface shape associated with the
corresponding region of interest. In some examples, aligning may
further comprise rotating the 3D virtual model about a gaze line of
the user. The region of interest may be an anatomical landmark of
the anatomy of interest. The anatomy of interest may be a shoulder
joint. In some examples, the anatomical landmark is a center region
of a glenoid.
[0182] In some examples, after a registration process is complete,
a tracking process can be initiated that continuously and
automatically verifies the registration between 3D virtual bone
model 1008 and observed bone structure 2200 during the Augment
Surgery mode. During a surgery, many events can occur (e.g.,
patient movement, instrument movement, loss of tracking, etc.) that
may disturb the registration between the 3D anatomical model and
the corresponding observed patient anatomy or that may impede the
ability of MR system 212 to maintain registration between the model
and the observed anatomy. Therefore, by implementing a tracking
feature, MR system 212 can continuously or periodically verify the
registration and adjust the registration parameters as needed. If
MR system 212 detects an inappropriate registration (such as
patient movement that exceeds a threshold amount), the user may be
asked to re-initiate the registration process.
[0183] In some examples, tracking can be implemented using one or
more optical markers that is fixed to a particular location on the
anatomy. MR system 212 monitors the optical marker(s) in order to
track the position and orientation of the relevant anatomy in 3D
space. If movement of the marker is detected, MR system 212 can
calculate the amount of movement and then translate the
registration parameters accordingly so as to maintain the alignment
between the 3D model and the observed anatomy without repeating the
registration process.
[0184] In other examples, tracking is markerless. For example,
rather than using optical markers, MR system 212 implements
markerless tracking based on the geometry of the observed anatomy
of interest. In some examples, the markerless tracking may rely on
the location of anatomical landmarks of the bone that provide
well-defined anchor points for the tracking algorithm. In
situations or applications in which well-defined landmarks are not
available, a tracking algorithm can be implemented that uses the
geometry of the visible bone shape or other anatomy. In such
situations, image data from optical camera(s) 530 and/or depth
cameras(s) 532 and/or motion sensors 533 (e.g., IMU sensors) can be
used to derive information about the geometry and movement of the
visible anatomy. An example of a tracking algorithm that can be
used for markerless tracking is described in David J. Tan, et al.,
"6D Object Pose Estimation with Depth Images: A Seamless Approach
for Robotic Interaction and Augmented Reality," arXiv:1709.01459v1
[cs,CV] (Sept. 5, 2017), although any suitable tracking algorithm
can be used. In some examples, the markerless tracking mode of MR
system 212 can include a learning stage in which the tracking
algorithm learns the geometry of the visible anatomy before
tracking is initiated. The learning stage can enhance the
performance of tracking so that tracking can be performed in real
time with limited processing power.
[0185] As discussed elsewhere in this disclosure, orthopedic
surgical procedures may involve performing various work on a
patient's anatomy. Some examples of work that may be performed
include, but are not necessarily limited to, cutting, drilling,
reaming, screwing, adhering, and impacting. In general, it may be
desirable for a practitioner (e.g., surgeon, physician's assistant,
nurse, etc.) to perform the work as accurately as possible. For
instance, if a surgical plan for implanting a prosthetic in a
particular patient specifies that a portion of the patient's
anatomy is to be reamed at a particular diameter to a particular
depth, it may desirable for the surgeon to ream the portion of the
patient's anatomy to as close as possible to the particular
diameter and to the particular depth (e.g., to increase the
likelihood that the prosthetic will fit and function as planned and
thereby promote a good health outcome for the patient).
[0186] A visualization system, such as MR visualization system 212,
may be configured to display virtual guidance including one or more
virtual guides for performing work on a portion of a patient's
anatomy. For instance, the visualization system may display a
virtual cutting plane overlaid on an anatomic neck of the patient's
humerus. In some examples, a user such as a surgeon may view
real-world objects in a real-world scene. The real-world scene may
be in a real-world environment such as a surgical operating room.
In this disclosure, the terms real and real-world may be used in a
similar manner. The real-world objects viewed by the user in the
real-world scene may include the patient's actual, real anatomy,
such as an actual glenoid or humerus, exposed during surgery. The
user may view the real-world objects via a see-through (e.g.,
transparent) screen, such as see-through holographic lenses, of a
head-mounted MR visualization device, such as visualization device
213, and also see virtual guidance such as virtual MR objects that
appear to be projected on the screen or within the real-world
scene, such that the MR guidance object(s) appear to be part of the
real-world scene, e.g., with the virtual objects appearing to the
user to be integrated with the actual, real-world scene. For
example, the virtual cutting plane/line may be projected on the
screen of a MR visualization device, such as visualization device
213, such that the cutting plane is overlaid on, and appears to be
placed within, an actual, observed view of the patient's actual
humerus viewed by the surgeon through the transparent screen, e.g.,
through see-through holographic lenses. Hence, in this example, the
virtual cutting plane/line may be a virtual 3D object that appears
to be part of the real-world environment, along with actual,
real-world objects.
[0187] A screen through which the surgeon views the actual, real
anatomy and also observes the virtual objects, such as virtual
anatomy and/or virtual surgical guidance, may include one or more
see-through holographic lenses. The holographic lenses, sometimes
referred to as "waveguides," may permit the user to view real-world
objects through the lenses and display projected holographic
objects for viewing by the user. As discussed above, an example of
a suitable head-mounted MR device for visualization device 213 is
the Microsoft HOLOLENS.TM. headset, available from Microsoft
Corporation, of Redmond, Wash., USA. The HOLOLENS.TM. headset
includes see-through, holographic lenses, also referred to as
waveguides, in which projected images are presented to a user. The
HOLOLENS.TM. headset also includes an internal computer, cameras
and sensors, and a projection system to project the holographic
content via the holographic lenses for viewing by the user. In
general, the Microsoft HOLOLENS.TM. headset or a similar MR
visualization device may include, as mentioned above, LCoS display
devices that project images into holographic lenses, also referred
to as waveguides, e.g., via optical components that couple light
from the display devices to optical waveguides. The waveguides may
permit a user to view a real-world scene through the waveguides
while also viewing a 3D virtual image presented to the user via the
waveguides. In some examples, the waveguides may be diffraction
waveguides.
[0188] The presentation virtual guidance such as of a virtual
cutting plane may enable a surgeon to accurately resect the humeral
head without the need for a mechanical guide, e.g., by guiding a
saw along the virtual cutting plane displayed via the visualization
system while the surgeon views the actual humeral head. In this
way, a visualization system, such as MR system 212 with
visualization device 213, may enable surgeons to perform accurate
work (e.g., with the accuracy of mechanical guides but without the
disadvantages of using mechanical guides). This "guideless" surgery
may, in some examples, provide reduced cost and complexity.
[0189] The visualization system (e.g., MR system 212/visualization
device 213) may be configured to display different types of virtual
guides. Examples of virtual guides include, but are not limited to,
a virtual point, a virtual axis, a virtual angle, a virtual path, a
virtual plane, and a virtual surface or contour. As discussed
above, the visualization system (e.g., MR system 212/visualization
device 213) may enable a user to directly view the patient's
anatomy via a lens by which the virtual guides are displayed, e.g.,
projected.
[0190] The visualization system may obtain parameters for the
virtual guides from a virtual surgical plan, such as the virtual
surgical plan described herein. Example parameters for the virtual
guides include, but are not necessarily limited to: guide location,
guide orientation, guide type, guide color, etc.
[0191] The techniques of this disclosure are described below with
respect to a shoulder arthroplasty surgical procedure. Examples of
shoulder arthroplasties include, but are not limited to, reversed
arthroplasty, augmented reverse arthroplasty, standard total
shoulder arthroplasty, augmented total shoulder arthroplasty, and
hemiarthroplasty. However, the techniques are not so limited, and
the visualization system may be used to provide virtual guidance
information, including virtual guides in any type of surgical
procedure. Other example procedures in which a visualization
system, such as MR system 212, may be used to provide virtual
guides include, but are not limited to, other types of orthopedic
surgeries; any type of procedure with the suffix "plasty," "stomy,"
"ectomy," "clasia," or "centesis,"; orthopedic surgeries for other
joints, such as elbow, wrist, finger, hip, knee, ankle or toe, or
any other orthopedic surgical procedure in which precision guidance
is desirable.
[0192] A typical shoulder arthroplasty includes various work on a
patient's scapula and performing various work on the patient's
humerus. The work on the scapula may generally be described as
preparing the scapula (e.g., the glenoid cavity of the scapula) for
attachment of a prosthesis and attaching the prosthesis to the
prepared scapula. Similarly, the work on the humerus may generally
be described as preparing the humerus for attachment of a
prosthesis and attaching the prosthesis to the prepared humerus. As
described herein, the visualization system may provide guidance for
any or all work performed in such an arthroplasty procedure.
[0193] As discussed above, a MR system (e.g., MR system 212 etc.)
may receive a virtual surgical plan for attaching a prosthetic to a
patient and/or preparing bones, soft tissue or other anatomy of the
patient to receive the prosthetic. The virtual surgical plan may
specify various work to be performed and various parameters for the
work to be performed. As one example, the virtual surgical plan may
specify a location on the patient's glenoid for performing reaming
and a depth for the reaming. As another example, the virtual
surgical plan may specify a surface for resecting the patient's
humeral head. As another example, the virtual surgical plan may
specify locations and/or orientations of one or more anchorage
locations (e.g., screws, stems, pegs, keels, etc.).
[0194] Many different techniques may be used to prepare a humerus
for prosthesis attachment and to perform actual prosthesis
attachment. Regardless of the technique used, MR system 212 may
provide virtual guidance to assist in one or both of the
preparation and attachment. As such, while the following techniques
are examples in which MR system 212 provides virtual guidance, MR
system 212 may provide virtual guidance for other techniques.
[0195] In an example technique, the work steps include resection of
a humeral head, creating a pilot hole, sounding, punching,
compacting, surface preparation, with respect to the humerus, and
attaching an implant to the humerus. Additionally, in some
techniques, the work steps may include bone graft work steps, such
as installation of a guide in a humeral head, reaming of the graft,
drilling the graft, cutting the graft, and removing the graft,
e.g., for placement with an implant for augmentation of the implant
relative to a bone surface such as the glenoid.
[0196] A surgeon may perform one or more steps to expose a
patient's humerus. For instance, the surgeon may make one or more
incisions to expose the upper portion of the humerus including the
humeral head. The surgeon may position one or more retractors to
maintain the exposure. In some examples, MR system 212 may provide
guidance to assist in the exposure of the humerus, e.g., by making
incisions, and/or placement of retractors.
[0197] Many different techniques may be used to prepare a scapula
for prosthesis attachment and to perform actual prosthesis
attachment. Regardless of the technique used, MR system 212 may
provide virtual guidance to assist in one or both of the
preparation and attachment. As such, while the following techniques
are examples in which MR system 212 provides virtual guidance, MR
system 212 may provide virtual guidance for other techniques.
[0198] In an example technique, the surgical procedure steps
include installation of a guide in a glenoid of the scapula,
reaming the glenoid, creating a central hole in the glenoid,
creating additional anchorage positions in the glenoid, and
attaching an implant to the prepared glenoid. As a guide pin is
used, the example technique may be considered a cannulated
technique. However, the techniques are similarly applicable to
non-cannulated techniques.
[0199] A surgeon may perform one or more steps to expose a
patient's glenoid. For instance, with the patient's arm abducted
and internally rotated, the surgeon may make one or more incisions
to expose the glenoid. The surgeon may position one or more
retractors to maintain the exposure. In some examples, MR system
212 may provide guidance to assist in the exposure and/or placement
of retractors.
[0200] FIG. 23 is a conceptual diagram illustrating an MR system
providing virtual guidance to a user for installation of a guide in
a glenoid of a scapula, in accordance with one or more techniques
of this disclosure. As shown in FIG. 23, MR system 212 may display
virtual guidance, e.g., in the form of virtual axis 5104, on
glenoid 5102 of scapula 5100. To display virtual axis 5104, MR
system 212 may determine a location on a virtual model of glenoid
5102 at which a guide is to be installed. MR system 212 may obtain
the location from a virtual surgical plan (e.g., the virtual
surgical plan described above). The location obtained by MR system
212 may specify one or both of coordinates of a point on the
virtual model and a vector. The point may be the position at which
the guide is to be installed and the vector may indicate the
angle/slope at which the guide is to be installed.
[0201] As discussed above, the virtual model of glenoid 5102 may be
registered with glenoid 5102 such that coordinates on the virtual
model approximately correspond to coordinates on glenoid 5102. As
such, by displaying virtual axis 5104 at the determined location on
the virtual model, MR system 212 may display virtual axis 5104 at
the planned position on glenoid 5102.
[0202] As also discussed above, the virtual model of glenoid 5102
may be selectively displayed after registration. For instance,
after the virtual model of glenoid 5102 is registered with glenoid
5102, MR system 212 may cease displaying of the virtual model.
Alternatively, MR system 212 may continue to display the virtual
model overlaid on glenoid 5102 after registration. The display of
the virtual model may be selective in that the surgeon may activate
or deactivate display of the virtual model.
[0203] MR system 212 may display the virtual model and/or virtual
guides with varying opacity (e.g., transparency). The opacity may
be adjusted automatically, manually, or both. As one example, the
surgeon may provide user input to MR system 212 to manually adjust
the opacity of the virtual model and/or virtual guides. As another
example, MR system 212 may automatically adjust the opacity based
on an amount of light in the viewing field (e.g., amount of light
where the surgeon is looking). For instance, MR system 212 may
adjust the opacity (e.g., increase the transparency) of the virtual
model and/or virtual guides to positively correlate with the amount
of light in the viewing field (e.g., brighter light results in
increased opacity/decreased transparency and dimmer light results
in decreased opacity/increased transparency).
[0204] The surgeon may attach a physical guide using the displayed
virtual guidance. As one example, where the guide is a guide pin
with a self-tapping threaded distal tip, the surgeon may align the
guide pin with the displayed virtual axis 5104 and utilize a drill
or other instrument to install the guide pin. As another example,
where the guide is a guide pin without a self-tapping tip, the
surgeon may align a drill bit of a drill with the displayed virtual
axis 5104 and operate the drill to form a hole to receive the guide
pin and then install the guide pin in the hole. In some examples,
MR system 212 may display depth guidance information to enable the
surgeon to install the guide pin to a planned depth. Examples of
depth guidance information are discussed in further detail herein
with reference to FIG. 66.
[0205] FIG. 24 is a conceptual diagram illustrating guide 5200,
i.e., a guide pin in this example, as installed in glenoid 5102. As
shown in FIGS. 51 and 52, by displaying virtual axis 5104, a
surgeon may drill in alignment with the virtual axis, which may be
referred to as a reaming axis, and thereby form a hole for
installation of guide 5200 at the planned position on glenoid 5102.
In this way, MR system 212 may enable the installation of a guide
without the need for an additional mechanical guide.
[0206] FIG. 25 is a conceptual diagram illustrating an MR system
providing virtual guidance for reaming a glenoid, in accordance
with one or more techniques of this disclosure. As shown in FIG.
25, reaming tool 5300 may be used to ream the surface of glenoid
5102. In this example, reaming tool 5300 may be a cannulated
reaming tool configured to be positioned and/or guided by a guide
pin, such as guide 5200. For example, the shaft of cannulated
reaming tool may receive guide 5200 such that the tool shaft is
mounted substantially concentrically with the pin. In other
examples, reaming tool 5300 may not be cannulated and may be guided
without the assistance of a physical guide pin.
[0207] The surgeon may attach reaming tool 5300 to guide 5200
(e.g., insert proximal tip of guide 5200 into reaming tool 5300),
and attach a drill or other instrument to rotate reaming tool 5300.
To perform the reaming, the surgeon may rotate reaming tool 5300 to
advance reaming tool 5300 down guide 5200 until reaming is
complete.
[0208] As discussed above, in some examples, the techniques of this
disclosure may reduce or eliminate the need to perform reaming of
the glenoid. In particular, by using a patient matched glenoid
implant (i.e., an implant with a surface shaped to conform to a
patient's glenoid), a surgeon may avoid (or reduce) the need to
perform reaming of the glenoid.
[0209] MR system 212 may display virtual guidance to assist in the
reaming process. As one example MR system 212 may provide depth
guidance. For instance, MR system 212 may display depth guidance to
enable the surgeon to ream to a target depth. As another example,
MR system 212 may provide targeting guidance. For instance, MR
system 212 may display an indication of whether reaming tool 5300
is aligned with a virtual reaming axis.
[0210] While described herein as a single reaming step, the surgery
may include multiple reaming steps. The various reaming steps may
use the same axis/guide pin or may use different axes/guide pins.
In examples where different reaming steps use different axes, MR
system 212 may provide virtual guidance for reaming using the
different axes.
[0211] FIGS. 26 and 27 are conceptual diagrams illustrating an MR
system providing virtual guidance for creating a central hole in a
glenoid, in accordance with one or more techniques of this
disclosure. As shown in FIGS. 26 and 27, drill bit 5400 may be used
to drill central hole 5500 in glenoid 5102. In this example, drill
bit 5400 may be a cannulated drill bit configured to be positioned
and/or guided by a guide pin, such as guide 5200. In other
examples, drill bit 5400 may not be cannulated and may be guided
without the assistance of a physical guide pin. For instance, MR
system 212 may provide virtual guidance to enable a surgeon to
drill glenoid 5102 without the use of guide 5200. As discussed in
further detail below, central hole 5500 may facilitate the
attachment of a prosthesis to glenoid 5102, e.g., via one or more
anchors.
[0212] MR system 212 may display virtual guidance to assist in the
creation of central hole 5500. For instance, MR system 212 may
display depth guidance to enable the surgeon to drill central hole
5500 to a target depth. As another example, MR system 212 may
provide targeting guidance. For instance, MR system 212 may display
an indication of whether drill bit tool 5400 is on a prescribed
axis selected to form the central hole 5500 at a proper position at
with a proper orientation.
[0213] In addition to a central hole (e.g., central hole 5500), it
may be desirable for the surgeon to create additional anchorage
positions in the glenoid. This additional anchorage positions may
improve the fixation between the prosthesis and the glenoid. For
instance, the additional anchorage positions may provide
anti-rotation support between the prosthesis and the glenoid.
Several different styles of anchorage may be used, depending on the
type of prosthesis to be installed. Some examples of anchorage
include, but are not necessarily limited to, keel and pegged
anchors. However, the virtual guidance techniques discussed herein
may be applicable to any type of anchorage. Example MR guidance for
keel type anchorage is discussed below with reference to FIGS.
26-29. Example MR guidance for pegged type anchorage is discussed
below with reference to FIGS. 30-32. In each case, the anchorage
may help in placing a glenoid implant, such as a glenoid base plate
for anatomic arthroplasty or a glenoid base plate and glenosphere
for reverse arthroplasty, onto the glenoid and fixing it in
place.
[0214] FIG. 28 is a conceptual diagram illustrating a glenoid
prosthesis with keel type anchorage. As shown in FIG. 28, glenoid
prosthesis 5600 includes rear surface 5602 configured to engage a
prepared surface of glenoid 5102 (e.g., a reamed surface), and a
keel anchor 5604 configured to be inserted in a keel slot created
in glenoid 5102 (e.g., keel slot 5902 of FIG. 31).
[0215] In some examples, glenoid prosthesis 5600 may be a patient
matched glenoid implant. For instance, at least a portion of rear
surface 5602 may be contoured to match a surface of glenoid 5102
using the techniques discussed above with reference to FIGS.
8-17B.
[0216] FIGS. 29-31 are conceptual diagrams illustrating an MR
system providing virtual guidance for creating keel type anchorage
positions in a glenoid, in accordance with one or more techniques
of this disclosure. As shown in FIG. 29, MR system 212 may provide
virtual guidance for drilling additional holes in glenoid 5102. MR
system 212 may provide the virtual guidance for drilling the
additional holes in any of a variety of manners. As one example, MR
system 212 may display virtual guidance such as virtual markers
having specified shapes (e.g., axes, arrows, points, circles, X
shapes, crosses, targets, etc.), sizes and/or colors, at the
locations the additional holes are to be drilled. For instance, in
the example of FIG. 29, MR system 212 may display virtual markers
5700A and 5700B at the locations the additional holes are to be
drilled. As another example, MR system 212 may display virtual axes
at the locations the additional holes are to be drilled to aid the
surgeon in properly aligning a drill bit to make the holes in the
glenoid bone.
[0217] MR system 212 may determine the locations of the additional
holes based on the virtual surgical plan. For instance, similar to
virtual axis 5104 of FIG. 23, MR system 212 may obtain, from the
virtual surgical plan, the location(s) of the additional holes to
be drilled on the virtual model of glenoid 5102. As such, by
displaying virtual markers 5700A and 5700B at the determined
locations on the virtual model, MR system 212 may display virtual
markers 5700A and 5700B at the planned positions on glenoid 5102.
As discussed above, the virtual surgical plan may be patient
specific in that the plan may be specifically developed for a
particular patient. As such, the planned positioned on glenoid 5102
at which MR system 212 displays virtual markers 5700A and 5700B may
be considered patient-specific planned positions. Therefore, the
locations of the planned positions will vary from patient to
patient according to individual patient-specific surgical
plans.
[0218] The surgeon may utilize a drill bit and a drill to create
the additional hole(s) at the location(s) indicated by MR system
212. For instance, as shown in FIG. 30, the surgeon may drill hole
5800A at the location of virtual marker 5700A and drill hole 5800B
at the location of virtual marker 5700B. The surgeon may use the
same drill bit for each hole or may use different drill bits for
different holes.
[0219] MR system 212 may provide virtual guidance for the drilling
in addition to or in place of the virtual markers, such as those
described above, which indicate the locations the additional holes
are to be drilled. As one example, MR system 212 may provide
targeting guidance to indicate whether the drill is on a target
axis. In this case, as an addition or alternative to the virtual
markers, MR system 212 may display guide axes that extend outward
from the locations of each of the respective holes to be drilled.
As another example, MR system 212 may display a mask with holes in
the mask that correspond to the locations at which the holes are to
be drilled. As another example, MR system 212 may display depth
guidance to enable the surgeon to drill holes 5800A and 5800B to
target depths.
[0220] MR system 212 may provide virtual guidance for working the
holes into a keel slot that may accept keel anchor 5604 of glenoid
prosthesis 5600. As an example, MR system 212 may display virtual
outline 5802 around holes 5800A, 5500, and 5800B. For instance, MR
system 212 may display virtual outline 5802 as approximately
corresponding to a final outline of the desired keel slot to be
created.
[0221] The surgeon may utilize a tool to work holes 5800A, 5500,
and 5800B into keel slot 5902. As shown in FIG. 29, the surgeon may
utilize keel punch 5900 to work holes 5800A, 5500, and 5800B into
keel slot 5902. For instance, the surgeon may impact keel punch
5900 into the area indicated by virtual outline 5802. In this case,
virtual outline 5802 defines a shape and dimension of the desired
keel slot 5902, permitting the surgeon to work the holes into a
form that visually matches or approximates the displayed virtual
outline of the keel slot.
[0222] MR system 212 may provide additional or alternative virtual
guidance for creating keel slot 5902. As one example, MR system 212
may display depth guidance to enable the surgeon to impact keel
punch 5900 to a target depth. As another example, MR system 212 may
provide targeting guidance to indicate whether keel punch 5900 is
on a target axis. As another example, MR system 212 may display a
mask with a cutout for virtual outline 5802.
[0223] FIG. 32 is a conceptual diagram illustrating a glenoid
prosthesis with pegged type anchorage. As shown in FIG. 32, glenoid
prosthesis 6000 includes rear surface 6002 configured to engage a
prepared surface of glenoid 5102 (e.g., a reamed surface), a
central peg anchor 6004 configured to be inserted in a central hole
created in glenoid 5102, and one or more peg anchors 6006A-6006C
(collectively, "peg anchors 6006") respectively configured to be
inserted in additional holes created in glenoid 5102.
[0224] In some examples, glenoid prosthesis 6000 may be a patient
matched glenoid implant. For instance, at least a portion of rear
surface 6002 may be contoured to match a surface of glenoid 5102
using the techniques discussed above with reference to FIGS.
8-17B.
[0225] FIGS. 33 and 34 are conceptual diagrams illustrating an MR
system providing virtual guidance for creating pegged type
anchorage positions in a glenoid, in accordance with one or more
techniques of this disclosure. As shown in FIG. 33, MR system 212
may provide virtual guidance for drilling additional holes in
glenoid 5102. MR system 212 may provide the virtual guidance for
drilling the additional holes in any of a variety of manners. As
one example, MR system 212 may display virtual markers (e.g., axes,
points, circles, X shapes, etc.) at the locations the additional
holes are to be drilled. For instance, in the example of FIG. 33,
MR system 212 may display virtual markers 5700A-5700C at the
locations the additional holes are to be drilled. As another
example, MR system 212 may display virtual axes extending from the
locations at which the additional holes are to be drilled. As
another example, MR system 212 may display a mask (effectively an
inverse of the virtual markers) that indicates where the holes are
to be drilled.
[0226] MR system 212 may determine the locations of the additional
holes based on the virtual surgical plan. For instance, similar to
virtual axis 5104 of FIG. 23, MR system 212 may obtain, from the
virtual surgical plan, which may be patient-specific, the
location(s) of the additional holes to be drilled on the virtual
model of glenoid 5102. As such, by displaying virtual markers
5700A-5700C at the determined locations on the virtual model, MR
system 212 may display virtual markers 5700A-5700C at the planned
positions on glenoid 5102.
[0227] The surgeon may utilize a drill bit (or multiple drill bits)
and a drill to create the additional hole(s) at the location(s)
indicated by MR system 212. For instance, as shown in FIG. 34, the
surgeon may drill hole 5800A at the location of virtual marker
5700A, drill hole 5800B at the location of virtual marker 5700B,
and drill hole 5800C at the location of virtual marker 5700C.
[0228] MR system 212 may provide virtual guidance for the drilling
in addition to or in place of the virtual markers that indicate the
locations the additional holes are to be drilled. As one example,
MR system 212 may provide targeting guidance to indicate whether
the drill is on a target axis. As another example, MR system 212
may display depth guidance to enable the surgeon to drill holes
5800A-5800C to target depths.
[0229] It is noted that different implants may have different
profiles, such as augmented profiles. Additionally, as discussed
herein, some implants may be implanted with additional materials
harvested from the patient, such as bone grafts. In some of such
examples, MR system 212 may provide virtual guidance for placement
of the additional materials. For instance, MR system 212 may
provide virtual guidance for attaching a bone graft to an implant
and guidance for attaching the graft/implant assembly to the
patient.
[0230] In some examples, regardless of the anchorage type being
used, the surgeon may utilize a trial component to determine
whether glenoid 5102 has been properly prepared. The trial
component may have a rear surface and anchors sized and positioned
identical to the rear surface and anchors of the prosthesis to be
implanted.
[0231] FIG. 35 is a conceptual diagram illustrating an MR system
providing virtual guidance for attaching an implant to a glenoid,
in accordance with one or more techniques of this disclosure. A
tool may be used to attach the implant (e.g., a pegged implant, a
keeled implant, or any other type of implant) to glenoid 5102. For
instance, the surgeon may utilize impactor 6302 to insert
prosthesis 6300 into the prepared glenoid 5102. In some examples,
one or more adhesives (e.g., glue, cement, etc.) may be applied to
prosthesis 6300 and/or glenoid 5102 prior to impaction.
[0232] In some examples, one or more fasteners may be used to
attach a prosthesis to glenoid 5102. For instance, as shown in
FIGS. 36 and 37, screws 6400A-6400D (collectively, "screws 6400")
and central stem 6402 may be used to attach prosthesis 6300 to
glenoid 5102. These fasteners may be used in addition to, or in
place of, any anchorages included in the prosthesis (e.g., pegs,
keels, etc.).
[0233] MR system 212 may provide virtual guidance to facilitate the
installation of the additional fasteners. For instance, as shown in
FIG. 35, MR system 212 may display virtual axes 6500A-6500D
(collectively, "virtual axes 6500"), which may be referred to as
"virtual screw axes," to guide the surgeon in the installation of
screws 6400. In examples where screws 6400 are not "self-tapping",
MR system 212 may display virtual guidance (e.g., virtual axes) to
guide drilling of pilot holes for screws 6400. For instance, MR
system 212 may display a virtual drilling axis obtained from the
virtual surgical plan that guides drilling of a pilot hole for a
screw of screws 6400.
[0234] To display the virtual guides for installation of the
fasteners, MR system 212 may register a virtual model of the
prosthesis to the actual observed prosthesis. For instance, MR
system 212 may obtain a virtual model of prosthesis 6300 from the
virtual surgical plan and perform the registration in a manner
similar to the registration process described.
[0235] MR system 212 may obtain locations for each of the fasteners
to be installed. For instance, MR system 212 may obtain, from the
virtual surgical plan, coordinates on the virtual model of the
prosthesis and vector for each of the fasteners. In some examples,
MR system 212 may determine that the coordinates for each fastener
are the centroid of a corresponding hole in the prosthesis. For
instance, MR system 212 may determine that the coordinates for
screw 6400A are the centroid of hole 6502.
[0236] The surgeon may install the fasteners using the displayed
virtual guidance. For instance, the surgeon may use a screwdriver
or other instrument to install screws 6400.
[0237] MR system 212 may display virtual guidance to assist in the
fastener attachment. As one example MR system 212 may provide depth
guidance. For instance, MR system 212 may display depth guidance to
enable the surgeon to install each of screws 6400 to a target
depth. As another example, MR system 212 may provide targeting
guidance. For instance, MR system 212 may display an indication of
whether each of screws 6400 is being installed on a prescribed
axis. As another example, MR system 212 may provide guidance on an
order in which to tighten screws 6400. For instance, MR system 212
may display a virtual marker on a particular screw of screws 6400
that is to be tightened.
[0238] As discussed above, MR system 212 may provide a wide variety
of virtual guidance. Example of virtual guidance that may be
provided by MR system 212 include, but are not limited to,
targeting guidance and depth guidance. MR system 212 may provide
targeting guidance to assist a surgeon in performing work (e.g.,
drilling a hole, reaming, installing a screw, etc.) along a
particular axis. MR system 212 may provide depth guidance to assist
a surgeon in performing work (e.g., drilling a hole, reaming,
installing a screw, etc.) to a desired depth.
[0239] While the techniques been disclosed with respect to a
limited number of examples, those skilled in the art, having the
benefit of this disclosure, will appreciate numerous modifications
and variations there from. For instance, it is contemplated that
any reasonable combination of the described examples may be
performed. It is intended that the appended claims cover such
modifications and variations as fall within the true spirit and
scope of the invention.
[0240] It is to be recognized that depending on the example,
certain acts or events of any of the techniques described herein
can be performed in a different sequence, may be added, merged, or
left out altogether (e.g., not all described acts or events are
necessary for the practice of the techniques). Moreover, in certain
examples, acts or events may be performed concurrently, e.g.,
through multi-threaded processing, interrupt processing, or
multiple processors, rather than sequentially.
[0241] In one or more examples, the functions described may be
implemented in hardware, software, firmware, or any combination
thereof If implemented in software, the functions may be stored on
or transmitted over as one or more instructions or code on a
computer-readable medium and executed by a hardware-based
processing unit. Computer-readable media may include
computer-readable storage media, which corresponds to a tangible
medium such as data storage media, or communication media including
any medium that facilitates transfer of a computer program from one
place to another, e.g., according to a communication protocol. In
this manner, computer-readable media generally may correspond to
(1) tangible computer-readable storage media which is
non-transitory or (2) a communication medium such as a signal or
carrier wave. Data storage media may be any available media that
can be accessed by one or more computers or one or more processors
to retrieve instructions, code and/or data structures for
implementation of the techniques described in this disclosure. A
computer program product may include a computer-readable
medium.
[0242] By way of example, and not limitation, such
computer-readable storage media can comprise RAM, ROM, EEPROM,
CD-ROM or other optical disk storage, magnetic disk storage, or
other magnetic storage devices, flash memory, or any other medium
that can be used to store desired program code in the form of
instructions or data structures and that can be accessed by a
computer. Also, any connection is properly termed a
computer-readable medium. For example, if instructions are
transmitted from a website, server, or other remote source using a
coaxial cable, fiber optic cable, twisted pair, digital subscriber
line (DSL), or wireless technologies such as infrared, radio, and
microwave, then the coaxial cable, fiber optic cable, twisted pair,
DSL, or wireless technologies such as infrared, radio, and
microwave are included in the definition of medium. It should be
understood, however, that computer-readable storage media and data
storage media do not include connections, carrier waves, signals,
or other transitory media, but are instead directed to
non-transitory, tangible storage media. Disk and disc, as used
herein, includes compact disc (CD), laser disc, optical disc,
digital versatile disc (DVD), floppy disk and Blu-ray disc, where
disks usually reproduce data magnetically, while discs reproduce
data optically with lasers. Combinations of the above should also
be included within the scope of computer-readable media.
[0243] Operations described in this disclosure may be performed by
one or more processors, which may be implemented as fixed-function
processing circuits, programmable circuits, or combinations
thereof, such as one or more digital signal processors (DSPs),
general purpose microprocessors, application specific integrated
circuits (ASICs), field programmable gate arrays (FPGAs), or other
equivalent integrated or discrete logic circuitry. Fixed-function
circuits refer to circuits that provide particular functionality
and are preset on the operations that can be performed.
Programmable circuits refer to circuits that can programmed to
perform various tasks and provide flexible functionality in the
operations that can be performed. For instance, programmable
circuits may execute instructions specified by software or firmware
that cause the programmable circuits to operate in the manner
defined by instructions of the software or firmware. Fixed-function
circuits may execute software instructions (e.g., to receive
parameters or output parameters), but the types of operations that
the fixed-function circuits perform are generally immutable.
Accordingly, the terms "processor" and "processing circuity," as
used herein may refer to any of the foregoing structures or any
other structure suitable for implementation of the techniques
described herein.
[0244] Various examples have been described. These and other
examples are within the scope of the following claims.
* * * * *