Augmented reality display systems for fitting, sizing, trialing and balancing of virtual implant components on the physical joint of the patient

• US 10,603,113 B2
• Filed: 07/29/2019
• Issued: 03/31/2020
• Est. Priority Date: 03/12/2016
• Status: Active Grant

• Current Assignee: Philipp K. Lang
• Original Assignee: Philipp K. Lang
• Inventors: Lang, Philipp K.
• Primary Examiner(s): Chang, Olivia C
• Application Number:
  US16/524,436
• Publication Number:
  US 20190380784A1
• Time in Patent Office: 246 Days
• CPC Class Codes: A61B 17/15 Guides therefor arrangements for guiding straight saw blades in general B23D51/025
• CPC Class Codes: A61B 17/155 Cutting femur
• CPC Class Codes: A61B 17/157 Cutting tibia
• CPC Class Codes: A61B 17/1666 for the acetabulum
• CPC Class Codes: A61B 17/17 Guides or aligning means for drills , mills, pins or wires
• CPC Class Codes: A61B 17/1703 using imaging means, e.g. by X-rays
• CPC Class Codes: A61B 17/1742 for the hip
• CPC Class Codes: A61B 17/1764 for the knee
• CPC Class Codes: A61B 17/1775 for the foot or ankle
• CPC Class Codes: A61B 17/1778 for the shoulder
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 2017/568 produced with shape and dimensions specific for an individual patient
• CPC Class Codes: A61B 2034/104 Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/368 changing the image on a display according to the operator's position
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/373 : using light, e.g. by using optical scanners
• CPC Class Codes: A61B 2090/374 : NMR or MRI
• CPC Class Codes: A61B 2090/376 : using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3762 : using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/502 : Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 34/10 : Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/74 : Manipulators with manual electric input means
• CPC Class Codes: A61B 90/36 : Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: A61F 2/32 : for the hip
• CPC Class Codes: A61F 2/3859 : Femoral components
• CPC Class Codes: A61F 2/389 : Tibial components A61F2/3868 takes precedence
• CPC Class Codes: A61F 2002/4018 : Heads or epiphyseal parts of humerus
• CPC Class Codes: A61F 2002/4205 : Tibial components
• CPC Class Codes: A61F 2002/4207 : Talar components
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.

Claims

1. A system for preparing a physical joint in a patient for a joint replacement, the system comprising:

at least one processor,

a see through optical head mounted display, and

a user interface,

wherein the system is configured to display two or more virtual implant components by the see-through optical head mounted display, wherein each virtual implant component is a three-dimensional digital representation corresponding to at least one portion of a corresponding physical implant component, a placement indicator of a corresponding physical implant component, or a combination thereof,

wherein the two or more virtual implant components are selected from a library of virtual implant components, wherein the two or more virtual implant components comprise at least a first virtual implant component selected for a first surface of the physical joint and a second virtual implant component selected for a second surface of the physical joint,

wherein the system is configured to display at least a portion of the first, the second or the first and the second virtual implant components onto the corresponding first, second or first and second surfaces visible directly through the see-through optical head mounted display,

wherein the system is configured to allow superimposition, alignment or superimposition and alignment of the at least the portion of the first, the second or the first and the second virtual implant components with the corresponding first, second or first and second surfaces,

wherein the user interface is configured to adjust the position, orientation or position and orientation of the first, the second or the first and the second virtual implant components in relationship to at least one anatomic structure of the physical joint,

wherein the see through optical head mounted display is registered in a coordinate system,

wherein the at least one anatomic structure is registered in the coordinate system, and

wherein the first, the second or the first and the second virtual implant components are registered in the coordinate system.

2. The system of claim 1, wherein the library of virtual implant components comprises two or more virtual implant components having different sizes, different shapes, or combinations thereof,

wherein the physical joint is a knee and the library comprises one or more virtual femoral components, one or more virtual tibial components, one or more patellar virtual components or any combination thereof for a knee joint replacement, or

wherein the physical joint is a hip and the library comprises one or more virtual femoral components and one or more virtual acetabular components for a hip joint replacement, or

wherein the physical joint is a shoulder and the library comprises one or more virtual humeral components and one or more virtual glenoid components for a shoulder joint replacement, or

wherein the physical joint is an ankle and the library comprises one or more virtual tibial components and one or more virtual talar components for an ankle joint replacement.

3. The system of claim 1, wherein the user interface comprises at least one of a graphical user interface, a voice recognition, a gesture recognition, a virtual interface displayed by the optical head mounted display, a virtual keyboard displayed by the optical head mounted display, a physical keyboard, a physical computer mouse, or a physical track pad.

4. The system of claim 1, wherein the physical joint is a knee and wherein the system is configured to maintain the display of the first, the second or the first and the second virtual implant components in relationship to the at least one anatomic structure when the knee is moved through a range of motion or when the knee is moved into different positions of at least one of flexion, extension, rotation, abduction, adduction or combinations thereof.

5. The system of claim 1, wherein the physical joint is a knee and wherein the system is configured to maintain the display of the first, the second or the first and the second virtual implant components in relationship to the corresponding first, second or first and second surfaces when the knee is moved through a range of motion or when the knee is moved into different positions of at least one of flexion, extension, rotation, abduction, adduction or combinations thereof.

6. The system of claim 1, wherein the system is for preparing the physical joint of the patient for a prosthesis and wherein the prosthesis is for a knee replacement, hip replacement, shoulder joint replacement, or ankle joint replacement.

7. The system of claim 1, wherein the user interface is configured to select the first, the second or the first and the second virtual implant components based on the size, the shape or combinations of thereof of the virtual implant component.

8. The system of claim 1, wherein the user interface is configured to select the first, the second or the first and the second virtual implant components based on the size, the shape or combinations thereof of the virtual implant components in relationship to the physical joint of the patient.

9. The system of claim 1, wherein the system is configured to select the first, the second or the first and the second virtual implant components based on the size, the shape or combinations of thereof of the virtual implant components in relationship to the physical joint of the patient and wherein the selection is automatic, semi-automatic or manual.

10. The system of claim 1, wherein the user interface is configured to adjust at least one of a color, brightness, intensity, contrast, or transparency of the display of the first, the second or the first and the second virtual implant components by the optical head mounted display, and wherein at least one portion of the first, second or first and second surfaces of the physical joint is visible through the superimposed first, second or first and second virtual implant components.

11. The system of claim 1, wherein the system is configured to display the two or more virtual implant components with a different color, a different degree of transparency or a combination thereof.

12. The system of claim 1, wherein the user interface is configured to allow movement of the first, the second or the first and the second virtual implant components in relationship to the physical joint of the patient, or wherein the user interface is configured to allow movement of the first, the second or the first and the second virtual implant components in relationship to the at least one anatomic structure.

13. The system of claim 1, wherein the user interface is configured to allow alignment of the display of the first, the second or the first and the second virtual implant components in relationship to the at least one anatomic structure.

14. The system of claim 1, wherein the system is configured to display the first, the second or the first and the second virtual implant components, by the optical head mounted display at a predetermined position, predetermined orientation, predetermined rotation, predetermined resection level, predetermined version, predetermined inclination, predetermined alignment or combination thereof, in relationship to the at least one anatomic structure.

15. The system of claim 1, wherein the system is configured to align the first, the second or the first and the second virtual implant components relative to at least one of a mechanical axis, a biomechanical axis, an anatomic axis, a rotation axis, a flexion axis, an extension axis, or a combination thereof.

16. The system of claim 1, wherein the user interface is configured to allow modification of the position or orientation or position and orientation of the first, the second or the first and the second virtual implant components to account for ligamentous laxity or tightness.

17. The system of claim 14, wherein the predetermined position, predetermined orientation, predetermined rotation, predetermined resection level, predetermined version, predetermined inclination, predetermined alignment or combination thereof of the first, the second or the first and the second virtual implant components comprises at least one of a predetermined varus correction, a predetermined valgus correction, a predetermined femoral component flexion, a predetermined femoral component extension, a predetermined femoral component rotation, a predetermined femoral component position relative to an anterior cortex, a predetermined tibial component slope, a predetermined tibial component rotation, or a predetermined tibial component position relative to a tibial cortical rim in a knee joint replacement, or comprises at least one of a predetermined femoral neck resection for a femoral component, a predetermined leg length, a predetermined femoral component anteversion, a predetermined femoral component offset, a predetermined acetabular component anteversion, or a predetermined acetabular component inclination in a hip joint replacement, or comprises at least one of a predetermined humeral neck resection for a humeral component, a predetermined arm length, a predetermined humeral component version, a predetermined glenoid component version, or a predetermined glenoid component inclination in a shoulder joint replacement, or comprises at least one of a coronal plane tibial or talar or tibial and talar implant component alignment, a sagittal plane tibial or talar or tibial and talar implant component alignment, an axial plane tibial or talar or tibial and talar implant component alignment, or a tibial or talar or tibial and talar implant component rotation in an ankle joint replacement.

18. The system of claim 14, wherein the user interface is configured to allow modification of the position, orientation or position and orientation of the display of the first, the second or the first and the second virtual implant components relative to the predetermined position, predetermined orientation, predetermined rotation, predetermined resection level, predetermined version, predetermined inclination, predetermined alignment or combinations thereof.

19. The system of claim 1, wherein the at least one anatomic structure comprises at least one anatomic landmark, anatomic plane, articular surface, cartilage surface, subchondral bone surface, cortical bone surface, cut bone surface, reamed bone surface, milled bone surface, impacted bone surface, tissue resection, tissue surface, altered tissue surface, or combinations thereof.

20. The system of claim 1, wherein the at least one anatomic structure comprises information about an anatomic axis, a biomechanical axis, a mechanical axis, a distance between two or more anatomic landmarks, a dimensions between two or more anatomic landmarks, a dimension between an anterior and a posterior anatomic structure, a dimension between a medial and a lateral anatomic structure, a dimension between a superior and an inferior anatomic structure, a curvature, a surface, an edge, a shape, a length, a width, a depth, or combinations thereof.

21. The system of claim 19, wherein the at least one anatomic landmark comprises one or more of a portion of or an entire acetabulum, a portion of or an entire edge of an acetabulum, multiple portions of an edge of an acetabulum, a portion of an iliac wall, a portion of a pubic bone, a portion of an ischial bone, an anterior superior iliac spine, an anterior inferior iliac spine, a symphysis pubis, a portion of or an entire greater trochanter, a portion of or an entire lesser trochanter, a center of the femoral head, a portion of or an entire femoral shaft, a portion of or an entire femoral neck, a portion of or an entire femoral head, a fovea capitis, a transverse acetabular ligament, a pulvinar, a ligamentum teres, a labrum, one or more femoral osteophytes, one or more acetabular osteophytes, a portion of or an entire medial femoral condyle, a portion of or an entire lateral femoral condyle, a portion of or an entire femoral notch, a portion of or an entire trochlea, a portion of an anterior cortex of the femur, a portion of an anterior cortex of the femur with adjacent portions of the trochlea, a portion of an anterior cortex of the femur with adjacent portions of the trochlea and osteophytes, one or more femoral osteophytes, one or more tibial osteophytes, an epicondylar eminence, a medial epicondylar eminence, a lateral epicondylar eminence, a portion of or an entire medial tibial plateau, a portion of or an entire lateral tibial plateau, a portion of or an entire tibial rim, a portion of or an entire medial tibial spine, a portion of or an entire lateral tibial spine, a portion of an anterior cortex of the tibia, a portion of an anterior cortex of the tibia and a portion of a tibial plateau, a portion of or an entire patella, a medial edge of a patella, a lateral edge of a patella, a superior pole of a patella, an inferior pole of a patella, a patellar osteophyte, a medial wall of the femoral notch, a lateral wall of the femoral notch, a roof of the femoral notch, an anterior cruciate ligament origin, an anterior cruciate ligament insertion, a medial wall of the medial condyle, a lateral wall of the lateral condyle, a medial femoral condyle shape, a medial femoral condyle radius, a medial femoral condyle convexity, a medial femoral condyle concavity, a lateral femoral condyle shape, a lateral femoral condyle radius, a lateral femoral condyle convexity, a lateral femoral condyle concavity, an intercondylar notch shape, an intercondylar notch surface feature, a medial tibial spine, a lateral tibial spine, an anteromedial tibial rim, an anterolateral tibial rim, a medial tibial rim, a lateral tibial rim, a lowest point of the medial plateau, a lowest point of the lateral plateau, a highest point of the medial plateau, a highest point of the lateral plateau, a medial tibial plateau shape, a lateral tibial plateau shape, a medial tibial plateau surface, a medial tibial plateau surface feature, a medial tibial plateau radius, a medial tibial plateau convexity, a medial tibial plateau concavity, a lateral tibial plateau shape, a lateral tibial plateau surface, a lateral tibial plateau surface feature, a lateral tibial plateau radius, a lateral tibial plateau convexity, a lateral tibial plateau concavity, a portion of or an entire glenoid, a portion of or an entire coracoid process, a portion of or an entire acromion, a portion of a clavicle, a portion of or an entire humeral head, a portion of or an entire humeral neck, a portion of a humeral shaft, one or more humeral osteophytes, one or more glenoid osteophytes, a portion of or an entire glenoid labrum, a portion of or an entire glenoid rim, a portion of or an entire shoulder ligament, a portion of or an entire coracoacromial ligament, or a portion of or an entire glenohumeral ligament.

22. The system of claim 1, wherein the first, the second or the first and the second virtual implant components are a virtual trial femoral component, a virtual trial tibial component, a virtual trial tibial insert, a virtual trial patellar component, or combinations thereof.

23. The system of claim 1, wherein the system is configured to maintain the display of the first, the second or the first and the second virtual implant components in relationship to the at least one anatomic structure when the physical joint is moved through a range of motion or when the physical joint is moved into different positions of flexion, extension, rotation, abduction, adduction or combinations thereof, wherein the at least one anatomic structure is located on the corresponding first, second or first and second surfaces of the physical joint.

24. The system of claim 1, wherein the joint is a knee and wherein the system is configured to maintain the display of the first and the second and optionally a third virtual implant component in relationship to at least a first, a second and optionally a third anatomic structure when the knee joint is moved through a range of motion or when the knee joint is moved into different positions of flexion, extension, rotation, abduction, adduction or combinations thereof,

wherein the first virtual implant component is a virtual femoral component and the first anatomic structure is located on a distal femur and the second virtual implant component is a virtual tibial component and the second anatomic structure is located on a proximal tibia, or

wherein the first virtual implant component is a virtual femoral component and the first anatomic structure is located on a distal femur and the second virtual implant component is a virtual patellar component and the second anatomic structure is located on a patella, or

wherein the first virtual implant component is a virtual femoral component and the first anatomic structure is located on a distal femur, the second virtual implant component is a virtual tibial component and the second anatomic structure is located on a proximal tibia, and the third virtual implant component is a virtual patellar component and the third anatomic structure is located on a patella.

25. The system of claim 1, wherein the joint is a hip and wherein the system is configured to maintain the display of the first and the second virtual implant components in relationship to at least two or more anatomic structures when the hip joint is moved through a range of motion or when the hip joint is moved into different positions of flexion, extension, rotation, abduction, adduction or combinations thereof, wherein the first virtual implant component is a virtual femoral component and a first anatomic structure is located on a proximal femur and the second virtual implant component is a virtual acetabular component and a second anatomic structure is located on an acetabulum, or

wherein the joint is a shoulder and wherein the system is configured to maintain the display of the first and the second virtual implant components in relationship to at least two or more anatomic structures when the shoulder joint is moved through a range of motion or when the shoulder joint is moved into different positions of flexion, extension, rotation, abduction, adduction or combinations thereof, wherein the first virtual implant component is a virtual humeral component and a first anatomic structure is located on a proximal humerus and the second virtual implant component is a virtual glenoid component and a second anatomic structure is located on a glenoid, or

wherein the joint is an ankle and wherein the system is configured to maintain the display of the first and the second virtual implant components in relationship to at least two or more anatomic structures when the ankle joint is moved through a range of motion or when the ankle joint is moved into different positions of flexion, extension, rotation, abduction, adduction or combinations thereof, wherein the first virtual implant component is a virtual tibial component and a first anatomic structure is located on a distal tibia and the second virtual implant component is a virtual talar component and a second anatomic structure is located on a talus.

26. The system of claim 1, wherein the at least one portion of the first, the second or the first and the second virtual implant components are tangent or intersecting with the at least one anatomic structure.

27. The system of claim 1, wherein the at least one processor is integrated into the optical head mounted display.

28. The system of claim 1, wherein the at least one processor is separate from the optical head mounted display.

29. The system of claim 1, wherein the first, second or the first and second surfaces of the physical joint comprise at least a portion of an articular surface, a cartilage surface, a subchondral bone surface, a cortical bone surface, a cut bone surface, a reamed bone surface, a milled bone surface, an impacted bone surface, a tissue resection, a tissue surface, an altered tissue surface, or combinations thereof.

30. The system of claim 1, wherein the placement indicator comprises an outline of the physical implant component.