Medical device navigation using a virtual 3D space

• US 10,772,686 B2
• Filed: 03/27/2019
• Issued: 09/15/2020
• Est. Priority Date: 10/27/2016
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: State, Andrei, Heaney, Brian, Kohli, Luv, Keller, Kurtis, Green, Caroline
• Primary Examiner(s): Osifade, Idowu O
• Application Number:
  US16/366,537
• Publication Number:
  US 20190321107A1
• Time in Patent Office: 538 Days
• Field of Search: None
• CPC Class Codes: A61B 17/068 Surgical staplers , e.g. containing multiple staples or clampsstaplers containing only one staple A61B17/10; magazines or containers for staples A61B17/105; for performing anastomosis A61B17/115; staplers in general B25C5/00
• CPC Class Codes: A61B 17/07207 the staples being applied sequentially
• CPC Class Codes: A61B 17/320016 Endoscopic cutting instruments, e.g. arthroscopes, resectoscopes A61B10/0233, A61B17/29, A61B17/3207 take precedence
• CPC Class Codes: A61B 2017/00106 ultrasonic
• CPC Class Codes: A61B 2017/07214 Stapler heads
• CPC Class Codes: A61B 2017/07271 characterised by its cartridge
• CPC Class Codes: A61B 2017/320052 Guides for cutting instruments by marking A61B2090/395
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/302 specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities
• CPC Class Codes: A61B 2090/3612 with images taken automatically
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 2090/3782 transmitter or receiver in catheter or minimal invasive instrument
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 2090/502 : Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 34/20 : Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 : User interfaces for surgical systems
• CPC Class Codes: A61B 90/10 : for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation

Abstract

A system and method for providing image guidance for placement of one or more medical devices at a target location. The system can determine one or more intersections between a medical device and an image region based at least in part on first emplacement data and second emplacement data. Using the determined intersections, the system can cause one or more displays to display perspective views of image guidance cues, including an intersection ghost in a virtual 3D space.

Claims

1. A computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:

determine an emplacement of a medical display object;

determine an emplacement of a plane-of-interest;

determine an emplacement of a ghost medical display object based at least in part on the determined emplacement of the medical display object and the determined emplacement of the plane-of-interest; and

cause the display to concurrently display, in a virtual 3D space;

at least a portion of the medical display object, and

at least a portion of the ghost medical display object on the plane-of-interest.

2. The system of claim 1, wherein the computer system is further configured to:

receive first emplacement data associated with a first medical device, wherein to determine the emplacement of the medical display object, the computer system is configured to determine the emplacement of the medical display object based at least in part on the received first emplacement data; and

receive second emplacement data associated with a second medical device, wherein to determine the emplacement of the plane-of-interest, the computer system is configured to determine the emplacement of the plane-of-interest based at least in part on the received second emplacement data.

3. The system of claim 2, wherein the computer system is further configured to:

receive a medical image corresponding to the second medical device; and

determine an emplacement of the medical image based at least in part on the received second emplacement data,

wherein to determine the emplacement of the plane-of-interest, the computer system is configured to determine the emplacement of the plane-of-interest based at least in part on the determined emplacement of the medical image.

4. The system of claim 1, wherein to determine the emplacement of the ghost medical display object, the computer system is further configured to, at least one of:

determine at least one of portions, points, or coordinates of the plane-of-interest that are closest to the medical display object,

determine an arc path corresponding to the medical display object and the plane-of-interest,

determine which of a plurality of connecting lines intersect the medical display object, wherein the plurality of connecting lines extend orthogonally from the plane-of-interest to the medical display object, or

use a virtual light source that is at least as large as the medical display object to be projected, wherein the virtual light source is distal to the plane-of-interest relative to the medical display object.

5. The system of claim 1, wherein the ghost medical display object comprises at least one of:

a virtual medical device that corresponds to a physical medical device, wherein the physical medical device comprises a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;

a medical image slice that corresponds to a real-time medical image of a medical imaging device;

or

an affected region of the virtual medical device, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.

6. The system of claim 1, wherein the plane-of-interest corresponds to at least one of a plane of a medical device or an image plane.

7. The system of claim 1, wherein the medical display object corresponds to an affected region of a medical device, wherein the medical device comprises a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis, and wherein the affected region corresponds to at least one of:

a volume located between the first limb member and the second limb member,

a stapling path of at least one staple of the medical device, or

a cutting path of a knife of the medical device.

8. The system of claim 1, wherein the medical display object corresponds to at least one of a transecting stapler, a grasper, or a vessel sealer, and wherein the plane-of-interest corresponds to an imaging plane of a medical imaging device.

9. A computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:

determine an emplacement of a virtual medical device corresponding to a physical medical device, the physical medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis;

determine an emplacement of an affected region based at least in part on the determined emplacement of the virtual medical device; and

cause the display to display, in a virtual 3D space, at least one of;

at least a portion of the affected region based at least in part on the determined emplacement of the affected region, or

an intersection indicator indicating an intersection between an image plane and a trajectory of the affected region along a drive path that is parallel to the joint member axis.

10. The system of claim 9, wherein the computer system is further configured to:

receive emplacement data associated with the physical medical device, wherein to determine the emplacement of the virtual medical device, the computer system is configured to determine the emplacement of the virtual medical device based at least in part on the received emplacement data; and

receive operating parameters corresponding to the physical medical device, wherein to determine the emplacement of the affected region, the computer system is configured to determine emplacement of the affected region is further based at least in part on the received operating parameters.

11. The system of claim 9, wherein the computer system is further configured to cause the display to further display, in the virtual 3D space, at least a portion of the virtual medical device based at least in part on the determined emplacement of the virtual medical device.

12. The system of claim 9, wherein the affected region corresponds to a volume located between the first limb member and the second limb member.

13. The system of claim 9, wherein the physical medical device comprises a medical stapler, wherein the affected region comprises a stapling path of the medical stapler.

14. The system of claim 9, wherein the affected region comprises a cutting path of a knife of the physical medical device.

15. The system of claim 9, wherein the physical medical device comprises at least one of a transecting stapler, a grasper, or a vessel sealer.

16. A computer system in communication with a display, the computer system comprising a computer processor and a non-transitory storage medium, wherein the computer system is configured to:

determine an emplacement of a virtual medical device;

determine an emplacement of a medical image slice;

determine a first intersection between an image plane corresponding to the medical image slice and a trajectory of a first portion of the virtual medical device along a first trajectory path of the virtual medical device;

determine a second intersection between the image plane and a trajectory of a second portion of the virtual medical device along the first trajectory path of the virtual medical device; and

cause the display to concurrently display, in a virtual 3D space;

at least a portion of the medical image slice based at least in part on the determined emplacement of the medical image slice,

a first intersection indicator corresponding to the first intersection, and

a second intersection indicator corresponding to the second intersection.

17. The system of claim 16, wherein the virtual medical device corresponds to a physical medical device, the physical medical device comprising a joint member having a joint member axis, a first limb member having a first axis, and a second limb member having a second axis that is different from the joint member axis and the first axis, wherein the first portion of the virtual medical device corresponds to the first limb member, wherein the second portion of the virtual medical device corresponds to the second limb member.

18. The system of claim 17, wherein the physical medical device comprises at least one of a transecting stapler, a grasper, or a vessel sealer.

19. The system of claim 17, wherein the first trajectory path is parallel to at least one of the joint member axis, the first axis, or the second axis.

20. The system of claim 16, wherein the computer system is further configured to cause display to further concurrently display, in the virtual 3D space, at least one of:

a first trajectory indicator corresponding to the trajectory of the first portion of the virtual medical device along the first trajectory path, or

a second trajectory indicator corresponding to the trajectory of the second portion of the virtual medical device along the first trajectory path.