Systems and methods for navigation and visualization

• US 10,058,393 B2
• Filed: 10/21/2016
• Issued: 08/28/2018
• Est. Priority Date: 10/21/2015
• Status: Active Grant

• Current Assignee: P TECH, LLC
• Original Assignee: P TECH, LLC
• Inventors: Bonutti, Peter M., Beyers, Justin E.
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/299,981
• Publication Number:
  US 20170112577A1
• Time in Patent Office: 676 Days
• Field of Search: 382128, 600424, 700245
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 2090/392 Radioactive markers
• CPC Class Codes: A61B 2090/3941 Photoluminescent markers
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: A61B 90/98 using electromagnetic means, e.g. transponders
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30012 Spine; Backbone
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 : involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A system for visualizing a surgical site is provided. The system includes a robotic mechanism for performing a procedure on a patient, an imaging device coupled to the robotic mechanism, the imaging device configured to provide image data of a site of interest, and a computing device coupled to the imaging device. The computing device includes one or more processors and at least one memory device configured to store executable instructions. The executable instructions, when executed by the processor, are configured to receive the image data of the site of interest, track motion patterns of the site of interest in the received image data, filter the received image data to remove line-of-sight restrictions therein and alter pixels therein based on the tracked motion patterns, and generate an output frame from the filtered image data. The system also includes a presentation interface device coupled to the computing device and configured to present the output frame for visualization of the site of interest.

Claims

1. A system for visualizing a surgical site, the system comprising:

a robotic mechanism for performing a procedure on a patient;

an imaging device coupled to the robotic mechanism, the imaging device configured to provide image data of a site of interest;

a computing device coupled to the imaging device, the computing device including one or more processors and at least one memory device configured to store executable instructions, the executable instructions, when executed by the processor, configured to;

receive the image data of the site of interest;

differentiate between at least one object of interest and at least one detected obstruction based on movement patterns and color variations;

filter the received image data to remove the at least one detected obstruction and recreate pixels related to the at least one detected obstruction based on at least one of (i) one or more buffered frames and (ii) one or more pixels adjacent to the pixels related to the at least one detected obstruction;

generate an output frame from the filtered image data; and

at least one of cease filtering and alert when the at least one detected obstruction exceeds a predetermined size threshold of the image of the site of interest; and

a presentation interface device coupled to the computing device, the presentation interface device configured to present the output frame for visualization of the site of interest.

2. The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on both (i) one or more buffered frames and (ii) one or more pixels adjacent to the at least one detected obstruction.

3. The system of claim 1, wherein the at least one detected obstruction is at least one of blood, synovium, tissue fats, debris, bone fragments, and smoke.

4. The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on (i) one or more buffered frames.

5. The system of claim 4, wherein the one or more buffered frames comprise a baseline image.

6. The system of claim 5, wherein the computing system is configured to compare the received image data to the baseline image.

7. The system of claim 6, wherein the computing system is configured to determine based on the comparison of the received image data to the baseline image that the detected obstruction is obstructing the object of interest in the received image data.

8. The system of claim 5, wherein the computing system is configured to mask the received image data to superimpose pixels related to the object of interest based on the baseline image onto the received image data over the pixels related to the detected obstruction.

9. The system of claim 1, wherein pixels related to the at least one detected obstruction are recreated based on (ii) pixels surrounding the at least one detected obstruction.

10. The system of claim 1, wherein the computing device is configured to cease filtering when the at least one detected obstruction exceeds the predetermined size threshold.

11. The system of claim 1, wherein the computing device is configured to alert when the at least one detected obstruction exceeds the predetermined size threshold.

12. The system of claim 1, wherein the object of interest comprises a source of a bleed and the detected obstruction comprises an opaque blood flow.

13. The system of claim 1, wherein the detected obstruction comprises opaque blood.

14. The system of claim 13, wherein the computing device is configured to adjust a color value of the pixels related to the opaque blood based on a color value of one or more adjacent pixels.

15. The system of claim 13, wherein the computing device is configured to adjust an intensity of the pixels related to the opaque blood based on an intensity of one or more adjacent pixels.

16. The system of claim 1, wherein the computing device is configured to adjust a color value of the pixels related to the detected obstruction based on color values of one or more adjacent pixels.

17. The system of claim 1, wherein computing device is configured to adjust the intensity of the pixels related to the detected obstruction based on color values of one or more adjacent pixels.