AUGMENTED SURGICAL REALITY ENVIRONMENT SYSTEM

• US 20150366628A1
• Filed: 05/12/2015
• Published: 12/24/2015
• Est. Priority Date: 06/18/2014
• Status: Active Application

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Ingmanson, Michael
• Application Number:
  US14/709,800
• Publication Number:
  US 20150366628A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/0804 Counting number of instruments used; Instrument detectors
• CPC Class Codes: A61B 2090/0805 automatically, e.g. by means of magnetic, optical or photoelectric detectors
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/366 using projection of images directly onto the body
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/02 Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/14542 for measuring blood gases A61B5/14551 takes precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: A61B 90/96 : using barcodes

Abstract

The present disclosure is directed to an augmented surgical reality environment system and methods. The system includes an image capture device to capture an image of a surgical environment. At least one biometric sensor obtains biometric data from a patient. A controller includes a memory configured to store a plurality of anatomical images and a processor. The processor receives at least one of the captured image, the biometric data, or one or more anatomical images from the plurality of anatomical images and generates an augmented image from at least one of the captured image, the biometric data, or the one or more anatomical images. A display device displays the augmented image.

Claims

1. An augmented surgical reality environment system comprising:

an image capture device configured to capture an image of a surgical environment;

at least one biometric sensor configured to obtain biometric data from a patient;

a controller including;

a memory configured to store a plurality of anatomical images;

a processor configured to;

(i) receive at least one of the captured images of the surgical environment, the biometric data, or one or more anatomical images from the plurality of anatomical images; and

(ii) generate an augmented image from at least one of the captured images of the surgical environment, the biometric data, or the one or more anatomical images; and

a display device configured to display the augmented image.

2. The augmented surgical reality environment system according to claim 1, wherein the display device is a projector.

3. The augmented surgical reality environment system according to claim 1, wherein the display device includes:

a frame;

at least one lens; and

a projector configured to project the augmented image onto the lens.

4. The augmented surgical reality environment system according to claim 1, wherein the controller determines a position or orientation of a surgical tool relative to the patient.

5. The augmented surgical reality environment system according to claim 4, wherein the augmented image includes a virtual image of a portion of the surgical tool disposed within the patient.

6. The augmented surgical reality environment system according to claim 4, wherein the position or orientation of the surgical tool is determined based on an image of the surgical tool captured by the image capture device.

7. The augmented surgical reality environment system according to claim 4, wherein the position or orientation of the surgical tool is determined by data provided by the surgical tool.

8. The augmented surgical reality environment system according to claim 7, wherein the data provided by the surgical tool includes accelerometer data or gyroscopic data.

9. The augmented surgical reality environment system according to claim 1, wherein the processor determines a position of an object relative to the patient based on the image of the surgical environment.

10. The augmented surgical reality environment system according to claim 9, wherein the augmented image includes an enhanced representation of the object and the display device displays the enhanced representation of the object on the patient at the position determined by the processor.

11. The augmented surgical reality environment system according to claim 1, wherein the controller is configured to receive a position signal from an object.

12. The augmented surgical reality environment system according to claim 11, wherein the processor determines a position of the object based on the received position signal.

13. The augmented surgical reality environment system according to claim 12, wherein the augmented image includes an enhanced representation of the object and the display device displays the enhanced representation of the object on the patient at the position determined by the processor.

14. The augmented surgical reality environment system according to claim 1, wherein the plurality of anatomical images are obtained from an x-ray, a computed tomography scan, or magnetic resonance imaging data.

15. The augmented surgical reality environment system according to claim 14, wherein the anatomical images are processed by the processor to enhance a portion of the anatomical image.

16. The augmented surgical reality environment system according to claim 15, wherein the enhanced portion of the anatomical image is displayed on the patient by the display device.

17. The augmented surgical reality environment system according to claim 16, wherein the enhanced portion of the anatomical image includes a heat map.

18. The augmented surgical reality environment system according to claim 1, wherein the augmented image includes a surgical plan includes one or more of a cut path, an incision location, an implant location, or notes.

19. The augmented surgical reality environment system according to claim 1, including a surgical device, wherein the augmented image includes a status of the surgical device.

20. The augmented surgical reality environment system according to claim 1, wherein the captured image of the surgical environment includes a direction and magnitude of a first cut and the processor determines a desired cut path and a distance for a second cut based on:

(i) the direction and magnitude of the first cut; and

(ii) the plurality of anatomical images stored in the memory.

21. The augmented surgical reality environment system according to claim 20, wherein the augmented image includes an image representing a direction and magnitude of the second cut.

22. The augmented surgical reality environment system according to claim 1, wherein the image capture device captures a first image of the surgical environment and a second image of the surgical environment,

wherein the controller determines if an object has moved based on a difference between the first image and the second image, and

wherein the controller highlights the object in the augmented image to be displayed on the display.