Surgery system

• US 9,733,463 B2
• Filed: 05/27/2015
• Issued: 08/15/2017
• Est. Priority Date: 05/27/2014
• Status: Active Grant

• Current Assignee: Carl Zeiss Meditec AG (Carl-Zeiss-Stiftung)
• Original Assignee: Carl Zeiss Meditec AG (Carl-Zeiss-Stiftung)
• Inventors: Eslami, Abouzar, Maier-Matic, Corinna
• Primary Examiner(s): Flory, Christopher A
• Application Number:
  US14/722,233
• Publication Number:
  US 20150342698A1
• Time in Patent Office: 811 Days
• Field of Search: 606 5
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/3618 with a mirror
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/3735 Optical coherence tomography [OCT]
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/20 Surgical microscopes characterised by non-optical aspects
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61F 9/00727 Apparatus for retinal reattachment
• CPC Class Codes: G02B 21/0012 Surgical microscopes counterbalanced structures for surgical microscopes G02B7/001
• CPC Class Codes: G02B 21/22 Stereoscopic arrangements

Abstract

A surgery system comprises a camera obtaining camera images, an OCT system, a data memory storing geometry data of a surgical tool 133 and a controller configured to identify a first portion of the tool in the camera images by object recognition using the geometry data; to determine, in the field of view, first locations where the tool is located and second locations aside of the tool; to trigger the OCT system to perform depth scans at the first and second locations; to identify a second portion 153 of the tool in the depth scans by object recognition using the geometry data; and to generate a first image 154 representing a third portion 157 of the tool based on the geometry data and the depth scans.

Claims

1. A surgery system comprising:

a camera and optics, wherein the optics are configured to image a field of view onto the camera, wherein the camera is configured to obtain two-dimensional camera images of the field of view and to generate camera image data representing the camera images;

an OCT system configured to perform depth scans at selectable locations contained in the field of view and to generate depth scans data representing the depth scans;

a data memory storing geometry data of at least one surgical tool;

a controller configured to;

identify a first portion of the at least one surgical tool in the camera images by object recognition using the geometry data of the at least one surgical tool;

determine first locations and second locations in the field of view, wherein the at least one surgical tool is located at the first locations and wherein the second locations are located aside of the at least one surgical tool;

trigger the OCT system to perform first depth scans at the determined first and second locations;

identify a second portion of the at least one surgical tool in the first depth scans by object recognition using the geometry data of the at least one surgical tool;

generate a first image representing at least one third portion of the at least one surgical tool, wherein a shape of a representation of the at least one third portion of the at least one surgical tool is extracted from the geometry data and wherein a position of the representation of the at least one third portion of the at least one surgical tool in the generated first image is based on the identified second portion of the at least one surgical tool.

2. The surgery system according to claim 1, wherein the at least one third portion of the at least one surgical tool is unidentifiable in the first depth scans.

3. The surgery system according to claim 1, wherein the controller is further configured to determine the first and second locations based on at least one of the camera images and second depth scans.

4. The surgery system according to claim 1, wherein the generated first image further represents the second portion of the at least one surgical tool, wherein the representation of the second portion of the at least one surgical tool is based on the geometry data and wherein a position of the representation of the second portion of the at least one surgical tool in the generated first image is based on the first depth scans.

5. The surgery system according to claim 1, wherein the generated first image further represents the first depth scans.

6. The surgery system according to claim 5, wherein the first depth scans and the at least one third portion of the at least one surgical tool are represented by different colors in the generated first image;

or

wherein the first depth scans and the second portion of the at least one surgical tool are represented by different colors in the generated first image.

7. The surgery system according to claim 1, wherein the data memory further stores geometry data of at least one anatomic structure,

wherein the controller is further configured to identify a first portion of the at least one anatomic structure in the depths scans by object recognition using the geometry data of the at least one anatomic structure, and

wherein the generated first image further represents at least one second portion of the at least one anatomic structure, wherein the representation of the at least one second portion of the at least one anatomic structure is based on the geometry data of the at least one anatomic structure and wherein a position of the representation of the at least one second portion of the at least one anatomic structure in the generated first image is based on the first depth scans.

8. The surgery system according to claim 7, wherein the first depth scans and the at least one second portion of the at least one anatomic structure are represented by different colors in the generated first image.

9. The surgery system according to claim 7, wherein the geometry data of the at least one anatomic structure are geometry data of tissue layers of a retina of an eye.

10. The surgery system according to claim 1, wherein the at least one surgical tool has a distal end, and

wherein the distal end of the at least one surgical tool is located at the first locations.

11. The surgery system according to claim 10, wherein the controller triggers the OCT system to perform the first depth scans at the first locations at a greater rate compared to those performed at locations which are located at a distance from the distal end of the at least one surgical tool in the field of view.

12. The surgery system according to claim 1, wherein the first locations and the second locations are located at a distance from each other and on at least one line, wherein the at least one surgical tool has a long axis, and wherein the at least one line is orientated either parallel to the long axis or orthogonal to the long axis.

13. The surgery system according to claim 1, wherein the controller comprises a user interface.

14. The surgery system according to claim 13, wherein the user interface comprises a display, and wherein the controller is further configured to display the generated first image on the display.

15. The surgery system according to claim 13, wherein the controller is further configured to generate a second image representing the camera image and one of the first portion and the second portion of the at least one surgical tool.

16. The surgery system according to claim 15, wherein the user interface comprises an ocular, wherein the optics are further configured to image at least a portion of the field of view via the ocular.

17. The surgery system according to claim 16, wherein the controller is further configured to generate a third image representing the first portion and the second portion of the at least one surgical tool and to project the generated second image into a beam path to the ocular.

18. The surgery system according to claim 13, wherein the data memory stores the geometry data of at least three surgical tools,

wherein the user interface comprises a component configured to obtain a selection of at least one of the at least three surgical tools from a user,

wherein the geometry data of the at least one selected surgical tool is used by the controller for the object recognition, and wherein the geometry data of the at least one not selected surgical tool is not used by the controller for the object recognition.

19. The surgery system according to claim 1, wherein a beam path of the OCT system traverses optical components of the optics.