Surgical stereo vision systems and methods for microsurgery

• US 9,330,477 B2
• Filed: 09/19/2012
• Issued: 05/03/2016
• Est. Priority Date: 09/22/2011
• Status: Active Grant

• Current Assignee: Digital Surgicals Pte Ltd.
• Original Assignee: Digital Surgicals Pte Ltd.
• Inventors: Rappel, James K.
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Aghevli, Reza
• Application Number:
  US13/622,820
• Publication Number:
  US 20130076863A1
• Time in Patent Office: 1,322 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 21/22 Stereoscopic arrangements
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

Surgical stereo vision systems and methods for microsurgery are described that enable hand-eye collocation, high resolution, and a large field of view. A digital stereo microscope apparatus, an operating system with a digital stereo microscope, and a method are described using a display unit located over an area of interest such that a human operator places hands, tools, or a combination thereof in the area of interest and views a magnified and augmented live stereo view of the area interest with eyes of the human operator substantially collocated with the hands of the human operator.

Claims

1. A digital stereo microscope apparatus, comprising:

a display unit fixed and located over an area of interest such that a human operator places hands, tools, or a combination thereof in the area of interest and views a magnified and augmented live stereo view of the area of interest with eyes of the human operator having a view substantially collocated and in a direct field and line of view with the hands, tools, or combination thereof of the human operator in the area of interest, wherein the human operator is able perform head movement without losing the live stereo view on the display unit;

an image display located on a front side of the display unit relative to the human operator providing the magnified and augmented live stereo view;

an optical system and an image acquisition system located in part on a back side of the display unit providing image capture and data capture of the area of interest;

wherein the optical system and the image acquisition system comprises at least one objective lens and at least one image sensor; and

a processing system communicatively coupled to the image acquisition system and the image display providing the magnified and augmented live stereo view to the image display based on the image capture and the data capture of the area of interest from the optical system and the image acquisition system, wherein the optical system and the image acquisition system is configured to provide the image capture with adjustments to provide the magnified and augmented live stereo view, in real-time, performed by the processing system utilizing stereopsis.

2. The digital stereo microscope apparatus of claim 1, wherein the optical system, the image acquisition system, and the processing system provide continuous zoom without an intermediate lens assembly with the continuous zoom being performed by the processing system.

3. The digital stereo microscope apparatus of claim 1, wherein the optical system and the image acquisition system are configured to capture image data at a maximum resolution of associated hardware with augmentation, magnification, and reduction of resolution of the image data being performed by the processing system.

4. The digital stereo microscope apparatus of claim 1, wherein the image display comprise a stereoscopic flat display panel that allows head movement of the human operator without losing a stereoscopic view of the area of interest.

5. The digital stereo microscope apparatus of claim 1, further comprising:

at least one property sensor capturing data related to the area of interest and communicating the captured data to the processing system, wherein the processing system is configured to process the captured data and display portions of the captured data on the image display in addition to the magnified and augmented live stereo view.

6. The digital stereo microscope apparatus of claim 1, wherein the image display comprises a first image display, and further comprising:

a second image display located apart from the display unit and communicatively coupled to the processing system, wherein the second image display is associated with a second human operator, wherein the second image display comprises an independent magnified and augmented live stereo view from the magnified and augmented live stereo view of the first image display, and wherein the independent magnified and augmented live stereo view is formed by the processing system.

7. The digital stereo microscope apparatus of claim 1, further comprising:

a display control controlling the processing system responsive to the human operator for zoom, pan, and focus of the optical system and the image acquisition system via the processing system, wherein the zoom, pan, and focus is performed virtually by the processing system.

8. The digital stereo microscope apparatus of claim 7, wherein the display control is configured to track eye movements of the human operator and generate commands to configure the optical system and the image acquisition system accordingly.

9. The digital stereo microscope apparatus of claim 8, wherein the eye movement is approximated to the movement of the head and tracked.

10. The digital stereo microscope apparatus of claim 7, wherein ambient light intensity and direction of the light is controlled by the display control.

11. The digital stereo microscope apparatus of claim 7, wherein the area of interest is remote from the human operator such that hands of the human operator and eyes of the human operator are virtually collocated.

12. The digital stereo microscope apparatus of claim 1, further comprising:

cameras capturing a left eye view and a right eye view of a field of view of the area of interest, wherein the processing system is configured to time synchronize the left eye view and the right eye view for presentation on the image display; and

additional image sensing elements fixed to articulate arms for alternate views within the area of interest.

13. The digital stereo microscope apparatus of claim 12, wherein the processing system is configured to utilize a sum of squared differences algorithm to identify pixel correspondence between the cameras;

wherein the sum of squared differences algorithm selectively processes different regions of the area of interest based on which regions are updated more frequently.

14. The digital stereo microscope apparatus of claim 1, further comprising:

at least one flexible arm adjustable separately from the display unit to the area of interest, wherein the at least one flexible arm comprises at least one component of the image acquisition system.

15. The digital stereo microscope apparatus of claim 1, wherein the processing system comprises:

property data processors that produce a visual representation of a property of objects in the area of interest as a time varying graphical view or as a time varying number display for display on the image display;

control data processors that process commands from the human operator for configuration and orientation of the optical system and the image acquisition system;

storage processors that store and retrieve data in an external storage system; and

display processors that produce a stereo view of the area of interest and user interaction therewith and combine with the visual representations of the property data processors and storage processors to produce an augmented stereo view.

16. The digital stereo microscope apparatus of claim 1, wherein the processing system is configured to present the magnified and augmented live stereo view with sufficient disparity between the left and right eye view images such that the human operator sees a three dimensional representation through stereopsis.

17. A surgical operating system with a digital stereo microscope, comprising:

an articulated arm comprising a display mounting system;

a display unit connected to the display mounting system;

an operating site over which the display unit is fixed and located such that a human operator places hands, tools, or a combination thereof in the operating site and views a magnified and augmented live stereo view of the operating site with eyes of the human operator having a view substantially collocated and in a direct field and line of view with the hands, tools, or a combination thereof of the human operator in the area of interest, wherein the human operator is able perform head movement without losing the live stereo view on the display unit;

an image display located on a front side of the display unit relative to the human providing the magnified and augmented live stereo view;

an optical system and an image acquisition system located on a back side of the display unit providing image capture and data capture of the operating site; and

a processing system communicatively coupled to the image acquisition system and the image display providing the magnified and augmented live stereo view, in real-time, to the image display utilizing stereopsis based on the image capture and the data capture of the operating site from the optical system and the image acquisition system.

18. The surgical operating system of claim 17, wherein the optical system and the image acquisition system comprises at least one objective lens, at least one zoom lens, and at least one image sensor, and wherein the optical system and the image acquisition system provide continuous zoom without an intermediate lens assembly;

wherein the image display comprise a stereoscopic flat display panel that allows head movement of the human operator without losing a stereoscopic view of the area of interest.

19. A method using a digital stereo microscope, comprising:

positioning and adjusting a display unit above an operating site to a fixed position;

enabling the display unit, wherein the display unit comprises an optical system and an image acquisition system located on a back side of the display unit adjacent to the operating site, and wherein the display unit comprises an image display on a front side of the display unit opposite to the operating site;

providing image capture and data capture of the area of interest via the optical system and the image acquisition system;

processing the image capture and the data capture via a processing system;

positioning a user'"'"'s hands, tools, or a combination thereof in the operating site while maintaining the user'"'"'s eyes having a view in a collocated manner and in a direct field and line of view looking at the image display, wherein the front side and the back side are relative to the user; and

presenting a magnified and augmented live stereo view of the operating site, in real-time, via the image display utilizing stereopsis based on the processed image capture and the processed data capture of the area of interest from the optical system and the image acquisition system, wherein the human operator is able perform head movement without losing the live stereo view on the display unit.