System and method for displaying concurrent video and reconstructed surgical views

• US 5,961,456 A
• Filed: 03/28/1996
• Issued: 10/05/1999
• Est. Priority Date: 05/12/1993
• Status: Expired due to Term

• Current Assignee: Franklin Education And Development Co., Houston Stereotactic Concepts, Inc., Sherwood Services AG
• Original Assignee: Franklin Education And Development Co.
• Inventors: Gildenberg, Philip L.
• Primary Examiner(s): Smith, Ruth S.
• Application Number:
  US08/625,572
• Time in Patent Office: 1,286 Days
• Field of Search: 128/653.1, 128/665, 606/130, 600/425, 600/429, 600/476, 600/426, 600/414, 600/417
• US Class Current: 600/429
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/20 Surgical microscopes characterised by non-optical aspects
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A video camera is positioned stereotactically to view a surgical opening in a patient and provide data related to stereotactic space and is stereotactically representative of a current actual view, which data is processed and displayed as a current actual image along with a reference image that is computer generated from graphics image data stored on the patient. The two images appear in a display variously, for ease of comparison and are coordinated by stereotactically tracking the camera to maintain position and orientation. A composite camera structure incorporates a surgical microscope and stereoscopic viewing along with light tracking coordination. Displays included side-by-side and overlapped.

Claims

1. An extracorporeal surgical viewing system for displaying coordinated multiple images relating to a surgical opening in a patient'"'"'s body, said system comprising:

a source of stored scan data representative of a patient'"'"'s body, said stored scan data being provided in stereotactic space established relative to said patient'"'"'s body;

a camera system for providing current video signals representative of a viewed subject;

a stereotactic apparatus including means for mounting said camera system in a position to view into the depths of said surgical opening in a patient'"'"'s body and including means for stereotactically tracking the position of said camera system relative to said stereotactic space established with reference to said patient'"'"'s body, said stereotactic apparatus including means for releasable substantially rigid attachment to said patient'"'"'s body, whereby when said stereotactic apparatus is attached to said patient'"'"'s body and said camera system is mounted on said mounting means, said camera system will be in a known relationship to said stereotactic space used for said stored scan data;

means for providing reference scan data signals from said stored scan data, said reference scan data signals being representative of the patient'"'"'s body at the location in said stereotactic space viewed by said camera system when mounted on said mounting means; and

display apparatus coupled to receive said current video signals from said camera system and said reference scan data signals representative of the patient'"'"'s body at the location viewed by said camera system to provide a coordinated multiple image display.

2. An extracorporeal surgical viewing system in accordance with claim 1 wherein said stereotactic apparatus includes a head ring adapted to be affixed to said patient and wherein said camera system includes a video camera.

3. An extracorporeal surgical viewing system in accordance with claim 1 wherein said means for providing reference scan data signals from said stored scan data, includes a frame buffer.

4. An extracorporeal surgical viewing system for providing actual and reference images in a coordinated display relating to a surgical opening in a patient'"'"'s body, said system comprising:

a source of stored reference image scan data representative of the patient'"'"'s body said reference image scan data being provided in stereotactic space established relative to said patient'"'"'s body;

a camera system, said camera system being independent of said source of stored reference image scan data for providing video signals representative of a view into the surgical opening in the patient'"'"'s body;

a stereotactic apparatus releasably attachable to said patient'"'"'s body, said stereotactic apparatus including means for mounting said camera system in a known position relative to said stereotactic space used for said reference image scan data to provide, when mounted on said stereotactic apparatus, a view of said surgical opening in the patient'"'"'s body and including means for tracking the position of said camera system in said stereotactic space established with reference to the patient'"'"'s body;

computer graphics means coupled to said source of stored reference image scan data and to said means for tracking the position of said camera system for providing reference image display data of the patient'"'"'s body related to the current position of said camera system and representing a reconstructed view corresponding to the view of said camera system of said surgical opening; and

display means for processing said video signals from said camera system and said reference image display data of the patient'"'"'s body related to the current position of said camera system to provide a coordinated display related to said stereotactic space of the view of said camera system and the reconstructed view.

5. An extracorporeal surgical viewing system in accordance with claim 4 wherein said stereotactic apparatus includes a head ring adapted to be affixed to said patient.

6. An extracorporeal surgical viewing system according to claim 4 wherein said camera system includes a microscope.

7. An extracorporeal surgical viewing system according to claim 4, wherein said camera system comprises a pair of cameras to provide video signals representative of a stereoscopic image.

8. An extracorporeal surgical viewing system according to claim 4, wherein said stereotactic apparatus includes light elements affixed to said camera system and sensors for receiving light from said light elements to provide signals indicative of the position and orientation of said camera system to control said computer graphics means.

9. An extracorporeal viewing system according to claim 4, wherein said stereotactic apparatus includes a head frame including a stereotactic arc structure.

10. An extracorporeal viewing system according to claim 4 wherein said camera system includes a camera and wherein said stereotactic apparatus includes a support structure for said camera and physical markers adapted to be placed on said patient for use with said camera to register said camera with respect to the patient thereby tracking the positional relationship of said camera and the patient.

11. An extracorporeal viewing system according to claim 10 wherein said support structure allows movement of said camera and the viewing system further includes a tracking system to provide said computer graphics means with signals indicative of the spatial position of said camera in reference to the patient.

12. An extracorporeal viewing system according to claim 4, wherein said camera system includes a camera and said viewing system further includes a microscope integral with said camera for alternatively viewing said surgical field of the patient'"'"'s body.

13. An extracorporeal viewing system according to claim 12 further including a beam splitter structure, and wherein said camera, said microscope and said beam splitter structure are affixed together to provide similar views of said surgical field to said camera and said microscope.

14. An extracorporeal viewing system according to claim 12 wherein said camera and said microscope are stereo optical.

15. A process for displaying coordinated multiple images relating to a surgical opening in a patient'"'"'s body, including the steps of:

storing scan data representative of the patient'"'"'s body said stored scan data being provided in stereotactic space established relative to said patient'"'"'s body;

mounting a camera system on a support, said support being fixed in a known spatial relationship to said stereotactic space established relative to said patient'"'"'s body;

operating said camera system to provide current video signals representative of said depths of said surgical opening;

tracking the position of said camera system with reference to said stereotactic space established relative to said patient'"'"'s body as said camera system provides current video signals representative of said depths of said surgical opening;

providing reference scan data signals from said stored scan data representative of the patient'"'"'s body at the location viewed by said camera system, said reference scan data signals not being provided by said camera system; and

displaying multiple images of said depths of said surgical opening in accordance with said current video signals and said reference scan data signals, said multiple images being coordinated with respect to said stereotactic space.

16. A process according to claim 15 wherein said step of providing mounting said camera system includes fixing at least a part of said camera system to said patient'"'"'s body.

17. A process according to claim 15 wherein said multiple images are displayed in an overlapping relationship.

18. An extracorporeal surgical viewing system for displaying coordinated multiple images relating to a surgical opening in a patient'"'"'s body, said system compromising:

a stereotactic apparatus releasably and substantially rigidly attachable to said patient'"'"'s body,

a source of stored scan data representative of said patient'"'"'s body, said stored scan data being provided in stereotactic space established relative to said patient'"'"'s body;

a camera system for providing current video signals representative of said surgical opening, said camera system being mounted on said stereotactic apparatus in known dimensional relationship to said stereotactic space, which known dimensional relationship is determinable by said stereotactic apparatus independently of said video signals;

means for providing reference scan data signals from said stored scan data, said reference scan data signals being representative of said patient'"'"'s body at the location in said stereotactic space viewed by said camera system; and

display apparatus coupled to receive said current video signals from said camera system and said reference scan data signals from said source of stored scan data to provide a multiple image display of said reference scan data signals and said current video signals, said multiple image display being coordinated with respect to said stereotactic space.

19. A process for displaying coordinated multiple images related to a surgical opening in a patient'"'"'s body, said process including the steps of:

storing scan data, said scan data being representative of said patient'"'"'s body in stereotactic space established relative to said patient'"'"'s body;

mounting a camera system in a position to view said surgical opening and in a known dimensional relationship to said stereotactic space;

operating said camera system to provide current video signals representative of said surgical opening;

tracking the position of said camera system with reference to said stereotactic space as said camera system provides current video signals representative of said surgical opening said tracking being performed without utilizing said current video signals;

providing reference scan data signals from said stored scan data; and

displaying multiple images of said surgical opening in accordance with said current video signals and said reference scan data signals, said multiple images being coordinated with respect to said stereotactic space.