Display system for enhancing visualization of body structures during medical procedures

• US 5,526,812 A
• Filed: 10/27/1995
• Issued: 06/18/1996
• Est. Priority Date: 06/21/1993
• Status: Expired due to Fees

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Adams, William J., Dumoulin, Charles L., Darrow, Robert D.
• Primary Examiner(s): Smith, Ruth S.
• Application Number:
  US08/549,320
• Time in Patent Office: 235 Days
• Field of Search: 128/653.1, 128/653.2, 606/130, 348/77, 345/1, 345/7-9, 345/113, 345/114, 364/413.13, 364/413.22
• US Class Current: 600/407
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 for local operation

Abstract

An interactive display system superimposes images of internal structures on a semi-transparent screen through which a surgeon views a patient during a medical procedure. The superimposed image is derived from image data obtained with an imaging system. An invasive device is also tracked and displayed on the semi-transparent screen. A ray extending through the invasive device can also be displayed which shows the intended path of the invasive device. The image is registered with the surgeon'"'"'s view of the patient and displayed in real-time during a medical procedure. This allows the surgeon to view internal and external structures, the relation between them, the proposed path of the invasive device, and adjust the procedure accordingly. A second embodiment employs stereoscopic viewing methods to provide three-dimensional representations of the radiological images superimposed on the semi-transparent screen through which the surgeon views the patient.

Claims

1. A interactive medical apparatus for displaying interactive images of image data of internal structures of a patient at a dynamically changing subject position, to be viewed at an dynamically changing observer viewpoint coinciding with an external view of said patient comprising:

a) imaging means for acquiring image data of the internal structures of the patient

b) a semi-transparent screen for displaying an image of image data provided to it with an adjustable degree of transparency to appear superimposed on structures seen through the screen;

c) a tracking device for repeatedly measuring location and orientation of the semi-transparent screen relative to the dynamically changing subject position, and the dynamically changing observer viewpoint;

d) a mechanical arm for adjustably holding the semi-transparent screen in a selected position between said observer viewpoint and said patient position such that a selected region is visible through the semi-transparent screen; and

e) a workstation means coupled to the tracking device for receiving the location and orientation of the screen, operator viewpoint and patient position, and for creating an image from said imaging data, on the semi-transparent screen of internal structures consistent with the relative location and orientation of the semi-transparent screen the operator viewpoint, and the patient position.

2. The interactive medical apparatus of claim 1 further comprising:

a) an invasive device having a position and orientation which is also measured by the tracking device; and

b) a symbol generation means for superimposing a symbol on the semi-transparent screen indicating the position and orientation of the invasive device.

3. The interactive medical apparatus of claim 2 wherein the symbol generation means further comprises a means for displaying a ray indicating the current orientation of the invasive device and its proposed path into the internal structures of the patient.

4. The interactive medical apparatus of claim 1 wherein tracking device includes means to determine a first and second stereoscopic observer viewpoint and the workstation means creates an image relating to the first stereoscopic observer viewpoint, and further comprising:

a) a second workstation means for creating a second image of said internal structures from the imaging data as viewed from the second stereoscopic observer viewpoint;

b) stereoscopic display means having first and second apertures having adjustable opacity, synchronized with the semi-transparent screen for making the first aperture transparent and the second aperture opaque when the first computer generated image is being displayed, and for making the first aperture opaque and the second aperture transparent when the second computer generated image is being displayed, thereby simulating a 3D image of internal structures superimposed upon and external structures of said patient.

5. The interactive medical apparatus of claim 1 wherein the semi-transparent screen has a surface comprised of a material which may be written upon and erased.

6. A method of aiding an operator at a dynamically changing observer viewpoint to perform a medical procedure upon a patient at a dynamically changing subject position comprising the steps of:

a) acquiring multi-dimensional medical imaging data from internal structures of said patient;

b) positioning a free-standing semi-transparent screen at a selected location and orientation allowing a selected region of said patient to be viewed through the semi-transparent screen;

c) measuring locations (x,y,z) and orientation angles (α

,φ

,θ

) of the semi-transparent screen, said subject position, and said observer viewpoint;

d) creating a computer generated image of the internal structures from the medical imaging data consistent with the measured locations (x,y,z) and orientation angles (α

,φ

,θ

);

e) displaying the computer generated image on the semi-transparent screen with a desired degree of transparency to create the illusion of internal structures superimposed upon said patient to assist the operator in said medical procedure.

7. The method of aiding an operator in a medical procedure of claim 6 wherein the step of acquiring multi-dimensional medical imaging data is performed in real-time by a medical imaging system.

8. The method of claim 6 wherein the step of creating a computer image comprises the step of:

a) creating a pair of computer generated stereo images of the internal structures from the medical imaging data consistent with the measured locations and orientation angles; and

b) the step of displaying the computer generated image comprises the step of;

displaying the pair of computer generated images on the semi-transparent screen with a desired degree of transparency to create a stereographic illusion of internal structures superimposed upon said patient to assist the operator in said medical procedure.

9. The method of aiding an operator in a medical procedure of claim 8 further comprising the step of adjusting the distances and orientation angles of the pair of computer generated stereo images to correspond to the view of the patient by the operator.