Computer graphic and live video system for enhancing visualization of body structures during surgery
First Claim
1. A real-time surgery apparatus for augmenting a surgeon'"'"'s view of a patient comprising:
- a) a medical imaging system for obtaining three-dimensional (3D) imaging data of internal structures of said patient;
b) a video camera means for providing real-time video images of exposed surfaces of said patient from a camera viewpoint being a location (x,y,z) and orientation angle (α
,φ
,θ
) with respect to the patient, and being substantially the same as a viewpoint of the patient as viewed by said surgeon;
c) a workstation for receiving the 3D imaging data creating a segmented model of internal structures of said patient and creating computer images from selected surfaces of the segmented model as viewed from the same location (x,y,z) and orientation (α
,φ
,θ
) as the video camera means;
d) mixer means for creating a composite image being a desired mixture of the video image of exposed surfaces, and the computer image of the selected surfaces of the internal structures for most of the composite image, with a moveable window being a region of the composite image showing predominantly the video image, or the computer image; and
e) display means for displaying the composite image resulting in an image of semi-transparent exposed surfaces correctly registered with the internal structures, and a view of the patient as viewed by said surgeon to allow said surgeon to locate internal structures relative to visible external surfaces in surgery.
0 Assignments
0 Petitions
Accused Products
Abstract
An interactive surgery planning and display system mixes live video of external surfaces of the patient with interactive computer generated models of internal anatomy obtained from medical diagnostic imaging data of the patient. The computer images and the live video are coordinated and displayed to a surgeon in real-time during surgery allowing the surgeon to view internal and external structures and the relation between them simultaneously, and adjust his surgery accordingly. In an alternative embodiment, a normal anatomical model is also displayed as a guide in reconstructive surgery. Another embodiment employs three-dimensional viewing.
507 Citations
14 Claims
-
1. A real-time surgery apparatus for augmenting a surgeon'"'"'s view of a patient comprising:
-
a) a medical imaging system for obtaining three-dimensional (3D) imaging data of internal structures of said patient; b) a video camera means for providing real-time video images of exposed surfaces of said patient from a camera viewpoint being a location (x,y,z) and orientation angle (α
,φ
,θ
) with respect to the patient, and being substantially the same as a viewpoint of the patient as viewed by said surgeon;c) a workstation for receiving the 3D imaging data creating a segmented model of internal structures of said patient and creating computer images from selected surfaces of the segmented model as viewed from the same location (x,y,z) and orientation (α
,φ
,θ
) as the video camera means;d) mixer means for creating a composite image being a desired mixture of the video image of exposed surfaces, and the computer image of the selected surfaces of the internal structures for most of the composite image, with a moveable window being a region of the composite image showing predominantly the video image, or the computer image; and e) display means for displaying the composite image resulting in an image of semi-transparent exposed surfaces correctly registered with the internal structures, and a view of the patient as viewed by said surgeon to allow said surgeon to locate internal structures relative to visible external surfaces in surgery. - View Dependent Claims (2, 3, 4)
-
-
5. A real-time surgery apparatus for displaying to a surgeon interactive three-dimensional (3D) semi-transparent internal images correctly registered with semi-transparent external images of a patient comprising:
-
a) a medical imaging system for obtaining three-dimensional (3D) imaging data of internal structures of said patient; b) a video camera means for providing stereoscopic right and left real-time images of exposed surfaces of said patient viewed from a right and left location (x1,y1,z1) (x2,y2,z2) and right and left orientation angle (α
1,φ
1,θ
1) (α
2,φ
2,θ
2), respectively, being substantially the locations and orientations of the eyes of said surgeon relative to the patient;c) tracking means for measuring locations (x1,y1,z1) (x2,y2,z2) and orientation angle (α
1,φ
1,θ
1) (α
2,φ
2,θ
2) of the imaging device relative to said patient;d) a first workstation for receiving right location (x1,y1,z1) and orientation angle (α
1,φ
1,θ
1) from the tracking means, and for receiving 3D imaging data from the medical imaging system and creating a right computer generated image of internal structures of said patient as viewed from location (x1,y1,z1) and orientation angle (α
1,φ
1,θ
1);e) a second workstation for receiving left location (x2,y2,z2) and orientation angle (α
2,φ
2,θ
2) from the tracking means, and for receiving 3D imaging data from the medical imaging system and creating a left computer generated image of internal structures of said patient as viewed from locations (x2,y2,z2) and orientation angle (α
2,φ
2,θ
2);f) mixer means for creating a plurality of composite right view images and a plurality of composite left view images, each of the plurality of composite images being a semi-transparent mixture of the real-time video images of exposed surfaces superimposed upon a semi-transparent mixture of the computer generated images of the internal structures, respectively, for most of the composite images, with a moveable window being a region of each composite image showing substantially only the video image, or the computer image; and g) stereoscopic display means for providing composite right view images to said surgeon'"'"'s right eye, and composite left view images to said surgeon'"'"'s left eye thereby simulating a 3D image having the desired degree of visibility of internal structures correctly registered with external structures, and a view of said patient by said surgeon.
-
-
6. A method of augmenting a surgeon'"'"'s view of a patient comprising the steps of:
-
a) acquiring 3D medical imaging data from internal structures of a patient; b) acquiring real-time video images of exposed surfaces of said patient viewed from a location (x,y,z) and orientation angle (α
,φ
,θ
) with respect to said patient, being substantially the same location and orientation angle of the surgeon'"'"'s view;c) creating a computer generated segmented model of the internal structures from the 3D medical imaging data and creating computer images from selected surfaces of the segmented model as viewed from the same location (x,y,z) and orientation angle (α
,φ
,θ
) as the exposed surfaces;d) mixing the real-time video images of the exposed surfaces with the computer images to produce a composite image of a semi-transparent image of the exposed surfaces superimposed upon a semi-transparent image of the internal structures as viewed from the same location and orientation for most of the composite image, and generating a moveable window being a region of the composite image showing substantially only one of the video image and, or the computer image; and e) displaying the composite image to provide desired visibility of exposed structures correctly registered relative to internal structures, and a view of said patient by said surgeon to allow said surgeon to locate internal structures relative to visible external surfaces in surgery. - View Dependent Claims (7, 8, 9, 10)
-
-
11. A method of augmenting a surgeon'"'"'s view of a patient comprising the steps of:
-
a) obtaining three-dimensional (3D) imaging data of internal structures of said patient; b) providing stereoscopic left and right real-time video images of exposed surfaces of said patient viewed from a left and right locations (x1,y1,z1) (x2,y2,z2) and left and right orientation angles (α
1,φ
1,θ
1) (α
2,φ
2,θ
2), respectively relative to the patient;c) measuring the locations (x1,y1,z1) (x2,y2,z2) and orientation angles (α
1,φ
1,θ
1) (α
2,φ
2,θ
2), respectively;d) creating a left computer generated image and a right computer generated image of internal structures from the 3D imaging data of said patient viewed from the same locations (x1,y1,z1) (x2,y2,z2) and orientation angles (α
1,φ
1,θ
1) (α
2,φ
2,θ
2) as the video images of exposed surfaces, respectively;e) mixing a desired portion of the real-time left video image and left computer generated image to result in a left composite image such that a semi-transparent image of internal structures is superimposed upon a semi-transparent image of exposed surfaces both viewed from location (x1,y1,z1) and orientation angle (α
1,φ
1,θ
1), and mixing a desired portion of the real-time right video image and right computer generated image to result in a right composite image such that a semi-transparent image of internal structures is superimposed upon a semi-transparent image of exposed surfaces both viewed from location (x2,y2,z2) and orientation angle (α
2,φ
2,θ
2);
generating a moveable window being a region of each composite image showing substantially only one of the video image and, or the computer image; andf) alternately providing left composite images to said surgeon'"'"'s left eye, and providing the right composite images to said surgeon'"'"'s right eye, thereby simulating a 3D image having the desired degree of visibility of internal structures relative to external structures of said patient to allow said surgeon to locate internal structures relative to visible external surfaces in surgery. - View Dependent Claims (12)
-
-
13. A method of representing relative locations of internal structures of a subject to a physician relative to visible external structures comprising the steps of:
-
a) acquiring video images of external surfaces of the subject from a selected viewpoint as would be visible to the unaided eyes of said physician; b) acquiring a 3D imaging set of imaging data of the internal structures of said subject; c) creating a computer model having identifiable surfaces of internal structures from the 3D image data set; d) rendering the 3D model to produce computer images of the internal structures as viewed from the selected viewpoint; e) selecting a relative degree of visibility of the external surfaces and internal structures, and a region of a composite image desired to be covered by a moveable window; and f) displaying the external surfaces and internal structures to said physician in correct registration with the selected relative degree of visibility for regions of the composite image other than that covered by the moveable window, to produce a relative representation of external surfaces to internal structures, correctly registered with external surfaces as viewed by the surgeon and displaying substantially only one of the video images and the computer images in the region of the moveable window.
-
-
14. A method of planning medical procedures relative to visible external surfaces of a subject having internal structures comprising the steps of:
-
a) acquiring video images of external surfaces of the subject from a selected viewpoint as would be visible to the unaided eyes of said physician; b) acquiring a 3D imaging set of imaging data of the internal structures of said subject; c) creating a computer model having identifiable surfaces of internal structures from the 3D image data set; d) rendering the 3D model to produce a computer image of the internal structures as viewed from the selected viewpoint; e) selecting a relative degree of visibility of the external surfaces and internal structures, and a region of a composite image desired to be covered by a moveable window; f) displaying the external surfaces and internal structures to said physician in correct registration with the selected relative degree of visibility for regions of the composite image other than that covered by the moveable window, to produce a relative representation of external surfaces to internal structures, correctly registered with external surfaces as viewed by the surgeon, displaying substantially only the video images, or the computer images in the region of the moveable window; and g) tracing internal structures to identify an entry point on external surface and a path from the entry point to a desired internal structure, thereby avoiding selected internal structures.
-
Specification