Video-based surgical targeting system

• US 6,675,032 B2
• Filed: 12/21/2000
• Issued: 01/06/2004
• Est. Priority Date: 10/07/1994
• Status: Expired due to Term

• Current Assignee: Medical Media Systems
• Original Assignee: Medical Media Systems
• Inventors: Chen, David T., McKenna, Michael A., Pieper, Steven D.
• Primary Examiner(s): Jaworski, Francis J.
• Assistant Examiner(s): Qaderi, Runa S.
• Application Number:
  US09/746,992
• Publication Number:
  US 20030032876A1
• Time in Patent Office: 1,111 Days
• Field of Search: 600/407, 348/77, 128/920, 128/922
• US Class Current: 600/407
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G09B 23/28 for medicine
• CPC Class Codes: Y10S 128/92 Computer assisted medical diagnostics
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

A video-based surgical targeting system for targeting anatomical structures within a patient.

Claims

1. A video-based surgical targeting system comprising:

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

a patient-specific 3-D computer model of said anatomical structure of said patient, said patient-specific 3-D computer model being assembled from said plurality of 2-D images contained in said patient-specific database;

marker placement means for (i) inserting virtual planning markers into said 2-D images contained in said patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into said 2-D images contained in said patient-specific database and thereafter incorporated into said patient-specific 3-D computer model, or inserting virtual planning markers into said 3-D computer model;

an image generator for generating a virtual image of the anatomical structure modeled by said patient-specific 3-D computer model;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image.

2. A video-based surgical targeting system according to claim 1 wherein said system further comprises registration means for placing said virtual image in registration with said real image.

3. A video-based surgical targeting system according to claim 2 wherein said registration means include means for manually aligning said virtual image with said real image.

4. A video-based surgical targeting system according to claim 2 wherein said registration means include means for automatically aligning said virtual image with said real image.

5. A video-based surgical tracking system according to claim 4 wherein said registration means are adapted to use said real image to track and update said patient-specific 3-D computer model.

6. A video-based surgical targeting system according to claim 2 wherein said registration means include tracking means for tracking the position of said real-time image generating means.

7. A video-based surgical targeting system according to claim 2 wherein said system further comprises instrument tracking means for tracking the position of a surgical instrument.

8. A video-based surgical targeting system according to claim 2 wherein said system further comprises patient tracking means for tracking the position of said anatomical structure.

9. A video-based surgical targeting system according to claim 2 wherein said registration means comprise a surgical instrument and tracking means for tracking the position of said surgical instrument, and further wherein said registration means comprise means for effecting registration when said surgical instrument engages known fiducial marks located on said anatomical structure.

10. A video-based surgical tracking system according to claim 2 wherein said registration means comprise a surgical instrument and tracking means for tracking the position of said surgical instrument, and further wherein said registration means comprise means for effecting registration when said surgical instrument samples multiple surface points located on said anatomical structure, with said means for effecting registration using a data matching procedure to correlate said sampled points with said patient-specific database.

11. A video-based surgical targeting system according to claim 1 wherein said real-time image generating means comprise a video camera.

12. A video-based surgical targeting system according to claim 1 wherein said real-time image generating means comprise an endoscope.

13. A video-based surgical targeting system according to claim 1 wherein said real-time image generating means comprise an ultrasound device.

14. A video-based surgical targeting system according to claim 1 wherein said patient-specific 3-D computer model is constructed out of one or more polygonal models.

15. A video-based surgical targeting system according to claim 1 wherein said image generator generates said virtual image by a polygonal rendering process.

16. A video-based surgical targeting system comprising:

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

marker placement means for inserting virtual planning markers into said 2-D images contained in said patient-specific database;

an image generator for generating a virtual image of the anatomical structure represented by said plurality of 2-D images contained in said patient-specific database;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image.

17. A video-based surgical targeting system according to claim 16 wherein said image generator generates said virtual image by a volume rendering process.

18. A video-based surgical targeting system comprising:

a patient-specific database comprising a plurality of images of the anatomical structure of a patient;

a patient-specific 3-D computer model of said anatomical structure of said patient, said patient-specific 3-D computer model being assembled from said plurality of images contained in said patient-specific database;

marker placement means for (i) inserting virtual planning markers into said images contained in said patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into said images contained in said patient-specific database and thereafter incorporated into said patient-specific 3-D computer model, or inserting virtual planning markers into said 3-D computer model;

an image generator for generating a virtual image of the anatomical structure modeled by said patient-specific 3-D computer model;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image.

19. A video-based surgical targeting system comprising:

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

marker placement means for inserting virtual planning markers into said 2-D images contained in said patient-specific database;

an image generator for generating a virtual image of the anatomical structure defined by said patient-specific database;

real-time image generating means for generating a real image of the anatomical structure of a patient, said real-time image generating means being adapted so as to dynamically update said patient-specific database with said real image;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image.

20. A method for targeting an anatomical structure, said method comprising the steps of:

(1) providing a video-based surgical targeting system comprising;

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

a patient-specific 3-D computer model of said anatomical structure of said patient, said patient-specific 3-D computer model being assembled from said plurality of 2-D images contained in said patient-specific database;

marker placement means for (i) inserting virtual planning markers into said 2-D images contained in said patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into said 2-D images contained in said patient-specific database and thereafter incorporated into said patient-specific 3-D computer model, or inserting virtual planning markers into said 3-D computer model;

an image generator for generating a virtual image of the anatomical structure modeled by said patient-specific 3-D computer model;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image;

(2) inserting a visual planning marker into a 2-D image contained in said patient-specific database;

(3) generating a virtual image of the anatomical structure defined by said patient-specific 3-D model, and generating a real image of the anatomical structure of a patient;

(4) mixing said virtual image and said real image into an output image, whereby said output image comprises a composite of both images, and displaying said output image on said display means, wherein said virtual image and said real image are mixed so that said visual planning marker is exposed to view so as to be superimposed against said real image.

21. A method according to claim 20 wherein said virtual image and said real image are mixed so as to be in registration with one another.

22. A method according to claim 21 wherein said video-based surgical targeting system comprises means for maintaining said virtual image and said real image in registration with one another even when said real-time image generating means move relative to said anatomical structure.

23. A method for targeting an anatomical structure, said method comprising the steps of:

(1) providing a video-based surgical targeting system comprising;

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

marker placement means for inserting virtual planning markers into said 2-D images contained in said patient-specific database;

an image generator for generating a virtual image of the anatomical structure represented by said plurality of 2-D images contained in said patient-specific database;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image;

(2) inserting a visual planning marker into a 2-D image contained in said patient-specific database;

(3) generating a virtual image of the anatomical structure represented by said plurality of 2-D images contained in said patient-specific database, and generating a real image of the anatomical structure of a patient; and

(4) mixing said virtual image and said real image into an output image, whereby said output image comprises a composite of both images, and displaying said output image on said display means, wherein said virtual image and said real image are mixed so that said visual planning marker is exposed to view so as to be superimposed against said real image.

24. A method for targeting an anatomical structure, said method comprising the steps of:

(1) providing a video-based surgical targeting system comprising;

a patient-specific database comprising a plurality of images of the anatomical structure of a patient;

a patient-specific 3-D computer model of said anatomical structure of said patient, said patient-specific 3-D computer model being assembled from said plurality of images contained in said patient-specific database;

marker placement means for (i) inserting virtual planning markers into said images contained in said patient-specific database, and/or (ii) adjusting the positions of virtual planning markers inserted into said images contained in said patient-specific database and thereafter incorporated into said patient-specific 3-D computer model, or inserting virtual planning markers into said 3-D computer model;

an image generator for generating a virtual image of the anatomical structure modeled by said patient-specific 3-D computer model;

real-time image generating means for generating a real image of the anatomical structure of a patient;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image;

(2) inserting a visual planning marker into an image contained in said patient-specific database;

(3) generating a virtual image of the anatomical structure defined by said patient-specific 3-D model, and generating a real image of the anatomical structure of a patient; and

(4) mixing said virtual image and said real image into an output image, whereby said output image comprises a composite of both images, and displaying said output image on said display means, wherein said virtual image and said real image are mixed so that said visual planning marker is exposed to view so as to be superimposed against said real image.

25. A method for targeting an anatomical structure, said method comprising the steps of:

(1) providing a video-based surgical targeting system comprising;

a patient-specific database comprising a plurality of 2-D images of the anatomical structure of a patient;

marker placement means for inserting virtual planning markers into said 2-D images contained in said database;

an image generator for generating a virtual image of the anatomical structure defined by said patient-specific database;

real-time image generating means for generating a real image of the anatomical structure of a patient, said real-time image generating means being adapted so as to dynamically update said patient-specific database with said real image;

video mixing means for mixing said virtual image and said real image into an output image, whereby said output image may comprise either one of the two images exclusive of the other, or a composite of both images; and

display means for displaying said output image;

(2) inserting a visual planning marker into a 2-D image contained in said patient-specific database;

(3) generating a virtual image of the anatomical structure defined by said patient-specific 3-D model, and generating a real image of the anatomical structure of a patient; and

(4) mixing said virtual image and said real image into an output image, whereby said output image comprises a composite of both images, and displaying said output image on said display means, wherein said virtual image and said real image are mixed so that said visual planning marker is exposed to view so as to be superimposed against said real image.