Apparatus and method for photogrammetric surgical localization

• US 6,491,702 B2
• Filed: 05/29/2001
• Issued: 12/10/2002
• Est. Priority Date: 04/21/1992
• Status: Expired due to Term

• Current Assignee: Sofamor Danek Holdings, Inc
• Original Assignee: Sofamor Danek Holdings, Inc
• Inventors: McDonald, Paul, Peters, William, Wiker, J. Clayton, Heilbrun, Mark Peter, Koehler, Spencer
• Primary Examiner(s): TRUONG, KEVIN THAO
• Application Number:
  US09/865,626
• Publication Number:
  US 20010039421A1
• Time in Patent Office: 560 Days
• Field of Search: 606/130, 128/653.1, 128/630, 128/303, 128/653.2, 600/414, 600/421, 600/424
• US Class Current: 606/130
• CPC Class Codes: A61B 2017/00199 with a console, e.g. a control panel with a display
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/08 Auxiliary means for directing the radiation beam to a particular spot, e.g. using light beams
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/92 coded with colour
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/33 : using feature-based methods
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 2013/0077 : Colour aspects
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A method and apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region are described. Pairs of two-dimensional images are obtained, preferably by means of two video cameras making images of the workspace along different sightlines which intersect. A fiducial structure is positioned in the workspace for defining a three dimensional coordinate framework, and a calibration image pair is made. The calibration image pair comprises two 2D projections from different locations of the fiducial structure. After the calibration image pair is made, the fiducial structure is removed. A standard projection algorithm is used to reconstruct the 3D framework of the fiducial structure from the calibration image pair. Appropriate image pairs can then be used to locate and track any other feature such as a medical instrument, in the workspace, so long as the cameras remain fixed in their positions relative to the workspace. The computations are desirably performed with a computer workstation including computer graphics capability, image processing capability, and providing a real-time display of the workspace as imaged by the video cameras. Also, the 3D framework of the workspace can be aligned with the 3D framework of any selected volume scan, such as MRI, CT, or PET, so that the instrument can be localized and guided to a chosen feature. No guidance arc or other apparatus need be affixed to the patient to accomplish the tracking and guiding operations.

Claims

1. A method for determining a position of a region of interest of an anatomy, comprising:

providing volume scan data representative of an anatomy;

establishing a medical workspace, which includes the anatomy, using a pair of cameras calibrated to each other;

identifying portions of the anatomy through selective illumination of the anatomy;

computing positions of the identified portions with respect to the medical workspace;

deriving a relationship between the medical workspace and the volume scan data using the computed positions of the identified portions; and

determining a position of a region of interest in the medical workspace using the relationship and the volume scan data.

2. The method of claim 1, wherein the pair of cameras acquire 2D data along differing sightlines, and wherein the method further comprises computing a 3D coordinate framework of the medical workspace utilizing the 2D data.

3. The method of claim 2, further comprising:

projecting light onto a plurality of locations on a surface of the anatomy;

detecting the plurality of locations by the cameras; and

computing the 3D positions of the plurality of locations in the 3D coordinate framework of the medical workspace.

4. The method of claim 3, wherein the projecting includes projecting a light grid onto the surface of the anatomy.

5. The method of claim 3, further comprising:

computing a transform to align the 3D coordinate framework of the medical workspace to a 3D coordinate framework of the volume scan, wherein the computing of the transform includes using the plurality of locations described in the 3D coordinate framework of the medical workspace and the 3D coordinate framework of the volume scan; and

applying the transform to the position of the region of interest, described in the 3D coordinate frame work of the volume scan data, to obtain the 3D position of the region of interest in the medical workspace.

6. The method of claim 3, wherein the projecting includes projecting a laser light to provide a plurality of light spots on the surface of the anatomy.

7. The method of claim 6, further comprising selecting the locations on the surface of the anatomy manually.

8. The method of claim 6, further comprising selecting the locations on the surface of the anatomy automatically by a computing means.

9. The method of claim 1, further comprising:

removably placing a fiducial structure within the medical workspace; and

acquiring data relating to the medical workspace containing the fiducial structure.

10. The method of claim 9, wherein the fiducial structure includes a plurality of markers arranged to define a 3D coordinate reference.

11. The method of claim 1, further comprising:

determining the position of a medical instrument in the medical workspace; and

comparing the position of the medical instrument with the position of the region of interest.

12. The method of claim 11, further comprising displaying a representation of the medical instrument in relation to the region of interest.

13. The method of claim 11 wherein the medical instrument includes a fluoroscope.

14. The method of claim 11 further comprising determining the workspace coordinates of the medical instrument through color recognition.

15. The method of claim 14 further comprising determining the workspace coordinates of the medical instrument based upon the color of markers positioned on the medical instrument.

16. The method of claim 11 further comprising determining the workspace coordinates of the medical instrument based on geometric matching.

17. The method of claim 11 further comprising using pattern recognition data to recognize the medical instrument from a plurality of medical instruments.

18. The method as defined in claim 17 further comprising determining the pattern by detecting an edge of the medical instrument.

19. A method for determining a position of a medical instrument relative to a patient'"'"'s anatomy, comprising:

providing volume scan data representative of an anatomy;

establishing a medical workspace using sensors;

identifying portions of the anatomy through selective illumination of the anatomy;

computing positions of the identified portions with respect to the medical workspace;

deriving a relation between the medical workspace and the volume scan data using the computed positions of the identified portions;

tracking the position of a medical instrument using the sensors; and

aligning the position of a region of interest and the position of medical instrument by using the relation.

20. The method of claim 19, further comprising:

temporarily placing a fiducial structure within the medical workspace;

acquiring data of the medical workspace containing the fiducial structure using the sensors; and

removing the fiducial structure after the medical workspace is established.

21. The method of claim 20, wherein the fiducial structure includes a plurality of markers arranged to define a 3D coordinate reference.

22. The method of claim 19, wherein the sensors include two cameras acquiring 2D data from differing sightlines, and wherein the method further comprises computing a 3D coordinate framework of the medical workspace utilizing the 2D data and a projection algorithm.

23. The method of claim 22, further comprising:

projecting light from an illumination source to a plurality of locations on a surface of the anatomy;

detecting the plurality of locations by the cameras; and

computing the 3D positions of the plurality of locations in the 3D coordinate framework of the medical workspace.

24. The method of claim 23, wherein the illumination source is a light projector which projects a light grid onto the surface of the anatomy.

25. The method of claim 23, wherein the illumination source is a laser which projects a plurality of light spots onto the surface of the anatomy.

26. The method of claim 25, further comprising selecting the plurality of locations manually.

27. The method of claim 25, further comprising selecting the plurality of locations automatically with a computing means.

28. The method of claim 23, further comprising:

computing a transform to correlate the 3D coordinate framework of the medical workspace to a 3D coordinate framework of the volume scan data, wherein the computing of the transform includes using the plurality of locations described in the 3D coordinate framework of the medical workspace and the 3D coordinate framework of the volume scan data;

applying the transform to the position of the region of interest, described in the 3D coordinate framework of the volume scan data, to obtain the 3D position of the region of interest in the medical workspace; and

comparing the position of the region of interest and the position of the medical instrument in the 3D coordinate framework of the medical workspace.

29. The method of claim 19, further comprising displaying a representation of the medical instrument in relation to a representation of the region of interest.

30. The method of claim 19 further comprising determining the workspace coordinates of the medical instrument through color recognition.

31. The method of claim 30 further comprising determining the workspace coordinates of the medical instrument based upon the color of markers positioned on the medical instrument.

32. The method of claim 19 further comprising determining the workspace coordinates of the medical instrument based on geometric matching.

33. The method of claim 19 further comprising using pattern recognition data to recognize the medical instrument from a plurality of medical instruments.

34. The method as defined in claim 33 further comprising determining the pattern by detecting an edge of the medical instrument.

35. A method for determining the position of a medical instrument relative to a patient'"'"'s anatomy within a medical workspace, comprising:

providing volume scan data representative of a patient'"'"'s anatomy, the volume scan data having a first 3D coordinate framework;

arranging a first camera and a second camera along respective first and second sightlines of a medical workspace, wherein the first and second sightlines are arranged at an angle;

acquiring a first pair of images of the medical workspace, wherein each image of the first pair is acquired using the first and second video camera;

calibrating the first and second camera to each other and the medical workspace;

establishing a second 3D coordinate framework of the medical workspace utilizing the first pair of images;

illuminating a plurality of points on the surface of the patient'"'"'s anatomy with a laser, the selection of the plurality of points being one of manual and automatic selection;

acquiring subsequent pairs of images of the medical workspace using the first and second cameras taken along the first and second sightlines, respectively, as used by the first pair of images;

computing the positions of the plurality of points in the second 3D coordinate framework using the subsequent image pairs;

deriving a correspondence between the first 3D coordinate framework and the second 3D coordinate framework;

providing pattern recognition data and medical instrument structure data corresponding to a plurality of different medical instruments;

recognizing a medical instrument from the plurality of different medical instruments, appearing in the subsequent image pairs, using the recognition and medical instrument structure data;

tracking a position of an operative portion of the medical instrument in the second 3D coordinate framework using the subsequent pairs of images;

determining a position of a region of interest in the second 3D coordinate framework, using the correspondence and a position of the region of interest described in the first 3D coordinate framework; and

comparing the positions of the operative portion of the medical instrument and the region of interest in the first 3D coordinate framework.

36. The method of claim 35, further comprising:

providing a fiducial structure within a medical workspace containing the patient'"'"'s anatomy, wherein the fiducial structure includes a plurality of markers arranged to define a 3D coordinate system.

37. The method of claim 35 further comprising determining the workspace coordinates of the medical instrument through color recognition.

38. The method of claim 37 further comprising determining the workspace coordinates of the medical instrument based upon the color of markers positioned on the medical instrument.

39. The method of claim 35 further comprising determining the workspace coordinates of the medical instrument based on geometric matching.

40. The method of claim 35 further comprising using the pattern recognition data to recognize the medical instrument from a plurality of medical instruments.

41. The method as defined in claim 40 further comprising determining the pattern by detecting an edge of the medical instrument.

42. A method for determining the position of a medical instrument relative to a patient'"'"'s anatomy within a medical workspace, comprising:

providing volume scan data representative of a patient'"'"'s anatomy, the volume scan data having a first 3D coordinate framework;

providing a fiducial structure within a medical workspace containing the patient'"'"'s anatomy, wherein the fiducial structure includes a plurality of markers arranged to define a 3D coordinate system;

arranging a first camera and a second camera along respective first and second sightlines of a medical workspace, wherein the first and second sightlines are arranged at an angle;

acquiring a first pair of images of the medical workspace containing the fiducial structure, wherein each image of the first pair is acquired using the first and second cameras;

calibrating the first and second cameras to each other and the medical workspace;

establishing a second 3D coordinate framework of the medical workspace containing the fiducial structure utilizing the first pair of images;

illuminating a plurality of points on the surface of the patient'"'"'s anatomy with a laser, the selection of the plurality of points being one of manual and automatic selection;

acquiring subsequent pairs of images of the medical workspace using the first and second cameras taken along the first and second sightlines, respectively, as used by the first pair of images;

computing the positions of the plurality of points in the second 3D coordinate framework using the subsequent image pairs;

deriving a correspondence between the first 3D coordinate framework and the second 3D coordinate framework;

providing pattern recognition data and medical instrument structure data corresponding to a plurality of different medical instruments;

recognizing a medical instrument from the plurality of different medical instruments, appearing in the subsequent image pairs, using the recognition and medical instrument structure data;

tracking a position of an operative portion of the medical instrument in the second 3D coordinate framework using the subsequent pairs of images;

determining a position of a region of interest in the second 3D coordinate framework, using the correspondence and a position of the region of interest described in the first 3D coordinate framework; and

comparing the positions of the operative portion of the medical instrument and the region of interest in the first 3D coordinate framework.

43. The method of claim 42 further comprising determining the workspace coordinates of the medical instrument through color recognition.

44. The method of claim 43 further comprising determining the workspace coordinates of the medical instrument based upon the color of markers positioned on the medical instrument.

45. The method of claim 42 further comprising determining the workspace coordinates of the medical instrument based on geometric matching.

46. The method of claim 42 comprising using the pattern recognition data to recognize the medical instrument from a plurality of medical instruments.

47. The method as defined in claim 46 further comprising determining the pattern by detecting an edge of the medical instrument.