Apparatus and method for photogrammetric surgical localization

• US 6,165,181 A
• Filed: 10/15/1998
• Issued: 12/26/2000
• Est. Priority Date: 04/21/1992
• Status: Expired due to Fees

• Current Assignee: Sofamor Danek Holdings, Inc (Medtronic Plc)
• Original Assignee: Sofamor Danek Holdings, Inc (Medtronic Plc)
• Inventors: McDonald, Paul, Peters, William, Wiker, J. Clayton, Heilbrun, Mark Peter, Koehler, Spencer
• Primary Examiner(s): Buiz, Michael
• Assistant Examiner(s): TRUONG, KEVIN THAO
• Application Number:
  US09/173,138
• Time in Patent Office: 803 Days
• Field of Search: 606/130, 128/303, 128/630, 128/653.1
• US Class Current: 606/130
• CPC Class Codes: A61B 2017/00199 with a console, e.g. a control panel with a display
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/08 Auxiliary means for directing the radiation beam to a particular spot, e.g. using light beams
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/92 coded with colour
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/33 : using feature-based methods
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 2013/0077 : Colour aspects
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A method and apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region are described. Pairs of two-dimensional images are obtained, preferably by means of two video cameras making images of the workspace along different sightlines which intersect. A fiducial structure is positioned in the workspace for defining a three dimensional coordinate framework, and a calibration image pair is made. The calibration image pair comprises two 2D projections from different locations of the fiducial structure. After the calibration image pair is made, the fiducial structure is removed. A standard projection algorithm is used to reconstruct the 3D framework of the fiducial structure from the calibration image pair. Appropriate image pairs can then be used to locate and track any other feature such as a medical instrument, in the workspace, so long as the cameras remain fixed in their positions relative to the workspace. The computations are desirably performed with a computer workstation including computer graphics capability, image processing capability, and providing a real-time display of the workspace as imaged by the video cameras. Also, the 3D framework of the workspace can be aligned with the 3D framework of any selected volume scan, such as MRI, CT, or PET, so that the instrument can be localized and guided to a chosen feature. No guidance arc or other apparatus need be affixed to the patient to accomplish the tracking and guiding operations.

Claims

1. An apparatus for establishing a workspace coordinate reference framework for features of a medical workspace including a patient'"'"'s body region, the apparatus comprising:

a calibration fiducial structure having fiducial markers supported in a fixed, known spatial relationship relative to each other, said calibration fiducial structure being positionable in the medical workspace;

an imager positioned relative to the medical workspace and capable of producing calibration images of the medical workspace with the calibration fiducial structure positioned therein

a digitizer in communication with the optical imager and capable of receiving calibration information relating to the calibration images and producing digitized calibration image signals,

a computer in communication with the digitizer, the computer being programmed to

receive the calibration image signals,

establish, from the calibration image signals and the known spatial relationship of the fiducial markers, a workspace coordinate framework defining the workspace in three dimensional coordinates; and

determine workspace coordinates of the features of the workspace.

2. The apparatus of claim 1, further including a medical instrument positionable within the medical workspace and wherein the imager is capable of producing subsequent images of the medical workspace with the medical instrument positioned therein and the digitizer is capable of receiving subsequent information relating to subsequent images and producing digitized subsequent image signals.

3. The apparatus of claim 2, wherein the computer is programed to

receive a set of scan coordinates of a scan image in a scan coordinate framework, the scan image having identifiable features,

align the scan coordinate framework with the workspace coordinate framework,

receive the subsequent image signals,

determine workspace coordinates of the medical instrument from the subsequent image signals, and

determine a locational relationship between the medical instrument and the scan images.

4. The apparatus of claim 3, wherein the imager includes a first camera and a second camera having respective first and second sightlines of the medical workspace, the first and second sightlines being arranged at an angle and intersecting in the medical workspace.

5. The apparatus of claim 4, further comprising a display unit in communication with the computer for displaying video images of the medical workspace, wherein the computer supplies display signals to the display unit.

6. The apparatus of claim 1, wherein the imager includes a first camera and a second camera having respective first and second sightlines of the medical workspace, the first and second sightlines being arranged at an angle and intersecting in the medical workspace.

7. The apparatus of claim 6, wherein the first and second cameras include video cameras.

8. The apparatus of claim 1, further comprising a display unit in communication with the computer for displaying images of the medical workspace, wherein the computer supplies display signals to the display unit.

9. An apparatus for establishing a workspace coordinate reference framework for features of a medical workspace including a patient'"'"'s body region, the apparatus comprising:

a calibration fiducial structure having fiducials supported in a fixed, known spatial relationship relative to each other, said calibration fiducial structure being positionable in the medical workspace;

an imager positioned relative to the medical workspace and capable of producing calibration images of the medical workspace with the calibration fiducial structure positioned therein;

a digitizer in communication with the imager and capable of receiving calibration information relating to the calibration images and producing digitized calibration image signals; and

a computer in communication with the digitizer, the computer being configured to

receive the calibration image signals,

establish, from the calibration image signals and the known spatial relationship of the fiducials, a workspace coordinate framework defining the workspace in three dimensional coordinates,

receive a set of scan coordinates of a scan image in a scan coordinate framework, the scan image having identifiable features,

align the scan coordinate framework with the workspace coordinate framework, and

determine workspace coordinates of the identifiable features.

10. The apparatus of claim 9, wherein the imager is capable of producing subsequent images of the medical workspace, the digitizer is capable of receiving subsequent information relating to subsequent images and producing digitized subsequent image signals, and the computer is configured to receive the subsequent image signals and produce subsequent images.

11. The apparatus of claim 10, farther including a medical instrument positionable within the medical workspace and wherein the computer is configured to determine the workspace coordinates of the medical instrument from the subsequent image signals.

12. The apparatus of claim 10, further comprising a display unit in communication with the computer for displaying images of the medical workspace, wherein the computer produces display signal from the subsequent images and supplies display signals to the display unit.

13. The apparatus of claim 9, wherein the fiducial markers are substantially spherically shaped.

14. The apparatus of claim 9, wherein the calibration fiducial structure is configured so as to be separable from the patient'"'"'s body region after the calibration images are produced.

15. The apparatus of claim 9, wherein the imager includes a first camera and a second camera having respective first and second sightlines of the medical workspace, the first and second sightlines being arranged at an angle and intersecting in the medical workspace.

16. The apparatus of claim 15, wherein the first and second cameras include video cameras.

17. The apparatus of claim 15, wherein the calibration images include a calibration image pair produced by the first and second cameras.

18. The apparatus of claim 17, wherein the subsequent images include a subsequent image pair produced by the first and second cameras.

19. An apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region, comprising:

a workspace imager positionable for producing a plurality images of the medical workspace, wherein at least two of said images are made from one of each of two sightlines, said sightlines intersecting at an angle;

a digitizer operably disposed for digitizing each of said images to produce sets of digital image signals, one said set of digital image signals corresponding to each of said images;

a fiducial structure positionable in said workspace having fiducial markers held in fixed spatial relation to one another; and

a computer connected to said digitizer to receive said digital image signals therefrom, said computer being programed to

establish a workspace coordinate framework in three dimensions from said images made when said fiducial structure is positioned within said workspace,

determine workspace coordinates in said workspace coordinate framework of any selected point which can be identified from said images,

receive a set of scan coordinates for selected landmarks present in a scan made in a scan coordinate framework,

correlate said scan coordinates with the workspace coordinates of said landmarks as derived from one of the images, and

convert the scan coordinates of a selected feature in said scan to corresponding workspace coordinates in said workspace framework.

20. The apparatus of claim 19, further comprising an instrument, the instrument having fiducial points held in fixed and known spatial relation to one another.

21. The apparatus of claim 19, wherein the workspace imager includes a first camera and a second camera.

22. The apparatus of claim 21, wherein the first and second cameras include video cameras.

23. The apparatus of claim 19, further comprising a display unit in communication with the computer for displaying images of the medical workspace, wherein the computer provides display signals to the display unit.

24. An apparatus for localizing a medical instrument relative to features of a patient'"'"'s body region, comprising:

a fiducial structure positionable in a medical workspace having fiducial markers located in a known, fixed relation to each other;

a workspace imager positionable for producing at least a pair of images of said medical workspace, said image pair comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

a digitizer operably disposed for digitizing each of said images of said image pair to produce sets of digital image signals, one said set of digital image signals corresponding to each of said images; and

a computer connected to said digitizer to receive said digital image signals therefrom, said computer being operable to

determine a first 2D coordinate from one member of a calibration image pair and a second 2D coordinate from the other member of said calibration image pair, said calibration image pair being an image pair made with said fiducial structure positioned in said medical workspace,

compute a 3D workspace coordinate framework from the known, fixed relation of the fiducial markers in combination with said first and second 2D coordinates,

compute an algorithm for converting any 2D coordinates derived from a subsequent image pair made after said fiducial structure is removed from said medical workspace, to corresponding 3D workspace coordinates,

determine a set of 2D image coordinates reflective of a current position of a feature of interest present in both images of said subsequent image pair, and

convert said set of 2D image coordinates to a set of corresponding workspace 3D coordinates.

25. The apparatus of claim 24, wherein the imager includes a first and second camera.

26. The apparatus of claim 25, wherein the first and second cameras include an optical camera.

27. A method for establishing a workspace reference framework for features of a medical workspace including a patient'"'"'s body region, the method comprising the steps of:

providing a calibration fiducial structure having fiducials supported in a fixed, known spatial relationship relative to each other, an imager positioned relative to the medical workspace, a digitizer in communication with the imager, and a computer in communication with the digitizer;

producing calibration images of the medical workspace with the calibration fiducial structure positioned therein;

producing digitized calibration image signals from the calibration images;

establishing, from the calibration image signals and the known spatial relationship of the fiducials, a workspace coordinate framework defining the workspace in three dimensional coordinates;

determining the workspace coordinates of features of the medical workspace; and

producing subsequent images of the medical workspace.

28. The method of claim 27, further comprising the steps of providing a medical instrument positioned therein and wherein the subsequent images include images of the medical workspace with the medical instrument positioned therein.

29. The method of claim 28, further comprising the steps of:

producing digitized subsequent image signals from the subsequent images; and

determining workspace coordinates of the medical instrument from the subsequent image signals.

30. The method of claim 29 wherein the computer stores scan coordinates of a scan image in a scan coordinate framework, the scan image having identifiable features, and further including the steps of aligning the scan coordinate framework with the workspace coordinate framework and determining a locational relationship between the medical instrument and features identified from the scan image.

31. A method of localizing a feature of interest in a workspace, comprising the steps of:

providing a fiducial structure comprising fiducials and a support connecting said fiducials in a known, fixed relation to each other;

making calibration images of the workspace comprising two 2D images of the fiducial structure positioned in the workspace such that the fiducials are visible in the calibration images;

digitizing the 2D images of the calibration images;

determining a first 2D coordinate from at least one of the calibration images and a second 2D coordinate from at least a second of the calibration images;

computing a 3D workspace coordinate framework from said first and second 2D coordinate and the known, fixed relation of the fiducials;

making subsequent images comprising 2D images of the workspace having a patient'"'"'s body region therein;

digitizing the 2D images of the subsequent images and creating 3D images of the subsequent images; and

determining workspace 3D coordinates of a feature of interest in the workspace from the 3D images of the subsequent images.

32. The method of claim 31, wherein the at least one of the calibration images is made by a first camera and the at least second of the calibration images is made by a second camera, and further comprising the step of positioning the first and second cameras so as to have respective first and second sightlines of the workspace, the first and second sightlines being arranged at an angle and intersecting in the workspace.

33. The method of claim 31, further comprising the steps of displaying video images of the medical workspace.

34. The method of claim 31, wherein a medical instrument is provided with a plurality of instrument fiducials, and further comprising the step of determining workspace coordinates of the medical instrument from coordinates of the instrument fiducials.

35. The method of claim 34, further comprising the step of determining workspace coordinates of an operative portion of the medical instrument from the workspace coordinates of the instrument fiducials and a distance between the operative portion and the instrument fiducials.

36. A method for localization of features of a medical workspace including a patient'"'"'s body region, the method comprising the steps of:

providing a calibration fiducial structure having fiducial markers supported in a fixed, known spatial relationship relative to each other, a plurality of cameras positioned relative to the medical workspace, a digitizer in communication with the plurality of cameras, and a computer in communication with the digitizer and storing a set of scan coordinates of a scan image in a scan coordinate framework, the scan image having identifiable features;

producing at least one calibration image pair of the medical workspace with the calibration fiducial structure positioned therein;

producing digitized calibration image signals from calibration information relating to the calibration image pair;

establishing, from the calibration image signals and the known spatial relationship of the fiducial markers, a workspace coordinate framework defining the workspace in three dimensional coordinates;

producing at least a subsequent image pair of the medical workspace;

producing digitized subsequent image signals from subsequent information relating to subsequent image pairs;

determining workspace coordinates of a feature of the medical workspace from the subsequent image signals;

aligning the scan coordinate framework with the workspace coordinate framework; and

determining a locational relationship between the feature of the medical workspace and features identified from the scan image.

37. The method of claim 36, further comprising positioning a medical instrument in the medical workspace, determining the workspace coordinates of the medical instrument from the subsequent image signal, and determining the locational relationship between the medical instrument and features identified from the scan images.

38. The method of claim 37, wherein the medical instrument is provided with a plurality of instrument fiducial markers, and the step of determining workspace coordinates of the medical instrument includes determining workspace coordinates of the instrument fiducial markers.

39. The method of claim 38, further comprising the step of determining workspace coordinates of an operative portion of the medical instrument from the workspace coordinates of the instrument fiducial markers and a distance between the operative portion and the instrument fiducial markers.

40. The method of claim 36, wherein the fiducials are located on a substantially spherically shaped balls.

41. The method of claim 36, wherein the plurality of cameras include a first camera and a second camera, and further comprising the step of positioning the first and second cameras so as to have respective first and second sightlines of the medical workspace, the first and second sightlines being arranged at an angle and intersecting in the medical workspace.

42. The method of claim 41, wherein the first and second cameras include video cameras.

43. The method of claim 36, further comprising the step of removing the calibration fiducial structure from the medical workspace prior to producing the subsequent image pair.

44. The method of claim 36, further comprising:

providing a display unit in communication with the computer;

supplying display signals from the computer to the display unit; and

displaying video images of the medical workspace.

45. The method of claim 36, wherein the aligning step includes identifying workspace coordinates and scan coordinates of a plurality of points within the scan coordinate framework and the workspace coordinate framework.

46. A method of stereotaxic localization of a medical instrument with respect to a structure within a patient'"'"'s body region, comprising the steps of:

providing an imager positioned for making a plurality of image pairs of a medical workspace having a patient'"'"'s body region disposed therein, wherein each of said image pairs comprises a first 2D image made along a first sightline and a second 2D image made along a second sightline which intersects said first sightline at an angle;

providing a fiducial structure for establishing a workspace coordinate framework, and including fiducials which are fixed with respect to one another, and having known spacings from one another;

making a calibration image pair comprising calibration 2D images, wherein the fiducials are visible in first and second images of the calibration image pair;

digitizing the calibration 2D images to produced digitized calibration 2D images;

computing a 3D workspace coordinate framework from the digitized calibration 2D images and the known spacings of the fiducials;

providing an internal scan of the patient'"'"'s body region to identify one or more internal features;

providing a medical instrument having an operative portion and having known dimensions;

making and digitizing a subsequent image pair of the workspace having a portion of the medical instrument visible in both images of the subsequent image pair to produce a digitized subsequent image pair;

computing workspace coordinates of the visible portion of the medical instrument from the digitized subsequent image pair;

computing workspace coordinates of an operative portion of the medical instrument from the workspace coordinates of the visible portion; and

determining a locational relationship between the operative portion of the medical instrument and one or more internal features identified from the scan.

47. The method of claim 46, wherein the image pairs are made by a first camera and a second camera.

48. The method of claim 46, further comprising the steps of displaying video images of the operative portion of the medical instrument and the one or more internal features identified from the scan in the workspace.