Apparatus and method for photogrammetric surgical localization

• US 5,603,318 A
• Filed: 10/29/1993
• Issued: 02/18/1997
• Est. Priority Date: 04/21/1992
• Status: Expired due to Term

• Current Assignee: Sofamor Danek Holdings, Inc (Medtronic Plc)
• Original Assignee: University of Utah Research Foundation (State of Utah)
• Inventors: Heilbrun, Mark P., McDonald, Paul, Peters, William, Wiker, J. Clayton, Koehler, Spencer
• Primary Examiner(s): Sykes, Angela D.
• Assistant Examiner(s): Gilbert, Samuel
• Application Number:
  US08/145,777
• Time in Patent Office: 1,208 Days
• Field of Search: 128/653.1, 128/653.2, 606/130, 364/413.13
• US Class Current: 600/426
• CPC Class Codes: A61B 2017/00199 with a console, e.g. a control panel with a display
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/08 Auxiliary means for directing the radiation beam to a particular spot, e.g. using light beams
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/92 coded with colour
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/33 : using feature-based methods
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 2013/0077 : Colour aspects
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A method and apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region are described. Pairs of two-dimensional images are obtained, preferably by means of two video cameras making images of the workspace along different sightlines which intersect. A fiducial structure is positioned in the workspace for defining a three dimensional coordinate framework, and a calibration image pair is made. The calibration image pair comprises two 2D projections from different locations of the fiducial structure. After the calibration image pair is made, the fiducial structure is removed. A standard projection algorithm is used to reconstruct the 3D framework of the fiducial structure from the calibration image pair. Appropriate image pairs can then be used to locate and track any other feature such as a medical instrument, in the workspace, so long as the cameras remain fixed in their positions relative to the workspace. The computations are desirably performed with a computer workstation including computer graphics capability, image processing capability, and providing a real-time display of the workspace as imaged by the video cameras. Also, the 3D framework of the workspace can be aligned with the 3D framework of any selected volume scan, such as MRI, CT, or PET, so that the instrument can be localized and guided to a chosen feature. No guidance arc or other apparatus need be affixed to the patient to accomplish the tracking and guiding operations.

Claims

1. Apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory structure having pattern recognition data and instrument structure data Stored therein, said pattern recognition data and said instrument structure data both being correlated to individual ones of a plurality of different medical instruments;

said computing means being operable to;

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial means is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said pair,

use said pattern recognition data to recognize a selected one of said medical instruments when it is visible in both images of one of said pairs, and

compute workspace coordinates of an operative portion of said selected medical instrument.

2. The apparatus of claim 1, wherein said workspace imaging means comprises at least two video cameras.

3. The apparatus of claim 2, wherein said computing means is further constructed to provide a signal which identifies said selected medical instrument, and said apparatus further includes means operably associated with said computing means for communicating said signal to a user.

4. The apparatus of claim 2, wherein said computing means is further operable to receive and correlate scan coordinates derived from a scan coordinate framework for each of three or more selected scan markers with the workspace coordinates of said selected scan markers as derived from one of said image pairs, to compute conversion functions for converting scan coordinates of any selected feature in a scan made in said scan coordinate framework to workspace coordinates in said workspace framework and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to corresponding scan coordinates.

5. The apparatus of claim 4, wherein said computing means is further operable to extrapolate either workspace coordinates or scan coordinates of said operative portion of said instrument from selected instrument structure data and a set of observed instrument coordinates of said selected instrument.

10. A method of localizing a feature of interest in a workspace, comprising the steps of:

providing a fiducial structure as described in claim 1, having known 3D coordinates;

making a calibration image pair of a workspace comprising first and second 2D images of the workspace made along different sightlines, and with the fiducial structure positioned in the workspace such that a minimum number of fiducial points are visible in both images of the calibration image pair;

digitizing the respective 2D images of the calibration image pair;

determining a first plurality of 2D coordinates from the first 2D image of the calibration image pair and a second plurality of 2D coordinates from the second 2D image of the calibration image pair;

computing a 3D workspace coordinate framework from said first and second pluralities of 2D coordinates and the known 3D coordinates of the fiducial points;

making a subsequent image pair comprising two 2D images of the workspace having a medical instrument visible therein, the two 2D images being made along the same respective sightlines as the calibration image pair;

digitizing both the 2D images of the subsequent image pair;

identifying the medical instrument using a database of pattern recognition features corresponding to different instruments in a group of medical instruments; and

determining a set of workspace 3D coordinates of the identified instrument from the 2D images of the subsequent image pair.

11. The method of claim 10, further including the steps of:

providing volume scan data reflective of a volume scan

having a defined volume scan 3D framework;

aligning the volume scan 3D framework with the workspace 3D

framework to produce a function for mapping the scan 3D coordinates of a selected point to workspace 3D coordinates;

selecting a target in the volume scan and determining workspace 3D coordinates for the target using the mapping function; and

comparing the positions of the target feature and the feature of interest in terms of their respective workspace 3D coordinates.

12. The method of claim 11, wherein the feature of interest is a medical instrument having an operative portion, and further including the steps of computing an instrument vector representing a linear direction of the instrument, computing coordinates representative of a present position of the operative portion of the instrument, and extrapolating the instrument vector from the coordinates of the present position of the operative portion for a distance sufficient to determine whether the instrument vector will intersect the target.

13. The method of claim 12, further including a step of providing video imaging means for making the image pairs, said step of providing video imaging means being performed prior to said step of making a calibration image pair, and wherein the calibration and subsequent image pairs are obtained with the video imaging means.

6. Apparatus for defining a location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory structure having pattern recognition data and instrument structure data stored therein said pattern recognition data and said instrument structure both being correlated to each of a plurality of different medical instruments;

said computing means being operable to;

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial structure is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said image pair,

correlate scan coordinates for each of three or more selected scan markers with the workspace coordinates of the same said scan markers as derived from one of said image pairs,

compute conversion functions for converting the scan coordinates of any selected feature in a scan made in a scan coordinate framework to workspace coordinates in said workspace framework, and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to scan coordinates in said scan coordinate framework,

use said pattern recognition data to recognize a selected one of said medical instruments which is visible in both images of one of said pairs, and compute workspace coordinates of a visible portion of said selected medical instrument and an operative portion of said selected medical instrument, said visible portion being visible in both images of said one image pair, and said operative portion not being visible in said one image pair.

7. The apparatus of claim 6, wherein said computing means is further operable to compute an instrument vector for said selected instrument, and to extrapolate a path along said instrument vector in said workspace framework or in said scan framework to determine whether said operative portion of said selected medical instrument will intersect a selected feature in said scan.

8. The apparatus of claim 7, wherein said workspace imaging means comprises at least two video cameras.

9. The apparatus of claim 8, wherein said computing means is further constructed to provide a signal which identifies said selected medical instrument, and said apparatus further includes means operably associated with said computing means for communicating said signal to a user.

14. An apparatus for stereotaxic localization of a medical instrument relative to a structure within a patient'"'"'s body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising a first image made along a first sightline and a second image made along a second sightline,

a set of six fiducial markers associated with said workspace;

said first and second sightlines intersecting at an angle such that said set of six fiducial markers is visible in both said first image and said second image, said six fiducial markers being fixed with respect to one another and arranged to establish a 3 dimensional workspace coordinate framework;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

computing means connected to said digitizing means to receive said image signals therefrom, and including memory means for storing data including a plurality of instrument dimensions, each of said instrument dimensions comprising the distance between a fiducial mark on an instrument and a preselected operative portion of said instrument;

said computing means being operable to;

establish said 3 dimensional workspace framework in three dimension using set of six fiducial markers,

determine workspace coordinates in said workspace framework of an instrument fiducial point visible in both said first and second images of an image pair,

determine the instrument in use from stored instrument structure date,

compute an instrument vector representing a linear direction of the instrument bearing the instrument fiducial point, and

compute coordinates representative of a position of an operative portion of the instrument from said instrument vector and the workspace coordinates determined for said visible instrument fiducial point.

15. The apparatus of claim 14, wherein said computing means is further constructed to extrapolate a path of the instrument vector for a desired distance from said coordinates of the operative instrument portion.

16. The apparatus of claim 15, wherein said computing means is further constructed to:

receive a set of scan coordinates for each of a plurality of landmarks present in a scan made in a scan coordinate framework, and to correlate said scan coordinates with the workspace coordinates of at least three of said landmarks as derived from one of said image pairs, said landmarks being selected from the group of anatomic features, surgical implants, radiologic implants, and fiducial structures adapted to be affixed to said patient;

compute conversion functions for convening scan coordinates of a selected feature in said scan to a set of corresponding workspace coordinates, and for convening workspace coordinates of a visible feature which is visible in both said first and second images of a selected one of said image pairs to a set of corresponding scan coordinates; and

use said conversion functions to compute one or more additional sets of corresponding workspace coordinates for user-selected features observable in said scan, and to compute one or more additional sets of corresponding scan coordinates for user-selected features visible in both said first and second images of pairs.

17. The apparatus of claim 14, wherein said memory means further stores pattern recognition data and said computing means is further constructed to use said pattern recognition data to recognize one of said medical instruments when it is visible in both images of one of said pairs, and to compute said workspace coordinates representative of an operative instrument portion for said recognized instrument.

18. The apparatus of claim 14, wherein said sightline intersection angle is between about 5 degrees and 175 degrees.

19. The apparatus of claim 14, wherein said six fiducial markers are selected from the group consisting of:

fiducial markers on a fiducial structure adapted to be affixed to a patient;

fiducial marks adapted to be inscribed on a patient'"'"'s body;

fiducial markers on a free-standing fiducial structure;

or any combination thereof.

20. A method of stcreotaxic localization of a medical instrument with respect to a structure within a patient'"'"'s body region, comprising the steps of:

providing imaging means positioned for making a plurality of image pairs of a medical workspace having a patient'"'"'s body region disposed therein, wherein each of said image pairs comprises a first 2D image made along a first sightline and a second 2D image made along a second sightline which intersects said first sightline at an angle;

providing fiducial means for establishing a workspace coordinate framework, and including six or more fiducial points which are fixed with respect to one another, spaced and arranged in 3 dimensions, and having known 3D spacings from one another;

making a calibration image pair comprising calibration 2D images, wherein at least six of the fiducial points are visible in first and second images of the calibration image pair;

digitizing the calibration 2D images to produced digitized calibration 2D images;

computing a 3D workspace coordinate framework from the digitized calibration 2D images and the known 3D spacings of the fiducial points;

providing an internal scan of the patient'"'"'s body region to identify one or more internal features;

providing a medical instrument having an operative portion and having known dimensions;

making and digitizing a subsequent image pair of tile workspace having a portion of the medical instrument visible in both images of the subsequent image pair to produce a digitized subsequent image pair;

computing workspace coordinates of the visible portion of tile medical instrument from the digitized subsequent image pair;

computing workspace coordinates of an operative portion of the medical instrument from the workspace coordinates of the visible portion; and

determining a locational relationship between the operative portion of the medical instrument and one or more internal features identified from the scan.

21. The method of claim 20, wherein fiducial markers are selected from the group consisting of:

fiducial markers on a fiducial structure adapted to be affixed to a patient;

fiducial marks adapted to be inscribed on a patient'"'"'s body;

fiducial markers on a free-standing fiducial structure;

or any combination thereof.

22. The method of claim 20, further including a step of computing a conversion function for converting coordinates between the scan coordinate framework and the 3D workspace coordinate framework, said step of computing a conversion function being performed prior to said step of determining a locational relationship.

23. The method of claim 20, further including the steps of:

computing an instrument vector representing a present direction of the instrument; and

extrapolating the instrument vector into a space beyond a present location of the operative portion of the medical instrument.

24. The method of claim 20, wherein said step of making and digitizing a subsequent image pair, and all steps recited thereafter, are repeated in sequence to track the medical instrument for as long as desired by a user.