Apparatus and method for photogrammetric surgical localization

• US 20010039421A1
• Filed: 05/29/2001
• Published: 11/08/2001
• Est. Priority Date: 04/21/1992
• Status: Active Grant

• Current Assignee: Sofamor Danek Holdings, Inc (Medtronic Plc)
• Original Assignee: Sofamor Danek Holdings, Inc (Medtronic Plc)
• Inventors: McDonald, Paul, Peters, William, Wiker, J. Clayton, Koehler, Spencer, Heilbrun, Marl Peter
• Granted Patent: US 6,491,702 B2
• Time in Patent Office: Days
• US Class Current: 606/130
• CPC Class Codes: A61B 2017/00199 with a console, e.g. a control panel with a display
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/08 Auxiliary means for directing the radiation beam to a particular spot, e.g. using light beams
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/92 coded with colour
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/33 : using feature-based methods
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 2013/0077 : Colour aspects
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A method and apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region are described. Pairs of two-dimensional images are obtained, preferably by means of two video cameras making images of the workspace along different sightlines which intersect. A fiducial structure is positioned in the workspace for defining a three dimensional coordinate framework, and a calibration image pair is made. The calibration image pair comprises two 2D projections from different locations of the fiducial structure. After the calibration image pair is made, the fiducial structure is removed. A standard projection algorithm is used to reconstruct the 3D framework of the fiducial structure from the calibration image pair. Appropriate image pairs can then be used to locate and track any other feature such as a medical instrument, in the workspace, so long as the cameras remain fixed in their positions relative to the workspace. The computations are desirably performed with a computer workstation including computer graphics capability, image processing capability, and providing a real-time display of the workspace as imaged by the video cameras. Also, the 3D framework of the workspace can be aligned with the 3D framework of any selected volume scan, such as MRI, CT, or PET, so that the instrument can be localized and guided to a chosen feature. No guidance arc or other apparatus need be affixed to the patient to accomplish the tracking and guiding operations.

Claims

1. Apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said images;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory means for storing pattern recognition data and instrument structure data correlated to each of a plurality of different medical instruments, said computing means being operable to;

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial structure is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said pair,

recognize one of said medical instruments when it is visible in both images of one of said pairs, and

compute workspace coordinates of an operative portion of said recognized instrument.

2. The apparatus of

claim 1

, wherein said workspace imaging means comprises at least two video cameras.

3. The apparatus of

claim 2

, wherein said computing means is further constructed to provide a notification to a user of the identity of said instrument, wherein said notification may'"'"'be a visible or aural indicator.

4. The apparatus of

claim 2

, wherein said computing means is further operable to receive and correlate scan coordinates derived from a scan coordinate framework for each of three or more selected scan markers with the workspace coordinates of said selected scan markers as derived from one of said image pairs, to compute conversion functions for converting scan coordinates of any selected feature in a scan made in said scan coordinate framework to workspace coordinates in said workspace framework and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to corresponding scan coordinates.

5. The apparatus of

claim 4

, wherein said computing means is further operable to extrapolate either workspace coordinates or scan coordinates of said operative portion of said, instrument from said instrument dimensions and a set of observed instrument coordinates of said instrument.

10. A method of localizing a feature of interest in a workspace, comprising the steps of:

providing a fiducial structure as described in

claim 1

;

making a calibration image pair of a workspace comprising two 2D images of the workspace made along different sightlines, and with the fiducial structure positioned in the workspace such that a minimum number of fiducial points are visible in both images of the calibration image pair;

digitizing the respective 2D images of the calibration image pair;

determining a first plurality of 2D coordinates from one member of the calibration image pair and a second plurality of 2D coordinates from the other member of the calibration image pair;

computing a 3D workspace coordinate framework from said first and second pluralities of 2D coordinates and the known 3D coordinates of the fiducial points;

making a subsequent image pair comprising two 2D images of the workspace having a medical instrument visible therein, the two 2D images being made along the same respective sightlines as the calibration image pair;

digitizing the respective 2D images of the subsequent image pair;

identifying the medical instrument using a database of pattern recognition features corresponding to different instruments in a group of medical instruments; and

determining the workspace 3D coordinates of the identified instrument from the respective 2D images of the subsequent image pair.

11. The method of

claim 10

, further including the steps of:

providing volume scan data reflective of a volume scan having a defined volume scan 3D framework;

aligning the volume scan 3D framework with the workspace 3D framework to produce a function for mapping the scan 3D coordinates of a selected point to workspace 3D coordinates;

selecting a target in the volume scan and determining workspace 3D coordinates for the target using the mapping function; and

comparing the positions of the target feature and the feature of interest in terms of their respective workspace 3D coordinates.

12. The method of

claim 11

, wherein the feature of interest is a medical instrument having an operative portion, and further including the steps of computing an instrument vector representing the linear direction of the instrument, computing coordinates representative of the position of the operative portion of the instrument, and extrapolating the workspace vector from the coordinates of the operation portion for a distance sufficient to determine whether the instrument vector will intersect the target.

13. The method of

claim 12

, wherein the calibration and subsequent image pairs are made by video imaging means.

14. The method of

claim 10

, further including a step of providing notification to a user of the identity of the identified instrument.

6. Apparatus for defining the location of a medical instrument relative to features of a medical workspace including a patient'"'"'s body region, comprising:

workspace imaging means positionable for producing a plurality of pairs of images of a medical workspace, each of said image pairs comprising two images made along one of each of a different one of two sightlines, said sightlines intersecting at an angle;

digitizing means operably disposed for digitizing each of said images of said image pairs to produce sets of image signals, one said set of image signals corresponding to each of said members of said image pairs;

fiducial means removably positionable in said workspace for providing a series of fiducial points held in fixed spatial relation to one another; and

computing means connected to said digitizing means to receive said image signals therefrom, and including memory means for storing pattern recognition data and instrument structure data correlated to each of a plurality of different medical instruments, said computing means being operable to;

establish a workspace coordinate framework in three dimensions from one of said pairs of images made when said fiducial structure is positioned within said workspace,

determine workspace coordinates in said workspace framework of any selected point which can be identified from both images of said image pair,

correlate scan coordinates for each of three or more selected scan markers with the workspace coordinates of the same said scan markers as derived from one of said image pairs,

compute conversion functions for converting the scan coordinates of any selected feature in a scan made in a scan coordinate framework to workspace coordinates in said workspace framework, and for converting the workspace coordinates of any selected feature observable in both images of said image pairs to scan coordinates in said scan coordinate framework,

recognize one of said medical instruments when it is visible in both of said images, and

compute workspace coordinates of an operative portion of said recognized instrument.

7. The apparatus of

claim 6

, wherein said computing means is further operable to extrapolate either workspace coordinates or scan coordinates of said operative portion of said instrument from said instrument dimensions and a set of observed instrument coordinates of said instrument.

8. The apparatus of

claim 7

, wherein said workspace imaging means comprises at least two video cameras.

9. The apparatus of

claim 8

, wherein said computing means is further constructed to provide a notification to a user of the identity of said instrument, wherein said notification may be a visible or aural indicator.

15. A fiducial structure for establishing a three-dimensional coordinate framework from a photographic image pair, comprising:

a sufficient number of fiducial indicators arranged to a three-dimensional coordinate system; and

frame means for supporting and connecting said fiducial indicators, said frame means consisting of a frame constructed to hold said fiducial indicators in fixed relation to each other.

16. The fiducial structure of

claim 15

having six fiducial points arranged in two distinct planes.