Fluoroscopic tracking and visualization system

• US 6,484,049 B1
• Filed: 04/28/2000
• Issued: 11/19/2002
• Est. Priority Date: 04/28/2000
• Status: Expired due to Term

• Current Assignee: Stryker European Holdings I, LLC (Stryker Corporation)
• Original Assignee: GE Medical Systems Global Technology Company LLC (GE Healthcare Technologies, Inc.)
• Inventors: Kapur, Tina, Seeley, Teresa, Gregerson, Gene, Lin, Faith
• Primary Examiner(s): Lateef, Marvin M.
• Assistant Examiner(s): Lin, Jeoyuh
• Application Number:
  US09/560,940
• Time in Patent Office: 935 Days
• Field of Search: 600/407, 600/425, 600/421, 600/427, 600/429, 600/431, 600/426, 606/130, 250/362, 250/363.01, 250/363.02, 250/363.03, 250/363.04, 250/363.05, 250/363.07, 250/363.09, 250/368, 250/370.08, 250/370.09, 250/370.1, 250/370.4, 250/550, 378/6, 378/21, 378/42, 378/44, 378/46, 378/62, 378/63, 378/68
• US Class Current: 600/426
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2562/17 Comprising radiolucent components
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/5235 combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• CPC Class Codes: A61B 6/547 involving tracking of position of the device or parts of the device
• CPC Class Codes: A61B 6/583 using calibration phantoms
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for

Abstract

A system employs a tracker and a set of substantially non-shadowing point markers, arranged in a fixed pattern or set in a fluoroscope calibration fixture that is imaged in each shot. The fixture is preferably affixed to the image detector of the fluoroscope, and tracking elements secured with respect to the fixture and at least one of a tool and the patient, provide respective position data irrespective of movement. A marker detection module identifies markers imaged in each shot, and a processor applies the known marker positions to model the projection geometry, e.g., camera axis and focus, for the shot and, together with the tracked tool position, form a corrected tool navigation image. In one embodiment an inverting distortion correction converts the tracked or actual location of the tool and displays the tool on the fluoroscopic image to guide the surgeon in tool navigation. In another aspect of the invention, the fluoroscope takes a series of frames while rotating in a plane about the patient, and the camera models derived from the marker images in each frame are applied to define a common center and coordinate axes in the imaged tissue region to which all of the fluoroscope view may be registered. The processor then filters and back-projects the image data or otherwise forms a volume image data set corresponding to the region of tissue being imaged, and desired fluoro-CT planar images of a the imaged patient volume are constructed from this data set. Planes may then be constructed and displayed without requiring complex tracking and image correlation systems previously needed for operating-room management of MRI, CT or PET study image data. Further, the fluoro-CT images thus constructed may be directly registered to preoperative MRI, CT or PET 3D image data, or may obviate the need for such preoperative imaging. Preferably, the tracker employs electromagnetic tracking elements, as shown for example in U.S. Pat. No. 5,967,980, to generate and/or detect electromagnetic field components unobstructed by the patient and intervening structures, and to determine coordinates directly referenced to the patient, the tool or the camera. The calibration fixture may be implemented with BBs in a radiolucent block of structural foam, and/or may be implemented by microlithographic techniques, in which case magnetic tracking elements may be simultaneously formed in registry with the markers on a sheet that mounts to the camera, is incorporated in a radiographic support table, or otherwise positioned to be imaged in each shot.

Claims

1. A system for surgical imaging and display of the type indicating tool position together with an image of a patient undergoing surgery, and including a display and an image processor for displaying such image in coordination with an image of the tool for visual navigation of the tool during the surgical procedure, the system being configured for use with a fluoroscope that includes an x-ray source and an imaging system, and such that at least one image in the display is derived from the fluoroscope at the time of surgery, wherein the system comprises:

a fixture including a three-dimensional array of markers and adapted to be rigidly affixed to a fluoroscope for automatically imaging the array of markers as a fluoroscope image is acquired with the fluoroscope;

a tracking assembly including a first tracking element associated with the tool and a second tracking element associated with the fixture, the tracking assembly being effective to automatically determine relative position data of the marker array and the tool;

a marker determination unit operative on the fluoroscope image to identify at least a subset of said markers in the image;

a processor operative with said identification to calibrate the system; and

a correction unit responsive to the system calibration and to said position data to determine an image transformation such that a representation of the tool is correctly positioned and oriented in the fluoroscope image.

2. The surgical imaging and display system of claim 1, wherein said fixture includes arrays of markers arranged in two different planes along an imaging direction in substantially non-shadowing patterns, and said correction unit applies the marker position data to determine camera focus and center to register the representation of the tool in the fluoroscope image.

3. The surgical imaging and display system of claim 1, wherein said correction unit determines an image distortion inverse transform operative to convert tracking assembly spatial coordinates of the tool relative to the imaging system, to fluoroscope image coordinates.

4. The surgical imaging and display system of claim 1, wherein said processor determines a camera projection geometry for each of a plurality of fluoroscope images and applies said camera projection geometry to identify a tissue coordinate system for a common region of imaged tissue.

5. The surgical imaging and display system of claim 4, wherein said processor transforms image data of said plurality of fluoroscope images to a 3D fluoro-CT data set representative of said common region of imaged tissue.

6. The surgical imaging and display system of claim 1, wherein said processor determines a camera model that includes radial distortion of said fluoroscope image.

7. The surgical imaging and display system of claim 1, further comprising a table of calibration data representing static camera corrections that vary with camera position or orientation.

8. The surgical imaging and display system of claim 1, wherein the fixture is releasably attachable to an image detector assembly of the fluoroscope camera.

9. The surgical imaging and display system of claim 1, wherein the three dimensional array of markers includes a first array of markers disposed in a first plane, and a second array of markers disposed in a second plane parallel to the first plane, the planes having a defined separation.

10. The surgical imaging and display system of claim 1, wherein the three dimensional array of markers includes a first array of markers disposed in a first plane, and a second array of markers disposed in a second plane, wherein one of said planes is affixed to the camera, and a tracking element is secured against movement with respect to the other of said planes.

11. The surgical imaging and display system of claim 1, wherein said fixture comprises a radiolucent and substantially non-scattering body having a plurality of radio-opaque features at different depths arranged to constitute patterns of markers imaged by the image detector.

12. The surgical imaging and display system of claim 1, wherein one of said tracking elements is an electromagnetic tracking element that includes a lithographically formed sheet assembly comprising marker elements.

13. A system for use with a fluoroscope that includes a source and an image detector, wherein the system comprises:

a fixture including a three dimensional array of point-like fluoroscopically opaque markers disposed over a region in a rigid body, said body being configured for disposition between the source and camera such that the array of markers is automatically imaged as an image is acquired by the fluoroscope;

a tracking assembly including at least a first tracking element associated with the fixture and a second tracking element associated with a tool and configured for automatically determining position data of the fixture with respect to the tool;

a marker determination unit operative on the fluoroscope image to identify at least a subset of said markers in the image and operative with a table of predefined marker positions to determine camera imaging parameters for the fluoroscope image; and

a correction unit responsive to said camera imaging parameters and to said position data to form a corrected tool navigation fluoroscopic image.

14. A system for display of tissue images and probe images during a surgical procedure on a patient, such system comprising

a fixture holding a plurality of radiopaque markers disposed in predefined patterns, said fixture being positionable between a source and an imaging side of a fluoroscope;

a stored table of position of said markers;

an image processor operative on a fluoroscopic image to identify imaged markers in the fluoroscopic image and operative with said stored table to model at least camera image plane piercing point and focus data for said image;

a tracker including a first tracking element securable against movement with respect to said fixture and at least one further tracking element securable against movement with respect to an object of interest such as a tool or a patient, said tracker determining coordinates of said fixture and said object of interest,

wherein said image processor applies said coordinates together with image positions of the identified imaged markers to model the camera data; and

an image display that applies said camera data to display a corrected fluoroscope image for tool guided surgery.

15. The tissue image display system of claim 14, wherein said processor and image display remove images of the markers from the fluoroscope image to display an interpolated unobstructed fluoroscope image.

16. The tissue image display system of claim 14, wherein the stored table is stored in non-volatile memory.

17. The tissue image display system of claim 16, wherein the tracker assembly includes a table of tracker position correction data for a given fluoroscope.

18. The tissue image display system of claim 14, wherein the tracker assembly includes a field generating tracking element and a plurality of field sensing tracking elements, respective ones of the tracking elements being securable against movement with respect to the patient, a tool and the fluoroscope imaging detector such that the system models said camera data and corrects images of the patient and tool irrespective of movement of the patient, the tool and the fluoroscope.

19. A system for use with a fluoroscope camera device for intraoperative tracking during a procedure on a patient, such system being of the type including a display for displaying one or more fluoroscope images together with an indication of probe or tissue status or position, and wherein the system comprises:

a fixed multi-dimensional array of fluoroscopically-imageable markers affixed on or proximate to an imaging assembly of the fluoroscope such that the markers are imaged in each view;

a tracking assembly including at least a first tracking element referenced to a patient and a second tracking element referenced to said fixed array;

a marker localization and identification unit operative on image data to identify in real time the position of markers in each fluoroscopic image and to replace the marker images with smooth image values to thereby create an unobstructed real time patient fluoroscopic view of patient tissue; and

a processor operative on detected image markers for integrating said view with patient position, tool position or preexisting image data to depict a trajectory of the surgical probe in said patient tissue intraoperatively in real time.

20. The system of claim 19, wherein said tracking assembly tracks position of a patient with respect to the marker array, and an image registration unit operates with data from the tracking assembly and the tracked markers to register the dimensionally corrected image to the patient.

21. The system of claim 20, wherein said processor determines a projection geometry camera calibration for each of a plurality of fluoroscope images, and further determines a coordinate system about a region of tissue imaged by the fluoroscope, said coordinate system being common to the tracking assembly and the fluoroscopic view of the said of tissue,

said processor further applying a transformation of image data of said plurality of images to define a fluoro-CT image data set representative of said region of tissue.

22. Apparatus for surgical imaging and probe guidance, such apparatus comprising:

a fluoroscope having a radiation emitting side and an imaging side, said fluoroscope being moveable to provide two-dimensional projection images through the body of a patient placed in its imaging field

a tracking system effective to simultaneously determine relative position of a patient, the fluoroscope and a surgical probe

a calibration fixture securing imageable registration markers positioned such that a plurality of said markers are identifiably imaged in every image taken by the fluoroscope, said calibration fixture being affixed to the fluoroscope so that the fixture is tracked by the tracking system

at least some of said markers lying at different coordinate positions along an imaging axis and being characterized by defined positions with respect to each other such that angle and spacing of images of said markers are effective to characterize camera parameters for relating a volume region of the patient imaged in each fluoroscopic image to camera focus and optical center, and

an image processor that operates on images of said markers to model fluoroscope image projection relative to tracked coordinates for each fluoroscope view such that imaged tool position in successive fluoroscope images taken from different orientations is accurately registered in successive images to track actual movement of the probe with respect to the patient.

23. The surgical imaging system of claim 22, wherein said image processor corrects image coordinates to projected tracked coordinate of a probe, to provide accurate tracking and display of a probe by measurement of probe position and change of display coordinates so as to display probe position on a display image without re-imaging of the tissue or probe.

24. A calibration fixture for use with a fluoroscope, such fixture comprising a radiolucent sheet or body having a major surface and a thickness, said fixture including an array of closely spaced radioopaque markers distributed over a caregion at or below said major surface at defined coordinate positions and being configured to secure in a fixed position in the imaging beam a fluoroscope such that a plurality of the markers are imaged in each pose and images of the markers are effective to define a camera model for the pose,

wherein the marker array is lithographically formed together with a co-registered magnetic element.

25. The calibration fixture of claim 24, wherein the sheet or body is formed of a rigid foam configured to have low x-ray absorption and scattering characteristics.

26. The calibration fixture of claim 25, wherein the array includes first and second sub-arrays of markers disposed at first and second offsets from said major surface.

27. The calibration fixture of claim 24, mounted in a radiographic patient support so as to be imaged together with the patient.

28. The calibration fixture of claim 24, adapted to be releasably affixed to a fluoroscope image detector to image the markers in each image formed by the fluoroscope.

29. The calibration fixture of claim 24, wherein the marker array is lithographically formed together with a co-registered magnetic element.

30. A calibration fixture comprising

a sheet body having

(i) an array of radioopaque markers distributed over a major region of the sheet, and

(ii) a magnetic tracking element attached to the sheet in defined relation to the array of markers.

31. The calibration fixture of claim 30, wherein the magnetic tracking element is at least partly formed on the sheet in registration with a pattern of said markers.