METHOD AND SYSTEM FOR DETERMINATION OF 3D POSITIONS AND ORIENTATIONS OF SURGICAL OBJECTS FROM 2D X-RAY IMAGES

• US 20110282189A1
• Filed: 05/12/2011
• Published: 11/17/2011
• Est. Priority Date: 05/12/2010
• Status: Abandoned Application

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Graumann, Rainer
• Application Number:
  US13/106,102
• Publication Number:
  US 20110282189A1
• Time in Patent Office: Days
• US Class Current: 600/427
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/487 involving fluoroscopy
• CPC Class Codes: A61B 6/5235 combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• CPC Class Codes: A61B 6/547 involving tracking of position of the device or parts of the device
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/75 involving models

Abstract

In a method, a system and a computer readable storage medium encoded with programming instructions, as well as a calculation module for three-dimensional presentation of at least two separate surgical objects within a medical procedure, access to three-dimensional models for the surgical objects takes place based on an acquired two-dimensional x-ray image with the surgical objects. The accessed three-dimensional models are integrated into the acquired two-dimensional x-ray image in order to be shown as a modified x-ray image. The modified x-ray image includes position information, relative positions and orientations of the surgical objects.

Claims

1. A method for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

with an imaging apparatus, acquiring a two-dimensional x-ray image in which at least two surgical objects are both shown;

from a computerized processor, accessing a three-dimensional model in electronic form respectively for each of said at least two surgical objects;

in said computerized processor, for each three-dimensional model, determining a plurality of two-dimensional projections of the three-dimensional model and selecting the two-dimensional projection, from among said plurality of two-dimensional projections, that is best adapted to the showing of the respective surgical object in the two-dimensional x-ray image;

in said computerized processor, calculating, for each of said at least two surgical objects, a three-dimensional position thereof for the selected best-adapted two-dimensional projection thereof; and

from said computerized processor, displaying the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

2. A method as claimed in claim 1 comprising selecting said surgical objects from the group consisting of surgical tools, surgical instruments, and medical implants.

3. A method as claimed in claim 1 comprising, for each of said surgical objects, selecting the best-adapted two-dimensional projection by an image registration procedure comprising a coordinate transformation, selected from the group consisting of two-dimensional image registration procedures and three-dimensional image registration procedures.

4. A method as claimed in claim 1 wherein said calculated three-dimensional positions of each of said at least two surgical objects are displayed in a modified x-ray image, and comprising selecting said modified x-ray image as a virtual presentation of said at least two surgical objects and a plurality of slice images.

5. A method as claimed in claim 1 comprising, in said display, overlaying at least one item selected from the group consisting of information other than said surgical objects, method data, and three-dimensional representations of items other than said three-dimensional objects.

6. A method as claimed in claim 1 comprising generating said display as an image fusion with an additional image.

7. A method as claimed in claim 1 comprising generating said display to include data for navigation of at least one of said surgical objects in a form allowing navigation-guidance of said medical procedure.

8. A method as claimed in claim 1 comprising generating said display repeatedly or continuously during said medical procedure and updating said display when a change selected from the group consisting of a relative position and a relative orientation is detected between said at least two surgical objects.

9. A method as claimed in claim 8 comprising generating and updating said display in real time.

10. A method as claimed in claim 1 wherein said display of the calculated three-dimensional positions of said at least two surgical objects is said modified x-ray image, and comprising localizing said at least two surgical objects in a coordinate system of the imaging apparatus using said modified x-ray image.

11. A method as claimed in claim 1 comprising acquiring said two-dimensional x-ray image pre-operatively, before beginning said medical procedure.

12. A method as claimed in claim 1 comprising acquiring said two-dimensional x-ray image at respective points in time during said medical procedure, selected from the group consisting of predetermined points in time and points in time respectively triggered by events occurring in said medical procedure.

13. A method as claimed in claim 1 comprising automatically detecting a change in a relative position or angle between said at least two surgical objects and, upon detecting said change, automatically acquiring said two-dimensional x-ray image and implementing said calculations in said computerized processor to generate said display.

14. A method for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

attaching a plurality of x-ray-opaque markers with a known geometry to each of at least two surgical objects;

with an imaging apparatus, acquiring a two-dimensional x-ray image in which at least two surgical objects are both shown;

from a computerized processor, accessing a three-dimensional model in electronic form respectively for each of said at least two surgical objects;

in said computerized processor, for each three-dimensional model, determining a plurality of two-dimensional projections of the three-dimensional model and selecting the two-dimensional projection, from among said plurality of two-dimensional projections, that is best adapted to the showing of the respective surgical object in the two-dimensional x-ray image;

in said computerized processor, calculating, for each of said at least two surgical objects, a three-dimensional position thereof for the selected best-adapted two-dimensional projection thereof; and

from said computerized processor, displaying the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

15. A method for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

attaching a plurality of x-ray-opaque markers with a known geometry to each of at least two surgical objects;

with an imaging apparatus, acquiring a two-dimensional x-ray image in which said at least two surgical objects, with said markers respectively attached thereto, are both shown;

in a computerized processor, calculating, from said markers for each of said at least two surgical objects shown in said two-dimensional x-ray image, three-dimensional positions in space between the at least two surgical objects; and

from said computerized processor, displaying the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

16. A system for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

an imaging apparatus that acquires a two-dimensional x-ray image in which at least two surgical objects are both shown;

a computerized processor configured to access a three-dimensional model in electronic form respectively for each of said at least two surgical objects;

said computerized processor being configured to determine, for each three-dimensional model, a plurality of two-dimensional projections of the three-dimensional model and to select the two-dimensional projection, from among said plurality of two-dimensional projections, that is best adapted to the showing of the respective surgical object in the two-dimensional x-ray image;

said computerized processor being configured to calculate, for each of said at least two surgical objects, a three-dimensional position thereof for the selected best-adapted two-dimensional projection thereof; and

said computerized processor being configured to display the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

17. A system for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

a plurality of x-ray-opaque markers with a known geometry to each of at least two surgical objects;

an imaging apparatus that acquires a two-dimensional x-ray image in which said at least two surgical objects, with said markers respectively attached thereto, are both shown;

a computerized processor configured to calculate, from said markers for each of said at least two surgical objects shown in said two-dimensional x-ray image, three-dimensional positions in space between the at least two surgical objects; and

said computerized processor being configured to display the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

18. A computerized processor for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

a processor input supplied with a two-dimensional x-ray image in which at least two surgical objects are both shown;

a calculation module configured to access a three-dimensional model in electronic form respectively for each of said at least two surgical objects;

said calculation module being configured to determine, for each three-dimensional model, a plurality of two-dimensional projections of the three-dimensional model and to select the two-dimensional projection, from among said plurality of two-dimensional projections, that is best adapted to the showing of the respective surgical object in the two-dimensional x-ray image;

said calculation module being configured to calculate, for each of said at least two surgical objects, a three-dimensional position thereof for the selected best-adapted two-dimensional projection thereof;

a processor output; and

said computation module being configured to produce and emit a data file, at said processor output, in a form that allows displaying the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

19. A computerized processor for three-dimensional presentation of at least two separate surgical objects in an image-assisted medical procedure, comprising:

a processor input supplied with a two-dimensional x-ray image in which at least two surgical objects are both shown with a plurality of x-ray-opaque markers with a known geometry attached to each of at least two surgical objects;

a computation module configured to calculate, from said markers for each of said at least two surgical objects shown in said two-dimensional x-ray image, three-dimensional positions in space between the at least two surgical objects; and

said computation module being configured to produce and emit a data file, at said processor output, in a form that allows displaying the respective calculated three-dimensional positions of each of said at least two surgical objects in at least one of a virtual three-dimensional space and a modified x-ray image.

20. A non-transitory computer-readable storage medium encoded with programming instructions, said storage medium being loadable into a computerized processor and causing said computerized processor to:

receive a two-dimensional x-ray image in which at least two surgical objects are both shown;

access a three-dimensional model in electronic form respectively for each of said at least two surgical objects;

for each three-dimensional model, determine a plurality of two-dimensional projections of the three-dimensional model and select the two-dimensional projection, from among said plurality of two-dimensional projections, that is best adapted to the showing of the respective surgical object in the two-dimensional x-ray image;

calculate, for each of said at least two surgical objects, a three-dimensional position thereof for the selected best-adapted two-dimensional projection thereof; and

cause the respective calculated three-dimensional positions of each of said at least two surgical objects to be displayed in at least one of a virtual three-dimensional space and a modified x-ray image.

21. A non-transitory computer-readable storage medium encoded with programming instructions, said storage medium being loadable into a computerized processor and causing said computerized processor to:

receive a two-dimensional x-ray image in which at least two surgical objects are both shown with a plurality of x-ray-opaque markers with a known geometry attached to each of at least two surgical objects;

calculate, from said markers for each of said at least two surgical objects shown in said two-dimensional x-ray image, three-dimensional positions in space between the at least two surgical objects; and

cause the respective calculated three-dimensional positions of each of said at least two surgical objects to be displayed in at least one of a virtual three-dimensional space and a modified x-ray image.