Method for 2D/3D registration, computational apparatus, and computer program

• US 9,811,913 B2
• Filed: 05/11/2016
• Issued: 11/07/2017
• Est. Priority Date: 05/13/2015
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Borsdorf, Anja, Wang, Jian
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/151,670
• Publication Number:
  US 20160335777A1
• Time in Patent Office: 545 Days
• Field of Search: 382128, 382131, 382132, 382154
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/487 involving fluoroscopy
• CPC Class Codes: A61B 6/5223 generating planar views from image data, e.g. extracting a coronal view from a 3D image
• CPC Class Codes: A61B 6/5235 combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• CPC Class Codes: A61B 90/00 Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges protective face masks A41D13/11; surgeon's or patient's gowns or dresses A41D13/12; devices for carrying-off, for treatment of, or for carrying-over, body liquids A61M1/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10124 Digitally reconstructed radiograph [DRR]
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A method for registering a three-dimensional image data record of a target region of a patient with a two-dimensional x-ray image is provided. The method includes selecting at least one rigid reference structure with an associated contour; establishing a two-dimensional gradient x-ray image and a three-dimensional gradient data record of the image data record; finding a neighborhood in the gradient x-ray image from a plurality of neighborhoods extending about test points for a plurality of contour points; establishing local two-dimensional displacement information by comparison of the contour points with the associated comparison points; establishing movement parameters of a three-dimensional movement model describing a movement of the target region between recording of the image data record and the x-ray image from the local two-dimensional displacement information; and establishing a registration transformation describing the registration by correcting the test transformation based on the movement parameters.

Claims

1. A method for registering a three-dimensional image data record of a target region of a patient with a two-dimensional x-ray image of the target region recorded in a recording geometry, wherein, after prescribing an initial transformation as a test transformation to be optimized, the method comprises:

selecting at least one rigid reference structure with an associated contour, visible in the x-ray image, from anatomical structures contained in the image data record, which were established during an evaluation;

establishing a two-dimensional gradient x-ray image and a three-dimensional gradient data record of the image data record, wherein at least one two-dimensional gradient comparison image, which is forward projected in the recording geometry of the x-ray image using the test transformation, is established from the gradient data record;

finding a neighborhood in the gradient x-ray image from a plurality of neighborhoods extending about test points for a plurality of contour points in the gradient comparison image corresponding to initial points on the three-dimensional contour of the at least one selected reference structure in the recording geometry of the x-ray image, which neighborhood best corresponds to a local neighborhood of the contour point due to a comparison measure and extends about a comparison point;

establishing local two-dimensional displacement information by comparison of the contour points with the associated comparison points;

establishing movement parameters of a three-dimensional movement model describing a movement of the target region between recording of the image data record and the x-ray image from the local two-dimensional displacement information; and

establishing a registration transformation describing the registration by correcting the test transformation based on the movement parameters.

2. The method of claim 1, wherein the method is repeated iteratively with the established registration transformation being used as a new test transformation until an abort criterion is satisfied.

3. The method of claim 2, wherein the abort criterion comprises dropping below a threshold for a norm of the movement parameters, exceeding a predetermined number of iteration acts, or combination thereof.

4. The method of claim 1, wherein one or both of an approximate transformation established from a known position of the patient in relation to the x-ray apparatus recording the x-ray image or an approximate transformation established from a registration process using forward projected virtual x-ray images is/are used as an initial transformation.

5. The method of claim 4, wherein either or both of the approximate transformations is carried out with a coarser resolution, an approximate transformation having an accuracy of less than 20 mm in respect of translations, an accuracy of less than 10°

in respect of rotations, or a combination thereof.

6. The method of claim 1, wherein a time series of x-ray images is recorded, wherein each x-ray image is registered and the registration transformation established in each case for the previously recorded x-ray image is used as an initial transformation.

7. The method of claim 6, wherein a smaller neighborhood, a smaller number of test points, or both the smaller neighborhood and the smaller number of test points is/are selected in the case of a registration transformation as an initial transformation determined for an x-ray image recorded immediately therebefore than in the case of any other initial transformation.

8. The method of claim 1, wherein the anatomical structures are established by segmentation.

9. The method of claim 8, wherein the anatomical structures are established by an edge detection algorithm, intensity windowing, or a combination thereof.

10. The method of claim 1, wherein the reference structure is selected manually or automatically in a manner dependent on background data related to a medical application.

11. The method of claim 1, wherein the initial points are selected such that the three-dimensional gradient given there is selected to be substantially perpendicular to a beam direction emerging from the recording geometry at this point.

12. The method of claim 11, wherein, for the purposes of selection as an initial point, the angle of the three-dimensional gradient at that point in relation to the beam direction lies in a tolerance interval extending by at most 10°

about the perpendicular.

13. The method of claim 1, wherein initial points that do not appear as a point lying on a contour in the gradient comparison image and/or the two-dimensional gradient of which does not correspond to that of the three-dimensional initial point in terms of direction is/are discarded.

14. The method of claim 1, wherein, in addition to a base test point corresponding to the contour point, test points are selected along the direction of the gradient, projected onto the image plane, at the initial point associated with the contour point.

15. The method of claim 1, wherein comparison measures are determined for a certain number of test points, and

wherein the test point assigned to the comparison measure indicating the greatest correspondence is selected as a comparison point, or the test points are selected by an optimization algorithm working to maximize the comparison measure.

16. The method of claim 15, wherein each test point of the certain number of test points corresponds with a pixel.

17. The method of claim 15, wherein the test points are selected along the direction of the projected three-dimensional gradient at the initial point associated with the contour point.

18. The method of claim 1, wherein the test points or a certain number of test points to be used is/are established in a manner dependent on deviation information describing the degree of the present deviation of the test transformation from reality.

19. The method of claim 18, wherein movement parameters of at least one registration process relating to an x-ray image recorded at an earlier time, prediction information derived therefrom, or both the movement parameters and the prediction information is/are used as deviation information.

20. The method of claim 1, wherein the associated displacement information is discarded if a minimum value for the comparison measure at the comparison point is undershot.

21. The method of claim 1, wherein the gradient correlation is used as a comparison measure.

22. The method of claim 1, wherein a movement model describing rigid movement of all reference structures and/or in relation to the individual reference structures and permitting a rotation and a translation is used as a movement model.

23. The method of claim 1, wherein, taking into account displacement information describing the observable portion of the movement, a target plane is established for each pair of contour point and initial point for establishing the movement parameters, with the initial point displaced in three dimensions, and

wherein the comparison point and the focal point of the beam source in the recording geometry lie in the target plane, whereupon, by way of the scalar products of the normal vectors of the target planes and of a vector describing the initial point displaced in three dimensions by the movement and containing the movement parameters, a system of equations is formed and solved for the movement parameters.

24. The method of claim 23, wherein the focal point of the beam source is selected as an origin of the coordinate system.

25. The method of claim 23, wherein the normal vector of the respective target planes is formed as a cross product of a vector, which is formed as a cross product of the gradient vector of the gradient data record perpendicular to the course of the contour in the three-dimensional image data record at the initial point with the vector describing the three-dimensional position of the initial point in the three-dimensional image data record or as a cross product of the gradient vector of the gradient comparison image at the contour point with the vector describing the three-dimensional position of the contour point, with the vector describing the three-dimensional position of the comparison point.

26. The method of claim 23, wherein the system of equations is established proceeding from setting the scalar product to equal zero.

27. The method of claim 23, wherein a rotational portion of the movement is considered to be linear for the purposes of establishing a linear system of equations.

28. The method of claim 23, wherein the system of equations is solved by applying an iterative, unweighted optimization algorithm using a method of least squares by virtue of the sum of terms of the equations respectively provided with a weighting factor and yielding zero being minimized for the contour points.

29. The method of claim 28, wherein each weighting factor is established based on the comparison measure for the respective comparison point, is established as the comparison measure for the respective comparison point, or both.

30. The method of claim 29, wherein the weighting factors are updated during iteration acts as a product of original weighting factors with a residue confidence.

31. The method of claim 1, wherein the target region is subdivided into portions covering individual reference structures and/or the direction of the central ray of the recording geometry is subdivided into portions covering depth intervals,

wherein dedicated gradient comparison images are established for each portion and only the gradient comparison image assigned to the portion containing the initial point is used during the local comparison.

32. The method of claim 31, wherein depth information known due to the portions is taken into account when establishing the movement parameters, when formulating the movement model, or when establishing the movement parameters and formulating the movement model.

33. An x-ray apparatus comprising:

a radiation source;

an x-ray detector configured to, with the radiation source, generate an x-ray image; and

a control apparatus, wherein the control apparatus is configured to;

select at least one rigid reference structure with an associated contour, visible in the x-ray image, from anatomical structures contained in an image data record, which were established during an evaluation;

establish a two-dimensional gradient x-ray image and a three-dimensional gradient data record of the image data record, wherein at least one two-dimensional gradient comparison image, which is forward projected in a recording geometry of the x-ray image using a test transformation, is established from the gradient data record;

find a neighborhood in the gradient x-ray image from a plurality of neighborhoods extending about test points for a plurality of contour points in the gradient comparison image corresponding to initial points on the three-dimensional contour of the at least one selected reference structure in the recording geometry of the x-ray image, which neighborhood best corresponds to a local neighborhood of the contour point due to a comparison measure and extends about a comparison point;

establish local two-dimensional displacement information by comparison of the contour points with the associated comparison points;

establish movement parameters of a three-dimensional movement model describing a movement of the target region between recording of the image data record and the x-ray image from the local two-dimensional displacement information; and

establish a registration transformation describing the registration by correcting the test transformation based on the movement parameters.

34. A computer program, when executed on a computational apparatus, is configured to:

select at least one rigid reference structure with an associated contour, visible in an x-ray image, from anatomical structures contained in an image data record, which were established during an evaluation;

establish a two-dimensional gradient x-ray image and a three-dimensional gradient data record of the image data record, wherein at least one two-dimensional gradient comparison image, which is forward projected in a recording geometry of the x-ray image using a test transformation, is established from the gradient data record;

find a neighborhood in the gradient x-ray image from a plurality of neighborhoods extending about test points for a plurality of contour points in the gradient comparison image corresponding to initial points on the three-dimensional contour of the at least one selected reference structure in the recording geometry of the x-ray image, which neighborhood best corresponds to a local neighborhood of the contour point due to a comparison measure and extends about a comparison point;

establish local two-dimensional displacement information by comparison of the contour points with the associated comparison points;

establish movement parameters of a three-dimensional movement model describing a movement of the target region between recording of the image data record and the x-ray image from the local two-dimensional displacement information; and

establish a registration transformation describing the registration by correcting the test transformation based on the movement parameters.