Tree structure based 2D to 3D registration

• US 7,787,683 B2
• Filed: 12/16/2005
• Issued: 08/31/2010
• Est. Priority Date: 12/20/2004
• Status: Active Grant

• Current Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Florin, Charles, Sauer, Frank, Khamene, Ali, Williams, James P.
• Primary Examiner(s): Le; Vu
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US11/303,564
• Publication Number:
  US 20060188139A1
• Time in Patent Office: 1,719 Days
• Field of Search: 382/130
• US Class Current: 382/130
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images
• CPC Class Codes: G06V 40/14 Vascular patterns

Abstract

A system and method for tree structure based two-dimensional to three-dimensional registration are provided for receiving three-dimensional (3D) data indicative of vessels, segmenting the 3D data with a vessel tree, simulating two-dimensional (2D) data responsive to the segmented 3D data to form a simulated 2D image, receiving 2D data indicative of vessels, segmenting the received 2D data with a vessel tree, finding a distance transform of the segmented 2D data to form a 2D distance map image, considering a set of poses as state vectors with the corresponding probability computed using the similarity measure computation, re-sampling the pose vector to find a set of most probable poses and considering them as hypothesized poses, recomputing the projection and re-evaluating the probability of the hypothesized poses and updating the state vector until the optimal pose is found.

Claims

1. A method for tree structure based two-dimensional to three-dimensional registration comprising:

receiving three-dimensional (3D) image data indicative of vessels;

segmenting the 3D image data with a vessel tree to generate a segmented 3D image;

simulating two-dimensional (2D) image from the segmented 3D image data and a hypothesized pose to form a simulated 2D image;

receiving 2D data indicative of vessels;

segmenting the received 2D image data with a vessel tree to generate a segmented 2D image;

extracting a 2D distance map image from a distance transform of the segmented 2D image;

comparing the simulated 2D image with the 2D distance map image to generate a probability of the hypothesized pose;

optimizing the hypothesized pose based on a particle filtering scheme that uses a plurality of poses of the received 2D image data relative to the received 3D image data and the probability; and

registering the received 2D image data with the received 3D image data based on the optimized hypothesized pose,

wherein weights of the particle filtering scheme are based on the distance transform of the segmented 2D data, and

wherein the method is performed by a processor.

2. The method as defined in claim 1, further comprising calibrating an imaging system that generates the simulated 2D image prior to simulating the 2D image.

3. The method as defined in claim 1, wherein the received 3D image data is computed tomographic angiography data or magnetic resonance angiography data and the received 2D image data is contrast enhanced x-ray data.

4. The method as defined in claim 1, wherein the distance transform is applied to a binary image.

5. The method as defined in claim 1, wherein the probability is quantified by a similarity measure.

6. The method as defined in claim 1, wherein the particle filtering is performed on pose parameters comprising a state vector.

7. The method as defined in claim 1, wherein the hypothesized pose includes at least one of a translation or a rotation.

8. The method as defined in claim 1, wherein the received 3D image data comprises pre-procedural data and the received 2D image data comprises intra-procedural data.

9. The method as defined in claim 1, wherein the received 2D image data comprises at least one image.

10. The method as defined in claim 1, wherein the particle filtering is based on a Bayesian framework as an engine for performing a maximization or minimization of a similarity or metric quantifier.

11. The method as defined in claim 1, wherein the distance transform is used to define a similarity or metric measure to quantify a distance of a synthetic projection image that is responsive to the hypothesized pose corresponding to the segmented 2D image.

12. The method as defined in claim 1, wherein the distance transform is used to compute a similarity or metric measure responsive to segmented structures in the simulated 2D image.

13. The method as defined in claim 1, wherein the simulated 2D image is calculated or projected only from segmented structures in the 3D image data at a frame rate greater than that of the projection of the full 3D image data.

14. The method as defined in claim 1, wherein the vessel trees are indicative of a human vessel tree exhibiting vascular network changes and non-rigid deformations.

15. The method as defined in claim 1, wherein segmenting the 3D image data comprises:

segmenting a vascular structure by using particle filters for segmentation of vascular structures with a Kalman-directed prior knowledge; and

extracting a centerline of the segmentation and saving it in a multi-element tree data structure format.

16. The method as defined in claim 1, wherein segmenting the 2D image data comprises:

segmenting the vascular structure from 2D images using particle filters for segmentation of vascular structures with a Kalman-directed prior knowledge; and

extracting a binary mask indicative of the segmentation.

17. The method as defined in claim 1, wherein extracting the 2D distance map image comprises defining a metric that quantifies the closeness of a synthetically projected vessel tree image points given calibration parameters, wherein the calibration parameters are estimated off-line through a calibration process or are available based on geometrical properties of an imaging device.

18. The method as defined in claim 1, wherein the 2D distance map image is computed using a linear time algorithm.

19. The method as defined in claim 1, wherein the comparing comprises:

traversing a multi-element tree array of the segmented 3D image data; and

computing projected points based on the hypothetical pose, where a corresponding location in the 2D distance map is looked up and distance values are accumulated to provide a geometrically correct distance measure that is valid for large discrepancies between the hypothetical pose and the 3D image data.

20. The method as defined in claim 1, wherein the particle filtering is used on registration parameters as an optimizer to maximize or minimize a similarity metric.

21. The method as defined in claim 1, wherein the particle filtering comprises a sequential Monte-Carlo technique that is used to estimate a Bayesian posterior density function with a set of samples, and includes estimating a probability density function (PDF) of pose parameters considered as feature space in the context of a rigid 2D to 3D registration.

22. A method for two-dimensional to three-dimensional registration, the method comprising:

receiving three-dimensional (3D) data indicative of vessels;

segmenting the 3D data with a vessel tree;

simulating two-dimensional (2D) data responsive to the segmented 3D data to form a simulated 2D image by the computer;

receiving 2D data indicative of vessels;

segmenting the received 2D data with a vessel tree;

finding a distance transform of the segmented 2D data to form a 2D distance map image;

using a similarity metric based on the distance map to compare the simulated 2D projection image with the segmented 2D image;

computing a probability based on the similarity metric, and re-sampling poses of the received 2D data relative to the received 3D data based on a particle filtering scheme to keep the most probable poses generating the highest similarity between the simulated 2D image and segmented 2D image; and

finding the pose with the highest similarity for registering the received 2D data to the corresponding projection of the received 3D data, wherein the particle-filtering comprises;

using an initial particle distribution as a prior vector of pose parameters and probability and considering these particles as current state particles;

adding noise to the state particles to dislocate them randomly and computing new weights associated with the modified pose particles, where the weight computation is based on a similarity measure computed from the distance transform of the 2D segmented projection image that is only computed once at the beginning of the operation, the weights indicating the probability of a particle'"'"'s pose parameters being accurate; and

processing the current state and the modified particles into a re-sampling step to normalize the weights, while considering these particles as the new current state particles,

wherein the method is performed by a processor.

23. A system for tree structure based two-dimensional to three-dimensional registration comprising:

a processor;

a segmentation software module for segmenting 2D data and 3D data with respective vessel trees; and

a distance transform computation software module for computing a distance transform of the segmented 2D data to form a 2D distance map image;

a similarity measure computation software module to generate a similarity metric based on the 2D distance map image and a projection of the 3D data; and

an optimization module, based on a particle filtering scheme with pose parameters as a state vector to hypothesize a set of poses, wherein the optimization module is configured to re-sample poses of the pose parameters based on the similarity metric to find an optimal pose that registers the received 2D data relative to the projection of the 3D data,

wherein weights of the particle filtering scheme are based on the distance transform of the segmented 2D data.

24. The system as defined in claim 23, further comprising at least one of an imaging adapter and a communications adapter in signal communication with the processor for receiving 3D image data before receiving 2D image data.

25. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform program steps for tree structure based two-dimensional to three-dimensional registration, the program steps comprising:

receiving three-dimensional (3D) data indicative of vessels;

segmenting the 3D data with a vessel tree;

simulating two-dimensional (2D) data responsive to the segmented 3D data to form a simulated 2D image;

receiving 2D data indicative of vessels;

segmenting the received 2D data with a vessel tree;

finding a distance transform of the segmented 2D data to form a 2D distance map image;

computing a similarity measure based on the simulated 2D image and the 2D distance map image;

optimizing pose parameters based on a particle filtering scheme using the pose parameters as state vectors and by evaluating a probability based the similarity measure, and re-sampling the state vectors to register the simulated 2D image with the segmented 2D data,

wherein weights of the particle filtering scheme are based on the distance transform of the segmented 2D data.

26. The program storage device as defined in claim 25 wherein the optimizing and re-sampling is done until convergence.

27. The program storage device as defined in claim 26, wherein the convergence is defined by setting a threshold on an acceptable amount of variation in the pose parameters computed from a probability density function (PDF) of the state vectors.

28. The program storage device as defined in claim 27, wherein the threshold for convergence of translation parameters is less than one millimeter, and the threshold for convergence of rotation parameters is less than one degree.

29. The program storage device as defined in claim 27, wherein the PDF of the most probable poses indicates an uncertainty associated with the registration process.

30. The program storage device as defined in claim 27, wherein the threshold for convergence of translation parameters is less than five millimeters, and the threshold for convergence of rotation parameters is less than two degrees.