System and method for tracking anatomical structures in three dimensional images

• US 7,555,151 B2
• Filed: 09/01/2005
• Issued: 06/30/2009
• Est. Priority Date: 09/02/2004
• Status: Expired due to Fees

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Zhou, Xiang, Georgescu, Bogdan, Xiao, Jing, Comaniciu, Dorin
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): FUJITA, KATRINA R
• Application Number:
  US11/218,018
• Publication Number:
  US 20060064007A1
• Time in Patent Office: 1,398 Days
• Field of Search: 382/128, 128/922, 600/437
• US Class Current: 382/128
• CPC Class Codes: A61B 8/08 Detecting organic movements or changes, e.g. tumours, cysts, swellings A61B8/02 - A61B8/06 take precedence
• CPC Class Codes: A61B 8/0883 for diagnosis of the heart
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10136 3D ultrasound image
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 2201/031 of internal organs
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

A system and method for defining and tracking a deformable shape of a candidate anatomical structure wall in a three dimensional (3D) image is disclosed. The shape of the candidate anatomical structure is represented by a plurality of labeled 3D landmark points. At least one 3D landmark point of the deformable shape in an image frame is defined. A 3D cuboid is defined around the detected 3D landmark point. For each landmark point associated with the anatomical structure, its location and location uncertainty matrix is estimated in subsequent frames relative to the reference anatomical structures. A shape model is generated to represent dynamics of the deformable shape in subsequent image frames. The shape model includes statistical information from a training data set of 3D images of representative anatomical structures. The shape model is aligned to the deformable shape of the candidate anatomical structure. The shape model is fused with the deformable shape. A current shape of the candidate anatomical structure is estimated.

Claims

1. A method for tracking 3-dimensional (3D) motion of a deformable shape of a candidate anatomical structure across a sequence of three dimensional (3D) images the method comprising the steps of:

specifying a plurality of landmark points on a deformable shape of a candidate anatomical structure in a present 3D image of said structure, wherein the landmark points are distributed evenly on said deformable shape;

defining a 3D cuboid around at least one 3D landmark point;

computing a Jacobian matrix from a temporal gradient between the present image and a reference images;

computing motion parameters from said Jacobian matrix and present and reference image values in said cuboid;

warping the cuboid in the reference image according to the motion parameters, and redefining the warped image to be the reference image;

repeating said steps of computing a temporal gradient and a Jacobian matrix, computing motion parameters, and warping the reference image until convergence; and

using the motion parameters to track 3D motion of the anatomical structure.

2. The method of claim 1 wherein localization uncertainties of the plurality of 3D landmark points are represented by covariance matrices.

3. The method of claim 1, further comprising the steps of:

matching the 3D landmark points of the candidate anatomical structure to corresponding landmark points of a reference anatomical structure.

4. The method of claim 3, further comprising the step of:

transforming an orientation of the candidate anatomical structure to a same orientation of the reference anatomical structure.

5. The method of claim 4 wherein the transformation includes translation of the image of the candidate anatomical structure.

6. The method of claim 4 wherein the transformation includes rotation of the image of the candidate anatomical structure.

7. The method of claim 4 wherein the transformation includes scaling of the image of the candidate anatomical structure.

8. The method of claim 1 wherein the anatomical structure is a left ventricle.

9. The method of claim 8, wherein specifying a plurality of landmark points on said deformable shape comprises:

identifying a center point of the LV in a short axis view slice of the mid-ventricular region;

specifying an intersecting point of the inferior right ventricle (RV) and the inferior ventricular septum (VS) in the short axis view slice;

connecting the center point and the intersecting point with a first line that separates the inferior VS from the inferior left ventricle free wall (LVFW) and the anterior LVFW from the antero-lateral LVFW;

determining second, third and fourth lines by rotating said first line in 45 degrees increments with respect to the LV center point;

dividing the LVFW and VS into eight segments by the first, second third and fourth lines wherein each of the first, second third and fourth lines represents a 3D slice along the short axis view;

extracting a slice between an antero-lateral LVFW and a postero-lateral LVFW as a four chamber view;

specifying an LV apex and first and second bottom corner points of an LV wall;

segmenting the LV wall into an apical part, a mid part, and a basal part between said apex and said first and second bottom points, each LV wall part having substantially equal height;

adjusting a 3D pose of the LV wall;

determining the LV pose along a long-axis view;

for each 3D slice along the short axis view, specifying one point at the apex and ten points evenly distributed along an interior surface of the LV wall and VS; and

averaging the coordinates of the apex points to determine a final apex location.

10. The method of claim 9, wherein adjusting a 3D pose of the LV wall comprises:

defining an orientation line connecting the apex point and a center point between the first and second bottom points; and

defining first, second, third, and fourth segmenting slices along the four chamber view as slices substantially perpendicular to the orientation line with substantially equal distances between the apex and a bottom line connecting the first and second bottom corner points.

11. The method of claim 9, wherein determining the LV pose along a long-axis view comprises:

extracting a slice between an inferior LVFW and an infero-posterior LVFW as a long axis view; and

specifying the LV apex in said long axis view and determining the LV pose along the long-axis view based on the LV apex and the LV center point.

12. The method of claim 1 wherein the image is a 3D ultrasound image.

13. The method of claim 1, wherein computing motion parameters includes using estimated motion parameters from a previous iteration.

14. The method of claim 1, wherein computing motion parameters from said Jacobian matrix and present and reference image values in said cuboid comprises calculating

15. A method for tracking 3-dimensional (3D) motion of a left ventricle (LV) all across a sequence of three dimensional (3D) images of a heart, said method comprising the steps of:

identifying a center point of the LV in a short axis view slice of the mid-ventricular region;

specifying an intersecting point of the inferior right ventricle (RV) and the inferior ventricular septum (VS) in the short axis view slice;

connecting the center point and the intersecting point with a first line that separates the inferior VS from the inferior left ventricle free wall (LVFW) and the anterior LVFW from the antero-lateral LVFW;

determining second, third and fourth lines by rotating said first line in 45 degrees increments with respect to the LV center point;

dividing the LVFW and VS into eight segments by the first, second third and fourth lines, wherein each of the first, second third and fourth lines represents a 3D slice along the short axis view;

extracting a slice between an antero-lateral LVFW and a postero-lateral LVFW as a four chamber view;

specifying an LV apex and first and second bottom corner points of an LV wall;

segmenting the LV wall into an apical part, a mid part, and a basal part between said apex and said first and second bottom points, each LV wall part having substantially equal height;

adjusting a 3D pose of the LV wall;

determining the LV pose along a long-axis view;

for each 3D slice along the sho axis view specifying as landmark points one point a the apex and ten points evenly distributed along an interior surface of the LV wall and VS;

averaging the coordinates of the apex points to determine a final landmark point;

individually tracking the pose of the landmark points across a time sequence of images; and

using trajectories of the landmark points to analyze 3D motion and deformation of the LV wall.

16. The method of claim 15, wherein tracking the pose of a landmark point comprises:

defining a 3D cuboid around at least one 3D landmark point;

computing a Jacobian matrix from a temporal gradient between the present image and a reference images;

computing motion parameters from said Jacobian matrix and present and reference image values in said cuboid;

warping the cuboid in the reference image according to the motion parameters, and redefining the warped image to be the reference image; and

repeating said steps of computing a temporal gradient and a Jacobian matrix, computing motion parameters, and warping the reference image until convergence.

17. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for tracking 3-dimensional (3D) motion of a deformable shape of a candidate anatomical structure across a sequence of three dimensional (3D) images, the method comprising the steps of:

specifying a plurality of landmark points on a deformable shape of a candidate anatomical structure in a present 3D image of said structure, wherein the landmark points are distributed evenly on said deformable shape;

defining a 3D cuboid around at least one 3D landmark point;

computing a Jacobian matrix from a temporal gradient between the present image and a reference images;

computing motion parameters from said Jacobian matrix and present and reference image values in said cuboid;

warping the cuboid in the reference image according to the motion parameters, and redefining the warped image to be the reference image

repeating said steps of computing a temporal gradient and a Jacobian matrix, computing motion parameters, and warping the reference image until convergence; and

using the motion parameters to track 3D motion of the anatomical structure.

18. The computer readable program storage device of claim 17, the method further comprising:

matching the 3D landmark points of the candidate anatomical structure to corresponding landmark points of a reference anatomical structure.

19. The computer readable program storage device of claim 17, the method further comprising:

transforming an orientation of the candidate anatomical structure to a same orientation of the reference anatomical structure.

20. The computer readable program storage device of claim 17, wherein the anatomical structure is a left ventricle.

21. The computer readable program storage device of claim 20, wherein specifying a plurality of landmark points on said deformable shape comprises:

identifying a center point of the LV in a short axis view slice of the mid-ventricular region;

specifying an intersecting point of the inferior right ventricle (RV) and the inferior ventricular septum (VS) in the short axis view slice;

connecting the center point and the intersecting point with a first line that separates the inferior VS from the inferior left ventricle free wall (LVFW) and the anterior LVFW from the antero-lateral LVFW;

determining second, third and fourth lines by rotating said first line in 45 degrees increments with respect to the LV center point;

dividing the LVFW and VS into eight segments by the first, second third and fourth lines, wherein each of the first, second third and fourth lines represents a 3D slice along the short axis view;

extracting a slice between an antero-lateral LVFW and a postero-lateral LVFW as a four chamber view;

specifying an LV apex and first and second bottom corner points of an LV wall;

segmenting the LV wall into an apical part, a mid part, and a basal part between said apex and said first and second bottom points, each LV wall part having substantially equal height;

adjusting a 3D pose of the LV wall;

determining the LV pose along a long-axis view;

for each 3D slice along the short axis view, specifying one point at the apex and ten points evenly distributed along an interior surface of the LV wall and VS; and

averaging the coordinates of the apex points to determine a final apex location.

22. The computer readable program storage device of claim 21, wherein adjusting a 3D pose of the LV wall comprises:

defining an orientation line connecting the apex point and a center point between the first and second bottom points; and

defining first second, third, and fourth segmenting slices along the four chamber view as slices substantially perpendicular to the orientation line with substantially equal distances between the apex and a bottom line connecting the first and second bottom corner points.

23. The computer readable program storage device of claim 21, wherein determining the LV pose along a long-axis view comprises:

extracting a slice between an inferior LVFW and an infero-posterior LVFW as a long axis view; and

specifying the LV apex in said long axis view and determining the LV pose along the long-axis view based on the LV apex and the LV center point.

24. The computer readable program storage device of claim 17, wherein the image is a 3D ultrasound image.

25. The computer readable program storage device of claim 17, wherein computing motion parameters includes using estimated motion parameters from a previous iteration.

26. The computer readable program storage device of claim 17, wherein computing motion parameters from said Jacobian matrix and present and reference image values in said cuboid comprises calculating