System and method for detecting and matching anatomical structures using appearance and shape

• US 7,747,054 B2
• Filed: 03/11/2009
• Issued: 06/29/2010
• Est. Priority Date: 11/19/2003
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Gupta, Alok, Zhou, Xiang, Georgescu, Bogdan, Rao, R. Bharat, Comaniciu, Dorin
• Primary Examiner(s): Strege; John B
• Application Number:
  US12/401,899
• Publication Number:
  US 20090226081A1
• Time in Patent Office: 475 Days
• Field of Search: 382/118, 382/128, 382/159, 382/209, 382/224, 382/282, 382/286
• US Class Current: 382/128
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Abstract

A detection framework that matches anatomical structures using appearance and shape is disclosed. A training set of images are used in which object shapes or structures are annotated in the images. A second training set of images represents negative examples for such shapes and structures, i.e., images containing no such objects or structures. A classification algorithm trained on the training sets is used to detect a structure at its location. The structure is matched to a counterpart in the training set that can provide details about the structure'"'"'s shape and appearance.

Claims

1. A method for detecting and matching anatomical structures in a candidate image to one or more anatomical structures in a training set of images comprising the steps of:

receiving the candidate image;

calculating feature values in the candidate image;

extracting the feature values from the candidate image;

applying a classification function to detect an anatomical structure;

comparing the extracted feature values from the candidate image to feature values of one or more images in the training set of images to identify one or more matching counterpart images in the training set upon detecting the anatomical structure; and

using one or more shapes of anatomical structures in the matching counterpart images from the training set to determine a shape of the anatomical structure in the candidate image;

wherein calculating feature values in the candidate image comprises approximating a mean intensity value for a rectangular feature, wherein the rectangular feature comprises invalid pixels corresponding to an occlusion and the mean intensity value is approximated using only a number of valid pixels in the rectangular feature corresponding to pixels that do not comprise the occlusion.

2. The method of claim 1 wherein the step of comparing the extracted feature values from the candidate image to feature values of one or more images in the training set of images further comprises the steps of:

generating a feature vector associated with the candidate image; and

comparing the feature vector associated with the candidate image to a feature vector associated with the one or more images in the training set of images.

3. The method of claim 2 wherein the feature vector associated with the candidate image comprises a plurality of weak classifier outputs h_i'"'"'s and associated weights a_i'"'"'s.

4. The method of claim 3 wherein the step of comparing extracted feature values from the candidate image to feature values of one or more images in the training set of images further comprises using different distance measures, including using a weighting matrix in the Euclidean distance function.

5. The method of claim 1 wherein the step of applying a classification function further comprises:

scanning a window through the candidate image.

6. The method of claim 1 wherein the step of applying a classification function is performed in a hierarchical manner.

7. The method of claim 1 wherein the classification function uses boosting.

8. The method of claim 1 wherein the classification function uses one of AdaBoosting or RankBoosting.

9. The method of claim 1 wherein the classification function uses a support vector machine.

10. The method of claim 1 wherein the candidate image is an ultrasound image.

11. The method of claim 10 wherein the anatomical structure is a left ventricle.

12. The method of claim 1 wherein the anatomical structure is a face.

13. A method for matching an anatomical structure in an image to one or more similarly shaped anatomical structures in a training set of images comprising the steps of:

receiving an image of a candidate anatomical structure;

calculating feature values of the candidate anatomical structure in the image;

extracting the feature values from the image;

comparing the extracted feature values from the image to feature values associated with similarly shaped anatomical structures in the training set; and

determining the shape of the candidate anatomical structure by using a shape of at least one nearest neighbor from the training set

wherein calculating feature values in the candidate image comprises approximating a mean intensity value for a rectangular feature, wherein the rectangular feature comprises invalid pixels corresponding to an occlusion and the mean intensity value is approximated using only a number of valid pixels in the rectangular feature corresponding to pixels that do not comprise the occlusion.

14. The method of claim 13 wherein the candidate anatomical structure has its contour annotated by a set of control points.

15. The method of claim 13 wherein each of the similarly shaped anatomical structures has its contour annotated by a set of control points.

16. The method of claim 13 wherein the step of comparing the extracted features values from the image to feature values associated with similarly shaped anatomical structures in the training set further comprises the steps of:

generating a feature vector associated with the candidate anatomical structure; and

comparing the feature vector associated with the candidate anatomical structure to a feature vector associated with a similarly shaped anatomical structure.

17. The method of claim 16 wherein the feature vector associated with the candidate anatomical structure comprises a plurality of weak classifier outputs h_i'"'"'s and associated weights a_i'"'"'s.

18. The method of claim 17 wherein the step of comparing the feature values from the image to feature values associated with similarly shaped anatomical structures in the training set further comprises using different distance measures, including using a weighting matrix in the Euclidean distance function.

19. The method of claim 13 wherein the image is an ultrasound image.

20. The method of claim 19 wherein the anatomical structure is a left ventricle.

21. The method of claim 13 wherein the anatomical structure is a face.

22. A system for detecting and matching anatomical structures in a candidate image to one or more anatomical structures in a training set of images comprising:

means for receiving the candidate image;

means for calculating feature values in the candidate image;

means for extracting the feature values from the candidate image;

means for applying a classification function to detect an anatomical structure;

means for comparing the extracted feature values from the candidate image to feature values of one or more images in the training set of images to identify one or more matching counterpart images in the training set; and

means for using a shape of an anatomical structure in a matching counterpart image from the training set to determine a shape of the anatomical structure in the candidate image;

wherein calculating feature values in the candidate image comprises approximating a mean intensity value for a rectangular feature, wherein the rectangular feature comprises invalid pixels corresponding to an occlusion and the mean intensity value is approximated using only a number of valid pixels in the rectangular feature corresponding to pixels that do not comprise the occlusion.

23. The system of claim 22 wherein the means for comparing the extracted feature values from the candidate image to feature values of one or more images in the training set of images further comprises:

means for generating a feature vector associated with the candidate image; and

means for comparing the feature vector associated with the candidate image to a feature vector associated with the one or more images in the training set of images.

24. The system of claim 23 wherein the feature vector associated with the candidate image comprises a plurality of weak classifier outputs h_i'"'"'s and associated weights a_i'"'"'s.

25. The system of claim 22 wherein said means for applying a classification function further comprises:

means for scanning a window through the candidate image.

26. The system of claim 22 wherein the classification function uses boosting.

27. The system of claim 22 wherein the classification function uses one of AdaBoosting or RankBoosting.

28. The system of claim 22 wherein the classification function uses a support vector machine.

29. The system of claim 22 wherein the candidate image is an ultrasound image.

30. The system of claim 29 wherein the anatomical structure is a left ventricle.

31. The system of claim 22 wherein the anatomical structure is a face.