Matching of Regions of Interest Across Multiple Views

• US 20120088981A1
• Filed: 10/06/2011
• Published: 04/12/2012
• Est. Priority Date: 10/07/2010
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Lu, Le, Raykar, Vikas C., Salganicoff, Marcos, Wolf, Matthias, Liu, Meizhu
• Granted Patent: US 8,885,898 B2
• Time in Patent Office: Days
• US Class Current: 600/300
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 8/13 Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30032 Colon polyp
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures

Abstract

Described herein is a framework for multi-view matching of regions of interest in images. According to one aspect, a processor receives first and second digitized images, as well as at least one CAD finding corresponding to a detected region of interest in the first image. The processor determines at least one candidate location in the second image that matches the CAD finding in the first image. The matching is performed based on local appearance features extracted for the CAD finding and the candidate location. In accordance with another aspect, the processor receives digitized training images representative of at least first and second views of one or more regions of interest. Feature selection is performed based on the training images to select a subset of relevant local appearance features to represent instances in the first and second views. A distance metric is then learned based on the subset of local appearance features. The distance metric may be used to perform matching of the regions of interest.

Claims

1. A method of matching images using a computer system, comprising:

receiving, by a processor, first and second digitized images;

receiving, by the processor, at least one CAD finding corresponding to a detected region of interest in the first image; and

automatically determining, by the processor, at least one candidate location in the second image that matches the CAD finding in the first image based on local appearance features extracted for the CAD finding and the candidate location.

2. The method of claim 1 wherein the first and second images are representative of first and second views of one or more candidate abnormalities of a patient.

3. The method of claim 2 wherein the first and second views comprise a prone, supine, lateral or decubitus view.

4. The method of claim 2 wherein the one or more candidate abnormalities comprise one or more polyps, lung nodules, liver tumors, breast cancer lesions, or prostate tumors.

5. The method of claim 2 wherein the first and second views are acquired at different times.

6. The method of claim 1 wherein the automatically determining the candidate location comprises automatically determining multiple candidate locations in the second image that match the CAD finding in the first image.

7. The method of claim 6 wherein the automatically determining the multiple candidate locations comprises:

for each candidate location, computing, based on the local appearance features, a distance from the CAD finding in the first image; and

ranking the multiple candidate locations according to respective computed distances.

8. The method of claim 7 further comprises presenting the ranked candidate locations in a sorted list.

9. The method of claim 1 wherein the automatically determining the candidate location comprises learning a distance metric and using the learned distance metric to compute, based on the local appearance features, a distance between the CAD finding in the first image and the candidate location in the second image.

10. The method of claim 9 wherein the learning the distance metric comprises performing metric boosting to optimally combine multiple weak learners to form a strong learner.

11. The method of claim 9 further comprising performing feature selection to select a subset of relevant local appearance features for learning the distance metric.

12. The method of claim 1 wherein the local appearance features comprise intensity, shape, texture-based, geometrical or contextual features.

13. The method of claim 1 wherein the automatically determining the at least one candidate location in the second image comprises:

computing, based on the local appearance features, a distance between the candidate location in the second image and the CAD finding in the first image using a learned distance metric;

if the computed distance is less than a predetermined distance threshold, identifying the candidate location as a true candidate.

14. The method of claim 13 wherein if the computed distance is more than a predetermined distance threshold, identifying the candidate location as belonging to a class determined by a first classification score.

15. The method of claim 14 further comprising identifying the candidate location as a false candidate if the first classification score is less than a predetermined score threshold.

16. The method of claim 14 further comprising computing the first classification score by combining second and third classification scores associated with the CAD finding and the candidate location respectively.

17. The method of claim 16 wherein the computing the first classification score comprises computing 1−

(1−

ρ

_prone)*(1−

ρ

_supine), wherein ρ

_prone denotes the second classification score and ρ

_supine denotes the third classification score.

18. The method of claim 16 wherein the computing the first classification score comprises multiplying the second and third classification scores.

19. The method of claim 16 wherein the computing the first classification score comprises computing λ

*(ρ

_prone*ρ

_supine)+(1−

λ

)*(1−

(1−

ρ

_prone)*(1−

ρ

_supine), wherein ρ

_prone denotes the second classification score, ρ

_supine denotes the third classification score and λ

denotes a trade-off variable.

20. A method of matching images across multiple views using a computer system, comprising:

receiving, by a processor, digitized training images representative of at least first and second views of one or more regions of interest;

performing, by the processor, feature selection based on the training images to select a subset of relevant local appearance features to represent instances in the first and second views; and

learning, by the processor, a distance metric based on the subset of local appearance features.

21. The method of claim 20 wherein the distance metric comprises a boosted distance metric.

22. The method of claim 21 wherein the boosted distance metric comprises a Mahalanobis distance metric.

23. The method of claim 21 wherein the learning the distance metric comprises performing metric boosting to optimally combine multiple weak learners to form a strong learner.

24. The method of claim 23 wherein the weak learners are constrained by an equivalence relationship.

25. The method of claim 23 wherein the weak learners are constrained by a proximity relationship.

26. The method of claim 23 wherein the weak learners comprise low rank positive semi-definite (PSD) matrices.

27. The method of claim 23 wherein an AdaBoost method is performed to combine the multiple weak learners.

28. The method of claim 20 wherein the performing the feature selection comprises performing a Minimum Redundancy Maximum Relevance (MRMR) method.

29. The method of claim 20 wherein the local appearance features comprise intensity, shape, texture-based, geometrical or contextual features.

30. A system for matching images, comprising:

a memory device for storing non-transitory computer readable program code; and

a processor in communication with the memory device, the processor being operative with the computer readable program code to;

receive first and second digitized images;

receive at least one CAD finding corresponding to a detected region of interest in the first image; and

automatically determine at least one candidate location in the second image that matches the CAD finding in the first image based on local appearance features for the CAD finding and the candidate location.

31. A non-transitory computer readable medium embodying a program of instructions executable by machine to perform steps for matching images, the steps comprising:

receiving first and second digitized images;

receiving at least one CAD finding corresponding to a detected region of interest in the first image; and

automatically determining at least one candidate location in the second image that matches the CAD finding in the first image based on local appearance features for the CAD finding and the candidate location.