Computer-assisted diagnosis method and system for automatically determining diagnostic saliency of digital images

• US 6,320,976 B1
• Filed: 04/01/1999
• Issued: 11/20/2001
• Est. Priority Date: 04/01/1999
• Status: Expired due to Term

• Current Assignee: Siemens AG
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Murthy, Sreerama K. V., Qian, Jianzhong, Novak, Carol L.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/283,550
• Time in Patent Office: 964 Days
• Field of Search: 382/128, 382/129, 382/131, 382/132, 382/133, 382/155, 382/159, 382/160, 382/181, 382/190, 382/195, 382/199, 382/203, 382/209, 382/210, 382/211, 382/215, 382/217, 382/218, 382/219, 382/220, 382/224, 382/225, 382/226, 382/227, 382/228, 382/232, 382/240, 382/260
• US Class Current: 382/128
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 6/502 for diagnosis of breast, i.e. mammography
• CPC Class Codes: A61B 6/56 Details of data transmission or power supply, e.g. use of slip rings
• CPC Class Codes: A61B 8/0825 for diagnosis of the breast, e.g. mammography
• CPC Class Codes: A61B 8/565 involving data transmission via a network
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 2201/032 of protuberances, polyps nodules, etc.

Abstract

A computer-assisted diagnosis method and system are provided for automatically determining diagnostic saliency of digital images. The method includes the step of providing filters for evaluating the image. Each filter is designed to identify a specific type of diagnostic finding, and is associated with the following: a virtual window for defining regions in the image at which the filter is applied; a set of training image patches corresponding to typical appearances of the specific type of diagnostic finding; a distance measure between the training image patches and the regions in the image defined by the virtual window; and a feature set corresponding to the distance measure. The filters are applied to the image to compute distances between the regions in the image defined by the virtual window and the training image patches based on the distance measure and the feature set, for each of the plurality of filters. Regions in the image are ranked as corresponding to a particular type of diagnostic finding based the computed distances.

Claims

1. A computer assisted diagnosis system for automatically determining diagnostic saliency of regions in a digital image comprised of a plurality of pixels, the system comprising:

a memory unit;

a plurality of filters stored in said memory unit, each of the plurality of filters designed to identify a specific type of diagnostic finding;

a virtual window for defining regions in the image at which each of the plurality of filters is applied;

a set of training image patches corresponding to typical appearances of the specific type of diagnostic finding;

a distance measure set corresponding to the training image patches and the regions in the image defined by the virtual window;

a feature set corresponding to the distance measure set; and

a processor for applying each of the plurality of filters to the image to compute distances between the regions in the image defined by the virtual window and the training image patches bases on the distance measure set and the feature set, and ranking regions in the image corresponding to a particular type of diagnostic finding based on the computed distances.

2. The system according to claim 1, wherein said processor applies each of the plurality of filters so as to evaluate each of the plurality of pixels.

3. The system according to claim 1, wherein a response of a filter is inversely proportional to a distance between a region in the image defined by the virtual window and a closest training image patch in the set of training image patches.

4. The system according to claim 1, wherein said processor applies some of the plurality of filters to a multi-resolution image pyramid constructed from the image, and aggregates results between pyramid levels.

5. The system according to claim 1, wherein said processor normalizes outputs of each of the plurality of filters.

6. The system according to claim 1, further comprising an adjusting device for one of uniformly and independently adjusting a sensitivity of each of the plurality of filters.

7. The system according to claim 1, wherein said processor identifies and ranks foci of attention belonging to a particular type of diagnostic finding using individual filter responses.

8. The system according to claim 1, wherein said processor combines outputs of the plurality of filters into a single diagnostic saliency map for the image.

9. The system according to claim 8, wherein said processor combines the outputs of the plurality of filters using winner-take-all operators between the regions in the image defined by the virtual windows, and one of aggregation operators and winner-take-all operators between different filters.

10. The system according to claim 8, wherein the diagnostic map comprises a numeric diagnostic saliency score for each of the regions in the image defined by the virtual windows associated with the plurality of filters, the specific type of diagnostic finding at each of the regions and characteristics of the diagnostic finding.

11. The system according to claim 8, wherein said processor identifies and ranks diagnostic foci of attention in the image at a given level of sensitivity, using the diagnostic saliency map.

12. The system according to claim 1, wherein said processor generates a numeric diagnostic saliency score for the regions in the image defined by the virtual windows associated with the plurality of filters.

13. The system according to claim 1, wherein said processor pre-processes each of the training image patches in each set of training image patches to optimize a time for distance computation with a region in the image defined by the virtual window.

14. The system according to claim 1, wherein said processor removes salt-and-pepper type of noise from outputs of the filters.

15. The system according to claim 1, wherein said processor identifies an individual training image patch based on an output of the individual training image patch.

16. The system according to claim 1, wherein said processor integrates domain knowledge rules into one of normalizing outputs of the plurality of filters and generating a diagnostic saliency map for the image.

17. The system according to claim 1, wherein said distance measure set is specific to the specific type of diagnostic finding identified by said plurality of filters.

18. The system according to claim 1, wherein said feature set is specific to the specific type of diagnostic finding identified by the plurality of filters.

19. The system according to claim 1, wherein at least some of said plurality of filters are specific to at least one predetermined resolution of the digital image.

20. The system according to claim 1, wherein each of said plurality of filters is disease specific.

21. The system according to claim 1, wherein the digital image is a digital mammogram and the specific type of diagnostic finding corresponds to breast cancer.

22. A computer assisted diagnosis method for automatically determining diagnostic saliency of locations in a digital image comprised of a plurality of pixels, the method comprising the steps of:

providing a plurality of filters for evaluating the image, wherein each of the plurality of filters is designed to identify a specific type of diagnostic finding;

providing a virtual window for defining regions in the image at which each of the plurality of filters is applied;

providing a set of training image patches corresponding to typical appearances of the specific type of diagnostic finding;

providing a distance measure set corresponding to the training image patches and the regions in the image defined by the virtual window;

providing a feature set corresponding to the distance measure set;

applying the plurality of filters to the image to compute distances between the regions in the image defined by the virtual window and the training image patches based on the distance measure set and the feature set, for each of the plurality of filters; and

ranking regions in the image as corresponding to a particular type of diagnostic finding based the computed distances.

23. The method according to claim 22, wherein said applying step applies each of the plurality of filters so as to evaluate each of the plurality of pixels.

24. The method according to claim 22, wherein a response of a filter is inversely proportional to a distance between a region in the image defined by the virtual window and a closest training image patch in the set of training image patches.

25. The method according to claim 22, further comprising the step of pre-processing each of the training image patches in each set of training image patches to optimize a time for distance computation with a region in the image defined by the virtual window.

26. The method according to claim 22, further comprising the steps of:

constructing a multi-resolution image pyramid from the image;

applying some of the plurality of filters to the multi-resolution image pyramid; and

aggregating results between pyramid levels for each filter.

27. The method according to claim 22, further comprising the step of normalizing outputs of each of the plurality of filters.

28. The method according to claim 22, further comprising the step of adjusting sensitivities of the plurality of filters to be one of uniform and independent.

29. The method according to claim 22, further comprising the steps of identifying and ranking foci of attention belonging to a particular type of diagnostic finding using individual filter responses.

30. The method according to claim 22, further comprising the step of combining outputs of the plurality of filters into a single diagnostic saliency map for the image.

31. The method according to claim 22, further comprising the steps of identifying and ranking diagnostic foci of attention in the image at a given level of sensitivity, using the diagnostic saliency map.

32. The method according to claim 22, further comprising the step of combining the outputs of the plurality of filters using winner-take-all operators between the regions in the image defined by the virtual windows, and one of aggregation operators and winner-take-all operators between different filters.

33. The method according to claim 22, further comprising the step of removing salt-and-pepper type of noise from outputs of the filters.

34. The method according to claim 22, further comprising the step of generating a numeric diagnostic saliency score for the regions in the image defined by the virtual windows associated with the plurality of filters based on the computed distances.

35. The method according to claim 22, further comprising the step of combining outputs of the plurality of filters into a single diagnostic saliency map for the image, the map comprising a numeric diagnostic saliency score for each of the regions in the image defined by the virtual windows associated with the plurality of filters, the specific type of diagnostic finding at each of the regions and characteristics of the diagnostic finding.

36. The method according to claim 22, further comprising the step of identifying an individual training image patch based on an output of the individual training image patch.

37. The method according to claim 22, further comprising the step of integrating domain knowledge rules into one of normalizing outputs of the plurality of filters and generating a diagnostic saliency map for the image.