Computer-aided image analysis

• US 6,996,549 B2
• Filed: 01/23/2002
• Issued: 02/07/2006
• Est. Priority Date: 05/01/1998
• Status: Expired due to Term

• Current Assignee: Curtis Anderson, Health Discovery Corporation, James Roberts, Joe Mckenzie, Jules B. Paderewski, Julian N. Stern, Memorial Health Systems Incorporated, Timothy P. O'Hayer
• Original Assignee: Health Discovery Corporation
• Inventors: Zhang, Hong, Carls, Garry, Barnhill, Stephen D.
• Primary Examiner(s): Knight, Anthony
• Assistant Examiner(s): Holmes, Michael B.
• Application Number:
  US10/056,438
• Publication Number:
  US 20020165837A1
• Time in Patent Office: 1,476 Days
• Field of Search: 706/16
• US Class Current: 706/16
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Digitized image data are input into a processor where a detection component identifies the areas (objects) of particular interest in the image and, by segmentation, separates those objects from the background. A feature extraction component formulates numerical values relevant to the classification task from the segmented objects. Results of the preceding analysis steps are input into a trained learning machine classifier which produces an output which may consist of an index discriminating between two possible diagnoses, or some other output in the desired output format. In one embodiment, digitized image data are input into a plurality of subsystems, each subsystem having one or more support vector machines. Pre-processing may include the use of known transformations which facilitate extraction of the useful data. Each subsystem analyzes the data relevant to a different feature or characteristic found within the image. Once each subsystem completes its analysis and classification, the output for all subsystems is input into an overall support vector machine analyzer which combines the data to make a diagnosis, decision or other action which utilizes the knowledge obtained from the image.

Claims

1. A computer-implemented method for analysis of a digitized image, the method comprising:

(a) inputting a training set of image data and a test set of image data into a processor;

(b) pre-processing each set of image data to detect and extract the presence of at least one feature of interest within the image data;

(c) training and testing at least one learning machine having at least one kernel using the pre-processed sets of image data to classify the at least one feature of interest into at least one of a plurality of classes of possible feature characteristic;

(d) comparing the classified features from the test set of image data with known results of the test set of image data to determine if an optimal solution is obtained;

(e) repeating steps (c) and (d) if the optimal solution is not obtained;

(f) if the optimal solution is obtained, inputting a live set of image data into the processor;

(g) pre-processing the live set of image data to detect and extract the presence of features of interest within the image data;

(h) classifying the at least one feature of interest; and

(i) generating an output comprising the classified at least one feature of interest from the live set of image data.

2. The method of claim 1, wherein steps (a) and (f) further comprise inputting each of the training, test and live sets of data into each of a plurality of detection subsystems, each detection subsystem adapted to detect and classify one of a plurality of features of interest, wherein each feature of interest has a plurality of possible feature characteristics, and wherein each subsystem generates an output for its corresponding feature of interest.

3. The method of claim 2, further comprising:

(j) combining outputs from each of the plurality of subsystems;

(k) inputting the combined outputs into at least one overall learning machine having at least one kernel; and

(l) generating an overall output comprising a classification of the digitized image.

4. The method of claim 3, wherein the overall learning machine is a soft margin support vector machine.

5. The method of claim 4, wherein the soft margin support vector machine is enhanced by applying a variable penalty for classification errors.

6. The method of claim 3, wherein the digitized image comprises a mammogram and the plurality of subsystems comprises a calcification detection subsystem, a mass detection subsystem, and a structure distortion subsystem.

7. The method of claim 1, wherein pre-processing steps (b) and (g) comprise segmenting the feature of interest to separate the feature of interest from a background and generating a numerical value for the segmented feature of interest.

8. The method of claim 7, wherein segmenting comprises identifying local extremes corresponding to each segmented feature of interest in the image data.

9. The method of claim 8, wherein the feature of interest comprises a spot having a brightness and identifying local extremes comprises classifying the brightness of the spot into one or more of a plurality of brightness levels.

10. The method of claim 9, wherein geometry is a possible feature characteristic and geometry is determined measuring a change in slope between borders of the spot at two different brightness levels.

11. The method of claim 1, wherein pre-processing steps (b) and (g) comprise segmenting the feature of interest and transforming the segmented feature to a fixed dimensional vector.

12. The method of claim 11, wherein transforming comprises:

computing a centroid of the feature of interest;

sampling a contour of the feature of interest using a polar coordinate system having an origin at the centroid to provide a plurality of radial measures;

forming a vector using the plurality of radial measures; and

applying a Fourier transform to the vector to provide the fixed dimensional vector.

13. The method of claim 1, wherein the at least one feature of interest comprises a plurality of features of interest and pre-processing steps (b) and (g) comprise segmenting a first feature of interest from a second, at least partially overlapping feature of interest by applying a gravitation model to each feature of interest to contract each feature into a distinct body.

14. The method of claim 1, wherein pre-processing steps (b) and (g) comprise applying a transform to the image data, the transform selected from the group consisting of wavelet transforms, Radon transforms, and Hough transforms.

15. The method of claim 1, wherein the at least one kernel is a Fourier kernel.

16. A method for computer-aided analysis of a digitized image having a plurality of features of interest, the method comprising”

(a) inputting a training set of image data and a test set of image data into a processor comprising a plurality of processing modules;

(b) assigning a processing module for each feature of interest;

(c) for each feature of interest, pre-processing each set of image data to detect and extract the presence of that feature of interest within the image data;

(d) for each feature of interest, training and testing at least one first-level support vector machine using the pre-processed sets of image data to classify the corresponding feature of interest into at least one of a plurality of possible feature characteristics;

(e) comparing the classified feature from the test set of image data with known results of the test set of image data to determine if an optimal solution is obtained;

(f) repeating steps (d) and (e) if the optimal solution is not obtained;

(g) if the optimal solution is obtained, inputting a live set of image data into the processor;

(h) pre-processing the live set of image data to detect and extract the presence of features of interest within the image data;

(i) classifying each feature of interest according to its possible feature characteristics to generate an output;

(j) combining the outputs for the plurality of features of interest

(k) inputting the combined outputs into at least one second-level support vector machine; and

(l) generating an overall output comprising a classification of the digitized image.

17. The method of claim 16, wherein the second-level support vector machine is a soft margin support vector machine.

18. The method of claim 17, wherein the soft margin support vector machine is enhanced by applying a variable penalty for classification errors.

19. The method of claim 16, wherein each first-level support vector machine uses a Fourier kernel.

20. The method of claim 16, wherein the digitized image comprises a mammogram and the plurality of processing modules comprises a calcification detection subsystem, a mass detection subsystem, and a structure distortion subsystem.

21. The method of claim 16, wherein pre-processing steps (c) and (h) comprise segmenting the feature of interest to separate the feature of interest from a background and generating a numerical value for the segmented feature of interest.

22. The method of claim 21, wherein segmenting comprises identifying local extremes corresponding to each segmented feature of interest in the image data.

23. The method of claim 22, wherein the feature of interest comprises a spot having a brightness and identifying local extremes comprises classifying the brightness of the spot into one or more of a plurality of brightness levels.

24. The method of claim 23, wherein geometry is a possible feature characteristic and geometry is determined by measuring a change in slope between borders of the spot at two different brightness levels.

25. The method of claim 16, wherein pre-processing steps (c) and (h) comprise segmenting the feature of interest and transforming the segmented feature to a fixed dimensional vector.

26. The method of claim 25, wherein transforming comprises:

computing a centroid of the feature of interest;

sampling a contour of the feature of interest using a polar coordinate system having an origin at the centroid to provide a plurality of radial measures;

forming a vector using the plurality of radial measures; and

applying a Fourier transform to the vector to provide the fixed dimensional vector.

27. The method of claim 16, wherein each digitized image includes a plurality of a single type of feature of interest and pre-processing steps (c) and (h) comprise segmenting a first feature of interest from a second, at least partially overlapping feature of interest by applying a gravitation model to each feature of interest to contract each feature into a distinct body.

28. The method of claim 16, wherein pre-processing steps (c) and (h) comprise applying a transform to the image data, the transform selected from the group consisting of wavelet transforms, Radon transforms, and Hough transforms.

29. A method for computer-aided analysis of a digitized mammogram, the method comprising:

(a) inputting a training set of mammogram data and a test set of mammogram data into a processor comprising a plurality of detection subsystems, each detection subsystem for analyzing one of a plurality of features of interest;

(b) assigning a processing module for each of the plurality of detection subsystems;

(c) in each detection subsystem, pre-processing each set of mammogram data to detect and extract the presence of a feature of interest corresponding to that detection subsystem;

(d) in each detection subsystem, training and testing at least one first-level support vector machine using the pre-processed sets of mammogram data to classify the corresponding feature of interest into at least one of a plurality of possible feature characteristics;

(e) comparing the classified feature from the test set of mammogram data with known analysis of the test set of mammogram data to determine if an optimal solution is obtained;

(f) repeating steps (d) and (e) if the optimal solution is not obtained;

(g) if the optimal solution is obtained, inputting a live set of mammogram data into the processor;

(h) pre-processing the live set of mammogram data to detect and extract the presence of features of interest within the mammogram data;

(i) classifying each feature of interest according to its possible feature characteristics to generate an output;

(j) combining the outputs for the plurality of features of interest

(k) inputting the combined outputs into at least one second-level support vector machine; and

(l) generating an overall output comprising an analysis of the digitized mammogram.

30. The method of claim 29, wherein the features of interest are calcification, mass and structure distortion.

31. The method of claim 29, wherein the second-level support vector machine is a soft margin support vector machine.

32. The method of claim 31, wherein the soft margin support vector machine is enhanced by applying a variable penalty for classification errors.

33. The method of claim 29, wherein each first-level support vector machine uses a Fourier kernel.

34. The method of claim 29, wherein pre-processing steps (c) and (h) comprise segmenting the feature of interest to separate the feature of interest from a background and generating a numerical value for the segmented feature of interest.

35. The method of claim 34, wherein segmenting comprises identifying local extremes corresponding to each segmented feature of interest in the image data.

36. The method of claim 35, wherein the feature of interest comprises a spot having a brightness and identifying local extremes comprises classifying the brightness of the spot into one or more of a plurality of brightness levels.

37. The method of claim 36, wherein geometry is a possible feature characteristic and geometry is determined by measuring a change in slope between borders of the spot at two different brightness levels.

38. The method of claim 29, wherein pre-processing steps (c) and (h) comprise segmenting the feature of interest and transforming the segmented feature to a fixed dimensional vector.

39. The method of claim 38, wherein transforming comprises:

computing a centroid of the feature of interest;

sampling a contour of the feature of interest using a polar coordinate system having an origin at the centroid to provide a plurality of radial measures;

forming a vector using the plurality of radial measures; and

applying a Fourier transform to the vector to provide the fixed dimensional vector.

40. The method of claim 29, wherein each digitized image includes a plurality of a single type of feature of interest and pre-processing steps (c) and (h) comprise segmenting a first feature of interest from a second, at least partially overlapping feature of interest by applying a gravitation model to each feature of interest to contract each feature into a distinct body.

41. The method of claim 29, wherein pre-processing steps (c) and (h) comprise applying a transform to the image data, the transform selected from the group consisting of wavelet transforms, Radon transforms, and Hough transforms.

42. A computer system for analysis of a digitized image having a plurality of features of interest, the computer system comprising:

a processor;

an input device for receiving image data to be processed;

a memory device in communication with the processor having a plurality of detection subsystems stored therein, each of the plurality of detection subsystems comprising;

a pre-processing component for detecting and extracting one of the features of interest within the image data;

a classification component comprising at least one first-level support vector machine for classifying the feature of interest into at least one of a plurality of possible features characteristics;

an output for outputting the classified feature of interest;

an overall analyzer for combining the outputs of the plurality of detection subsystems and generating an analysis of the digitized image, the overall analyzer comprising a second-level support vector machine.

43. The computer system of claim 42, wherein the at least one first-level support vector machine uses a Fourier kernel.

44. The computer system of claim 42, wherein the second-level support vector machine is a soft margin support vector machine.

45. The computer system of claim 44, wherein the soft margin support vector machine is enhanced by applying a variable penalty for classification errors.

46. The computer system of claim 42, wherein the digitized image comprises a mammogram and the plurality of detection subsystems comprises a calcification detection subsystem, a mass detection subsystem, and a structure distortion subsystem.

47. The computer system of claim 42, wherein pre-processing component applies a segmenting routine to separate the feature of interest from a background and generates a numerical value for the segmented feature of interest.

48. The computer system of claim 47, wherein segmenting routine identifies local extremes corresponding to each segmented feature of interest in the image data.

49. The computer system of claim 48, wherein the feature of interest comprises a spot having a brightness and local extremes are identified by classifying the brightness of the spot into one or more of a plurality of brightness levels.

50. The computer system of claim 49, wherein geometry is a possible feature characteristic and geometry is determined by measuring a change in slope between borders of the spot at two different brightness levels.

51. The computer system of claim 42, wherein the pre-processing component segments the feature of interest and applies a transform to the segmented feature to a fixed dimensional vector.

52. The computer system of claim 51, wherein transform comprises:

computing a centroid of the feature of interest;

sampling a contour of the feature of interest using a polar coordinate system having an origin at the centroid to provide a plurality of radial measures;

forming a vector using the plurality of radial measures; and

applying a Fourier transform to the vector to provide the fixed dimensional vector.

53. The computer system of claim 42, wherein each digitized image includes a plurality of a single type of feature of interest and the pre-processing component segments a first feature of interest from a second, at least partially overlapping feature of interest by applying a gravitation model to each feature of interest to contract each feature into a distinct body.

54. The computer system of claim 42, wherein the pre-processing component applies a transform to the image data, wherein the transform is selected from the group consisting of wavelet transforms, Radon transforms, and Hough transforms.