Classifying image features

• US 7,555,155 B2
• Filed: 01/27/2006
• Issued: 06/30/2009
• Est. Priority Date: 01/27/2005
• Status: Active Grant

• Current Assignee: Akoya Biosciences, Inc.
• Original Assignee: Cambridge Research & Instrumentation Incorporated (Revvity, Inc.)
• Inventors: Levenson, Richard, Hoyt, Clifford C., Gossage, Kirk William
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/342,272
• Publication Number:
  US 20060245631A1
• Time in Patent Office: 1,250 Days
• Field of Search: 382/128, 382/129, 382/130, 382/131, 382/132, 382/133, 382/134, 382/162, 600/407, 600/476, 356/36, 356/300, 356/369, 356/453, 356/465
• US Class Current: 382/133
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

Methods are disclosed for classifying different parts of a sample into respective classes based on an image stack that includes one or more images.

Claims

1. A method comprising using one or more processors to perform the following steps:

decomposing a set of spectral images of a sample into an unmixed image set, wherein each member of the unmixed image set corresponds to a spectral contribution from a different component in the sample;

generating a composite image based on the set of spectral images of the sample, wherein the spatial intensities of two or more different members of a group consisting of the spectral images and the unmixed images are weighted differently and combined to produce the composite image, and

classifying different parts of the sample into respective classes based on an image stack comprising one or more of the unmixed images and further comprising the composite image.

2. The method of claim 1, wherein the images in the image stack used for classification comprise two or more of the unmixed images.

3. The method of claim 1, wherein classification involves both spectral and spatial information in the image stack.

4. The method of claim 1, wherein the set of spectral images comprises n images, and wherein the one or more images in the image stack used for classification comprises fewer than n images.

5. The method of claim 1, wherein the composite image is generated according to a weighting function that changes monotonically with a spectral wavelength.

6. The method of claim 1, wherein the spatial intensities of two or more different ones of the unmixed images are weighted differently and combined to produce the composite image.

7. The method of claim 1, wherein a neural network is used for the classifying.

8. The method of claim 7, wherein classifying different regions of the sample into the different classes comprises identifying selected regions of the image stack that correspond to each of the individual classes, training the neural network to recognize the classes based on the selected regions, and applying the trained neural network to the additional regions of the image stack.

9. The method of claim 1, further comprising generating an output image showing classified regions of the sample.

10. The method of claim 1, further comprising obtaining the one or more images in the image stack.

11. The method of claim 1, wherein the weighting function is selected to enhance a contrast between features contributed to the composite image from the two or more different members of the group consisting of the spectral images and the unmixed images.

12. The method of claim 1, wherein the sample is a tissue section.

13. Apparatus comprising a computer readable medium storing a program that causes a processor to carry out the method of claim 1.

14. A method for classifying different parts of a sample into respective classes based on an image stack comprising one or more images, the method comprising using one or more processors to perform the following steps:

positioning a sampling window within the image stack to select a portion of the image stack for classification, the selected portion comprising multiple pixels;

classifying the selected portion into one of several classes, wherein each of the pixels in the selected portion are provisionally classified as having the same class as that of the selected portion;

translating the sampling window to select a second portion of the image stack for classification and classifying the second portion into one of several classes, wherein each of the pixels in the second portion are provisionally classified as having the same class as that of the second portion;

repeating the translating and classifying for additional portions of the image stack until at least some of the pixels in the image stack have been provisionally classified multiple times as part of different portions selected by the sampling window; and

classifying each of at least some of the pixels that have been provisionally classified multiple times into one of the several classes based on their multiple provisional classifications.

15. The method of claim 14, wherein the different portions selected by the sampling window comprise the same number of pixels.

16. The method of claim 14, wherein at least some of the different portions selected by the sampling window overlap with one another.

17. The method of claim 14, wherein the provisional classifications of each pixel can be expressed as a histogram indicating the number of times the pixel was provisionally classified in each class.

18. The method of claim 14, wherein the final classification of each pixel corresponds to the class to which it was most frequently provisionally classified.

19. The method of claim 14, wherein the number of times at least some of the pixels are provisionally classified is more than two and no larger than the number of pixels in the sampling window.

20. The method of claim 19, wherein the number of times at least some of the pixels are provisionally classified equals the number of pixels in the sampling window.

21. The method of claim 14, wherein a neural network is used for the classifying.

22. The method of claim 21, wherein classifying different regions of the sample into the different classes comprises identifying selected regions of the image stack that correspond to each of the individual classes, training the neural network to recognize the classes based on the selected regions, and applying the trained neural network to additional regions of the image stack.

23. The method of claim 21, wherein the input into the neural network is a feature vector having one or more elements based on calculating at least one spatial gray level dependency matrix for a corresponding one of the images in the image stack.

24. The method of claim 21, wherein the input into the neural network is a feature vector having one or more elements based on calculating a two-dimensional Fourier transform for a corresponding one of the images in the image stack.

25. The method of claim 14, wherein the one or more images in the image stack comprise at least one spectral image and wherein the input information for the classifying includes both spectral and spatial information.

26. The method of claim 25, wherein the spectral images are images of sample emission according to different spectral indices for the emission.

27. The method of claim 25, wherein the spectral images are images of sample emission according to different spectral indices for illumination of the sample causing the emission.

28. The method of claim 14, wherein the image stack comprises only one image.

29. The method of claim 14, wherein the image stack comprises an RGB image.

30. The method of claim 14, further comprising generating an output image showing the classified regions of the sample.

31. The method of claim 14, further comprising:

decomposing a set of spectral images of the sample into an unmixed image set, wherein each member of the unmixed image set corresponds to a spectral contribution from a different component in the sample,

wherein the images in the image stack used for classifications comprise one or more of the unmixed images.

32. The method of claim 31, wherein the images in the image stack used for classification comprise two or more of the unmixed images.

33. The method of claim 14, further comprising:

generating a composite image based on the set of spectral images of the sample,

wherein the spatial intensities of two or more different members of a group consisting of the spectral images and unmixed images derived from the spectral images are weighted differently and combined to produce the composite image, and

wherein the one or more images in the image stack comprise the composite image.

34. The method of claim 33, wherein the set of spectral images comprises n images and wherein the one or more images in the image stack used for the classifications comprises fewer than n images.

35. The method of claim 33, wherein the weighting function is selected to enhance a contrast between features contributed to the composite image from the two or more different spectral images.

36. Apparatus comprising a computer readable medium storing a program that causes a processor to carry out the method of claim 14.

37. A method for classifying different parts of a sample into respective classes based on an image stack comprising more than three spectral images, the method comprising using one or more processors to perform the following steps:

generating a composite image based on the spectral images of the sample, wherein the spatial intensities of two or more different members of the spectral images are weighted differently and combined to produce the composite image; and

classifying different regions of the image stack into respective classes based on the set of more than three spectral images, wherein each region comprises multiple pixels so that each classification involves both spectral and spatial information and wherein the image stack comprise the composite image.

38. Apparatus comprising a computer readable medium storing a program that causes a processor to carry out the method of claim 37.

39. Apparatus comprising:

a means for obtaining a set of spectral images of a sample; and

an electronic processor coupled to the means and configured to;

i) decompose the set of spectral images into an unmixed image set, wherein each member of the unmixed image set corresponds to a spectral contribution from a different component in the sample;

ii) generate a composite image based on the set of spectral images of the sample, wherein the spatial intensities of two or more different members of a group consisting of the spectral images and the unmixed images are weighted differently and combined to produce the composite image, and iii) classify different parts of the sample into respective classes based on an image stack comprising one or more of the unmixed images and further comprising the composite image.

40. Apparatus for classifying different parts of a sample into respective classes based on an image stack comprising one or more images comprising:

a means for obtaining the one or more images; and

an electronic processor coupled to the means and configured to;

i) position a sampling window within the image stack to select a portion of the image stack for classification, the selected portion comprising multiple pixels;

ii) classify the selected portion into one of several classes, wherein each of the pixels in the selected portion are provisionally classified as having the same class as that of the selected portion;

iii) translate the sampling window to select a second portion of the image stack for classification and classifying the second portion into one of several classes, wherein each of the pixels in the second portion are provisionally classified as having the same class as that of the second portion;

iv) repeat the translating and classifying for additional portions of the image stack until at least some of the pixels in the image stack have been provisionally classified multiple times as part of different portions selected by the sampling window; and

v) classify each of at least some of the pixels that have been provisionally classified multiple times into one of the several classes based on their multiple provisional classifications.

41. Apparatus for classifying different parts of a sample into respective classes based on an image stack comprising more than three spectral images comprising:

a means for obtaining the spectral images; and

an electronic processor coupled to the means and configured to;

i) generate a composite image based on the spectral images of the sample, wherein the spatial intensities of two or more different members of the spectral images are weighted differently and combined to produce the composite image; and

ii) classify different regions of the image stack into respective classes based on the set of more than three spectral images, wherein each region comprises multiple pixels so that each classification involves both spectral and spatial information and wherein the image stack comprise the composite image.

42. Apparatus comprising:

an optical system for obtaining a set of spectral images of a sample; and

an electronic processor coupled to the optical system and configured to;

i) decompose the set of spectral images into an unmixed image set, wherein each member of the unmixed image set corresponds to a spectral contribution from a different component in the sample;

ii) generate a composite image based on the set of spectral images of the sample, wherein the spatial intensities of two or more different members of a group consisting of the spectral images and the unmixed images are weighted differently and combined to produce the composite image, and iii) classify different parts of the sample into respective classes based on an image stack comprising one or more of the unmixed images and further comprising the composite image.

43. Apparatus for classifying different parts of a sample into respective classes based on an image stack comprising one or more images comprising:

an optical system for obtaining the one or more images; and

an electronic processor coupled to the optical system and configured to;

i) position a sampling window within the image stack to select a portion of the image stack for classification, the selected portion comprising multiple pixels;

ii) classify the selected portion into one of several classes, wherein each of the pixels in the selected portion are provisionally classified as having the same class as that of the selected portion;

iii) translate the sampling window to select a second portion of the image stack for classification and classifying the second portion into one of several classes, wherein each of the pixels in the second portion are provisionally classified as having the same class as that of the second portion;

iv) repeat the translating and classifying for additional portions of the image stack until at least some of the pixels in the image stack have been provisionally classified multiple times as part of different portions selected by the sampling window; and

v) classify each of at least some of the pixels that have been provisionally classified multiple times into one of the several classes based on their multiple provisional classifications.

44. Apparatus for classifying different parts of a sample into respective classes based on an image stack comprising more than three spectral images comprising:

an optical system for obtaining the spectral images; and

an electronic processor coupled to the optical system and configured to;

i) generate a composite image based on the spectral images of the sample, wherein the spatial intensities of two or more different members of the spectral images are weighted differently and combined to produce the composite image; and

ii) classify different regions of the image stack into respective classes based on the set of more than three spectral images, wherein each region comprises multiple pixels so that each classification involves both spectral and spatial information and wherein the image stack comprise the composite image.