Analyzing pixel data using image, thematic and object layers of a computer-implemented network structure

• US 7,801,361 B2
• Filed: 02/21/2007
• Issued: 09/21/2010
• Est. Priority Date: 10/15/2002
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schmidt, Guenter, Binnig, Gerd
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/709,601
• Publication Number:
  US 20080008349A1
• Time in Patent Office: 1,308 Days
• Field of Search: 382/100, 382/158, 382/190, 382/195, 382/224, 382/226, 382/227
• US Class Current: 382/227
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

An analysis system analyzes and measures patterns present in the pixel values of digital images using a computer-implemented network structure that includes a process hierarchy, a class network and a data network. The data network includes image layers, thematic layers and object networks. Various types of processing are performed depending on whether the data of the digital images is represented in the image, thematic or object layers. Pixel-oriented and object-oriented processing is combined so that fewer computations and less memory are used to analyze the digital images. Pixel-oriented processes, such as filtering, are selectively performed only at pixel locations that are assigned to a specified thematic class of a thematic layer or that are linked to a particular object of the object network. Similarly, object-oriented processing is performed at pixel locations linked to objects generated using thematic layers or using image layers on which pixel-oriented processes have already been performed.

Claims

1. A method comprising:

(a) specifying an object class, wherein a first plurality of objects belongs to the object class;

(b) specifying a filter;

(c) specifying process steps of a process hierarchy;

(d) acquiring first pixel values, wherein the first pixel values form a first image layer, and wherein the first pixel values are associated with first pixel locations;

(e) executing the process steps on a computer to generate the first plurality of objects by linking some of the first pixel locations to each of the first plurality of objects, wherein the first plurality of objects forms a first object network;

(f) linking each of the first plurality of objects to the object class;

(g) generating a second image layer by applying the filter to those first pixel values associated with first pixel locations that are linked to one of the first plurality of objects, wherein filtered pixel values form the second image layer, and wherein the filtered pixel values are associated with second pixel locations;

(h) generating a second object network by linking some of the second pixel locations to each of a second plurality of objects;

(i) determining a characteristic of one of the second plurality of objects by measuring the object; and

(j) displaying the measured object on a graphical user interface of the computer.

2. The method of claim 1, wherein the first image layer is stored in a memory space, and wherein the first image layer is overwritten by the second image layer in the memory space.

3. The method of claim 1, wherein the first object network comprises a third plurality of objects, wherein the second object network is the first object network in which the second plurality of objects is substituted for the first plurality of objects, and wherein the second object network comprises the third plurality of objects.

4. The method of claim 1, wherein the filter has a parameter, and wherein the parameter is updated for each of the first plurality of objects before those first pixel values associated with first pixel locations that are linked to each object are filtered.

5. The method of claim 1, wherein each of the first plurality of objects has a diameter, wherein the filter is an edge detection filter, and wherein the parameter is the diameter of each object.

6. The method of claim 1, wherein the one of the second plurality of objects has an area, and wherein the measuring measures the area.

7. The method of claim 1, wherein the object class represents nuclei in a microscopic image of a cell, wherein each nucleus has an area, and wherein the measuring the object in (i) measures the area of a nucleus.

8. A method comprising:

(a) specifying an object class, wherein an object of a data network belongs to the object class;

(b) specifying a thematic class;

(c) specifying process steps of a process hierarchy;

(d) acquiring first pixel values, wherein the first pixel values form a first image layer, and wherein the first pixel values are associated with first pixel locations;

(e) executing the process steps on a computer to generate a second image layer of second pixel values by filtering the first pixel values of the first image layer, wherein the second pixel values are associated with second pixel locations;

(f) generating a thematic layer by assigning the thematic class to certain of the second pixel locations;

(g) generating a third image layer of third pixel values by processing those second pixel values associated with second pixel locations that are classified as belonging to the thematic class, wherein the third pixel values are associated with third pixel locations;

(h) generating an object network based on the third pixel values, wherein the object network comprises the object, and wherein certain of the third pixel locations are linked to the object;

(i) determining a characteristic of the object of the object network by measuring the object; and

(j) displaying the measured object on a graphical user interface of the computer.

9. The method of claim 8, wherein the thematic layer generated in (f) is modified using a morphological operation before the third image layer is generated in (g).

10. The method of claim 9, wherein the morphological operation reclassifies certain of the second pixel locations as belonging to the thematic class.

11. The method of claim 8, wherein the object has a border with a length, and wherein the characteristic of the object is the length of the border.

12. The method of claim 8, wherein the thematic class is assigned to certain of the second pixel locations in (f) according to a classifier algorithm, and wherein the classifier algorithm determines whether each of the second pixel values associated with the second pixel locations exceeds a threshold value.

13. The method of claim 8, wherein the thematic class is assigned to certain of the second pixel locations in (f) according to a classifier algorithm, and wherein the classifier algorithm compares each of the second pixel values to its neighboring second pixel values of the second image layer.

14. A method comprising:

(a) specifying an object class and a thematic class, wherein an object of a data network belongs to the object class;

(b) specifying a first algorithm and a second algorithm;

(c) acquiring pixel values, wherein a first image layer comprises the pixel values;

(d) generating a second image layer by filtering the pixel values, wherein the filtered pixel values are associated with pixel locations;

(e) generating an object network of the data network by linking a first plurality of the pixel locations to the object based on the filtered pixel values;

(f) generating a thematic layer by assigning the thematic class to a second plurality of the pixel locations based on the filtered pixel values;

(g) executing the first algorithm on a computer to process those filtered pixel values that are associated with the first plurality of the pixel locations;

(h) executing the second algorithm on the computer to process those filtered pixel values that are associated with the second plurality of the pixel locations; and

(i) displaying the object on a graphical user interface of the computer.

15. The method of claim 14, wherein the second image layer is a digital image that includes an edge, and wherein the second algorithm detects the edge in the digital image by identifying each of the second plurality of the pixel locations that has a predetermined number of neighboring pixel locations belonging to the thematic class.

16. The method of claim 15, wherein the neighboring pixel locations consist of eight pixel locations that are contiguous to each pixel location belonging to the thematic class.

17. The method of claim 15, further comprising:

(j) assigning a second thematic class to each pixel location belonging to the thematic class that also has the predetermined number of neighboring pixel locations belonging to the thematic class; and

(k) increasing the number of pixel locations belonging to the second thematic class by assigning the second thematic class to each pixel location belonging to the thematic class that borders a pixel location belonging to the second thematic class.

18. The method of claim 14, wherein the first plurality of the pixel locations is a subset of the second plurality of the pixel locations, and wherein the thematic layer is generated in (f) only at pixel locations that are linked to the object.

19. The method of claim 14, wherein the object has a size, wherein the first algorithm has a parameter, wherein the parameter is based on the size of the object.

20. A method comprising:

(a) acquiring pixel values of a digital image that includes an edge, and wherein the pixel values are associated with pixel locations;

(b) generating an object network based on the pixel values, wherein the object network comprises an object, and wherein the object is linked to certain of the pixel locations;

(c) generating a thematic layer based on the pixel values, wherein the thematic layer has a first thematic class and a second thematic class, wherein the thematic layer is generated only at pixel locations that are linked to the object, and wherein the pixel locations are assigned the first thematic class or the second thematic class;

(d) detecting the edge in the digital image by identifying each pixel location belonging to the first thematic class that has a predetermined number of neighboring pixel locations belonging to the second thematic class; and

(e) displaying the edge in the digital image on a graphical user interface.

21. The method of claim 20, wherein the neighboring pixel locations consist of N2-1 pixel locations in a neighborhood surrounding each pixel location that belongs to the first thematic class, and wherein N is an odd integer greater than one.

22. The method of claim 21, wherein N is three, and wherein eight pixel locations in the neighborhood are contiguous to each pixel location that belongs to the first thematic class.

23. The method of claim 21, wherein N is nine, and wherein each pixel location belonging to the first thematic class is centered in a 9-by-9 matrix of pixel locations, eighty pixel locations of which comprise the neighborhood.

24. The method of claim 20, further comprising:

(f) assigning a third thematic class to each pixel location that belongs to the first thematic class and that is identified as having the predetermined number of neighboring pixel locations belonging to the second thematic class; and

(g) assigning the third thematic class to each pixel location that belongs to the second thematic class that neighbors a pixel location that has been assigned the third thematic class.

25. A method comprising:

(a) acquiring a digital image comprising pixel values;

(b) generating a thematic layer based on the pixel values, wherein a first thematic class and a plurality of other thematic classes are assigned to pixel locations associated with the pixel values;

(c) generating an object network by segmenting pixel locations associated with contiguous thematic classes of the thematic layer;

(d) assigning a first object class to objects of the object network that are linked to pixel locations assigned to the first thematic class;

(e) modifying the thematic layer classes for those pixel locations that are linked to the objects of the first object class;

(f) modifying the objects of the first object class by segmenting contiguous thematic classes of the thematic layer;

(g) classifying the objects of the object network according to membership in a second object class;

(h) measuring the objects that are assigned to the second object class; and

(i) displaying the measured objects on a graphical user interface.

26. The method of claim 25, wherein after the thematic layer is generated in (b), the thematic layer is modified using a mophological operation.

27. The method of claim 25, wherein the modifying in (f) is a morphological operation whose parameters depend on characteristics of the objects.

28. A method comprising:

(a) acquiring a digital image comprising pixel values;

(b) generating an object network by segmenting contiguous regions in the digital image;

(c) classifying the objects of the object network according to a membership function of a first object class;

(d) assigning a first thematic class to those pixel locations that are linked to the objects of the first object class;

(e) identifying pixel locations that are assigned the first thematic class and that fulfill a criterion of a second thematic class;

(f) resegmenting the objects of the first object class by grouping contiguous pixel locations assigned to the first thematic class and the second thematic class;

(g) classifying the objects of the object network according to a membership function of a second object class;

(h) measuring the objects that are assigned to the second object class; and

(i) displaying the measured objects on a graphical user interface.

29. The method of claim 28, wherein the criterion in (e) is determined based on both the number of pixel locations in a neighborhood of each pixel location of interest that are assigned the first thematic class and on whether each pixel value associated with the pixel locations in the neighborhood has a brightness below a specified threshold.