Cognition integrator and language

• US 20070122017A1
• Filed: 08/28/2006
• Published: 05/31/2007
• Est. Priority Date: 10/15/2002
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schmidt, Guenter, Schaepe, Arno, Binnig, Gerd
• Granted Patent: US 7,873,223 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

In a specification mode, a user specifies classes of a class network and process steps of a process hierarchy using a novel scripting language. The classes describe what the user expects to find in digital images. The process hierarchy describes how the digital images are to be analyzed. Each process step includes an algorithm and a domain that specifies the classes on which the algorithm is to operate. A Cognition Program acquires table data that includes pixel values of the digital images, as well as metadata relating to the digital images. In an execution mode, the Cognition Program generates a data network in which pixel values are linked to objects, and objects are categorized as belonging to classes. The process steps, classes and objects are linked to each other in a computer-implemented network structure in a manner that enables the Cognition Program to detect target objects in the digital images.

Claims

1. A method comprising:

in a specification mode;

specifying a class network having a class, wherein a membership function defines a likelihood that an object of a data network belongs to the class;

specifying a process step having a domain and an algorithm, wherein the domain designates the class, and wherein the process step is part of a process hierarchy;

specifying a link type that defines a relation between the class and the object;

in an execution mode;

acquiring table data values; and

generating the data network by generating the object of the data network and by selectively linking selected table data values to the object according to the class network and the process hierarchy.

2. The method of claim 1, wherein the table data values include patient data and pixel values of a medical image of a patient, wherein the patient has a gender, an age, a weight, a height, blood values, prescribed medications, a number of children, a family history of ailments, a history of breastfeeding, a history of smoking and a history of drug use, and wherein the patient data is taken from the group consisting of:

the gender, the age, the weight, the height, the blood values, the prescribed medications, the number of children, the family history of ailments, the history of breastfeeding, the history of smoking, the history of drug use, and tissue analysis results of the patient.

3. The method of claim 2, wherein the medical image of the patient is a mammographic image of the patient'"'"'s breasts.

4. The method of claim 1, wherein the specifying the process step is performed by a user by selecting the process step from a library of predefined process steps.

5. The method of claim 1, wherein a first plurality of the table data values are spectral intensity values of a digital image, and wherein a second plurality of the table data values are items of metadata relating to the digital image.

6. The method of claim 5, wherein the digital image is an image from an airborne sensor.

7. The method of claim 5, wherein the class network, the process hierarchy and the data network together form a semantic network, and wherein the process step is linked to one of the items of metadata.

8. The method of claim 1, wherein the domain designates a scheme for navigating from the object to a second object.

9. The method of claim 8, wherein the object and the second object exhibit object properties, and wherein the scheme for navigating uses constraints on the object properties to navigate from the object to the second object.

10. The method of claim 1, wherein the table data values comprise pixel values from a plurality of parallel planar scans of a three-dimensional physical object, wherein the generating the data network includes generating sub-objects by linking selected pixel values of the scans to the sub-objects, and wherein the generating the data network includes linking the sub-objects of adjacent scans.

11. The method of claim 10, wherein the linked sub-objects of adjacent scans represent an internal organ of a human body.

12. The method of claim 1, wherein the table data values comprise values associated with a plurality of scans taken at different time intervals, wherein the generating the data network includes generating sub-objects by linking selected values in each of the scans to the sub-objects, and wherein the generating the data network includes linking the sub-objects of scans taken in adjacent time intervals.

13. The method of claim 12, wherein the linked sub-objects of scans taken in adjacent time intervals depict a movement of a cell.

14. The method of claim 12, wherein the plurality of scans taken at different time intervals is a video movie depicting movement of a moving object, and wherein the moving object is taken from the group consisting of:

a worm, a fish, a bacterium, a cell, a person, a motor vehicle, a boat, and an airplane.

15. The method of claim 1, wherein the table data values comprise values from a first group of parallel planar slices of a three-dimensional data set of a physical object taken at one time and from a second group of parallel planar slices of the three-dimensional data set of the physical object taken at a different time, and wherein the values from the first group and the second group of parallel planar slices depict movement of the physical object.

16. The method of claim 1, wherein the generating the data network comprises:

classifying whether each of a plurality of objects of the data network belongs to the class designated by the domain of the process step;

performing the algorithm on each of the plurality of objects that belong to the class;

automatically generating a second class that is part of the class network, wherein the domain designates the second class instead of the class; and

performing the algorithm on each object of the plurality of objects that belong to the second class.

17. The method of claim 16, wherein the generating the data network further comprises:

deleting the class from the class network.

18. The method of claim 1, wherein program instructions stored on a computer-readable medium perform the algorithm on the object that belongs to the class, further comprising, after the generating the data network:

in the specification mode;

specifying a second class of the class network; and

in the execution mode;

re-generating the data network by selectively linking table data values to objects that belong to the second class, wherein the program instructions are not re-compiled before the program instructions perform the algorithm on each of the objects that belong to the second class.

19-49. -49. (canceled)

50. A computer-implemented network structure, comprising:

a data network including a pixel value, an item of metadata, a first link, a first object, a second link and a second object, wherein the pixel value is part of a digital image and the item of metadata relates to the digital image, wherein the first link links the pixel value to the first object, and wherein the second link links the first object to the second object;

a class network including a class, wherein a membership function determines that the first object belongs to the class and that the second object does not belong to the class;

a process hierarchy including a process step, wherein the process step includes a domain specification and an algorithm, wherein the domain specification specifies the class; and

a third link linking the algorithm to the item of metadata, wherein the algorithm is performed based on the item of metadata.

51. The network structure of claim 50, further comprising:

a fourth link linking the domain specification to the second object.

52. The network structure of claim 50, wherein the item of metadata is in the form of text.

53. The network structure of claim 50, further comprising:

a second digital image including a second pixel value, wherein the second pixel value is linked to a third object, wherein the class is linked to a subclass, wherein the class is linked to the first object, and wherein the subclass is linked to the third object.

54. The network structure of claim 51, wherein the item of metadata is patient data taken from the group consisting of:

gender, age, weight, height, blood values, prescribed medications, number of children, history of breastfeeding, family history of ailments, history of smoking and history of drug use.

55. A computer-readable medium comprising program instructions for performing the steps of:

receiving a specification of a class network having a class, wherein a membership function defines a likelihood that an object of a data network belongs to the class;

receiving a specification of a link type that defines a relation between the class and the object;

receiving a specification of a process step having a domain and an algorithm, wherein the domain designates the class, and wherein the process step is part of a process hierarchy;

acquiring table data values; and

performing process steps of the process hierarchy to generate the data network, wherein the data network is generated by generating the object of the data network and by selectively linking selected table data values to the object according to the class network and the process hierarchy.

56. The computer-readable medium of claim 55, wherein the process hierarchy and the class network are depicted in a graphical user interface, and wherein the process hierarchy and the class network can be edited in the graphical user interface.

57. The computer-readable medium of claim 56, wherein the specification of the process step is selected by a user from a library of predefined process steps.

58. The computer-readable medium of claim 55, wherein a first plurality of the table data values are spectral intensity values of a digital image, and wherein a second plurality of the table data values are items of metadata relating to the digital image.

59. The computer-readable medium of claim 58, wherein the digital image is a mammogram.

60. The computer-readable medium of claim 58, wherein the class network, the process hierarchy and the data network together form a semantic network, and wherein the process step is linked to one of the items of metadata.

61. The computer-readable medium of claim 55, wherein the process step performs the algorithm on each object of the class, wherein a user inputs a second membership function at run time that defines whether each object of the data network belongs to a second class of the class network, and wherein the process step is performed by program instructions that are not re-compiled before the program instructions perform the algorithm on each object of the second class.