System and methods for extracting semantics from images

• US 20030219149A1
• Filed: 05/22/2002
• Published: 11/27/2003
• Est. Priority Date: 05/22/2002
• Status: Active Grant

• Current Assignee: Agilent Technologies, Inc.
• Original Assignee: Agilent Technologies, Inc.
• Inventors: Vailaya, Aditya, Kuchinsky, Allan, Adler, Annette Marie
• Granted Patent: US 7,146,030 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: G16B 45/00 ICT specially adapted for bioinformatics-related data visualisation, e.g. displaying of maps or networks
• CPC Class Codes: G16B 5/00 ICT specially adapted for modelling or simulations in systems biology, e.g. gene-regulatory networks, protein interaction networks or metabolic networks
• CPC Class Codes: G16B 5/10 Boolean models

Abstract

System, tools and methods for extracting the contents from a graphical image, such as the representation of a biological process, and converting the contents into a standardized, machine readable format. The standard format can be used to create editable graphical representations of images, such as biological models.

Claims

1. A method of extracting semantics from a static graphic image of a biological model and for converting the static image to an editable biological model, said method comprising the steps of:

accessing stored information defining predefined constraints used to create the static image;

identifying at least one entity or relationship represented in the static image, based upon the stored information;

extracting information describing the at least one entity or relationship; and

converting at least a portion of the biological model to an editable, local format using the extracted information.

2. The method of claim 1, wherein the predefined constraints are site-specific vocabulary.

3. The method of claim 1, wherein the predefined constraints include assignments of different colors to entities and relationships, and said identifying includes color processing of the static image.

4. The method of claim 1, wherein the predefined constraints include assignments of different shapes to entities and relationships, and said identifying includes template matching.

5. The method of claim 1, wherein the static image is digitally created or scanned from a paper source, said method further comprising the step of converting the static image into a standard format prior to the identifying and extracting steps.

6. The method of claim 5, wherein the standard format is selected from the group consisting of JPEG, PBM, GIF, PPM, TIFF and Bitmap.

7. The method of claim 1, further comprising the steps of:

accessing an image map used to create the static image;

identifying at least one entity or relationship represented in the static image, based upon information in the image map;

extracting information describing the at least one entity or relationship identified based upon the image map information; and

converting at least a portion of the biological model to the editable, local format using the information extracted based upon the image map information.

8. The method of claim 7 wherein the image map comprises HTML.

9. The method of claim 1, further comprising the step of manually converting at least a portion of the biological model to an editable, local format.

10. A method of extracting semantics from a static graphic image of a biological model and for converting the static image to an editable biological model, said method comprising the steps of:

accessing an image map used to create the static image;

identifying at least one entity or relationship represented in the static image, base upon information in the image map;

extracting information describing the at least one entity or relationship; and

converting at least a portion of the biological model to the editable, local format using the extracted information.

11. The method of claim 10, wherein the image map comprises HTML.

12. The method of claim 10, wherein the static image is digitally created or scanned from a paper source, said method further comprising the step of converting the static image into a standard format prior to the identifying and extracting steps.

13. The method of claim 12, wherein the standard format is selected from the group consisting of JPEG, PBM, GIF, TIFF, PPM, and Bitmap.

14. The method of claim 10, further comprising the steps of:

accessing stored information defining predefined constraints used to create the static image;

identifying at least one entity or relationship represented in the static image, baseD upon the stored information;

extracting information describing the at least one entity or relationship, based upon the stored information; and

converting at least a portion of the biological model to an editable, local format using the information that was extracted based upon the stored information.

15. The method of claim 10, further comprising the step of manually converting at least a portion of the biological model to an editable, local format.

16. A method of extracting semantics from a static graphic image of a biological model and for converting the static image to an editable biological model, said method comprising the steps of:

converting the static image to a standard format;

extracting information from the standard formatted static image based using image processing based upon content defining specific constraints on the static image; and

mapping the extracted information according to a local format, thereby making an editable map.

17. The method of claim 16, wherein the image processing used in the extracting step includes one or more of the techniques selected from the group consisting of:

color processing, shape processing, morphological analysis with open operators, morphological analysis with closed operators, connected component analysis, edge detection, detecting geometrical shapes, template matching and text detection.

18. The method of claim 16, wherein the method is applied to the entire static image.

19. The method of claim 16, wherein the method is applied to manually preselected regions of the static image.

20. The method of claim 16, wherein the method is applied to automatically preselected regions of the static image.

21. The method of claim 20, wherein the automatically preselected regions are selected based upon at least one of color-processing, shape-processing, and connected component analysis.

22. The method of claim 16, wherein the content defining specific constraints on the static image is site-specific content.

23. The method of claim 16, further comprising the steps of:

extracting other information relating to the static image;

converting the other information to the local format; and

mapping at least a portion of the locally formatted information on the editable map or linking the other information in the local format to the editable map.

24. The method of claim 23, wherein the other information comprises HTML tags on the static image map.

25. The method of claim 23, wherein the other information comprises information in an image map used to create the static image.

26. The method of claim 16, further comprising the step of applying OCR to text regions of the static image to convert the text to the local format.

27. A system for extracting semantics from a static graphic image of a biological model and for converting the static image to an editable biological model, said system comprising:

means for accessing stored information defining predefined constraints used to create the static image;

means for identifying at least one entity or relationship represented in the static image, based upon the stored information;

means for extracting information describing the at least one entity or relationship; and

means for converting at least a portion of the biological model to an editable, local format using the extracted information.

28. The system of claim 27, further comprising:

means for converting the static image into a standard format prior to extracting semantics therefrom.

29. The system of claim 27, further comprising:

means for accessing an image map used to create the static image;

means for identifying at least one entity or relationship represented in the static image, based upon information in the image map;

means for extracting information describing the at least one entity or relationship identified based upon the image map information; and

means for converting at least a portion of the biological model to the editable, local format using the information extracted based upon the image map information.

30. The method of claim 27, further comprising means for manually converting at least a portion of the biological model to an editable, local format.

31. A system for extracting semantics from a static graphic image of a biological model and for converting the static image to an editable biological model, said system comprising:

a database-specific content extraction module adapted to extract content from the static image based on predefined constraints specific to a database from which the static image originated; and

a conversion module adapted to convert the extracted content to a local format used to create an editable biological map corresponding to the static image.

32. The system of claim 31, further comprising an image mapping module adapted to extract content in or relating to the static image, based upon an image map, and input the extracted content from or relating to the static image to the conversion module.

33. The system of claim 31, further comprising an image pre-processing module adapted to convert the static image to a standard format, prior to further processing by the system.

34. A computer readable medium carrying one or more sequences of instructions from a user of a computer system for identifying data relevant to the context of a user'"'"'s local data, wherein the execution of the one or more sequences of instructions by one or more processors cause the one or more processors to perform the steps of:

accessing stored information defining predefined constraints used to create the static image;

identifying at least one entity or relationship represented in the static image, based upon the stored information;

extracting information describing the at least one entity or relationship; and

converting at least a portion of the biological model to an editable, local format using the extracted information.

35. The computer readable medium of claim 34, wherein the static image is digitally created or scanned from a paper source, wherein the following further step is performed:

converting the static image into a standard format prior to the identifying and extracting steps.

36. The computer readable medium of claim 34, wherein the following further steps are performed:

accessing an image map used to create the static image;

identifying at least one entity or relationship represented in the static image, based upon information in the image map;

extracting information describing the at least one entity or relationship identified based upon the image map information; and

converting at least a portion of the biological model to the editable, local format using the information extracted based upon the image map information.

37. A computer readable medium carrying one or more sequences of instructions from a user of a computer system for identifying data relevant to the context of a user'"'"'s local data, wherein the execution of the one or more sequences of instructions by one or more processors cause the one or more processors to perform the steps of:

accessing an image map used to create the static image;

identifying at least one entity or relationship represented in the static image, base upon information in the image map;

extracting information describing the at least one entity or relationship; and

converting at least a portion of the biological model to the editable, local format using the extracted information.