AUTOMATICALLY LINKING DOCUMENTS WITH RELEVANT STRUCTURED INFORMATION

• US 20110131216A1
• Filed: 02/01/2011
• Published: 06/02/2011
• Est. Priority Date: 09/08/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chakravarthy, Venkat, Roy, Prasan, Mohania, Mukesh Kumar, Gupta, Himanshu
• Granted Patent: US 8,126,892 B2
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A method of associating a given text document with relevant structured data is disclosed. The method receives as inputs a text document, and structured data in the form of a relational database. The method then identifies terms in the text document, and searches and queries the structured data using the terms to identify fragments of the structured data that are relevant to the document. Finally, the text document and the identified fragments of structured data are output to a user.

Claims

1. A computer-implemented method of associating a text document with relevant structured data, said method comprising:

receiving, by a computer, a text document;

receiving, by said computer, structured data in the form of entities from a relational database;

simultaneously identifying, by said computer, segments in said text document related to said entities in said structured data, and linking each identified segment in said text document with a most relevant entity in said structured data; and

outputting, by said computer, said identified segments of said text document and associated entities in said structured data.

2. The method according to claim 1, wherein said simultaneously identifying segments and linking said each identified segment comprises:

identifying a set of entities from said relational database;

identifying context information associated with each of said set of entities;

parsing said text document to identify a set of terms, where each of said set of terms is a noun phrase found in a segment including context information from said text document; and

searching and querying said structured data based on said identified set of terms to find an annotation that, based upon a predefined criterion, has a best score among all annotations, where an annotation consists of a set of entities, non-overlapping segments of said text document, and assignment of an entity in said set of entities to each segment.

3. The method according to claim 2, wherein said searching and querying said structured data comprises:

calculating a score according to said predefined criterion for each annotation, wherein said score is based on said terms found in said segments and said terms contained in said context information for said entity associated with said segments in the annotation; and

finding said annotation with a maximum score.

4. The method according to claim 2, wherein identifying a set of entities and identifying context information comprise:

receiving one or more entity templates, where each entity template is a rooted tree with a designated root node, each node in said rooted tree is labeled with a table from said relational database, an edge linking two nodes exists when said tables labeling each of said two linked nodes have a foreign-key relationship in a relational schema of said relational database, each tuple in said relational schema labeling said root node of any template constitutes an entity, and for an entity in an entity template, said context information of said entity consists of said tuples that have a path to said entity through one or more foreign-keys covered by said entity template.

5. The method according to claim 3, wherein finding said annotation with said maximum score comprises:

maintaining a context cache comprising containment relationships between entities and terms, where each containment relationship is an entity-term pair such that said term is contained in said context information of said entity;

based on said containment relationships present in said context cache, issuing a query to said relational database based on contents of said context cache to identify further containment relationships between said entities and said terms;

updating said context cache with results from said query;

finding a current best annotation based on said context cache; and

repeating;

issuing a query, updating said context cache, and finding a current best annotation, until said annotation having said maximum score is found.

6. A method of associating a text document with relevant structured data comprising:

receiving, using a computer, a text document;

receiving, using said computer, structured data in the form of entities from a relational database;

simultaneously, using said computer, identifying segments in said text document related to said entities in said structured data, and linking each identified segment in said text document with a most relevant entity in said structured data; and

outputting, using said computer, said identified segments of said text document and associated entities in said structured data.

7. The method according to claim 6, wherein said simultaneously identifying segments and linking said each identified segment of said method comprises:

identifying a set of entities from said relational database;

identifying context information associated with each of said set of entities;

parsing said text document to identify a set of terms, where each of said set of terms is a noun phrase found in a segment including context information from said text document; and

searching and querying said structured data based on said identified set of terms to find an annotation that, based upon a predefined criterion, has a best score among all annotations, where an annotation consists of a set of entities, non-overlapping segments of said text document, and assignment of an entity in said set of entities to each segment.

8. The method according to claim 7, wherein said searching and querying said structured data of said method comprises:

calculating a score according to said predefined criterion for each annotation, wherein said score is based on said terms found in said segments and said terms contained in said context information for said entity associated with said segments in the annotation; and

finding said annotation with a maximum score.

9. The method according to claim 8, wherein finding said annotation with said maximum score of said method comprises:

maintaining a context cache comprising containment relationships between entities and terms, where each containment relationship is an entity-term pair such that said term is contained in said context information of said entity;

based on said containment relationships present in said context cache, issuing a query to said relational database based on contents of said context cache to identify further containment relationships between said entities and said terms;

updating said context cache with results from said query;

finding a current best annotation based on said context cache; and

repeating;

issuing a query, updating said context cache, and finding a current best annotation, until said annotation having said maximum score is found.

10. The method according to claim 7, wherein identifying a set of entities and identifying context information of the method comprises:

receiving one or more entity templates, where each entity template is a rooted tree with a designated root node, each node in said rooted tree is labeled with a table from said relational database, an edge linking two nodes exists when said tables labeling each of said two linked nodes have a foreign-key relationship in a relational schema of said relational database, each tuple in said relational schema labeling said root node of any template constitutes an entity, and for an entity in an entity template, said context information of said entity consists of said tuples that have a path to said entity through one or more foreign-keys covered by said entity template.

11. A computer-implemented method of partitioning a text document into segments with respect to structured data, said method comprising:

receiving, by a computer, said text document;

receiving, by said computer, said structured data from a relational database;

identifying, by said computer, terms in said text document;

identifying, by said computer, a set of entities from said relational database;

receiving, by said computer, context information, comprising said terms, associated with said set of entities;

identifying, by said computer, said entities that are most relevant to segments within said text document by using said context information and said terms in said text document;

finding, by said computer, an annotation that is most relevant to said text document among all annotations, where an annotation consists of a set of entities, non-overlapping segments of said text document, and assignment of an entity in said set of entities to each segment; and

outputting, by said computer, said segments of said text document and said identified entities that partition said text document.

12. The method according to claim 11, further comprising:

said simultaneously identifying segments and linking said each identified segment comprises;

identifying a set of entities from said relational database;

identifying context information associated with each of said set of entities;

parsing said text document to identify a set of terms, where each of said set of terms is a noun phrase found in a segment including context information from said text document; and

searching and querying said structured data based on said identified set of terms to find an annotation that, based upon a predefined criterion, has a best score among all annotations, where an annotation consists of a set of entities, non-overlapping segments of said text document, and assignment of an entity in said set of entities to each segment.

13. The method according to claim 11, wherein said searching and querying said structured data comprises:

calculating a score according to said predefined criterion for each annotation, wherein said score is based on said terms found in said segments and said terms contained in said context information for said entity associated with said segments in the annotation; and

finding said annotation with a maximum score.

14. The method according to claim 11, wherein identifying a set of entities and identifying context information comprise:

receiving one or more entity templates, where each entity template is a rooted tree with a designated root node, each node in said rooted tree is labeled with a table from said relational database, an edge linking two nodes exists when said tables labeling each of said two linked nodes have a foreign-key relationship in a relational schema of said relational database, each tuple in said relational schema labeling said root node of any template constitutes an entity, and for an entity in an entity template, said context information of said entity consists of said tuples that have a path to said entity through one or more foreign-keys covered by said entity template.

15. The method according to claim 11, wherein finding said annotation with said maximum score comprises:

maintaining a context cache comprising containment relationships between entities and terms, where each containment relationship is an entity-term pair such that said term is contained in said context information of said entity;

based on said containment relationships present in said context cache, issuing a query to said relational database based on contents of said context cache to identify further containment relationships between said entities and said terms;

updating said context cache with results from said query;

finding a current best annotation based on said context cache; and

repeating;

issuing a query, updating said context cache, and finding a current best annotation, until said annotation having said maximum score is found.