Reference resolution for text enrichment and normalization in mining mixed data

• US 20080027893A1
• Filed: 07/26/2006
• Published: 01/31/2008
• Est. Priority Date: 07/26/2006
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Renders, Jean-Michel, Cavestro, Bruno
• Granted Patent: US 8,595,245 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/94 Hypermedia Hyperlinking G06F40/134

Abstract

A method for enrichment of text which enables mixed data mining includes generating a model for structured data found in tables of a database. In the model, semantically-linked terms are associated with referents, such as field names or cell content of the fields, of the structured data. The referents may be a business object or refer to a business object. A plurality of candidate referring entities in textual data in the database, such as chunks of free text, is identified. For each candidate referring entity, a similarity measure between the candidate referring entity in the textual data and the model is computed to identify referring entities of the candidate referring entities and corresponding business objects/referents to which the referring entities refer. The textual data is enriched with information derived from the business objects.

Claims

1. a method for enrichment of text comprising:

generating a model for structured data in a database, comprising associating referents in the database with designating terms which each describe an object, the referents each comprising or referring to one of the objects;

identifying a plurality of candidate referring entities in textual data in the database;

for each candidate referring entity, computing a similarity measure which includes comparing the candidate referring entity in the textual data with the model to identify referring entities of the candidate referring entities and corresponding objects to which the referring entities refer; and

enriching the textual data with information derived from the objects.

2. The method of claim 1, wherein the objects comprise business objects.

3. The method of claim 1, wherein the referents comprise contents of fields for the structured data.

4. The method of claim 1, wherein the generation of the model includes for each of the referents, identifying designating terms which are in a semantic relationship with the referent.

5. The method of claim 4, wherein the identification of designating terms which are in a semantic relationship with the referent comprises accessing a lexical resource.

6. The method of claim 1, wherein one of the designating terms associated with each of the referents comprises a normalized unique identifier of the object.

7. The method of claim 1, wherein the identifying of candidate referring entities in the textual data comprises identifying noun phrases.

8. The method of claim 1, wherein the identifying of candidate referring entities comprises identifying a normalized form of the candidate referring entity.

9. The method of claim 1, wherein the identifying of candidate referring entities comprises at least one of:

expanding the textual data with external information relating to the textual data; and

co-reference resolution within the textual data or expanded textual data.

10. The method of claim 1, wherein the associating of designating terms with the referents comprises, for each referent, associating a plurality of designating terms with the referent, each of the designating terms having a weight.

11. The method of claim 10, wherein the computing of the similarity measure comprises computing the similarity measure between a candidate referring entity and a referent as a function of the weight of the each of designating terms.

12. The method of claim 1, wherein the computing of the similarity measure comprises computing at least one of a string kernel value and a minimum edit distance between a candidate referring entity and a designating term of the model.

13. The method of claim 1, wherein the computing of the similarity measure further includes comparing the candidate referring entity in the textual data with structured data of the same record of a table or with structured data of another record linked to that record.

14. The method of claim 1, wherein the enrichment comprises enriching the textual data with a normalized identifier of the object to which the identified referent refers.

15. The method of claim 14, wherein the normalized identifier comprises a name and an associated role or function.

16. The method of claim 1, wherein the enrichment comprises annotating the referring entities with a tag or additional text.

17. The method of claim 1, further comprising enriching the structured data with information derived from the object to which the referent refers.

18. A system comprising:

a database comprising records stored in memory which include structured data arranged in records comprising fields of structured data and textual data in fields of textual data, the textual data comprising annotations which identify business objects referred to by the structured data, developed by the method of claim 1; and

a processor which executes instructions for querying the database to analyze text from the textual data and structured data.

19. The system of claim 18, wherein the analysis of structured and unstructured data comprises retrieving text responsive to a query

20. A method of retrieving text responsive to a query comprising:

inputting a query;

retrieving information responsive to the query from stored structured and textual data, the textual data having been enriched according to the method of claim 1.

21. A system comprising:

a database comprising records stored in memory which include records each comprising structured data arranged in fields of structured data and textual data in fields of textual data, at least some of the fields of structured data comprising referents which refer to business objects;

a processor which annotates the textual data with annotations which identify business objects referred to by the referents of the structured data by computing a similarity measure between textual chunks of the textual data and a business model associated with the structured data.

22. A method comprising:

associating designating terms with referents in fields of a database table, the database table comprising a plurality of records for which the fields include structured data, the referents each comprising or referring to a business object;

identifying candidate referring entities in portions of textual data, the textual data portions being in textual data fields of the database table or linked thereto whereby a textual data portion is associated with fewer than all of the plurality of records;

for each of a plurality of candidate referring entities;

computing a similarity measure between the candidate referring entity and the designating terms associated with the referents of the same record of the database table; and

where the computed similarity measure exceeds a threshold, enriching the textual data with information derived from the business object for the referent.

23. The method of claim 22, further comprising:

retrieving text from the enriched textual data and structured data which is responsive to a query.