Automatic disambiguation based on a reference resource

• US 20080208864A1
• Filed: 02/26/2007
• Published: 08/28/2008
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cucerzan, Silviu-Petru, Schultz, Mike
• Granted Patent: US 8,112,402 B2
• Time in Patent Office: Days
• US Class Current: 707/8
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A novel system for automatically indicating the specific identity of ambiguous named entities is provided. An automatic disambiguation data collection is created using a reference resource. Explicit named entities are catalogued from the reference resource, together with various abbreviated, alternative, and casual ways of referring to the named entities. Entity indicators, such as labels and context indicators associated with the named entities in the reference resource, are also catalogued. The automatic disambiguation collection can then be used as a basis for evaluating ambiguous references to named entities in text content provided in different applications. The content surrounding the ambiguous reference may be compared with the entity indicators to find a good match, indicating that the named entity associated with the matching entity indicators is the intended identity of the ambiguous reference, which can be automatically provided to a user.

Claims

1. A method comprising:

identifying a surface form of a named entity in a text, wherein the surface form is associated in a surface form reference with one or more reference named entities, and each of the reference named entities is associated in a named entity reference with one or more entity indicators;

evaluating one or more measures of correlation among one or more of the entity indicators, and the text;

identifying one of the reference named entities for which the associated entity indicators have a relatively high correlation to the text; and

providing a disambiguation output that indicates the identified reference named entity to be associated with the surface form of the named entity in the text.

2. The method of claim 1, wherein the entity indicators comprise one or more labels applied to a respective named entity in an information resource.

3. The method of claim 2, wherein the one or more labels comprise at least one element selected from among:

a tag;

a category;

a heading;

a title;

a section name; and

a list name.

4. The method of claim 1, wherein the entity indicators comprise one or more context indicators applied to a respective named entity in an information resource.

5. The method of claim 4, wherein the context indicators comprise at least one element selected from among:

an attribute included in a title of a document about the respective named entity in the information resource;

an attribute appearing proximate to the respective named entity in the information resource;

an attribute that links to a document about the respective named entity in the information resource; and

an attribute that is linked from a document about the respective named entity in the information resource.

6. The method of claim 1, further comprising:

extracting a set of surface forms and entity indicators associated with a plurality of named entities from one or more information resources; and

storing the surface forms and named entities in a surface form reference, comprising a data collection indexed by the surface forms and indicating the named entities associated with each of the surface forms; and

storing the named entities and entity indicators in a named entity reference, comprising a data collection indexed by the named entities and containing the entity indicators associated with each of the named entities;

wherein the surface form of a named entity in a text is identified from among the surface forms stored in the surface form reference, and the one or more measures of correlation are evaluated between the text and the extracted entity indicators stored in the named entity reference.

7. The method of claim 6, wherein the one or more information resources are selected from among a group consisting of:

a website;

a related group of websites;

a web publication;

an encyclopedia;

a directory;

an atlas; and

a citation index.

8. The method of claim 6, wherein the surface forms of a respective named entity are extracted from at least one element from among a group consisting of:

a title of a document about the respective named entity;

a title of a document that redirects to the document about the respective named entity;

a title of a document that disambiguates a surface form to the respective named entity as one possible disambiguation; and

the text of a link to a document about the respective named entity.

9. The method of claim 6, wherein the extracted entity indicators comprise context indicators extracted from at least one of the information resources, wherein the context indicators comprise elements of text associated with the respective named entities, in at least one of the information resources.

10. The method of claim 6, wherein the extracted entity indicators comprise labels extracted from at least one of the information resources, wherein the labels comprise classifying identifiers applied to the respective named entities in at least one of the information resources.

11. The method of claim 1, further comprising:

identifying a plurality of surface forms that overlap within the text;

identifying one of the overlapping surface forms that is associated with one of the reference named entities in the surface form reference, for which the associated entity indicators have a relatively high correlation to the text; and

providing a disambiguation output that indicates the identified surface form and the reference named entity associated with the identified surface form.

12. The method of claim 1, further comprising receiving a user input indicating the text comprising the surface form for which a corresponding reference named entity is to be identified.

13. The method of claim 1, further comprising receiving an indication, from a text-associated application, of a block of text associated with the application, and providing one or more disambiguation outputs that indicate identified reference named entities to be associated with one or more surface forms within the block of text.

14. The method of claim 13, wherein the text-associated application comprises at least one application selected from among a group of options consisting of:

a word processing application, a presentation application, a spreadsheet application, a database application, an email application, an instant messaging application, an information manager application, a web page application, a web page editing application, a reference work application, a fixed-layout document application, a desktop publishing application, a diagramming application, a project management application, a form developer application, a customer relationship management application, an enterprise resource planning application, a field services application, a retail management application, a social networking application, a tax preparation application, a finance management application, an accounting application, and a game application.

15. The method of claim 1, wherein evaluating the one or more measures of similarity comprises applying different weights to at least one of:

different types of the entity indicators;

proximity of the entity indicators to the surface form of the named entity within the text;

number of common labels;

or number of links among each other of entity indicators extracted from documents that link to or are linked from a document about an associated reference named entity.

16. The method of claim 1, wherein evaluating the one or more measures of similarity comprises using at least one element from a group consisting of:

a vector space model;

cosine similarity;

Jensen-Shannon divergence;

Kullback-Liebler divergence; and

mutual information.

17. The method of claim 1, wherein the text to be evaluated for disambiguation of surface forms may be selected from among a group of optional units of text consisting of:

a document, a page, a paragraph, a sentence, and a manually selected portion of text.

18. The method of claim 17, further comprising first evaluating the one or more measures of similarity between a larger one of the units of text and the one or more entity indicators, and if more than one of the reference named entities is identified as having associated entity indicators with a relatively high correlation to the entity indicators in the text, then evaluating the one or more measures of correlation between an iteratively smaller one of the units of text and the one or more entity indicators, until a unique reference named entity is identified as having associated entity indicators with a relatively high correlation to the entity indicators in the text.

19. A computer-readable medium comprising computer-executable instructions which, when executed by a computing device, enable the computing device to prepare and apply an automatic disambiguation system, comprising steps of:

extracting a collection of surface forms associated with named entities from an information resource;

extracting a collection of labels associated with the named entities from the information resource;

extracting a collection of context indicators associated with the named entities from the information resource;

when provided with a surface form in a text sample, evaluating a measure of correlation of entity indicators associated with the surface form in the text sample with the labels and the context indicators associated with the named entities associated with the surface form in the collection of surface forms; and

providing an output, based on the measure of correlation, indicating one of the named entities to be a disambiguation of the surface form in the text sample.

20. A computer-readable medium comprising one or more databases, the one or more databases comprising a collection of reference surface forms and a collection of named entities, wherein each of the reference surface forms is associated with one or more of the named entities, and each of the named entities is associated with one or more entity indicators extracted from one or more reference works in which the one or more entity indicators are associated with the named entities;

wherein the computer-readable medium further comprises computer-executable instructions that, when executed by a computing device, configure the computing device to disambiguate one or more polysemic words in a sample of text, comprising steps of;

identifying a respective polysemic word with one of the reference surface forms;

evaluating a similarity measure between the entity indicators associated with the named entities associated with the reference surface forms, and entity indicators associated with the respective polysemic word in the sample of text;

identifying one of the named entities as having a relatively high similarity measure between its associated entity indicators and the entity indicators associated with the respective polysemic word; and

providing a disambiguation output indicating the polysemic word to be associated with the identified named entity.