AUTOMATIC DISAMBIGUATION BASED ON A REFERENCE RESOURCE

• US 20120102045A1
• Filed: 01/03/2012
• Published: 04/26/2012
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cucerzan, Silviu-Petru, Schultz, Mike
• Granted Patent: US 9,772,992 B2
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A novel system for automatically indicating the specific identity of ambiguous named entities is provided. An automatic disambiguation data collection is created using a reference resource. Explicit named entities are catalogued from the reference resource, together with various abbreviated, alternative, and casual ways of referring to the named entities. Entity indicators, such as labels and context indicators associated with the named entities in the reference resource, are also catalogued. The automatic disambiguation collection can then be used as a basis for evaluating ambiguous references to named entities in text content provided in different applications. The content surrounding the ambiguous reference may be compared with the entity indicators to find a good match, indicating that the named entity associated with the matching entity indicators is the intended identity of the ambiguous reference, which can be automatically provided to a user.

Claims

1. A computer-implemented disambiguation system comprising:

a surface forms database having a plurality of surface form records, each of the surface form records corresponding to a surface form that is an ambiguous orthographic representation of a common name for an entity, each of the surface form records having an indication of the corresponding surface form and indications of named entities that are associated with the surface form;

a named entities database have a plurality of named entity records, each of the named entity records corresponding to one of the named entities, each of the named entity records having labels representative of the named entity and contextual information associated with the named entity; and

a computer processor that identifies one of the surface forms in a text, the computer processor evaluating the identified one of the surface forms against the labels and the contextual information to determine which one of the named entities is most associated with the one of the surface forms.

2. The system of claim 1, further comprising a display showing a representation of the text, the display including an indication of the one of the named entities that is most associated with the one of the surface forms proximate to the one of the surface forms in the text.

3. The system of claim 2, wherein the computer processor pre-processes the text to find boundaries of the one of the surface forms.

4. The disambiguation system of claim 2 wherein the computer processor identifies one of the surface forms in the text by identifying a plurality of surface forms that overlap within the text.

5. The disambiguation system of claim 4 wherein the computer processor identifies one of the overlapping surface forms that is associated with one of the surface form records in the surface forms database, for which the associated named entity indications have a relatively high correlation to the text.

6. The disambiguation system of claim 5 wherein the computer processor displays, on the display, a disambiguation output that indicates the surface form in the identified surface form record and the named entity indications associated with the identified surface form record for which the labels and contextual information in the named entities database have a highest degree of association.

7. The disambiguation system of claim 1, wherein the computer processor receives a user input indicating the text comprising the surface form for which a corresponding named entity is to be identified.

8. The disambiguation system of claim 1 wherein the computer processor evaluates the identified one of the surface forms against the labels and contextual information using one or more measures of similarity.

9. The disambiguation system of claim 8 wherein the one or more measures of similarity includes a measure that applies different weights to at least one of:

different types of the entity indications;

proximity of the entity indications to the surface form of the named entity within the text;

number of common labels;

or number of links among each other of entity indications extracted from documents that link to or are linked from a document about an associated named entity.

10. The disambiguation system of claim 9 wherein the one or more measures of similarity comprises at least one element from a group consisting of:

a vector space model;

Jensen-Shannon divergence;

Kullback-Liebler divergence; and

mutual information.

11. The disambiguation system of claim 1, wherein the text to be evaluated for disambiguation of surface forms is selected from among a group of optional units of text.

12. The disambiguation system of claim 11 wherein the group of optional units of text comprise one of:

a document, a page, a paragraph, a sentence, and a manually selected portion of text.

13. The disambiguation system of claim 12, wherein the computer processor identifies the one of the surface forms by first evaluating one or more measures of similarity between a larger one of the optional units of text and the one or more entity indications.

14. The disambiguation system of claim 13 wherein the computer processor identifies the one of the surface forms by, if more than one of the named entities associated with the surface form is identified as having associated entity indications with a relatively high correlation to the entity indications in the text, then evaluating the one or more measures of correlation between an iteratively smaller one of the optional units of text and the one or more entity indications, until a unique named entity is identified as having associated entity indications with a relatively high correlation to the entity indications in the text.

15. The disambiguation system of claim 1, wherein the computer processor receives the text by receiving an indication, from a text-associated application, of a block of text associated with the application.

16. The disambiguation system of claim 15 wherein the computer processor provides one or more disambiguation outputs on a display that indicate identified named entities to be associated with one or more surface forms within the block of text.

17. A computer-readable storage medium comprising one or more databases, the one or more databases comprising a collection of reference surface forms and a collection of named entities, wherein each of the reference surface forms is associated with one or more of the named entities that are different from the associated surface form, and each of the named entities is associated with one or more entity indicators extracted from one or more reference works in which the one or more entity indicators are associated with the named entities;

wherein the computer-readable medium further comprises computer-executable instructions that, when executed by a computing device having a processor, configure the computing device to disambiguate one or more polysemic words in a sample of text, comprising steps of;

identifying, with the processor, a plurality of overlapping polysemic words in the sample of text each having one of the reference surface forms;

evaluating, with the processor a similarity measure between the entity indicators associated with the named entities associated with the reference surface forms, and entity indicators associated with each of the overlapping polysemic words in the sample of text;

identifying, with the processor, one of the named entities as having a relatively high similarity measure between its associated entity indicators and the entity indicators associated with one of the overlapping polysemic words; and

providing a disambiguation output, with the processor, indicating the one of the polysemic words to be associated with the identified named entity.

18. The computer-readable storage medium of claim 17 wherein providing a disambiguation output comprises:

displaying the sample of text; and

displaying an indication of an identified named entity to be associated with the one of the polysemic words.

19. The computer readable storage medium of claim 18 wherein displaying the indication comprises:

displaying the identified named entity closely proximate the one of the polysemic words in the text.

20. The computer readable storage medium of claim 19 wherein displaying the identified named entity comprises:

displaying an element visually connecting the identified named entity to the one of the polysemic words.