HYBRID SYSTEM FOR NAMED ENTITY RESOLUTION

• US 20080319978A1
• Filed: 08/29/2007
• Published: 12/25/2008
• Est. Priority Date: 06/22/2007
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Jacquet, Guillaume, Brun, Caroline, Ehrmann, Maud
• Granted Patent: US 8,374,844 B2
• Time in Patent Office: Days
• US Class Current: 707/5
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A method for named entity resolution includes parsing an input text string to identify a context in which an identified named entity of the input text string is used. The identified context is compared with at least one stored context in which the named entity in the stored context is associated with a class of named entity, the named entity class being selected from a plurality of classes, at least one of the plurality of classes corresponding to a metonymic use of a respective named entity. A named entity class is assigned to the identified named entity from the plurality of named entity classes, based on at least one of the identified context and the comparison.

Claims

1. A method for named entity resolution comprising:

parsing an input text string to identify a context in which an identified named entity of the input text string is used;

comparing the identified context with at least one stored context in which the named entity in the stored context is associated with a class of named entity, the named entity class being selected from a plurality of classes, at least one of the plurality of classes corresponding to a metonymic use of a respective named entity;

assigning a named entity class from the plurality of named entity classes to the identified named entity based on at least one of the identified context and the comparison.

2. The method of claim 1, wherein the at least one stored context comprises a plurality of stored contexts and the comparing comprises:

for each of the plurality of stored contexts, computing a distance between the identified context and the stored context;

computing a score for each of the plurality of named entity classes based on the computed distances; and

assigning one of the named entity classes to the identified named entity, based on the computed scores.

3. The method of claim 2, wherein the distance is computed as a function of a difference between a frequency of occurrence of the identified named entity that is in the identified context in a distribution space derived from a training corpus and a frequency of occurrence of a lexical unit that is in the stored context in the distribution space.

4. The method of claim 2, wherein the comparing comprises:

from a stored global distribution space comprising triples, each triple comprising a lexical unit and a context in which the lexical unit is found in a training corpus, at least some of the lexical units being named entities, identifying a sub-space comprising a subset of the triples, each of the triples in the subset comprising a stored context and a lexical unit, the stored context being one that is also found in a triple with the identified named entity in the global distribution space;

for triples in the sub-space, determining whether the named entity in the stored context is associated with a class of named entity selected from the plurality of classes, and if so, assigning the class to the named entity;

and wherein the computing of the distance is computed for stored contexts in the sub-space.

5. The method of claim 1, wherein the parsing includes applying a set of dependency rules to the input text string, each of the dependency rules specifying a syntactic relation between a first lexical unit based on a named entity and a second lexical unit, the rule being satisfied when the relation is present in the input text string.

6. The method of claim 5, wherein the context is based on the syntactic relation and the named entity.

7. The method of claim 1, wherein the class is selected from a set of classes including a literal class and at least one metonymic class.

8. The method of claim 7, wherein the at least one metonymic class comprises at least one location-specific metonymic class and at least one organization-specific metonymic class.

9. The method of claim 8, wherein the at least one organization-specific metonymic class comprises at least two organization-specific classes selected from the group consisting of:

a class in which an organization name stands for its members;

a class in which the organization name refers to an event associated with the organization;

a class in which the organization name refers to its products;

a class in which the organization name stands for the facility that houses the organization;

a class in which the organization name is used as an index indicating its value;

a class in which the name is used as a string; and

a class in which the organization name refers to a representation;

the organization-specific classes optionally further including an additional class for all other types of organization-specific metonymy not otherwise covered.

10. The method of claim 8, wherein the at least one location-specific metonymic class comprises at least two location-specific classes selected from the group consisting of:

a class in which a location name stands for persons or an organization associated with it;

a class in which the location name stands for an event that happened there;

a class in which the location name stands for a product developed there;

a class in which the location name is used as a reference to another name;

a class in which the location name refers to a representation;

the location-specific classes optionally further including an additional class for all other types of location-specific metonymy not otherwise covered.

11. The method of claim 1, wherein the parsing includes assigning a preliminary class to the named entity selected from a set of preliminary classes including a literal class, at least one metonymic class, and an unknown class.

12. The method of claim 11, wherein when the preliminary class assigned is an unknown class, the assigning of the named entity class from the plurality of named entity classes is based on the comparison.

13. The method of claim 1, further comprising annotating a document in which the text string occurs in accordance with the assigned class.

14. A computer program product encoding instructions which, when executed on a computer, perform the method of claim 1.

15. A hybrid system for performing the method of claim 1, comprising:

a parser for identifying the context;

a distribution component for comparing the identified context with at least one stored context; and

memory storing a data structure which associates lexical units to the contexts in which they are found in a training corpus and their frequency of occurrence.

16. A hybrid system for named entity resolution comprising:

a symbolic component for identifying a context in which an identified named entity of an input test string is used; and

a distribution component for computing a distance between the identified context in which the named entity is being used and another context in which the named entity is used in a known metonymic sense;

a data structure which stores a plurality of contexts;

the system assigning a class to the identified named entity, based on at least one of the identified context and the computed distance.

17. The system of claim 16, wherein the symbolic component comprises a parser which applies a set of dependency rules to the input text string, each of the dependency rules specifying a syntactic relation between a first lexical unit based on a named entity and a second lexical unit, the rule being satisfied when the relation is present in the input text string.

18. The system of claim 17, wherein the context is based on the relation and the named entity.

19. The system of claim 16, wherein the class is selected from a set of classes including a literal class and at least one metonymic class.

20. The system of claim 19, wherein the at least one metonymic class comprises at least one location-specific metonymic class and at least one organization-specific metonymic class.

21. The system of claim 16, wherein the symbolic component assigns a class to the named entity selected from a set of classes including a literal class, at least one metonymic class, and an unknown class.

22. The system of claim 21, wherein when the class assigned by the symbolic component is an unknown class, the class assigned by the system to the identified named entity is based on the computed distance.

23. The system of claim 16, wherein the distribution component assigns a score to the named entity based on the computes the distance between the context in which the named entity is being used and another context in which the named entity is used in a known metonymic sense

24. A method for document annotation comprising:

inputting a document comprising at least one text string;

parsing the text string to identify a context in which an identified named entity of the text string is used;

comparing the identified context with at least one stored context in which the named entity in the stored context is associated with a class of named entity, the named entity class being selected from a plurality of classes, at least one of the plurality of classes corresponding to a metonymic use of a respective named entity;

assigning a named entity class from the plurality of named entity classes to the identified named entity based on at least one of the identified context and the comparison; and

annotating the document based on the assigned class.

25. A document annotated by the method of claim 24.