NLP-based entity recognition and disambiguation

• US 20090144609A1
• Filed: 10/15/2008
• Published: 06/04/2009
• Est. Priority Date: 10/17/2007
• Status: Active Grant

• Current Assignee: Fiver LLC
• Original Assignee: VCVC III LLC (Vulcan, Inc.)
• Inventors: Bhatti, Satish, Liang, Jisheng, Dhillon, Navdeep S., Koperski, Krzysztof, Tusk, Carsten
• Granted Patent: US 8,594,996 B2
• Time in Patent Office: Days
• US Class Current: 715/230
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

Methods and systems for entity recognition and disambiguation using natural language processing techniques are provided. Example embodiments provide an entity recognition and disambiguation system (ERDS) and process that, based upon input of a text segment, automatically determines which entities are being referred to by the text using both natural language processing techniques and analysis of information gleaned from contextual data in the surrounding text. In at least some embodiments, supplemental or related information that can be used to assist in the recognition and/or disambiguation process can be retrieved from knowledge repositories such as an ontology knowledge base. In one embodiment, the ERDS comprises a linguistic analysis engine, a knowledge analysis engine, and a disambiguation engine that cooperate to identify candidate entities from a knowledge repository and determine which of the candidates best matches the one or more detected entities in a text segment using context information.

Claims

1. A computer-implemented method for identifying one or more entities in an indicated text segment, comprising:

processing the indicated text segment to determine a plurality of terms and their associated parts-of-speech tags and grammatical roles;

performing linguistic analysis of the processed text segment to determine one or more potential entity names which are referred to in the text segment;

generating, for each potential entity name, an entity profile having one or more associated properties that characterize the entity based upon surrounding context; and

determining one or more mostly likely entities which are referred to in the text segment by comparing the entity profiles generated for each potential entity name with one or more candidate entities using both linguistic and contextual information.

2. The method of claim 1 wherein the candidate entities are entity entries retrieved from a knowledge repository.

3. The method of claim 2 wherein the knowledge repository is an ontology knowledge base.

4. The method of claim 1 wherein the determining of the one or more mostly likely entities which are referred to in the text segment by comparing the entity profiles generated for each potential entity name with one or more candidate entities further comprises:

searching a knowledge repository for a set of candidate entities that have similar characteristics to one or more of the generated entity profiles;

ranking the candidate entities in the set of candidate entities to determine a set of mostly likely entities which are referred to in the text segment; and

providing the determined set of mostly likely entities.

5. The method of claim 4 wherein the ranking weights the candidate entities according to contextual information surrounding portions of the text segment that refer to the potential entity names.

6. The method of claim 4 wherein the ranking weights the candidate entities according to preference information.

7. The method of claim 4 wherein the ranking the candidate entities further comprises using a classification model to classify the candidate entities and to order them based upon closest matches.

8. The method of claim 1 wherein the determining the one or more most likely entities which are referred to in the text segment by comparing the entity profiles generated for each potential entity name with one or more candidate entities using both linguistic and contextual information further comprises:

resolving the one or more most likely entities to a single identified entity by performing iterative comparisons reusing entity recognition information gained from a prior comparison until no new entity recognition information is gained.

9. The method of claim 1 wherein each entity profile comprises a feature vector of terms collected from modifiers and/or actions associated with the potential entity name based upon the linguistic analysis of the processed text segment.

10. The method of claim 8 wherein the terms are weighted based upon at least one of a binary weight or a frequency weight.

11. The method of claim 1, further comprising providing a ranked list of the determined one or more most likely entities which are referred to in the text segment.

12. The method of claim 1, further comprising using the determined one or more likely entities to inform a relationship search.

13. The method of claim 1, further comprising:

performing the method in response to selection of a user interface component.

14. The method of claim 1 wherein the method is embedded in code that supports a widget presented on a web page.

15. The method of claim 1, further comprising:

Invoking the method to annotate information on a web page.

16. A computer-readable medium containing contents that, when executed causes a computing system to perform a method comprising:

processing the indicated text segment to determine a plurality of terms and their associated parts-of-speech tags and grammatical roles;

performing linguistic analysis of the processed text segment to determine one or more potential entity names which are referred to in the text segment;

generating, for each potential entity name, an entity profile having one or more associated properties that characterize the entity based upon surrounding context; and

automatically determining one or more mostly likely entities which are referred to in the text segment by comparing the entity profiles generated for each potential entity name with one or more candidate entities from a knowledge base using both linguistic and contextual information.

17. The computer-readable medium of claim 16, wherein the medium is a memory in a computing system containing the contents.

18. The computer-readable medium of claim 16 wherein the contents are instructions that, when executed, cause a computer processor in the computing device to perform the method.

19. The computer-readable medium of claim 16 embedded in a computing system configured to perform indexing and storing of a corpus of documents for searching using natural language processing.

20. An entity recognition and disambiguation computing system, comprising;

a memory; and

a recognition and disambiguation module stored in the memory that is configured, when executed, to

receive a text segment for processing;

recognize one or more candidate named entities which are referred to by a detected entity in a received text segment based, at least in part, upon a natural language analysis of the text segment; and

disambiguate the candidate named entities to determine a single named entity to which the detected entity in the received text segment is deemed to refer based upon a combination of linguistic analysis, contextual information gleaned from surrounding text, and information retrieved from one or more knowledge repositories.

21. The system of claim 20, wherein the module is further configured, when executed, to disambiguate the candidate named entities using information based upon a relationship search.

22. The system of claim 20 wherein the module is further configured, when executed, to disambiguate the candidate named entities using a classification modeling approach.

23. A computer-implemented method for presenting information regarding named entities comprising:

receiving an indication of a segment of text;

invoking the recognition and disambiguation module of claim 20 to process the indicated text segment to automatically determine one or more named entities referred to in the text segment; and

for each determined one or more named entities, presenting a link to information associated with the named entity.

24. The method of claim 23 wherein the information is based in part upon an ontology entry.