Entity analysis system

• US 9,202,176 B1
• Filed: 08/08/2011
• Issued: 12/01/2015
• Est. Priority Date: 08/08/2011
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: The Gravity Cue Company Limited
• Inventors: Kapur, Amit R., Pearman, Steven F., Benedetto, James R.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): FINK, THOMAS ANDREW
• Application Number:
  US13/205,585
• Time in Patent Office: 1,576 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A method for building a factual database of concepts and entities that are related to the concepts through a learning process. Training content (e.g., news articles, books) and a set of entities (e.g., Bill Clinton and Barack Obama) that are related to a concept (e.g., Presidents) is received. Groups of words that co-occur frequently in the textual content in conjunction with the entities are identified as templates. Templates may also be identified by analyzing parts-of-speech patterns of the templates. Entities that co-occur frequently in the textual content in conjunction with the templates are identified as additional related entities (e.g., Ronald Reagan and Richard Nixon). To eliminate erroneous results, the identified entities may be presented to a user who removes any false positives. The entities are then stored in association with the concept.

Claims

1. A computer-implemented method of learning related entities, the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more potential word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each potential word template is one or more words, and wherein each potential word template is tagged with a part-of-speech tag based on grammatical use of the one or more words in the training content;

identifying one or more word templates from the one or more potential word templates based on a frequency of occurrence of the one or more potential word templates and based on the part-of-speech tag of the one or more potential word templates compared to part-of-speech tags of word templates of a set of word templates, wherein the one or more identified word templates are added to the set of word templates;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, a confidence score of the identified word template when the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of a set of word templates;

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value;

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity;

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from a part-of-speech tag of the set of entities;

receiving an external input selecting candidate entities for removal if the selected candidate entities do not relate to the first concept from the set of candidate entities;

removing candidate entities from the set of candidate entities based on the received external input;

adding one or more candidate entities remaining in the set of candidate entities to the set of entities; and

storing the set of entities in association with the first concept.

2. The method of claim 1, wherein identifying one or more candidate entities that relate to the first concept further comprises:

removing one or more candidate entities from the set of candidate entities based on whether the one or more candidate entities are related to a second concept,

wherein the second concept does not relate to the first concept.

3. A computer-implemented method of learning related entities, the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each word template is one or more words, and wherein the one or more identified word templates are added to a set of word templates;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, a confidence score of the identified word template when the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of a set of word templates;

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value;

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more identified candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity;

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from the part-of-speech tag of the set of entities;

receiving an external input selecting candidate entities for removal if the selected candidate entities do not relate to the first concept from the set of candidate entities;

removing candidate entities from the set of candidate entities based on the received external input; and

adding one or more candidate entities remaining in the set of candidate entities to the set of entities.

4. The method of claim 3, wherein identifying one or more word templates from the training content further comprises:

tagging each of the one or more words in the training content that occur with the entity of the set of entities with a part-of-speech tag based on grammatical use of the one or more words in the training content that occur with the entity of the set of entities;

identifying the one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities.

5. The method of claim 4, wherein identifying one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities comprises:

identifying the one or more word templates based on similarities between the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities and part-of-speech tags of word templates in the set of word templates.

6. The method of claim 4, wherein identifying one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities comprises:

identifying the one or more word templates based on similarities between the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities in a current iteration and part-of-speech tags of word templates in the set of word templates identified in a previous iteration.

7. The method of claim 3, wherein identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates further comprises:

identifying the one or more candidate entities based a grammatical use of the candidate entities and a part-of-speech tag of the entities of the set of entities.

8. The method of claim 3, wherein learning additional entities that are related to the first concept further comprises:

receiving an external input to remove candidate entities that do not relate to the first concept from the set of candidate entities; and

removing candidate entities from the set of candidate entities based on the received input.

9. The method of claim 8, wherein learning additional entities that are related to the first concept further comprises:

searching an ontology database of related entities and concepts for a second concept that relates to a candidate entity in the set of candidate entities, wherein the ontology database includes the set of entities that relate to the first concept;

determining whether the second concept relates to the first concept; and

removing candidate entities from the set of candidate entities when the second concept of the candidate entity does not relate to the first concept.

10. The method of claim 8, wherein learning additional entities that are related to the first concept further comprises:

searching an ontology database of related entities and concepts for a second concept that relates to a candidate entity in the set of candidate entities, wherein the ontology database includes the set of entities that relate to the first concept;

determining whether the second concept relates to the first concept; and

adding the candidate entities to the set of entities that relates to the first concept when the second concept of the candidate entity relates to the first concept.

11. The method of claim 3, wherein the first concept is a classification of each entity in the set of entities.

12. The method of claim 3, wherein each entity in the set of entities is a proper noun.

13. The method of claim 8, wherein the received input to remove candidate entities from the set of candidate entities is a selection by a user of candidate entities to remove from the set of candidate entities.

14. The method of claim 8, wherein the received input to remove candidate entities from the set of candidate entities is a selection by a user of candidate entities to keep in the set of candidate entities.

15. The method of claim 8, wherein the received input to remove candidate entities from the set of candidate entities is received in fewer than all iterations.

16. The method of claim 3, wherein a word template of the set of word templates includes one or more words that precede an entity of the set of entities within the training content.

17. The method of claim 3, wherein a word template of the set of word templates includes one or more words that follow an entity of the set of entities within the training content.

18. The method of claim 3, wherein a word template of the set of word templates includes one or more words that surround an entity of the set of entities within the training content.

19. The method of claim 3, further comprising storing the set of entities in association with the first concept.

20. A computer product for learning related entities, the computer product comprising a non-transitory computer-readable medium containing computer program code for performing the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each word template is one or more words, and wherein the one or more identified word templates are added to a set of word templates;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, a confidence score of the identified word template when the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of a set of word templates;

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value;

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more identified candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity;

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from a part-of-speech tag of the set of entities;

receiving an external input selecting candidate entities for removal if the selected candidate entities do not relate to the first concept from the set of candidate entities;

removing candidate entities from the set of candidate entities based on the received external input; and

adding the one or more candidate entities of the set of candidate entities to the set of entities.