Entity analysis system

• US 9,558,456 B2
• Filed: 10/29/2015
• Issued: 01/31/2017
• Est. Priority Date: 08/08/2011
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: The Gravity Cue Company Limited
• Inventors: Kapur, Amit R., Pearman, Steven F., Benedetto, James R.
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): FINK, THOMAS ANDREW
• Application Number:
  US14/926,059
• Publication Number:
  US 20160048773A1
• Time in Patent Office: 460 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A method for building a factual database of concepts and entities that are related to the concepts through a learning process. Training content (e.g., news articles, books) and a set of entities (e.g., Bill Clinton and Barack Obama) that are related to a concept (e.g., Presidents) is received. Groups of words that co-occur frequently in the textual content in conjunction with the entities are identified as templates. Templates may also be identified by analyzing parts-of-speech patterns of the templates. Entities that co-occur frequently in the textual content in conjunction with the templates are identified as additional related entities (e.g., Ronald Reagan and Richard Nixon). To eliminate erroneous results, the identified entities may be presented to a user who removes any false positives. The entities are then stored in association with the concept.

Claims

1. A computer-implemented method of learning related entities, the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more potential word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each potential word template is one or more words, and wherein each potential word template is tagged with a part-of-speech tag based on grammatical use of the one or more words in the training content;

identifying one or more word templates from the one or more potential word templates based on a frequency of occurrence of the one or more potential word templates and based on the part-of-speech tag of the one or more potential word templates compared to part-of-speech tags of word templates of a set of word templates, wherein the one or more identified word templates are added to the set of word templates;

generating, for each identified word template, a confidence score for the identified word template based on a frequency of occurrence of the identified word template;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, the confidence score of the identified word template based on whether the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of IM set of word templates;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value; and

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value.

2. The method of claim 1, wherein learning additional entities that are related to the first concept further comprises:

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity; and

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from a part-of-speech tag of the set of entities,

wherein the method of claim 1 further comprises;

adding one or more candidate entities remaining in the set of candidate entities to the set of entities; and

storing the set of entities in association with the first concept.

3. The method of claim 2, wherein identifying one or more candidate entities that relate to the first concept further comprises:

removing one or more candidate entities from the set of candidate entities based on whether the one or more candidate entities are related to a second concept,

wherein the second concept does not relate to the first concept.

15. The method of claim 1, wherein the received input to remove candidate entities from the set of candidate entities is a selection by a user of candidate entities to remove from the set of candidate entities.

4. A computer-implemented method of learning related entities, the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each word template is one or more words, and wherein the one or more identified word templates are added to a set of word templates;

generating, for each identified word template, a confidence score for the identified word template based on a frequency of occurrence of the identified word template;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, the confidence score of the identified word template based on whether the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of the set of word templates;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value; and

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value.

5. The method of claim 4, wherein learning additional entities that are related to the first concept further comprises:

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more identified candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity; and

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from the part-of-speech tag of the set of entities,

wherein the method of claim 4 further comprises;

adding one or more candidate entities remaining in the set of candidate entities to the set of entities.

6. The method of claim 4 wherein identifying one or more word templates from the training content further comprises:

tagging each of the one or more words in the training content that occur with the entity of the set of entities with a part-of-speech tag based on grammatical use of the one or more words in the training content that occur with the entity of the set of entities;

identifying the one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities.

7. The method of claim 6, wherein identifying one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities comprises:

identifying the one or more word templates based on similarities between the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities and part-of-speech tags of word templates in the set of word templates.

8. The method of claim 6, wherein identifying one or more word templates based on the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities comprises:

identifying the one or more word templates based on similarities between the part-of-speech tag of the one or more words in the training content that occur with the entity of the set of entities in a current iteration and part-of-speech tags of word templates in the set of word templates identified in a previous iteration.

9. The method of claim 5, wherein identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates further comprises:

identifying the one or more candidate entities based a grammatical use of the candidate entities and a part-of-speech tag of the entities of the set of entities.

10. The method of claim 5, wherein learning additional entities that are related to the first concept further comprises:

receiving an external input to remove candidate entities that do not relate to the first concept from the set of candidate entities; and

removing candidate entities from the set of candidate entities based on the received input.

11. The method of claim 10, wherein learning additional entities that are related to the first concept further comprises:

searching an ontology database of related entities and concepts for a second concept that relates to a candidate entity in the set of candidate entities, wherein the ontology database includes the set of entities that relate to the first concept;

determining whether the second concept relates to the first concept; and

removing candidate entities from the set of candidate entities when the second concept of the candidate entity does not relate to the first concept.

12. The method of claim 10, wherein learning additional entities that are related to the first concept further comprises:

searching an ontology database of related entities and concepts for a second concept that relates to a candidate entity in the set of candidate entities, wherein the ontology database includes the set of entities that relate to the first concept;

determining whether the second concept relates to the first concept; and

adding the candidate entities to the set of entities that relates to the first concept when the second concept of the candidate entity relates to the first concept.

13. The method of claim 4, wherein the first concept is a classification of each entity in the set of entities.

14. The method of claim 4, wherein each entity in the set of entities is a proper noun.

16. The method of claim 10, wherein the received input to remove candidate entities from the set of candidate entities is a selection by a user of candidate entities to keep in the set of candidate entities.

17. The method of claim 10, wherein the received input to remove candidate entities from the set of candidate entities is received in fewer than all iterations.

18. The method of claim 4, wherein a word template of the set of word templates includes one or more words that precede an entity of the set of entities within the training content.

19. A computer product for learning related entities, the computer product comprising a non-transitory computer-readable medium containing computer program code for performing the method comprising:

receiving a set of entities, the set of entities including a plurality of entities and each entity in the set of entities relating to a first concept;

receiving training content that includes textual content that is organized and that includes the plurality of entities of the set of entities; and

learning additional entities that are related to the first concept by iteratively performing the following steps;

identifying one or more word templates from the training content based on occurrences of one or more words in the training content with an entity of the set of entities, wherein each word template is one or more words, and wherein the one or more identified word templates are added to a set of word templates;

generating, for each identified word template, a confidence score for the identified word template based on a freauencv of occurrence of the identified word template;

identifying, for each identified word template, one or more part-of-speech tags of the identified word templates;

adjusting, for each identified word template, the confidence score of the identified word template based on whether the one or more part of speech tags of the identified word template is similar to the part-of-speech tags of word templates of the set of word templates;

comparing, for each identified word template, the confidence score of the identified word template to a threshold value; and

removing the identified word template from the set of word templates when the confidence score of the identified word template is outside the threshold value.

20. The computer product of claim 19, wherein learning additional entities that are related to the first concept further comprises:

adjusting, for each identified word template, the confidence score of the identified word template when the identified word template is identified as being a false positive;

identifying one or more candidate entities that relate to the first concept based on occurrences of each of the one or more candidate entities in the training content with at least one of the word templates of the set of word templates, wherein the one or more identified candidate entities are added to a set of candidate entities;

identifying a part-of-speech tag for each candidate entity; and

removing a candidate entity from the set of candidate entities when the part-of-speech tag of the candidate entity is different from a part-of-speech tag of the set of entities,

wherein the computer product of claim 19 further comprises;

adding the one or more candidate entities of the set of candidate entities to the set of entities.