METHOD AND SYSTEM FOR ROBUST TAGGING OF NAMED ENTITIES IN THE PRESENCE OF SOURCE OR TRANSLATION ERRORS

• US 20160012020A1
• Filed: 07/14/2014
• Published: 01/14/2016
• Est. Priority Date: 07/14/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: George, Brancovici, Gabriel, Nicolae, Hiep, Luong, Sindhu, Vijaya-Raghavan, Yonghoon, Lee
• Granted Patent: US 10,073,673 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A system includes a storage device configured to store a plurality of named entities collected from a plurality of sources, tokenized, and associated with a label. The system includes a tagging apparatus configured to receive textual communications, identify each of the textual communications, tokenize the textual communications, match the textual tokens with the named entities tokens stored in the storage device in order to assign the textual tokens to the labels associated with the named entity tokens, tag the textual communications based on the matching between the textual tokens and the named entity tokens in order to identify the intended meaning of each of the textual communications, and identify the intended meaning of the textual communications based on applying the tags to the textual communications. A method capable of disambiguating named entities using a common sense reasoning approach is also disclosed.

Claims

1. A robust named entity tagging system capable of identifying an intended meaning of a communication source regardless of the source or translation errors, the system comprising:

a storage device configured to store a plurality of named entities collected from a plurality of sources, wherein each of the named entities are tokenized into a common format of named entity tokens, wherein each of the named entities are associated with a label, and wherein each of the named entity tokens are one of a word or a syllable of a word; and

a tagging apparatus configured to;

receive one or more textual communications,

identify each of the one or more textual communications,

tokenize the one or more textual communications into a common format of textual tokens,

match the one or more textual tokens with one or more of the named entities tokens stored in the storage device in order to assign the textual tokens to the labels associated with the named entity tokens,

tag the one or more textual communications based on the matching between the textual tokens and the named entity tokens in order to identify the intended meaning of each of the one or more textual communications, and

identify the intended meaning of the one or more textual communications based on applying the tags to the one or more textual communications.

2. The system as set forth in claim 1, wherein the named entities comprise at least one of object names, names associated with objects or people, proper names associated with objects or people, titles, phrases, and a combination thereof.

3. The system as set forth in claim 1, wherein the tagging apparatus is configured to apply the tags to the one or more textual communications in order to correct textual mistakes.

4. The system as set forth in claim 1, wherein the tagging apparatus is configured to utilize a beam traversal with a partial solution set of a configurable size on a prefix tree using a custom similarity measure as comparisons occur at sequentially traversed nodes in the prefix tree in order to match the one or more textual communications with one or more of the named entities stored in the storage device.

5. The system as set forth in claim 4, wherein the partial solution set comprises a limited size.

6. The system as set forth in claim 1, wherein the tagging apparatus is configured to partially disambiguate at least one of the one or more named entities matched with the one or more textual tokens to ensure that a subset of the one or more named entities of a high-confidence can be selected from the full set of one or more named entities.

7. The system as set forth in claim 1, wherein the tagging apparatus is configured to partially disambiguate at least one of the one or more named entities matched with the one or more textual tokens by filtering out at least one of the one or more named entities having the lowest confidence level of match detection.

8. The system as set forth in claim 1, wherein the tagging apparatus is configured to partially disambiguate at least one of the one or more named entities matched with the one or more textual tokens by ranking the confidence levels of at least two of the one or more named entities matched with the one or more textual tokens.

9. A method implemented in a robust named entity tagging system capable of identifying an intended meaning of a communication source regardless of the source or translation errors, the method comprising:

receiving, by a tagging apparatus, one or more textual communications;

identifying each of the one or more textual communication;

tokenizing the one or more textual communications into a common format of textual tokens;

matching the one or more textual tokens with one or more named entities tokens stored in a storage device in order to assign the textual tokens to labels associated with the named entity tokens, and wherein each of the one or more named entities is associated with a label;

tagging the one or more textual communications based on the matching between the textual tokens and the named entity tokens in order to identify the intended meaning of each of the one or more textual communications; and

identifying the intended meaning of the one or more textual communications based on applying the tags to the one or more textual communications.

10. The method as set forth in claim 9, wherein the named entities comprise at least one of object names, names associated with objects or people, proper names associated with objects or people, titles, phrases, and a combination thereof.

11. The method as set forth in claim 9, further comprising applying the tags to the one or more textual communications in order to correct textual mistakes.

12. The method as set forth in claim 9, further comprising utilizing a beam traversal with a partial solution set of a configurable size on a prefix tree using a custom similarity measure as comparisons occur at sequentially traversed nodes in the prefix tree in order to match the one or more textual communications with one or more of the named entities stored in the storage device.

13. The method as set forth in claim 12, wherein the partial solution set comprises a limited size.

14. The method as set forth in claim 9, further comprising partially disambiguating at least one of the one or more named entities matched with the one or more textual tokens to ensure that a subset of the one or more named entities of a high-confidence can be selected from the full set of one or more named entities.

15. The method as set forth in claim 9, further comprising partially disambiguating at least one of the one or more named entities matched with the one or more textual tokens by filtering out at least one of the one or more named entities having the lowest confidence level of match detection.

16. The method as set forth in claim 9, further comprising partially disambiguating at least one of the one or more named entities matched with the one or more textual tokens by ranking the confidence levels of at least two of the one or more named entities matched with the one or more textual tokens.

17. For use in an electronic device, a method capable of disambiguating named entities, the method comprising the steps of:

receiving textual communication by an electronic device, wherein the textual communication comprises one or more named entities;

grouping concepts associated with the one or more name entities in order to extract relevant information from a knowledge base;

inferring relevant information from the extracted relevant information from the knowledge base;

calculating and ranking the inferred relevant knowledge for the one or more named entities; and

identifying the most relevant label based on the highest rank of inferred relevant knowledge for each of the one or more named entities.

18. The method as set forth in claim 17, wherein the textual communication comprises one or more ambiguous entities.

19. The method as set forth in claim 17, wherein the textual communication is preprocessed to remove stop-words and punctuation.

20. The method as set forth in claim 17, wherein calculating and ranking the inferred relevant knowledge includes calculating a score based on the following function: