METHODS AND COMPUTER SYSTEMS FOR NAMED ENTITY VERIFICATION, NAMED ENTITY VERIFICATION MODEL TRAINING, AND PHRASE EXPANSION

• US 20180173694A1
• Filed: 07/19/2017
• Published: 06/21/2018
• Est. Priority Date: 12/21/2016
• Status: Abandoned Application

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Liu, Chao-Hong, Chiueh, Tzi-Cker, Kuo, Chih-Chung, Lee, Chung-Han, Hung, Jian-Yung
• Application Number:
  US15/653,536
• Publication Number:
  US 20180173694A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

The disclosure provides methods and computer systems for named entity verification, named entity verification model training, and phrase expansion. The method for named entity verification includes to receive an unknown type phrase, to generate a query phrase according to the unknown type phrase, to perform auto-completion on the query phrase to receive one or more returned phrases, to extract feature information from the returned phrases, and to determine a named entity type of the unknown type phrase based on the feature information verify whether or not the unknown type phrase belongs to the target named entity type based on the feature information and a target verification model to accordingly output a verification result.

Claims

1. A computer-implemented method for named entity verification comprising:

receiving an unknown type phrase;

generating a query phrase according to the unknown type phrase;

performing auto-completion on the query phrase to receive at least one returned phrase;

extracting feature information from the at least one returned phrase; and

determining a named entity type of the unknown type phrase based on the feature information and a target verification model to accordingly output a verification result.

2. The method according to claim 1, wherein the step of generating the query phrase according to the unknown type phrase comprises:

generating the query phrase according to a string extraction or a string concatenation of the unknown type phrase.

3. The method according to claim 1, wherein before the step of generating the query phrase according to the unknown type phrase, the method further comprises:

receiving a target named entity type.

4. The method according to claim 3, wherein the target named entity type is received from a user input or selected from a set of pre-stored named entity types.

5. The method according to claim 3, wherein the step of generating the query phrase according to the unknown type phrase comprises:

generating the query phrase according to the unknown type phrase and at least one key phrase of the target named entity type.

6. The method according to claim 3, wherein the step of determining the named entity type of the unknown type phrase based on the feature information and the target verification model to accordingly output the verification result comprises:

determining whether or not the unknown type phrase belongs to the target named entity type based on the feature information and the target verification model to accordingly output the verification result.

7. The method according to claim 6, wherein the step of extracting the feature information from the at least one returned phrase comprises:

obtaining and setting at least one related phrase from the at least one returned phrase as the feature information.

8. The method according to claim 7, wherein the target verification model is a supervised rule-based model, and wherein the step of determining whether or not the unknown type phrase belongs to the target named entity type based on the feature information and the target verification model to accordingly output the verification result comprises:

obtaining a plurality of base phrases in associated with the target named entity type;

inputting the feature information into the target verification model; and

obtaining the verification result from an output of the target verification model, wherein the output is associated with an existence of any of the base phrases within the at least one related phrase and indicates whether or not the unknown type phrase belongs to the target named entity type.

9. The method according to claim 6, wherein the step of extracting the feature information from the at least one returned phrase comprises:

obtaining at least one related phrase from the at least one returned phrase;

obtaining a plurality of base phrases in associated with the target named entity type;

calculating a plurality of feature values according to the at least one related phrase and the base phrases, wherein each of the feature values is a binary value and determined by whether there exists each of the base phrases within the at least one related phrase; and

converting the feature values to a feature vector as the feature information.

10. The method according to claim 9, wherein the target verification model is a supervised machine learning model, and wherein the step of determining whether or not the unknown type phrase belongs to the target named entity type based on the feature information and the target verification model to accordingly output the verification result comprises:

inputting the feature vector into the target verification model; and

obtaining the verification result from an output of the target verification model, wherein the output indicates whether or not the unknown type phrase belongs to the target named entity type or indicates that the unknown type phrase belongs to any of the named entity types.

11. The method according to claim 1, wherein after the step of receiving the unknown type phrase and the target named entity type, the method further comprises:

determining a language or a geographical region in associated with the unknown type phrase so as to accordingly generate the at least one query phrase and extract the feature information from the at least one returned phrase.

12. A computer-implemented method for training a named entity verification model comprising:

receiving known type training data, wherein the known type training data comprises a plurality of training phrases with a target named entity type;

generating a plurality of query phrases according to the training phrases;

performing auto-completion on each of the query phrases to receive a plurality of returned phrases;

extracting feature information from the returned phrases corresponding to each of the query phrases; and

training a target verification model associated with the target named entity type according to the feature information.

13. The method according to claim 12, wherein the step of generating the query phrases according to the training phrases comprises:

setting each of the training phrases or each of the training phrases with a whitespace character as the query phrases.

14. The method according to claim 12 further comprising:

generating a plurality of key phrases from the returned phrases corresponding to a target named entity type.

15. The method according to claim 14, wherein the step of generating the plurality of key phrases from the returned phrases corresponding to the target named entity type comprises:

obtaining a rank list of the returned phrases according to term frequency scores; and

selecting a predetermined number of returned phrases from the rank list as the plurality of key phrases.

16. The method according to claim 14, wherein the step of generating the plurality of key phrases from the returned phrases corresponding to the target named entity type comprises:

obtaining a rank list of the returned phrases according to term frequency-inverse document frequency scores; and

selecting a predetermined number of returned phrases from the rank list as the plurality of key phrases.

17. The method according to claim 14, wherein the steps of extracting the feature information from the returned phrases and training the target verification model associated with the target named entity type according to the feature information comprise:

obtaining the plurality of key phrases as the feature information in associated with the target named entity type; and

training the target verification model according to the feature information based on a supervised rule-based model.

18. The method according to claim 14, wherein the steps of extracting the feature information from the returned phrases and training the target verification model associated with the target named entity type according to the feature information comprise:

obtaining at least one related phrase from the returned phrases;

obtaining the plurality of key phrases as a plurality of base phrases in associated with the target named entity type;

calculating a plurality of feature values as the feature information according to the at least one related phrase and the base phrases; and

training the target verification model according to the feature information based on a supervised machine learning model.

19. The method according to claim 12, wherein the known type training data further comprises a plurality of other training phrases with a non-target named entity type to train the target verification model.

20. The method according to claim 12, wherein after the step of receiving the known type training data, the method further comprises:

determining a language or a geographical region in associated with the known type training data so as to accordingly generate the query phrases and extract the feature information from the returned phrases.

21. A method for phrase expansion comprising:

receiving a phrase set from a phrase database;

generating a plurality of query phrases according to the phrase set;

performing auto-completion on each of the query phrases to receive at least one returned phrase;

extracting a new candidate phrase from the at least one returned phrase, wherein the new candidate phrase does not exist in the phrase set;

adding the new candidate phrase to expand the phrase set; and

performing an iterative expansion control process to iteratively expand the phrase set based on the new candidate phrase.

22. The method according to claim 21 further comprising:

receiving a maximum phrase length; and

limiting the length of each of the query phrases not to exceed the maximum phrase length.

23. The method according to claim 21 further comprising:

receiving a maximum phrase number; and

limiting the number of phrases each of the query phrases not to exceed the maximum phrase number.

24. The method according to claim 21 further comprising:

terminating the iterative expansion control process when no new candidate phrase is received.

25. The method according to claim 21, wherein after the step of receiving the phrase set from the phrase database, the method further comprises:

determining a language or a geographical region in associated with the phrase set so as to accordingly receive the at least one returned phrase.

26. A computer system comprising:

a memory, configured to store data and a plurality of instructions;

at least one processor, coupled to the memory, and configured to access and execute the instructions to perform steps of;

receiving an unknown type phrase;

generating a query phrase according to the unknown type phrase;

performing auto-completion on the query phrase to receive at least one returned phrase;

extracting feature information from the at least one returned phrase; and

determining a named entity type of the unknown type phrase based on the feature information and a target verification model to accordingly output a verification result.

27. A computer system comprising:

a memory, configured to store data and a plurality of instructions;

at least one processor, coupled to the memory, and configured to access and execute the instructions to perform steps of;

receiving known type training data, wherein the known type training data comprises a plurality of training phrases with a target named entity type;

generating a plurality of query phrases according to the training phrases;

performing auto-completion on each of the query phrases to receive a plurality of returned phrases;

extracting feature information from the returned phrases corresponding to each of the query phrases; and

training a target verification model associated with the target named entity type according to the feature information.

28. A computer system comprising:

a memory, configured to store data and a plurality of instructions;

at least one processor, coupled to the memory, and configured to access and execute the instructions to perform steps of;

receiving a phrase set from a phrase database;

generating a plurality of query phrases according to the phrase set;

performing auto-completion on each of the query phrases to receive at least one returned phrase;

extracting a new candidate phrase from the at least one returned phrase, wherein the new candidate phrase does not exist in the phrase set;

adding the new candidate phrase to expand the phrase set; and

performing an iterative expansion control process to iteratively expand the phrase set based on the new candidate phrase.