Recognition of target words using designated characteristic values

• US 8,744,839 B2
• Filed: 09/22/2011
• Issued: 06/03/2014
• Est. Priority Date: 09/26/2010
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Sun, Haibo, Yang, Yang, Chen, Yining
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): THOMAS-HOMESCU, ANNE L
• Application Number:
  US13/240,034
• Publication Number:
  US 20120078631A1
• Time in Patent Office: 985 Days
• Field of Search: 704 1- 10, 704231-257, 704/275, 707/748, 707/723, 707/730, 707/728, 707705-711, 707/722, 707740-742, 707/738, 707/737, 707/759, 707/758, 707/755, 707/756, 707/750, 707/749, 707/751, 707/754, 707/718, 707/752, 707765-770, 707/690, 707/692, 707/792, 707/794, 707/784, 707/697, 707/607, 707/602, 707/673, 707/661, 707/695, 707/803, 707/811, 707/827
• US Class Current: 704/10
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Target word recognition includes: obtaining a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set; performing segmentation of the characteristic computation data to generate a plurality of text segments; combining the plurality of text segments to form a text data combination set; determining an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations; determining a plurality of designated characteristic values for the plurality of text data combinations; based at least in part on the plurality of designated characteristic values and according to at least a criterion, recognizing among the plurality of text data combinations target words whose characteristic values fulfill the criterion.

Claims

1. A method of target word recognition, comprising:

obtaining a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

performing segmentation of the characteristic computation data to generate a plurality of text segments;

combining the plurality of text segments to form a text data combination set;

determining an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determining a plurality of designated characteristic values for the plurality of text data combinations;

determining, using a processor, a criterion, including;

obtaining a training sample word set and sample characteristic computation data, the sample characteristic computation data comprising a plurality of sample words and designated characteristic values of the plurality of sample words;

obtaining a sample text data combination set based on the plurality of sample words;

determining a plurality of designated characteristic values of sample text data combinations in an intersection of the sample text data combination set and the training sample word set; and

setting a threshold value of a designated characteristic value of a sample text data combination in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognizing among the plurality of text data combinations, target words whose characteristic values fulfill the criterion.

2. The method of claim 1, wherein the candidate word set is based on user-inputted query keywords to a website.

3. The method of claim 1, wherein the characteristic computation data is obtained from descriptions of search results in response to user-inputted query keywords to a website.

4. The method of claim 1, wherein at least some of the target words are unlisted words in a dictionary, and the method further comprises adding the unlisted words to the dictionary.

5. The method of claim 1, wherein the designated characteristic values include mutual information.

6. The method of claim 1, wherein the designated characteristic values include logarithmic likelihood.

7. The method of claim 1, wherein the designated characteristic values include context entropy.

8. The method of claim 1, wherein the designated characteristic values include position-based in-word probability.

9. The method of claim 1, wherein combining the plurality of sample segments includes applying an n-gram language model to the plurality of sample segments.

10. The method of claim 1, wherein setting the threshold value includes:

sorting a training sample word in the intersection using the threshold value to reach a determination of whether the training sample word in the intersection is a target word;

comparing the determination with a known result; and

adjusting the threshold value if the determination does not match the known result.

11. The method of claim 1, wherein combining the plurality of text segments to form a text data combination set includes:

adoption of an n-gram model based on n-gram windows, shifting of the n-gram windows according to a predetermined sequence, and performance of word segment combination of the word segments contained within the windows to obtain a post-combining text data combination.

12. A target word recognition system, comprising:

one or more processors configured to;

obtain a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

perform segmentation of the characteristic computation data to generate a plurality of text segments;

combine the plurality of text segments to form a text data combination set;

determine an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determine a plurality of designated characteristic values for the plurality of text data combinations;

determine a criterion, including;

obtain a training sample word set and sample characteristic computation data, the sample characteristic computation data comprising a plurality of sample words and designated characteristic values of the plurality of sample words;

obtain a sample text data combination set based on the plurality of sample words;

determine a plurality of designated characteristic values of sample text data combinations in an intersection of the sample text data combination set and the training sample word set; and

set a threshold value of a designated characteristic value of a sample text data combination in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognize among the plurality of text data combinations, target words whose characteristic values fulfill the criterion; and

one or more memories coupled to the one or more processors, configured to provide the one or more processors with instructions.

13. The system of claim 12, wherein the candidate word set is based on user-inputted query keywords to a website.

14. The system of claim 12, wherein the characteristic computation data is obtained from descriptions of search results in response to user-inputted query keywords to a website.

15. The system of claim 12, wherein combining the plurality of sample segments includes applying an n-gram language model to the plurality of sample segments.

16. The system of claim 12, wherein setting the threshold value includes:

sorting a training sample word in the intersection using the threshold value to reach a determination of whether the training sample word in the intersection is a target word;

comparing the determination with a known result; and

adjusting the threshold value if the determination does not match the known result.

17. The system of claim 12, wherein combining the plurality of text segments to form a text data combination set includes:

adoption of an n-gram model based on n-gram windows, shifting of the n-gram windows according to a predetermined sequence, and performance of word segment combination of the word segments contained within the windows to obtain a post-combining text data combination.

18. A non-transitory computer program product for target word recognition, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

obtaining a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

performing segmentation of the characteristic computation data to generate a plurality of text segments;

combining the plurality of text segments to form a text data combination set;

determining an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determining a plurality of designated characteristic values for the plurality of text data combinations;

determining a criterion, including;

obtaining a training sample word set and sample characteristic computation data, the sample characteristic computation data comprising a plurality of sample words and designated characteristic values of the plurality of sample words;

obtaining a sample text data combination set based on the plurality of sample words;

determining a plurality of designated characteristic values of sample text data combinations in an intersection of the sample text data combination set and the training sample word set; and

setting a threshold value of a designated characteristic value of a sample text data combination in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognizing among the plurality of text data combinations, target words whose characteristic values fulfill the criterion.

19. A method of target word recognition, comprising:

obtaining a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

performing segmentation of the characteristic computation data to generate a plurality of text segments;

combining the plurality of text segments to form a text data combination set;

determining an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determining a plurality of designated characteristic values for the plurality of text data combinations;

determining, using a processor, a criterion, including;

obtaining a training sample word set and sample characteristic computation data, the training sample word set comprising a plurality of sample words and sorting results indicating whether each of the plurality of sample words is a target word, and the sample characteristic computation data comprising the plurality of sample words and designated characteristic values of the plurality of sample words;

segmenting the plurality of sample words to obtain a plurality of sample segments of minimum granularity;

combining the plurality of sample segments to obtain a sample text data combination set;

determining an intersection of the sample text data combination set and the training sample word set;

determining a plurality of designated characteristic values of sample text data combinations in the intersection; and

setting a threshold value of a designated characteristic value of a sample text data combination in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognizing among the plurality of text data combinations, target words whose characteristic values fulfill the criterion.

20. A target word recognition system, comprising:

one or more processors configured to;

obtain a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

perform segmentation of the characteristic computation data to generate a plurality of text segments;

combine the plurality of text segments to form a text data combination set;

determine an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determine a plurality of designated characteristic values for the plurality of text data combinations;

determine a criterion, including;

obtaining a training sample word set and sample characteristic computation data, the training sample word set comprising a plurality of sample words and sorting results indicating whether each of the plurality of sample words is a target word, and the sample characteristic computation data comprising the plurality of sample words and designated characteristic values of the plurality of sample words;

segmenting the plurality of sample words to obtain a plurality of sample segments of minimum granularity;

combining the plurality of sample segments to obtain a sample text data combination set;

determining an intersection of the sample text data combination set and the training sample word set;

determining a plurality of designated characteristic values of sample text data combinations in the intersection; and

setting a threshold value of a designated characteristic value of a sample text data combination in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognize among the plurality of text data combinations, target words whose characteristic values fulfill the criterion; and

one or more memories coupled to the one or more processors, configured to provide the one or more processors with instructions.

21. A non-transitory computer program product for target word recognition, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

obtaining a candidate word set and corresponding characteristic computation data, the candidate word set comprising text data, and characteristic computation data being associated with the candidate word set;

performing segmentation of the characteristic computation data to generate a plurality of text segments;

combining the plurality of text segments to form a text data combination set;

determining an intersection of the candidate word set and the text data combination set, the intersection comprising a plurality of text data combinations;

determining a plurality of designated characteristic values for the plurality of text data combinations;

determining a criterion, including;

obtaining a training sample word set and sample characteristic computation data, the training sample word set comprising a plurality of sample words and sorting results indicating whether each of the plurality of sample words is a target word, and the sample characteristic computation data comprising the plurality of sample words and designated characteristic values of the plurality of sample words;

segmenting the plurality of sample words to obtain a plurality of sample segments of minimum granularity;

combining the plurality of sample segments to obtain a sample text data combination set;

determining an intersection of the sample text data combination set and the training sample word set;

determining a plurality of designated characteristic values of sample text data combinations in the intersection; and

setting a threshold value of a designated characteristic value of a sample text data combinations, in the intersection as a part of the criterion; and

based at least in part on the plurality of designated characteristic values for the plurality of text data combinations and according to at least the criterion, recognizing among the plurality of text data combinations target words whose characteristic values fulfill the criterion.