Analyzing textual data

• US 10,176,804 B2
• Filed: 01/12/2017
• Issued: 01/08/2019
• Est. Priority Date: 01/18/2016
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Jiang, Huixing, Sun, Jian, Chu, Min
• Primary Examiner(s): Sharma, Neeraj
• Application Number:
  US15/404,855
• Publication Number:
  US 20170206897A1
• Time in Patent Office: 726 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Analyzing textual data is disclosed, including by: receiving textual data; determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template; extracting a source substantive from the textual data; using the source substantive to determine a target substantive from a word vector model that is trained on a set of training data; and generating an answer including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template.

Claims

1. A method for analogy analysis, comprising:

receiving textual data;

determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extracting a source substantive from the textual data, wherein a substantive comprises a noun;

using, by a processor, the source substantive to determine a target substantive from a word vector model that is trained on a set of training data, wherein using the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises;

using the word vector model to generate a source word vector corresponding to a first source substantive;

determining a set of candidate target substantives that is similar to the first source substantive based at least in part on comparing target word vectors corresponding to the set of candidate target substantives to the source word vector;

identifying a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determining a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identifying a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

selecting a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives, wherein selecting the candidate target substantive comprises determining a score corresponding to each pair of candidate target substantive and computed target substantive;

generating an answer comprising an analogy including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template; and

outputting the answer.

2. The method of claim 1, further comprising:

receiving speech data; and

converting the speech data into the textual data.

3. The method of claim 1, wherein determining that the textual data is the candidate for analogy analysis comprises:

performing segmentation on the textual data to obtain a plurality of textual words;

comparing the plurality of textual words against one or more analogical question templates;

determining that at least some of the plurality of textual words match the analogical question template; and

determining that the matching analogical question template corresponds to the analogical answer template based on an established correspondence between the analogical question template and the analogical answer template.

4. The method of claim 1, further comprising generating the set of training data, including by crawling a plurality of webpages.

5. The method of claim 1,

wherein selecting comprises determining respective scores corresponding to pairs of candidate target substantives and computed target substantives.

6. The method of claim 1, wherein generating the answer comprises replacing a placeholder source substantive in the analogical answer template with the target substantive.

15. The method of claim 1, wherein outputting the answer comprises:

converting the answer into speech data; and

outputting the speech data.

7. A system for analogy analysis, comprising:

a processor; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to;

receive textual data;

determine that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extract a source substantive from the textual data, wherein a substantive comprises a noun;

use the source substantive to determine a target substantive from a word vector model that is trained on a set of training data, wherein to use the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises to;

use the word vector model to generate a source word vector corresponding to a first source substantive;

determine a set of candidate target substantives that is similar to the first source substantive based at least in part on comparing target word vectors corresponding to the set of candidate target substantives to the source word vector;

identify a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determine a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identify a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

select a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives, wherein to select the candidate target substantive comprises to determine a score corresponding to each pair of candidate target substantive and computed target substantive;

generate an answer comprising an analogy including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template; and

output the answer.

8. The system of claim 7, wherein the memory is configured to further provide the processor with instructions which when executed cause the processor to:

receive speech data; and

convert the speech data into the textual data.

9. The system of claim 7, wherein to determine that the textual data is the candidate for analogy analysis comprises to:

perform segmentation on the textual data to obtain a plurality of textual words;

compare the plurality of textual words against one or more analogical question templates;

determine that at least some of the plurality of textual words match the analogical question template; and

determine that the matching analogical question template corresponds to the analogical answer template based on an established correspondence between the analogical question template and the analogical template.

10. The system of claim 7, wherein to generate the set of training data includes crawling a plurality of webpages.

11. The system of claim 7,

wherein to select comprises to determine respective scores corresponding to pairs of candidate target substantives and computed target substantives.

12. The system of claim 7, wherein to generate the answer comprises to replace a placeholder source substantive in the analogical answer template with the target substantive.

16. The system of claim 7, wherein to output the answer comprises to:

convert the answer into speech data; and

output the speech data.

13. A computer program product for analogy analysis, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving textual data;

determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extracting a source substantive from the textual data, wherein a substantive comprises a noun;

using the source substantive to determine a target substantive from a word vector model that is trained on a set of training data, wherein using the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises;

using the word vector model to generate a source word vector corresponding to a first source substantive;

determining a set of candidate target substantives that is similar to the first source substantive based at least in part on comparing target word vectors corresponding to the set of candidate target substantives to the source word vector;

identifying a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determining a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identifying a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

selecting a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives, wherein selecting the candidate target substantive comprises determining a score corresponding to each pair of candidate target substantive and computed target substantive; and

generating an answer comprising an analogy including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template; and

outputting the answer.

14. The computer program product of claim 13, wherein determining that the textual data is the candidate for analogy analysis comprises:

performing segmentation on the textual data to obtain a plurality of textual words;

comparing the plurality of textual words against one or more analogical question templates;

determining that at least some of the plurality of textual words match the analogical question template; and

determining that the matching analogical question template corresponds to the analogical answer template.

17. The computer program product of claim 13, wherein outputting the answer comprises:

converting the answer into speech data; and

outputting the speech data.