ANALYZING TEXTUAL DATA

• US 20170206897A1
• Filed: 01/12/2017
• Published: 07/20/2017
• Est. Priority Date: 01/18/2016
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Jiang, Huixing, Sun, Jian, Chu, Min
• Granted Patent: US 10,176,804 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Analyzing textual data is disclosed, including by: receiving textual data; determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template; extracting a source substantive from the textual data; using the source substantive to determine a target substantive from a word vector model that is trained on a set of training data; and generating an answer including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template.

Claims

1. A method, comprising:

receiving textual data;

determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extracting a source substantive from the textual data;

using, by a processor, the source substantive to determine a target substantive from a word vector model that is trained on a set of training data; and

generating an answer including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template.

2. The method of claim 1, further comprising:

receiving speech data; and

converting the speech data into the textual data.

3. The method of claim 1, wherein determining that the textual data is the candidate for analogy analysis comprises:

performing segmentation on the textual data to obtain a plurality of textual words;

comparing the plurality of textual words against one or more analogical question templates;

determining that at least some of the plurality of textual words match the analogical question template; and

determining that the matching analogical question template corresponds to the analogical answer template based on an established correspondence between the analogical question template and the analogical answer template.

4. The method of claim 1, further comprising generating the set of training data, including by crawling a plurality of webpages.

5. The method of claim 1, wherein using the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises:

determining a set of candidate target substantives that is similar to the source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identifying a subset of the set of candidate target substantives that is associated with a same substantive type as the source substantive; and

selecting one or more target substantives from the subset of the set of candidate target substantives.

6. The method of claim 1, wherein using the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises:

determining a set of candidate target substantives that is similar to a first source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identifying a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determining a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identifying a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

selecting a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives.

7. The method of claim 1, wherein using the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises:

determining a set of candidate target substantives that is similar to a first source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identifying a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determining a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identifying a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

selecting a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives, wherein selecting comprises determining respective scores corresponding to pairs of candidate target substantives and computed target substantives.

8. The method of claim 1, wherein generating the answer comprises replacing a placeholder source substantive in the analogical answer template with the target substantive.

9. The method of claim 1, further comprising converting the answer into speech data and providing the speech data.

10. A system, comprising:

a processor; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to;

receive textual data;

determine that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extract a source substantive from the textual data;

use the source substantive to determine a target substantive from a word vector model that is trained on a set of training data; and

generate an answer including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template.

11. The system of claim 10, wherein the memory is configured to further provide the processor with instructions which when executed cause the processor to:

receive speech data; and

convert the speech data into the textual data.

12. The system of claim 10, wherein to determine that the textual data is the candidate for analogy analysis comprises to:

perform segmentation on the textual data to obtain a plurality of textual words;

compare the plurality of textual words against one or more analogical question templates;

determine that at least some of the plurality of textual words match the analogical question template; and

determine that the matching analogical question template corresponds to the analogical answer template.

13. The system of claim 10, wherein to generate the set of training data includes crawling a plurality of webpages.

14. The system of claim 10, wherein to use the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises to:

determine a set of candidate target substantives that is similar to the source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identify a subset of the set of candidate target substantives that is associated with a same substantive type as the source substantive; and

select one or more target substantives from the subset of the set of candidate target substantives.

15. The system of claim 10, wherein to use the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises to:

determine a set of candidate target substantives that is similar to a first source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identify a subset of the set of candidate target substantives that is are associated with a same first substantive type as the first source substantive;

determine a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identify a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

select a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives.

16. The system of claim 10, wherein to use the source substantive to determine the target substantive from the word vector model that is trained on the set of training data comprises to:

determine a set of candidate target substantives that is similar to a first source substantive based at least in part on target word vectors corresponding to the set of candidate target substantives and a source word vector corresponding to the source substantive;

identify a subset of the set of candidate target substantives that is associated with a same first substantive type as the first source substantive;

determine a set of computed target substantives from the first source substantive, a second source substantive from a plurality of source substantives, and the subset of the set of candidate target substantives;

identify a subset of the set of computed target substantives that is associated with a same second substantive type as the second source substantive; and

select a candidate target substantive from the subset of the set of candidate target substantives and a computed target substantive from the subset of the set of computed target substantives, wherein to select comprises to determine respective scores corresponding to pairs of candidate target substantives and computed target substantives.

17. The system of claim 10, wherein to generate the answer comprises to replace a placeholder source substantive in the analogical answer template with the target substantive.

18. The system of claim 10, wherein the memory is configured to further provide the processor with instructions which when executed cause the processor to convert the answer into speech data and provide the speech data.

19. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving textual data;

determining that the textual data is a candidate for analogy analysis based at least in part on at least a portion of the textual data matching an analogical question template;

extracting a source substantive from the textual data;

using the source substantive to determine a target substantive from a word vector model that is trained on a set of training data; and

generating an answer including the target substantive based at least in part on an analogical answer template corresponding to the analogical question template.

20. The computer program product of claim 19, wherein determining that the textual data is the candidate for analogy analysis comprises:

performing segmentation on the textual data to obtain a plurality of textual words;

comparing the plurality of textual words against one or more analogical question templates;

determining that at least some of the plurality of textual words match the analogical question template; and

determining that the matching analogical question template corresponds to the analogical answer template.