Splitting utterances for quick responses

• US 9,972,308 B1
• Filed: 11/08/2016
• Issued: 05/15/2018
• Est. Priority Date: 11/08/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Nagano, Tohru, Tachibana, Ryuki
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US15/346,676
• Publication Number:
  US 20180130460A1
• Time in Patent Office: 553 Days
• Field of Search: 704231-257, 704270-275
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 15/063 Training

Abstract

Methods, a system, and a classifier are provided. A method includes preparing, by a processor, pairs for an information retrieval task. Each pair includes (i) a training-stage speech recognition result for a respective sequence of training words and (ii) an answer label corresponding to the training-stage speech recognition result. The method further includes obtaining, by the processor, a respective rank for the answer label included in each pair to obtain a set of ranks. The method also includes determining, by the processor, for each pair, an end of question part in the training-stage speech recognition result based on the set of ranks. The method additionally includes building, by the processor, the classifier such that the classifier receives a recognition-stage speech recognition result and returns a corresponding end of question part for the recognition-stage speech recognition result, based on the end of question part determined for the pairs.

Claims

1. A method for building a classifier, comprising:

preparing, by a processor, a plurality of pairs for an information retrieval task, each of the plurality of pairs including (i) a training-stage speech recognition result for a respective sequence of training words and (ii) an answer label corresponding to the training-stage speech recognition result;

obtaining, by the processor using a search engine, a respective rank for the answer label included in each of the plurality of pairs to obtain a set of ranks;

determining, by the processor, for each of the plurality of pairs, an end of question part in the training-stage speech recognition result based on the set of ranks;

building, by the processor, the classifier such that the classifier receives a recognition-stage speech recognition result and returns a corresponding end of question part for the recognition-stage speech recognition result, based on the end of question part determined for the plurality of pairs;

splitting, by the processor, the recognition-stage speech recognition result at the corresponding end of question part for the recognition-stage speech recognition result;

generating, by the processor, a set of answer candidates for replying to the corresponding end of question part for the recognition-stage speech recognition result; and

providing, by a display device, the set of answer candidates to a user.

2. The method of claim 1, wherein the end of question part for a given one of the plurality of pairs is determined based on a rank change of a part of the respective sequence of training words included in the training-stage speech recognition result for the given one of the plurality of pairs.

3. The method of claim 1, wherein building the classifier comprises generating a plurality of clusters, each of the plurality of clusters comprising at least one partial set of words, each of the at least one partial set of words including a set of words from the beginning of the speech recognition result to the end of question part of the speech recognition result.

4. The method of claim 3, wherein in response to receiving the recognition-stage speech recognition result incrementally in a word by word manner as an input, the classifier calculates a similarity between the input and the plurality of clusters and returns the corresponding end of question part for the recognition-stage speech recognition result based the similarity between the input and any of the plurality of clusters being less than a predetermined threshold.

5. The method of claim 3, wherein the plurality of clusters are generated by grouping together end of question parts that include the same or similar words.

6. The method of claim 1, wherein for a given pair from among the plurality of pairs, the end of question part in the training-stage speech recognition result for the given pair is determined by checking the respective rank for the answer label included in the given pair against the respective rank for the answer label included in other ones of the plurality of pairs.

7. The method of claim 1, wherein a first encountered word in the training-stage speech recognition result that results in a correct answer label for the training stage speech recognition result is determined as the end of question part in the training-stage speech recognition result.

8. The method of claim 1, wherein said obtaining step comprises generating a data structure that correlates an occurrence number of a word in a word sequence to the rank for the word in the word sequence.

9. A computer program product for building a classifier, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:

preparing, by a processor, a plurality of pairs for an information retrieval task, each of the plurality of pairs including (i) a training-stage speech recognition result for a respective sequence of training words and (ii) an answer label corresponding to the training-stage speech recognition result;

obtaining, by the processor using a search engine, a respective rank for the answer label included in each of the plurality of pairs to obtain a set of ranks;

determining, by the processor, for each of the plurality of pairs, an end of question part in the training-stage speech recognition result based on the set of ranks;

building, by the processor, the classifier such that the classifier receives a recognition-stage speech recognition result and returns a corresponding end of question part for the recognition-stage speech recognition result, based on the end of question part determined for the plurality of pairs;

splitting, by the processor, the recognition-stage speech recognition result at the corresponding end of question part of the recognition-stage speech recognition result;

generating, by the processor, a set of answer candidates for replying to the corresponding end of question part for the recognition-stage speech recognition result; and

providing, by a display device, the set of answer candidates to a user.

10. The computer program product of claim 9, wherein building the classifier comprises generating a plurality of clusters, each of the plurality of clusters comprising at least one partial set of words, each of the at least one partial set of words including a set of words from the beginning of the speech recognition result to the end of question part of the speech recognition result.

11. The computer program product of claim 10, wherein in response to receiving the recognition-stage speech recognition result incrementally in a word by word manner as an input, the classifier calculates a similarity between the input and the plurality of clusters and returns the corresponding end of question part for the recognition-stage speech recognition result based the similarity between the input and any of the plurality of clusters being less than a predetermined threshold.

12. The computer program product of claim 9, wherein for a given pair from among the plurality of pairs, the end of question part in the training-stage speech recognition result for the given pair is determined by checking the respective rank for the answer label included in the given pair against the respective rank for the answer label included in other ones of the plurality of pairs.

13. The computer program product of claim 9, wherein a first encountered word in the training-stage speech recognition result that results in a correct answer label for the training stage speech recognition result is determined as the end of question part in the training-stage speech recognition result.

14. The computer program product of claim 9, wherein said obtaining step comprises generating a data structure that correlates an occurrence number of a word in a word sequence to the rank for the word in the word sequence.

15. A system, comprising:

a processor, configured to;

prepare a plurality of pairs for an information retrieval task, each of the plurality of pairs including (i) a training-stage speech recognition result for a respective sequence of training words and (ii) an answer label corresponding to the training-stage speech recognition result;

obtain, using a search engine, a respective rank for the answer label included in each of the plurality of pairs to obtain a set of ranks;

determine for each of the plurality of pairs, an end of question part in the training-stage speech recognition result based on the set of ranks;

build a classifier that receives a recognition-stage speech recognition result and returns a corresponding end of question part for the recognition-stage speech recognition result, based on the end of question part determined for the plurality of pairs;

split the recognition-stage speech recognition result at the corresponding end of question part of the recognition-stage speech recognition result; and

generate a set of answer candidates for replying to the corresponding end of question part for the recognition-stage speech recognition result; and

a display device for providing the set of answer candidates to a user.

16. A classifier for detecting an end of question part in speech recognition results, the classifier comprising;

a storage unit for storing a plurality of clusters, each of the plurality of clusters comprising at least one partial set of words, each of the partial set of words being a set of words from a beginning of a training-stage speech recognition result to the end of question part of the training-stage speech recognition result;

a processor unit for

calculating, in response to receiving a recognition-stage speech recognition result incrementally in a word by word manner as an input, a similarity between the input and the plurality of clusters; and

outputting the end of question part for the recognition-stage speech recognition result based on the similarity between the input and any of the plurality of clusters being less than a predetermined threshold;

splitting the recognition-stage speech recognition result at the corresponding end of question part of the recognition-stage speech recognition result;

generating a set of answer candidates for replying to the corresponding end of question part for the recognition-stage speech recognition result; and

a display device for providing the set of answer candidates to a user.

17. A method for detecting an end of question part in speech recognition results, the method comprising;

storing, in a storage unit, a plurality of clusters, each of the plurality of clusters comprising at least one partial set of words, each of the partial set of words being a set of words from a beginning of a training-stage speech recognition result to the end of question part of the training-stage speech recognition result;

calculating, by a processor, in response to receiving a recognition-stage speech recognition result incrementally in a word by word manner as an input, a similarity between the input and the plurality of clusters; and

outputting, by the processor, the end of question part for the recognition-stage speech recognition result based on the similarity between the input and any of the plurality of clusters being less than a predetermined threshold;

splitting, by the processor, the recognition-stage speech recognition result at the corresponding end of question part for the recognition-stage speech recognition result;

generating, by the processor, a set of answer candidates for replying to the corresponding end of question part for the recognition-stage speech recognition result; and

providing, by a display device, the set of answer candidates to a user.