Pre-processing partial inputs for accelerating automatic dialog response

• US 10,395,658 B2
• Filed: 05/22/2017
• Issued: 08/27/2019
• Est. Priority Date: 05/22/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Durham, Christopher M., Gattiker, Anne E., Hubregtsen, Thomas S., Hwang, Inseok, Lee, Jinho
• Primary Examiner(s): Azad, Abul K
• Application Number:
  US15/601,542
• Publication Number:
  US 20180336903A1
• Time in Patent Office: 827 Days
• Field of Search: None
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 25/27 characterised by the analysis technique

Abstract

An apparatus comprising a memory and a processor coupled to the memory. The processor receives input from a user, processes a first portion of the input via more than one service module while receiving a second portion of the input to determine a first speculative result, wherein processing the first portion of the input comprises executing at least one service module coupled to a corresponding speculation buffer, processes a second portion of the input via the more than one service module to determine a second speculative result, wherein processing the second portion of the input comprises executing the at least one service module coupled to the corresponding speculation buffer, processes the input via the more than one service module to determine a final output, wherein processing the input comprises executing the at least one service module coupled to the corresponding speculation buffer, and outputs the final output to the user.

Claims

1. An apparatus comprising:

a memory; and

a processor coupled to the memory and configured to;

receive a speech input from a user;

process a first portion of the input via more than one service module, including at least a speech-to-text service module, while continuing to receive more of the input to determine a first speculative result, wherein processing the first portion of the input comprises executing at least one service module coupled to a corresponding speculation buffer;

process a second portion of the input via the more than one service module, including at least the speech-to-text service module, to determine a second speculative result, wherein processing the second portion of the input comprises executing the at least one service module coupled to the corresponding speculation buffer, wherein the second portion of the input includes the first portion of the input as well as more of the input not included in the first portion of the input;

process the input via the more than one service module, including at least the speech-to-text service module, to determine a final output, wherein processing the input comprises executing the at least one service module coupled to the corresponding speculation buffer, wherein the final output is the first speculative result when the second speculative result and the first speculative result are the same, and wherein the final output is the second speculative result when the second speculative result and the first speculative result are not the same; and

output the final output to the user.

2. The apparatus of claim 1, wherein the input is verbal input, and wherein processing the first portion of the verbal input comprises:

determining a first transcript of the first portion of the verbal input according to speech-to-text processing;

determining a first intent expressed in the first transcript of the first portion of the verbal input according to natural language processing;

generating a first dialog response corresponding to the first intent expressed in the first transcript of the first portion of the verbal input according to dialog response generation; and

generating a first speculative output according to the first dialog response.

3. The apparatus of claim 2, wherein processing the second portion of the verbal input comprises:

determining a second transcript of the second portion of the verbal input according to speech-to-text processing;

determining a second intent expressed in the second transcript of the second portion of the verbal input according to natural language processing;

generating a second dialog response corresponding to the second intent expressed in the transcript of the second portion of the verbal input according to dialog response generation when the first intent does not match the second intent;

using the first dialog response as the second dialog response when the first intent matches the second intent; and

generating a second speculative output according to the second dialog response.

4. The apparatus of claim 3, wherein the first speculative output is used as the second speculative output when the first dialog response is the same as the second dialog response.

5. The apparatus of claim 3, wherein the second speculative output is used as the final output.

6. The apparatus of claim 3, wherein processing the verbal input to determine a final result comprises:

determining a final transcript of the verbal input according to speech-to-text processing after an entirety of the verbal input is received;

determining a final intent expressed in the final transcript of the verbal input according to natural language processing when the final transcript does not match the second transcript;

using the second intent as the final intent when the final transcript matches the second transcript;

generating a final dialog response corresponding to the final intent expressed in the final transcript of the verbal input according to dialog response generation when the final intent does not match the second intent;

using the second dialog response as the final dialog response when the second intent matches the final intent; and

generating the final output according to the final dialog response.

7. A computer-implemented method comprising:

obtaining a speech-to-text (STT) result based on an input;

providing the STT result as input to a next stage of a speculative processing system;

determining whether the input exists in a speculation buffer of the next stage resulting from previous processing of a portion of the input;

obtaining a speculative processing result corresponding to the input from the speculation buffer when the input exists in the speculation buffer from the previous processing of the portion of the input;

providing the speculative processing result as a second input to a subsequent stage of the speculative processing system when the input exists in the speculation buffer;

processing the input to determine an output when the input does not exist in the speculation buffer; and

providing the output as the second input to the subsequent stage of the speculative processing system when the input does not exist in the speculation buffer.

8. The computer-implemented method of claim 7, further comprising writing the input to the speculation buffer before processing the input to determine the output when the input does not exist in the speculation buffer.

9. The computer-implemented method of claim 7, further comprising writing the output to the speculation buffer before providing the output as the input to the subsequent stage of the speculative processing system.

10. The computer-implemented method of claim 7, wherein processing the input to determine the output comprises generating an audio file of a dialog response determined at least in part according to the speculative processing result.

11. The computer-implemented method of claim 7, wherein the subsequent stage of the speculative processing system is an output stage configured to play an audio file when the audio file is the input to the subsequent stage of the speculative processing system.

12. The computer-implemented method of claim 7, wherein the STT result is a partial STT result for a portion of a verbal input provided by a user, and wherein the STT result is obtained prior to a completion of the user providing the verbal input.

13. The computer-implemented method of claim 12, wherein the speculation buffer includes output determined according to processing of a previously received partial STT result.

14. A computer program product for performing domain adaptation of a domain, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

receive an input from a user;

perform speech-to-text (STT) processing to determine a transcript of the input;

perform natural language processing (NLP) to determine an intent expressed by the transcript;

determine an output corresponding to the intent and a speculative processing result of a portion of the input, wherein the speculative processing result is a result of speculative processing of the portion of the input determined before an entirety of the input is received, wherein the output corresponds to the speculative processing result when the speculative processing result corresponds to the intent, and wherein the output corresponds to the intent when the speculative processing result does not correspond to the intent; and

transmit the output to an output device to cause presentation of the output to the user.

15. The computer program product of claim 14, wherein the input is a verbal input, and wherein executing the program instructions further causes the processor to:

perform STT processing on the portion of the verbal input to determine a transcript of the portion of the verbal input;

perform NLP on the transcript of the portion of the verbal input to determine an intent expressed by the transcript of the portion of the verbal input;

generate a dialog response according to the intent expressed by the transcript of the portion of the verbal input; and

generate an audio file according to the dialog response.

16. The computer program product of claim 15, wherein determining the output corresponding to the intent and the speculative processing result of the portion of the verbal input comprises using at least one of the intent expressed by the transcript of the portion of the verbal input, the dialog response, or the audio file in determining the output corresponding to the intent.

17. The computer program product of claim 15, wherein executing the program instructions further causes the processor to present the output to the user by playing the audio file via a speaker.

18. The computer program product of claim 14, wherein the input is a verbal input, and wherein executing the program instructions further causes the processor to:

perform second STT processing on a second portion of the verbal input to determine a second transcript of the second portion of the verbal input;

perform second NLP on the second transcript of the second portion of the verbal input to determine a second intent expressed by the second transcript of the second portion of the verbal input;

generate a second dialog response according to the second intent expressed by the second transcript of the second portion of the verbal input; and

generate a second audio file according to the second dialog response.

19. The computer program product of claim 18, wherein determining the output corresponding to the intent and the speculative processing result of the portion of the verbal input comprises using at least one of the second intent expressed by the second transcript of the portion of the verbal input, the second dialog response, or the second audio file in determining the output corresponding to the intent.