Propagating conversational alternatives using delayed hypothesis binding

• US 10,249,297 B2
• Filed: 07/13/2015
• Issued: 04/02/2019
• Est. Priority Date: 07/13/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khan, Omar Zia, Sarikaya, Ruhi
• Primary Examiner(s): Serrou, Abdelali
• Application Number:
  US14/797,492
• Publication Number:
  US 20170018271A1
• Time in Patent Office: 1,359 Days
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Examples of the present disclosure describe processing by an input understanding system/service. A received input is processed to generate a set of alternatives for recognizing the received input. The set of alternatives is filtered. Filtering comprises ranking the set of alternatives and propagating a plurality of the ranked alternatives for additional processing. The propagated alternatives are processed to generate an expanded set of alternatives for potential hypotheses based on the received input. The expanded set of alternatives is filtered. Filtering comprises ranking alternatives of the expanded set and propagating a plurality of the ranked alternatives of the expanded set for additional processing. The propagated alternatives of the expanded set are evaluated based on application of knowledge data fetched from external resources. A response to the received input is generated. Generating of the response comprises ranking the evaluated alternatives and selecting the response based on the ranked and evaluated alternatives.

Claims

1. A computer-implemented method comprising:

processing, by a speech recognition system, a received input to generate a set of alternatives, wherein the set of alternatives corresponds to hypothetical interpretations of the received input, and wherein the received input is an utterance;

filtering the set of alternatives, wherein the filtering comprises ranking the set of alternatives using machine learning techniques and propagating a plurality of the ranked alternatives for additional processing, wherein the machine learning techniques include comparing confidence metrics for two or more alternatives in the set of alternatives;

processing the propagated alternatives to generate an expanded set of alternatives as potential hypotheses based on the received input, wherein generating the expanded set of alternatives comprises determining one or more domains associated with the input, and generating one or more alternatives associated with the one or more domains;

filtering the expanded set of alternatives, wherein the filtering comprises ranking alternatives of the expanded set and propagating a plurality of the ranked alternatives of the expanded set for additional processing;

evaluating the propagated alternatives of the expanded set based on application of knowledge data fetched from external resources;

generating a response to the received input, wherein the generating comprises ranking the evaluated alternatives and selecting a ranked and evaluated alternative for generating the response; and

providing the response to a user.

2. The computer-implemented method according to claim 1, further comprising retaining data associated with ranked and evaluated alternatives.

3. The computer-implemented method according to claim 2, further comprising utilizing the retained data to evaluate a received subsequent input.

4. The computer-implemented method according to claim 2, further comprising:

processing a received subsequent input to generate a set of subsequent input alternatives for recognition of the received subsequent input;

filtering the set of subsequent input alternatives, wherein the filtering comprises ranking the set of subsequent input alternatives and propagating a plurality of the ranked subsequent input alternatives for additional processing;

processing the propagated subsequent alternatives to generate an expanded set of subsequent input alternatives as potential hypotheses to the received subsequent input;

filtering the expanded set of subsequent input alternatives, wherein the filtering comprises ranking the expanded set and propagating a plurality of the ranked subsequent input alternatives of the expanded set for additional processing; and

processing the propagated subsequent input alternatives of the expanded set including evaluating whether a slot or entity associated with a previous received input is relevant to the received subsequent input and evaluating the propagated subsequent input alternatives based on user response during interaction with an input understanding service.

5. The computer-implemented method according to claim 4, wherein the processing of the propagated subsequent input alternatives comprises generating ranked output alternatives and propagating a plurality of the ranked output alternatives for additional processing.

6. The computer-implemented method according to claim 5, further comprising evaluating the propagated plurality of ranked output alternatives based on application of the knowledge data fetched from the external resources, and generating a response to the received subsequent input, wherein the generating ranks the evaluated output alternatives and selects the response for output from the ranked and evaluated output alternatives.

7. The computer-implemented method according to claim 1, wherein the plurality of propagated and ranked alternatives from the set of alternatives is determined using a threshold that is set based on at least one selected from a group consisting of:

a latency of an input processing component, a capacity of the input processing component, and processing of a set of ranking signals.

8. The computer-implemented method according to claim 5, wherein the plurality of the ranked output alternatives is determined using a threshold that is set based on at least one selected from a group consisting of:

a latency of an input processing component and a capacity of the input processing component, and processing of a set of ranking signals.

19. The computer-implemented method according to claim 1, wherein one or more alternatives in the plurality of the ranked alternatives are ranked based on a determined likelihood that an identified domain associated with the one or more alternatives satisfies an intention of the received utterance.

20. The computer-implemented method according to claim 1, wherein ranking the set of alternatives comprises determining a confidence value for one or more alternatives in the set of alternatives and comparing two or more of the determined confidence values to determine a ranking order.

21. The computer-implemented method according to claim 1, wherein processing the received input comprises at least one of producing an NBest list of alternatives and producing a word confusion network.

9. An input understanding system comprising:

at least one processor; and

a memory operatively connected with the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, perform a method comprising;

processing, using an input recognition component of a speech recognition system, a received input to generate a set of alternatives, wherein the set of alternatives corresponds to hypothetical interpretations of the received input, and wherein the received input is an utterance;

filtering the set of alternatives, wherein the filtering comprises ranking the set of alternatives using machine learning techniques and propagating a plurality of the ranked alternatives from the set to an input understanding component, wherein the machine learning techniques include comparing confidence metrics for two or more alternatives in the set of alternatives;

processing, using the input understanding component, the propagated alternatives to generate an expanded set of alternatives as potential hypotheses based on the received input, wherein generating the expanded set of alternatives comprises determining one or more domains associated with the input, and generating one or more alternatives associated with the one or more domains;

filtering the expanded set of alternatives, wherein the filtering comprises ranking alternatives of the expanded set and propagating a plurality of the ranked alternatives of the expanded set to a knowledge fetch component;

evaluating, using the knowledge fetch component, the propagated alternatives of the expanded set based on application of knowledge data fetched from external resources and propagating the evaluated alternatives to a policy determination component;

generating, using the policy determination component, a response to the received input, wherein the policy determination component ranks the evaluated alternatives and selects the response from the ranked and evaluated alternatives; and

providing the response to a user.

10. The input understanding system according to claim 9, wherein the executed operations further comprising retaining data associated with ranked and evaluated alternatives.

11. The input understanding system according to claim 10, wherein the executed operations further comprising utilizing the retained data to evaluate a received subsequent input.

12. The input understanding system according to claim 10, wherein the executed operations further comprising:

processing, using the input recognition component, a received subsequent input to generate a set of subsequent input alternatives for recognition of the received subsequent input,

filtering the set of subsequent input alternatives, wherein the filtering comprises ranking the set of subsequent input alternatives and propagating a plurality of the ranked subsequent input alternatives to the input understanding component,

processing, using the input understanding component, the propagated subsequent alternatives to generate an expanded set of subsequent input alternatives as the potential hypotheses,

filtering the expanded set of subsequent input alternatives, wherein the filtering comprises ranking the expanded set and propagating a plurality of the ranked subsequent input alternatives of the expanded set to a belief state component, and

processing, using the belief state component, the propagated subsequent input alternatives of the expanded set including evaluating whether a slot or entity associated with a previous received input is relevant to the received subsequent input and evaluating the propagated subsequent input alternatives based on user response during interaction with the input understanding system.

13. The input understanding system according to claim 12, wherein the belief state component processing comprises generating ranked output alternatives and propagating a plurality of the ranked output alternatives to the knowledge fetch component.

14. The input understanding system according to claim 13, wherein the executed operations further comprising evaluating, by the knowledge fetch component, the propagated plurality of ranked output alternatives based on application of the knowledge data fetched from the external resources and propagating the evaluated alternatives to the policy determination component, and

using the policy determination component, generating a response to the received subsequent input, wherein the policy determination component ranks the evaluated output alternatives received from the knowledge fetch component and selects the response for output from the ranked and evaluated output alternatives.

15. The input understanding system according to claim 9, wherein the input understanding system is conversational understanding system, the input recognition component is an automatic speech component, and the input understanding component is a language understanding component.

16. The input understanding system according to claim 9, wherein the plurality of propagated and ranked alternatives from the set of alternatives is determined using a threshold that is set based on at least one selected from a group consisting of:

a latency of the input understanding component and a capacity of the input understanding component, and processing of a set of ranking signals.

22. The input understanding system according to claim 9, wherein the input understanding component generates a triplet comprising one or more domains, one or more intents and one or more tagged slots, and wherein the triplet is used to determine the one or more alternatives.

17. A computer-readable storage device including executable instructions, that when executed on at least one processor, causing the processor to perform operations comprising:

using an automatic speech recognition component of a speech recognition system, processing a received utterance to generate a set of alternatives, wherein the set of alternatives corresponds to hypothetical interpretations of the received input;

filtering the set of alternatives by ranking the set of alternatives using machine learning techniques and propagating a plurality of the ranked alternatives to a language understanding component, wherein the machine learning techniques include comparing confidence metrics for two or more alternatives in the set of alternatives;

using the language understanding component, processing the propagated alternatives to generate an expanded set of alternatives as potential hypotheses based on the received utterance, wherein generating the expanded set of alternatives comprises determining one or more domains associated with the input, and generating one or more alternatives associated with the one or more domains;

filtering the expanded set of alternatives by ranking alternatives of the expanded set and propagating a plurality of the ranked alternatives of the expanded set to a knowledge fetch component;

using the knowledge fetch component, evaluating the propagated alternatives of the expanded set based on application of knowledge data fetched from external resources and propagating the evaluated alternatives to a policy determination component;

using the policy determination component, generating a response to the received utterance, wherein the policy determination component ranks the evaluated alternatives and outputs the response from the ranked and evaluated alternatives; and

providing the response to a user.

18. The computer-readable storage device according to claim 17, where the operations further comprising:

using the automatic speech recognition component, processing a received subsequent input to generate a set of subsequent input alternatives for recognition of the received subsequent input and filtering the set of subsequent input alternatives, wherein the filtering comprises ranking the set of subsequent input alternatives and propagating a plurality of the ranked subsequent input alternatives for additional processing;

using the language understanding component, processing the propagated subsequent alternatives to generate an expanded set of subsequent input alternatives as the potential hypotheses and filtering the expanded set of subsequent input alternatives, wherein the filtering comprises ranking the expanded set and propagating a plurality of the ranked subsequent input alternatives of the expanded set for additional processing;

using a belief state component, processing the propagated subsequent input alternatives of the expanded set including evaluating whether a slot or entity associated with a previous received utterance is relevant to the received subsequent input and evaluating the propagated subsequent input alternatives based on user response during interaction with an input understanding service, wherein processing of the propagated subsequent input alternatives comprises generating ranked output alternatives and propagating a number of the ranked output alternatives for additional processing;

using the knowledge fetch component, evaluating the propagated number of ranked output alternatives based on application of the knowledge data fetched from the external resources; and

using the policy determination component, generating a response to the received subsequent input, wherein the policy determination component ranks the evaluated output alternatives and outputs the response from the ranked and evaluated output alternatives.