System and method of providing intent predictions for an utterance prior to a system detection of an end of the utterance

• US 10,614,799 B2
• Filed: 11/24/2015
• Issued: 04/07/2020
• Est. Priority Date: 11/26/2014
• Status: Active Grant

• Current Assignee: Cerence Inc.
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Kennewick, Jr., Michael R., Carter, Daniel B.
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US14/951,384
• Publication Number:
  US 20160148610A1
• Time in Patent Office: 1,596 Days
• Field of Search: 704 9
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 2015/228 of application context

Abstract

In certain implementations, intent prediction is provided for a natural language utterance based on a portion of the natural language utterance prior to a system detection of an end of the natural language utterance. In some implementations, a first portion of a natural language utterance of a user may be received. Speech recognition may be performed on the first portion of the natural language utterance to recognize one or more words of the first portion of the natural language utterance. Context information for the natural language utterance may be obtained. Prior to a detection of an end of the natural language utterance, a first intent may be predicted based on the one or more words of the first portion and the context information. One or more user requests may be determined based on the first predicted intent.

Claims

1. A method of determining an intent prediction for a natural language utterance, prior to a system detection of an end of the natural language utterance, based on a portion of the natural language utterance and statistical information that correlates requests that are linked together in that one type of request statistically follows another type of request, the method being implemented on a computer system that includes one or more physical processors executing computer program instructions which, when executed by the one or more physical processors, perform the method, the method comprising:

receiving, by the computer system, a first natural language utterance of a user;

determining, by the computer system, a first type of user request based on the first natural language utterance;

identifying, by the computer system, at least a second type of user request based on the first type of user request and the statistical information that indicates that the first type of request is made via one or more first spoken words followed by the second type of request, wherein the statistical information indicates that users other than the user have made the second type of request after the first type of request, and wherein identifying the second type of user request comprises determining that the users other than the user made the second type of request after having made the first type of request;

receiving, by the computer system, a first portion of a second natural language utterance of the user;

performing, by the computer system, speech recognition on the first portion of the second natural language utterance to recognize one or more words of the first portion of the second natural language utterance;

receiving, by the computer system, visual inputs provided by the user, wherein the visual inputs are streamed to the computer system and processed in parallel;

determining, by the computer system, prior to a detection of an end of the second natural language utterance, a first intent associated with the second natural language utterance based on the first portion of the second natural language utterance and the identified type of second user request;

determining, by the computer system, an intent associated with the visual inputs provided by the user; and

generating, by the computer system, at least one response for presentation to the user, utilizing a pre-fetched result related to at least one of the first intent and the intent associated with the visual inputs.

2. The method of claim 1, further comprising:

obtaining, by the computer system, context information for the second natural language utterance, wherein determining the first intent comprises predicting, prior to the detection of the end of the second natural language utterance, the first intent based on the one or more words of the first portion and the context information.

3. The method of claim 2, further comprising:

determining, by the computer system, prior to the detection of the end of the second natural language utterance, one or more inferred words that the user will utter in the second natural language utterance based on the first determined intent; and

providing, by the computer system, prior to the detection of the end of the second natural language utterance, the one or more words of the first portion and the one or more inferred words for user selection.

4. The method of claim 3, the method further comprising:

receiving, by the computer system, a user selection of the one or more inferred words,

wherein generating the one or more user requests based on the first determined intent comprises generating a user request based on the user selection.

5. The method of claim 4, further comprising:

determining, by the computer system, prior to the detection of the end of the second natural language utterance, one or more second inferred words that the user will utter in the second natural language utterance based on the first determined intent; and

providing, by the computer system, prior to the detection of the end of the second natural language utterance, the one or more second inferred words for user selection.

6. The method of claim 1, further comprising:

obtaining, by the computer system, one or more first results related to the one or more user requests;

receiving, at the computer system, a second portion of the second natural language utterance;

determining, by the computer system, a second intent associated with the second natural language utterance based on the second portion of the second natural language utterance;

generating, by the computer system, a second user request based on the second determined intent;

obtaining, by the computer system, one or more second results related to the second user request;

determining, by the computer system, an intent of the user in speaking at least a portion of the second natural language utterance;

determining, by the computer system, whether at least one of the first intent or the second intent corresponds to the intent of the user; and

generating, by the computer system, a response for presentation to the user;

(a) using the one or more first results responsive to a determination that the first intent corresponds to the intent of the user; and

(b) using the one or more second results responsive to a determination that the second intent corresponds to the intent of the user.

7. The method of claim 6, wherein generating the response comprises generating the response using the one or more first results and without using the one or more second results responsive to a determination that the first intent corresponds to the intent of the user and that the second intent does not correspond to the intent of the user.

8. The method of claim 1, further comprising:

obtaining, by the computer system, one or more first results related to the one or more user requests; and

providing, by the computer system, prior to the detection of the end of the second natural language utterance, the one or more first results for presentation to the user.

9. The method of claim 8, wherein obtaining one or more first results related to the first type of user request comprises:

causing, by the computer system, a search query to be executed based on the first intent;

obtaining, by the computer system, one or more search results of the search query, wherein the one or more first results includes at least the one or more search results.

10. The method of claim 1, further comprising:

performing, by the computer system, speech recognition on one or more portions of the second natural language utterance to recognize one or more words of the one or more portions of the second natural language utterance; and

performing, by the computer system, semantic processing on the one or more words of the one or more portions to detect the end of the second natural language utterance.

11. The method of claim 1, wherein the one or more user requests comprise at least one of a command or a query.

12. The method of claim 1, further comprising:

identifying, by the computer system, one or more domain agents based on the first intent;

providing, by the computer system, the one or more user requests to the one or more domain agents for resolution of the one or more user requests.

13. A system for determining an intent prediction for a natural language utterance, prior to a system detection of an end of the natural language utterance, based on a portion of the natural language utterance and statistical information that correlates requests that are linked together in that one type of request statistically follows another type of request, the system comprising:

one or more physical processors programmed with computer program instructions which, when executed, cause the one or more physical processors to;

receive a first natural language utterance of a user;

determine a first type of user request based on the first natural language utterance;

identify at least a second type of user request based on the first type of user request and the statistical information that indicates that the first type of request is made via one or more first spoken words followed by the second type of request, wherein the statistical information indicates that users other than the user have made the second type of request after the first type of request, and wherein identifying the second type of user request comprises determining that the users other than the user made the second type of request after having made the first type of request;

receive a first portion of a second natural language utterance of the user;

perform speech recognition on the first portion of the second natural language utterance to recognize one or more words of the first portion of the second natural language utterance;

receive visual inputs provided by the user, wherein the visual inputs are streamed to the computer system and processed in parallel;

determine, prior to a detection of an end of the second natural language utterance, a first intent associated with the second natural language utterance based on the first portion of the second natural language utterance and the identified second type of user request;

determine an intent associated with the visual inputs provided by the user; and

generate at least one response for presentation to the user, utilizing a pre-fetched result related to at least one of the first intent and the intent associated with the visual inputs.

14. The system of claim 13, wherein the one or more physical processors are further caused to:

perform speech recognition on the first portion of the second natural language utterance to recognize one or more words of the first portion of the second natural language utterance; and

obtain context information for the second natural language utterance, wherein determining the first intent comprises predicting, prior to the detection of the end of the second natural language utterance, the first intent based on the one or more words of the first portion and the context information.

15. The system of claim 14, wherein the one or more physical processors are further caused to:

determine, prior to the detection of the end of the second natural language utterance, one or more inferred words that the user will utter in the second natural language utterance based on the first determined intent; and

provide, prior to the detection of the end of the second natural language utterance, the one or more words of the first portion and the one or more inferred words for user selection.

16. The system of claim 15, wherein the one or more physical processors are further caused to:

receive a user selection of the one or more inferred words, wherein generating the one or more user requests based on the first determined intent comprises generating a user request based on the user selection.

17. The system of claim 16, wherein the one or more physical processors are further caused to:

determine, prior to the detection of the end of the second natural language utterance, one or more second inferred words that the user will utter in the second natural language utterance based on the first determined intent; and

provide, prior to the detection of the end of the second natural language utterance, one or more second inferred words for user selection.

18. The system of claim 13, wherein the one or more physical processors are further caused to:

obtain one or more first results related to the one or more user requests;

receive a second portion of the second natural language utterance after the receipt of the first portion of the second natural language utterance;

determine a second intent associated with the natural language utterance based on the second portion of the second natural language utterance;

determine a second user request based on the second determined intent;

obtain one or more second results related to the second user request;

determine an intent of the user in speaking at least a portion of the second natural language utterance;

determine whether at least one of the first intent or the second intent corresponds to the intent of the user; and

generate a response for presentation to the user;

(a) using the one or more first results responsive to a determination that the first intent corresponds to the intent of the user; and

(b) using the one or more second results responsive to a determination that the second intent corresponds to the intent of the user.

19. The system of claim 18, wherein generating the response comprises generating the response using the one or more first results and without using the one or more second results responsive to a determination that the first intent corresponds to the intent of the user and that the second intent does not correspond to the intent of the user.

20. The system of claim 13, wherein the one or more physical processors are further caused to:

obtain one or more first results related to the one or more user requests; and

provide, prior to the detection of the end of the second natural language utterance, the one or more first results for presentation to the user.

21. The system of claim 13, wherein the one or more physical processors are further caused to:

perform speech recognition on one or more portions of the second natural language utterance to recognize one or more words of the one or more portions of the

second natural language utterance; and

perform semantic processing on the one or more words of the one or more portions to detect the end of the second natural language utterance.

22. The system of claim 13, wherein the one or more user requests comprise at least one of a command or a query.

23. A method of determining an intent prediction for a natural language utterance, prior to a system detection of an end of the natural language utterance, based on a portion of the natural language utterance and statistical information that correlates requests that are linked together in that one type of request statistically follows another type of request, the method being implemented on a computer system that includes one or more physical processors executing computer program instructions which, when executed by the one or more physical processors, perform the method, the method comprising:

receiving, by the computer system, a first natural language utterance of a user;

determining, by the computer system, a first type of user request based on the first natural language utterance;

identifying, by the computer system, at least a second type of user request based on the first type of user request and the statistical information that indicates that the first type of request is made via one or more first spoken words followed by the second type of request, wherein the statistical information indicates that users other than the user have made the second type of request after the first type of request, and wherein identifying the second type of user request comprises determining that the users other than the user made the second type of request after having made the first type of request;

receiving, by the computer system, a first portion of a second natural language utterance of a user;

performing, by the computer system, speech recognition on the first portion of the second natural language utterance to recognize one or more words of the first portion of the second natural language utterance;

obtaining, by the computer system, context information for the second natural language utterance;

receiving, by the computer system, visual inputs provided by the user, wherein the visual inputs are streamed to the computer system and processed in parallel;

determining, by the computer system, an intent associated with the visual inputs provided by the user;

determining, by the computer system, prior to the detection of the end of the second natural language utterance, a first intent based on the one or more words of the first portion, the context information, and the identified second type of user request;

determining, by the computer system, prior to the detection of the end of the second natural language utterance, one or more inferred words that the user will utter in the second natural language utterance based on the first determined intent;

providing, by the computer system, the one or more words of the first portion and the one or more inferred words for user selection;

receiving, at the computer system, a user selection of the one or more inferred words; and

determining, by the computer system, at least one response for presentation to the user, utilizing a pre-fetched result related to at least one of the first intent, the one or more inferred words and the intent associated with the visual inputs.

24. The method of claim 23, the method further comprising:

determining, by the computer system, prior to the detection of the end of the second natural language utterance, one or more second inferred words that the user will utter in the second natural language utterance based on the first determined intent; and

providing, by the computer system, prior to the detection of the end of the second natural language utterance, the one or more second inferred words for user selection.

25. The method of claim 23, further comprising:

receiving, at the computer system, a third utterance of the user;

determining, by the computer system, third context information for the third utterance based on the one or more words of the first portion of the second natural language utterance and, the one or more inferred words; and

determining, by the computer system, one or more user requests based on the third utterance and the third context information.

26. The method of claim 25, wherein the third utterance is received after the receipt of the second natural language utterance.

27. The method of claim 25, wherein the third utterance is received prior to the receipt of the second natural language utterance.

28. A method of determining an intent prediction for a natural language utterance, prior to a system detection of an end of the natural language utterance, based on a portion of the natural language utterance and statistical information that correlates requests that are linked together in that one type of request statistically follows another type of request, the method being implemented on a computer system that includes one or more physical processors executing computer program instructions which, when executed by the one or more physical processors, perform the method, the method comprising:

receiving, by the computer system, a first natural language utterance of a user;

determining, by the computer system, a first type of user request based on the first natural language utterance;

identifying, by the computer system, at least a second type of user request based on the first type of user request and the statistical information that indicates that the first type of request is made via one or more first spoken words followed by the second type of request, wherein the statistical information indicates that users other than the user have made the second type of request after the first type of request, and wherein identifying the second type of user request comprises determining that the users other than the user made the second type of request after having made the first type of request;

receiving, by the computer system, a first portion of a second natural language utterance of the user;

performing, by the computer system, speech recognition on the first portion of the second natural language utterance to recognize one or more words of the first portion of the second natural language utterance;

receiving, by the computer system, visual inputs provided by the user, wherein the visual inputs are streamed to the computer system and processed in parallel;

determining, by the computer system, prior to a detection of an end of the second natural language utterance, a first intent associated with the second natural language utterance based on the first portion of the second natural language utterance and the identified second type of user request;

determining, by the computer system, an intent associated with the visual inputs provided by the user;

determining, by the computer system, a first user request based on the first intent; and

obtaining, by the computer system, at least one response for presentation to the user, utilizing a pre-fetched result related to at least one of the first user request and the intent associated with the visual inputs.

29. The method of claim 28, further comprising:

receiving, at the computer system, a second portion of the second natural language utterance;

determining, by the computer system, a second intent associated with the second natural language utterance based on the second portion of the second natural language utterance;

determining, by the computer system, a second user request based on the second determined intent;

obtaining, by the computer system, one or more second results related to the first user request;

selecting, by the computer system, at least one of the first intent or the second intent; and

generating, by the computer system, a response for presentation to the user based on the selected one of the first intent or the second intent.

30. The method of claim 29, wherein generating the response comprises generating the response:

(a) using the one or more first results responsive to a determination that the first intent corresponds to the intent of the user; and

(b) using the one or more second results responsive to a determination that the second intent corresponds to the intent of the user.

31. The method of claim 30, wherein generating the response comprises generating the response using the one or more first results and without using the one or more second results responsive to a determination that the first intent corresponds to the intent of the user and that the second intent does not correspond to the intent of the user.