Dialog management context sharing

• US 9,754,591 B1
• Filed: 11/18/2013
• Issued: 09/05/2017
• Est. Priority Date: 11/18/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kumar, Nishant, Thomas, David Robert, Kshirsagar, Sumedha Arvind, Jain, Vikas, Beal, Jeff Bradley, Gopalakrishnan, Ajay, Bharathi, Shishir Sridhar
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Wang, Yi-Sheng
• Application Number:
  US14/083,332
• Time in Patent Office: 1,387 Days
• Field of Search: None
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Features are disclosed for performing functions in response to user requests based on contextual data regarding prior user requests. Users may engage in conversations with a computing device in order to initiate some function or obtain some information. A dialog manager may manage the conversations and store contextual data regarding one or more of the conversations. Processing and responding to subsequent conversations may benefit from the previously stored contextual data by, e.g., reducing the amount of information that a user must provide if the user has already provided the information in the context of a prior conversation. Additional information associated with performing functions responsive to user requests may be shared among applications, further improving efficiency and enhancing the user experience.

Claims

1. A system comprising:

a computer-readable memory storing executable instructions; and

one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least;

obtain first audio data regarding a first utterance of a user, the first utterance associated with a first user request;

perform speech recognition on the first audio data to generate first speech recognition results;

cause a first application to perform a first function responsive to the first user request based on the first speech recognition results, wherein performance of the first function comprises presenting information in a first modality;

receive, from the first application, contextual information associated with the first user request in a format associated with a second application and information regarding use of the contextual information in the format, wherein the contextual information is not provided to the user in response to performing the first function;

store the contextual information and the information regarding use of the contextual information in the format;

subsequently, obtain second audio data regarding a second utterance of the user, the second utterance associated with a second user request;

perform speech recognition on the second audio data to generate second speech recognition results;

determine that the second application is associated with the second speech recognition results;

provide, to the second application, the contextual information associated with the first user request and the information regarding use of the contextual information in the format; and

cause the second application to perform a second function responsive to the second user request based on the contextual information, the information regarding use of the contextual information in the format, and the second speech recognition results, wherein performance of the second function comprises presenting information in a second modality different than the first modality.

2. The system of claim 1, further comprising an electronic data store configured to store a plurality of contextual information received from a plurality of applications.

3. The system of claim 1, wherein the first user request is associated with a first dialog, and wherein the second user request is associated with a second dialog separate from the first dialog.

4. The system of claim 1, wherein the contextual information is associated with an access configuration indicating that the second application may access the contextual information.

5. The system of claim 1, wherein the contextual information comprises one of:

a visual representation of information associated with a response presented to the user audibly, or additional information associated with information presented to the user in response to the first utterance.

6. A computer-implemented method comprising:

under control of one or more computing devices configured with specific computer-executable instructions,

obtaining first speech recognition results associated with a first user request;

performing, using a first application, a first function responsive to the first user request based on the first speech recognition results, wherein performing the first function comprises presenting information in a first modality;

generating, using the first application, contextual information regarding the first user request in a format associated with a second application, wherein the contextual information is not provided to the user in response to the first request;

storing the contextual information and information regarding use of the contextual information in the format;

obtaining second speech recognition results associated with a second user request;

determining to perform a second function responsive to the second user request based on the second speech recognition results; and

performing, using the second application, the second function based at least in part on the contextual information regarding the first user request and the information regarding use of the contextual information in the format, wherein performing the second function comprises presenting the contextual information in a second modality different than the first modality.

7. The computer-implemented method of claim 6, wherein the contextual information comprises at least one of information regarding processing of the first user request, information regarding performance of the first function, information formatted for a different application;

information formatted for a different modality;

or metadata for consumption by a different application.

8. The computer-implemented method of claim 6, further comprising storing contextual information regarding a plurality of user requests.

9. The computer-implemented method of claim 8, wherein performing the second function is additionally based on contextual information regarding an additional user request separate from the first user request and the second user request.

10. The computer-implemented method of claim 8, further comprising limiting an amount of contextual information that is stored, the limiting based on at least one of:

a time associated with an item of contextual information, a size of an item of contextual information, or a number of user requests for which contextual information is stored.

11. The computer-implemented method of claim 6, wherein the first user request is associated with a first dialog, and wherein the second user request is associated with a second dialog separate from the first dialog.

12. The computer-implemented method of claim 6, wherein the contextual information is associated with an access configuration indicating that the second application may access the contextual information.

13. The computer-implemented method of claim 6, further comprising:

generating the contextual information based on the first user request;

determining that the contextual information is not suitable for presentation in the first modality; and

associating the contextual information with the second modality.

14. The computer-implemented method of claim 6, wherein:

presenting the information in the first modality comprises presenting the information through audio; and

presenting the contextual information in the second modality comprises presenting the contextual information visually.

15. The computer-implemented method of claim 6, wherein generating the contextual information regarding the first user request in the format associated with the second application comprises:

determining, by the first application, that the contextual information regarding the first user request is related to the second function, wherein the second function is performed by the second application;

determining, by the first application, a format utilized by the second application to perform the second function; and

generating, by the first application, the contextual information in the format utilized by the second application to perform the second function.

16. The computer-implemented method of claim 6, wherein generating the contextual information regarding the first user request in the format associated with the second application comprises:

determining, by the second application, that the contextual information regarding the first user request is related to the second function, wherein the second function is performed by the second application;

sending, from the second application to the first application, information regarding a format utilized by the second application to perform the second function; and

generating, by the first application, the contextual information in the format utilized by the second application to perform the second function.

17. The computer-implemented method of claim 6, further comprising tagging the contextual information with metadata indicating that the contextual information is related to the second function.

18. One or more non-transitory computer readable media comprising executable code that, when executed, causes one or more computing devices to perform a process comprising:

obtaining first speech recognition results associated with a first user request;

performing, using a first application, a first function responsive to the first user request based on the first speech recognition results, wherein performing the first function comprises presenting information in a first modality;

generating, using the first application, contextual information regarding the first user request in a format associated with a second function of a second application, wherein the contextual information is not provided to the user in response to the first request;

storing the contextual information and information regarding use of the contextual information in the format;

obtaining second speech recognition results associated with a second user request;

determining to perform the second function responsive to the second user request based on the second speech recognition results; and

performing, using the second application, the second function based at least in part on the contextual information regarding the first user request and the information regarding use of the contextual information in the format, wherein performing the second function comprises presenting the contextual information in a second modality different than the first modality.

19. The one or more non-transitory computer readable media of claim 18, wherein the contextual information comprises at least one of information regarding processing of the user request, information regarding performance of the first function, information formatted for a different application;

information formatted for a different modality;

or metadata for consumption by a different application.

20. The one or more non-transitory computer readable media of claim 18, the process further comprising storing contextual information regarding a plurality of user requests.

21. The one or more non-transitory computer readable media of claim 20, wherein performing the second function is additionally based on contextual information regarding an additional user request separate from the first user request and the second user request.

22. The one or more non-transitory computer readable media of claim 20, the process further comprising limiting an amount of contextual information that is stored, the limiting based on at least one of:

a time associated with an item of contextual information, a size of an item of contextual information, or a number of user requests for which contextual information is stored.

23. The one or more non-transitory computer readable media of claim 18, wherein the first user request is associated with a first dialog, and wherein the second user request is associated with a second dialog separate from the first dialog.

24. The one or more non-transitory computer readable media of claim 18, wherein the contextual information is associated with an access configuration indicating that the second application may access the contextual information.