Asynchronous processing of user requests

• US 10,339,934 B2
• Filed: 06/27/2016
• Issued: 07/02/2019
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sung, Sang Soo, Whipp, David P., Qian, Jing
• Primary Examiner(s): Tzeng, Feng-Tzer
• Application Number:
  US15/193,929
• Publication Number:
  US 20170372703A1
• Time in Patent Office: 1,100 Days
• Field of Search: 704275
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for asynchronous execution of client requests. In some implementations, data indicating a user request to a digital assistant is received. An action corresponding to the user request is determined. It is determined that the action is classified as an action to be performed asynchronously to the user request. A confirmation message is sent, for output, and the action is performed asynchronously to the user request.

Claims

1. A method performed by one or more processing devices of a server system, the method comprising:

storing, by the one or more processing devices of the server system, predetermined assignment data that specifies, for each of multiple different action types, whether asynchronous performance or synchronous performance is assigned for handling later-received user requests associated with the action type;

after storing the assignment data, receiving, by the one or more processing devices of the server system, data from a client device over a network, the data indicating a user request from a user to a digital assistant;

determining, by the one or more processing devices of the server system, (i) an action corresponding to the user request and (ii) a particular action type that is associated with the action, wherein the particular action type is one of the multiple different action types;

accessing, by the one or more processing devices of the server system, the predetermined assignment data stored prior to receiving the data indicating the user request, wherein the predetermined assignment data indicates whether asynchronous performance or synchronous performance is assigned for handling user requests associated with the particular action type;

based on the predetermined assignment data and the particular action type associated with the action corresponding to the user request, selecting, by the one or more processing devices of the server system, between (i) performance of the action synchronously with respect to the user request such that the action is performed before the server system performs another action requested by a subsequent user request from the user and (ii) performance of the action asynchronously with respect to the user request such that the server system is configured to perform another action requested by a subsequent user request from the user before or in parallel with the action;

selecting, by the one or more processing devices of the server system, one or more confirmation messages to send to the client device for output by the client device, the selected one or more confirmation messages comprising (i) when the action has been selected for performance synchronously with respect to the user request, an initial confirmation message only that indicates an intent to perform the action, and (ii) when the action has been selected for performance asynchronously with respect to the user request, an initial conformation message that indicates the intent to perform the action, and an additional confirmation message that indicates, in response to execution of the action or in response to a user request about a status of the action, that the action has been executed; and

causing, by the one or more processing devices of the server system, the action to be performed synchronously or asynchronously with respect to the user request according to the selection.

2. The method of claim 1, wherein receiving the data indicating user request to the digital assistant comprises receiving audio data comprising a voice command;

wherein the method comprises performing automated speech recognition to determine a transcription for the voice command; and

wherein determining the action corresponding to the user request comprises determining the action based on the transcription.

3. The method of claim 1, wherein selecting between (i) performance of the action synchronously with respect to the user request and (ii) performance of the action asynchronously with respect to the user request comprises selecting performance of the action asynchronously with respect to the user request;

and

wherein causing the action to be performed synchronously or asynchronously with respect to the user request according to the selection comprises causing the action to be performed asynchronously with respect to the user request.

4. The method of claim 3, wherein the user request is a first user request from a user, and wherein causing the action to be performed asynchronously with respect to the user request comprises receiving one or more other user requests from the user to the digital assistant before performing the action requested by the first user request, wherein the one or more other user requests correspond to actions different from the action requested by the first user request.

5. The method of claim 3, wherein the user request is a first user request, and wherein causing the action to be performed asynchronously with respect to the user request comprises performing actions corresponding to one or more other user requests from the user to the digital assistant before performing the action requested by the first user request, wherein the actions corresponding to one or more other user requests are different from the action requested by the first user request.

6. The method of claim 1, comprising sending the one or more confirmation messages for output.

7. The method of claim 1, further comprising:

using a text-to-speech system to generate audio data comprising synthesized speech; and

sending the generated audio data comprising the synthesized speech.

8. The method of claim 1, wherein the method further comprises determining that the action corresponds to the particular action type;

wherein accessing the predetermined assignment data comprises accessing predetermined assignment data that includes a value, for each specific action type of the multiple different action types, that indicates whether newly requested actions of the specific action type are to be executed synchronously or asynchronously with respect to corresponding user requests that request that the actions of the specific action type be performed; and

wherein selecting between (i) performance of the action synchronously with respect to the user request and (ii) performance of the action asynchronously with respect to the user request comprises determining whether the predetermined assignment data indicates that the actions of the particular action type are assigned to be performed asynchronously.

9. The method of claim 1, wherein the method further comprises:

identifying an application or server configured to perform the action; and

determining the particular action type for the action based on the identified application or server configured to perform the action.

10. The method of claim 3, wherein causing the action to be performed asynchronously with respect to the user request comprises:

storing a record indicating the action that is requested; and

based on the stored record, sending a request to a server system to perform the action.

11. The method of claim 3, wherein causing the action to be performed asynchronously with respect to the user request comprises:

selecting, by the server system, an area of a data storage system based on the determined action;

sending, by the server system, a write request to store data about the action in the selected area;

in response to the write request, issuing, by the server system, a remote procedure call to execute software corresponding to the selected area of the data storage system; and

generating, by the server system and using the executed software, a request to an application server to perform the action.

12. The method of claim 3, wherein causing the action to be performed asynchronously with respect to the user request comprises:

sending a request to a server system to perform the action;

setting a timeout period for completion of the action;

determining that the action is completed before the timeout period ends; and

storing data indicating execution of the action.

13. The method of claim 1, further comprising:

receiving data indicating a second user request to the digital assistant;

determining a second action corresponding to the second user request;

determining that the second action is not classified as an action to be performed asynchronously to the second user request; and

based on determining that the second action is not classified as an action to be performed asynchronously to the second user request, causing the second action to be performed synchronously with respect to the user request.

19. The method of claim 1, wherein storing the predetermined assignment data comprises storing predetermined assignment data that maps each specific action type of the multiple different action types to an execution mode to be used for executing later-requested actions of the specific action type, wherein the execution mode mapped to a specific action type specifies whether the actions of the specific action type will be performed synchronously or asynchronously with respect to respective future user requests that request the actions of the specific action type to be performed;

wherein selecting between (i) performance of the action synchronously with respect to the user request and (ii) performance of the action asynchronously with respect to the user request comprises;

identifying the execution mode that is mapped to the particular action type by the predetermined assignment data; and

assigning the action requested by the user request to be executed using the execution mode that is mapped to the particular action type by the predetermined assignment data.

20. The method of claim 3, wherein selecting performance of the action asynchronously with respect to the user request comprises:

determining that the predetermined assignment data associates an asynchronous execution mode with the particular action type;

identifying a time associated with the action corresponding to the user request;

determining a time range in which asynchronous execution is not allowed; and

determining that the time associated with the action corresponding to the user request is outside the time range in which asynchronous execution is not allowed.

14. A system comprising:

one or more processing devices of a server system and one or more storage devices storing instructions that are operable, when executed by the one or more processing devices, to cause the one or more processing devices to perform operations comprising;

storing, by the one or more processing devices of the server system, predetermined assignment data that specifies, for each of multiple different action types, whether asynchronous performance or synchronous performance is assigned for handling later-received user requests associated with the action type;

after storing the assignment data, receiving, by the one or more processing devices of the server system, data from a client device over a network, the data indicating a user request from a user to a digital assistant;

determining, by the one or more processing devices of the server system, (i) an action corresponding to the user request and (ii) a particular action type that is associated with the action, wherein the particular action type is one of the multiple different action types;

accessing, by the one or more processing devices of the server system, the predetermined assignment data stored prior to receiving the data indicating the user request, wherein the predetermined assignment data indicates whether asynchronous performance or synchronous performance is assigned for handling user requests associated with the particular action type;

based on the predetermined assignment data and the particular action type associated with the action corresponding to the user request, selecting, by the one or more processing devices of the server system, between (i) performance of the action synchronously with respect to the user request such that the action is performed before the server system performs another action requested by a subsequent user request from the user and (ii) performance of the action asynchronously with respect to the user request such that the server system is configured to perform another action requested by a subsequent user request from the user before or in parallel with the action;

selecting, by the one or more processing devices of the server system, one or more confirmation messages to send to the client device for output by the client device, the selected one or more confirmation messages comprising (i) when the action has been selected for performance synchronously with respect to the user request, an initial confirmation message only that indicates an intent to perform the action, and (ii) when the action has been selected for performance asynchronously with respect to the user request, an initial conformation message that indicates the intent to perform the action, and an additional confirmation message that indicates, in response to execution of the action or in response to a user request about a status of the action, that the action has been executed; and

causing, by the one or more processing devices of the server system, the action to be performed synchronously or asynchronously with respect to the user request according to the selection.

15. The system of claim 14, wherein receiving the data indicating user request to the digital assistant comprises receiving audio data comprising a voice command;

wherein the operations comprise performing automated speech recognition to determine a transcription for the voice command; and

wherein determining the action corresponding to the user request comprises determining the action based on the transcription.

16. The system of claim 14, wherein the user request is a first user request from a user, and wherein causing the action to be performed comprises causing the action to be performed asynchronously with respect to the user request by receiving one or more other user requests from the user to the digital assistant before performing the action requested by the first user request, wherein the one or more other user requests correspond to actions different from the action requested by the first user request.

17. The system of claim 14, wherein the operations further comprise:

using a text-to-speech system to generate audio data comprising synthesized speech; and

sending the generated audio data comprising the synthesized speech.

18. One or more non-transitory computer-readable storage media storing computer program instructions that, when executed by one or more processing devices of a server system, cause the one or more processing devices to perform operations comprising:

storing, by the one or more processing devices of the server system, predetermined assignment data that specifies, for each of multiple different action types, whether asynchronous performance or synchronous performance is assigned for handling later-received user requests associated with the action type;

after storing the assignment data, receiving, by the one or more processing devices of the server system, data from a client device over a network, the data indicating a user request to a digital assistant;

determining, by the one or more processing devices of the server system, (i) an action corresponding to the user request and (ii) a particular action type that is associated with the action, wherein the particular action type is one of the multiple different action types;

accessing, by the one or more processing devices of the server system, the predetermined assignment data stored prior to receiving the data indicating the user request, wherein the predetermined assignment data indicates whether asynchronous performance or synchronous performance is assigned for handling user requests associated with the particular action type;

based on the predetermined assignment data and the particular action type associated with the action corresponding to the user request, selecting, by the one or more processing devices of the server system, between (i) performance of the action synchronously with respect to the user request such that the action is performed before the server system performs another action requested by a subsequent user request from the user and (ii) performance of the action asynchronously with respect to the user request such that the server system is configured to perform another action requested by a subsequent user request from the user before or in parallel with the action;

selecting, by the one or more processing devices of the server system, one or more confirmation messages to send to the client device for output by the client device, the selected one or more confirmation messages comprising (i) when the action has been selected for performance synchronously with respect to the user request, an initial confirmation message only that indicates an intent to perform the action, and (ii) when the action has been selected for performance asynchronously with respect to the user request, an initial conformation message that indicates the intent to perform the action, and an additional confirmation message that indicates, in response to execution of the action or in response to a user request about a status of the action, that the action has been executed; and

causing, by the one or more processing devices of the server system, the action to be performed synchronously or asynchronously with respect to the user request according to the selection.