Providing a personal assistant module with a selectively-traversable state machine

• US 10,462,619 B2
• Filed: 06/08/2016
• Issued: 10/29/2019
• Est. Priority Date: 06/08/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Diaconu, Adrian, Qian, Jing, Whyte, Jack, Ma, Ning, Yehoshua, Shir
• Primary Examiner(s): Figueroa, Marisol
• Application Number:
  US15/177,327
• Publication Number:
  US 20170359707A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04W 4/16 Communication-related supplementary services, e.g. call-transfer or call-hold
• CPC Class Codes: H04W 76/10 Connection setup
• CPC Class Codes: H04W 88/02 Terminal devices

Abstract

Techniques are described herein for leveraging an online semantic processor to generate a finite state machine to be delivered to and implemented on a resource-constrained device. In various implementations, a user request may be received by a personal assistant module. Data indicative of the user request may be uploaded, e.g., by the personal assistant module, to an online semantic processor. Data may be received, e.g., by the personal assistant module, from the online semantic processor. The received data may represent a state machine having a plurality of candidate states of the personal assistant module. Each candidate state may be potentially responsive to the request. Resources local to the resource-constrained device may be analyzed to ascertain signal(s) unavailable to the online semantic processor. The personal assistant module may then transition to a given state of the plurality of candidate states. The given state may be selected based on the signal(s).

Claims

1. An online semantic processor, comprising:

a communication interface;

one or more processors operably coupled with the communication interface; and

memory operably coupled with the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to;

receive, from a personal assistant module operating on a resource-constrained device, via the communication interface, data indicative of a user request;

generate a state machine comprising a plurality of candidate states that are traversable by the personal assistant module at the resource-constrained device based on one or more signals that are unavailable to the online semantic processor and ascertainable by the personal assistant module from one or more sensors local to the resource-constrained device,

wherein at least a first candidate state of the plurality of candidate states comprises an output to be delivered by the personal assistant module that solicits input selecting between two or more distinct software application states that are potentially responsive to the user request, and

wherein at least a second candidate state of the plurality of candidate states is reachable by the personal assistant module by transitioning past the first candidate state based on one or more of the signals that are unavailable to the online semantic processor, and wherein in the second candidate state at least one of the two or more distinct software application states has been selected automatically based on the one or more of the signals; and

download, to the resource-constrained device, via the communication interface, data representing the state machine.

2. The online semantic processor of claim 1, further comprising instructions to analyze one or more online resources associated with the user to ascertain one or more information signals, wherein the state machine is generated based at least in part on the one or more information signals.

3. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a resource-constrained device, cause the resource constrained device to operate a personal assistant module that performs the following operations:

receiving a user request;

uploading data indicative of the user request to an online semantic processor;

receiving, from the online semantic processor, data representing a state machine comprising a plurality of candidate states of the personal assistant module, wherein at least a first candidate state of the plurality of candidate states comprises an output to be delivered by the personal assistant module that solicits input selecting between two or more software application settings for a given software application installed on the resource-constrained device that is potentially responsive to the request;

analyzing one or more resources local to the resource-constrained device to ascertain one or more signals unavailable to the online semantic processor, wherein the one or more resources include information, stored locally on the resource-constrained device, about use of the given software application installed on the resource-constrained device; and

transitioning, at the resource-constrained device without receiving disambiguating input from a user of the resource-constrained-device, past the first candidate state of the plurality of candidate states to a second state of the plurality of candidate states in which at least one of the two or more software application settings has been selected, wherein the transitioning is based on the one or more signals unavailable to the online semantic processor.

4. The at least one non-transitory computer-readable medium of claim 3, wherein the given software application is configured to operate a camera integral with the resource-constrained device, and the output to be delivered by the personal assistant module includes a request for the user to provide a camera flash setting.

5. The at least one non-transitory computer-readable medium of claim 3, wherein the given software application is configured to operate a camera integral with the resource-constrained device, and the output to be delivered by the personal assistant module includes a request for the user to select between photo and video.

6. The at least one non-transitory computer-readable medium of claim 3, wherein the given software application comprises a media playing application.

7. The at least one non-transitory computer-readable medium of claim 6, wherein the output to be delivered by the personal assistant module includes a request for the user to select between random playback and another playback mode.

8. The at least one non-transitory computer-readable medium of claim 6, wherein the output to be delivered by the personal assistant module includes a request for the user to select a volume level.

9. A method implemented using one or more processors, comprising:

receiving a user request;

uploading data indicative of the user request to an online semantic processor;

receiving, from the online semantic processor, data representing a state machine comprising a plurality of candidate states of the personal assistant module, wherein at least a first candidate state of the plurality of candidate states comprises an output to be delivered by the personal assistant module that solicits input selecting between two or more software application settings for a given software application installed on the resource-constrained device that is potentially responsive to the request;

analyzing one or more resources local to the resource-constrained device to ascertain one or more signals unavailable to the online semantic processor, wherein the one or more resources include information, stored locally on the resource-constrained device, about use of the given software application installed on the resource-constrained device; and

transitioning, at the resource-constrained device without receiving disambiguating input from a user of the resource-constrained-device, past the first candidate state of the plurality of candidate states to a second state of the plurality of candidate states in which at least one of the two or more software application settings has been selected, wherein the transitioning is based on the one or more signals unavailable to the online semantic processor.

10. The method of claim 9, wherein the given software application is configured to operate a camera integral with the resource-constrained device, and the output to be delivered by the personal assistant module includes a request for the user to provide a camera flash setting.

11. The method of claim 9, wherein the given software application is configured to operate a camera integral with the resource-constrained device, and the output to be delivered by the personal assistant module includes a request for the user to select between photo and video.

12. The method of claim 9, wherein the given software application comprises a media playing application.

13. The method of claim 12, wherein the output to be delivered by the personal assistant module includes a request for the user to select between random playback and another playback mode.

14. The method of claim 12, wherein the output to be delivered by the personal assistant module includes a request for the user to select a volume level.