Structuring verbal commands to allow concatenation in a voice interface in a mobile device

• US 8,452,602 B1
• Filed: 09/15/2012
• Issued: 05/28/2013
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bringert, Bjorn Erik, Pietryka, Pawel, Hodgson, Peter John, Penha, Henrique, Tickner, Simon, Zanolin, Luca, Cohen, Richard Zarek, LeBeau, Michael J.
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US13/621,018
• Time in Patent Office: 255 Days
• Field of Search: 704/235, 704/251, 704/270, 704/275
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A spoken utterance includes at least a first level of a multi-level command format, in which the first level identifies an application. The spoken utterance may also include a second level of the multi-level command format, in which the second level identifies an action. In response to receiving the spoken utterance at a computing device, a representation of the application identified by the first level is displayed on a display of the computing device. If the spoken utterance includes the second level of the multi-level command format, the action identified by the second level is initiated. If the spoken utterance does not include the second level of the multi-level command format, the computing device waits for a predetermined period of time and provides at least one of an audible or visual action prompt if the second level is not received within the predetermined period of time.

Claims

1. A method, comprising:

receiving a spoken utterance at a computing device, wherein the spoken utterance includes at least a first level of a multi-level command format, wherein the first level relates to an application;

causing speech recognition to be performed on the spoken utterance to obtain text corresponding to the spoken utterance;

based on the text, identifying the application;

displaying a representation of the identified application on a display of the computing device;

determining whether the text includes a second level of the multi-level command format, wherein the second level relates to an action;

if the text includes the second level of the multi-level command format, identifying the action;

if the text does not include the second level of the multi-level command format, waiting by the computing device for a first predetermined period of time to receive an additional spoken utterance that includes the second level of the multi-level command format and, if no spoken utterance that includes the second level is received within the first predetermined period of time, providing at least one of an audible or visual action prompt;

determining whether the text includes a third level of the multi-level command format, wherein the third level relates to slot input;

if the text includes both the second level and the third level of the multi-level command format, populating at least one slot of the identified action with the slot input and initiating the identified action with the at least one slot populated with the slot input; and

if the text includes the second level but not the third level of the multi-level command format, waiting for a second predetermined period of time to receive an additional spoken utterance that includes the third level of the multi-level command format and, if no additional spoken utterance that includes the third level is received within the second predetermined period of time, providing at least one of an audible or visual slot prompt.

2. The method of claim 1, wherein displaying a representation of the identified application comprises displaying indications of actions that are available for the identified application.

3. The method of claim 1, further comprising displaying a user interface based on the identified application on the display of the computing device.

4. The method of claim 3, wherein the user interface includes a plurality of slots.

5. The method of claim 4, wherein populating at least one slot of the identified action with the slot input comprises displaying text in at least one of the plurality of slots.

6. The method of claim 4, wherein the identified application is an email application.

7. The method of claim 6, wherein the plurality of slots include a recipient slot, a subject slot, and a message slot.

8. A non-transitory computer readable medium having stored thereon instructions that, when executed by a computing device having at least one processor, cause the computing device to perform functions comprising:

receiving a spoken utterance, wherein the spoken utterance includes at least a first level of a multi-level command format, wherein the first level identifies an application;

in response to receiving the spoken utterance, displaying a representation of the identified application on a display of the computing device;

determining whether the spoken utterance includes a second level of the multi-level command format, wherein the second level relates to an action;

if the spoken utterance includes the second level of the multi-level command format, identifying the action;

if the spoken utterance does not include the second level of the multi-level command format, waiting for a first predetermined period of time to receive an additional spoken utterance that includes the second level of the multi-level command format and, if no additional spoken utterance that includes the second level is received within the first predetermined period of time, providing at least one of an audible or visual action prompt;

determining whether the spoken utterance includes a third level of the multi-level command format, wherein the third level identifies slot input;

if the spoken utterance includes both the second level and the third level of the multi-level command format, populating at least one slot of the identified action with the slot input and initiating the identified action with the at least one slot populated with the slot input; and

if the spoken utterance includes the second level but not the third level of the multi-level command format, waiting for a second predetermined period of time to receive an additional spoken utterance that includes the third level of the multi-level command format and, if no additional spoken utterance that includes the third level is received within the second predetermined period of time, providing at least one of an audible or visual slot prompt.

9. The non-transitory computer readable medium of claim 8, wherein displaying a representation of the identified application comprises displaying indications of actions that are available for the identified application.

10. The non-transitory computer readable medium of claim 8, wherein the functions further comprise displaying a user interface based on the identified application on the display of the computing device.

11. The non-transitory computer readable medium of claim 10, wherein the user interface includes a plurality of slots.

12. The non-transitory computer readable medium of claim 11, wherein populating at least one slot of the identified action with the slot input comprises displaying text in at least one of the plurality of slots.

13. The non-transitory computer readable medium of claim 11, wherein the identified application is an email application.

14. A computing device, comprising:

at least one processor;

data storage;

a plurality of instructions stored in the data storage and executable by the at least one processor to cause the computing device to perform functions, the functions comprising;

receiving a spoken utterance, wherein the spoken utterance includes at least a first level of a multi-level command format, wherein the first level identifies an application;

in response to receiving the spoken utterance, displaying a representation of the identified application on a display of the computing device;

determining whether the spoken utterance includes a second level of the multi-level command format, wherein the second level relates to an action;

if the spoken utterance includes the second level of the multi-level command format, identifying the action;

if the spoken utterance does not include the second level of the multi-level command format, waiting for a first predetermined period of time to receive an additional spoken utterance that includes the second level of the multi-level command format and, if no additional spoken utterance that includes the second level is received within the first predetermined period of time, providing at least one of an audible or visual action prompt;

determining whether the spoken utterance includes a third level of the multi-level command format, wherein the third level identifies slot input;

if the spoken utterance includes both the second level and the third level of the multi-level command format, populating at least one slot of the identified action with the slot input and initiating the identified action with the at least one slot populated with the slot input; and

if the spoken utterance includes the second level but not the third level of the multi-level command format, waiting for a second predetermined period of time to receive an additional spoken utterance that includes the third level of the multi-level command format and, if no additional spoken utterance that includes the third level is received within the second predetermined period of time, providing at least one of an audible or visual slot prompt.

15. The computing device of claim 14, wherein displaying a representation of the identified application comprises displaying indications of actions that are available for the identified application on the display of the computing device.

16. The computing device of claim 14, wherein the functions further comprise displaying a user interface based on the identified application on the display of the computing device.

17. The computing device of claim 16, wherein the user interface includes a plurality of slots.

18. The computing device of claim 17, wherein populating at least one slot of the identified action with the slot input comprises displaying text in at least one of the plurality of slots.

19. The computing device of claim 17, wherein the identified application is an email application.

20. The computing device of claim 19, wherein the plurality of slots include a recipient slot, a subject slot, and a message slot.