Multi-level voice menu

• US 10,019,993 B2
• Filed: 07/06/2017
• Issued: 07/10/2018
• Est. Priority Date: 01/30/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Nass, Clifford Ivar
• Primary Examiner(s): Pullias, Jesse
• Application Number:
  US15/643,261
• Publication Number:
  US 20170309274A1
• Time in Patent Office: 369 Days
• Field of Search: 704275
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04R 1/08 Mouthpieces; Microphones; Attachments therefor

Abstract

Methods, apparatus, and computer-readable media are described herein related to a user interface (UI) that can be implemented on a head-mountable device (HMD). The UI can include a voice-navigable UI. The voice-navigable UI can include a voice navigable menu that includes one or more menu items. The voice-navigable UI can also present a first visible menu that includes at least a portion of the voice navigable menu. In response to a first utterance comprising one of the one or more menu items, the voice-navigable UI can modify the first visible menu to display one or more commands associated with the first menu item. In response to a second utterance comprising a first command, the voice-navigable UI can invoke the first command. In some embodiments, the voice-navigable UI can display a second visible menu, where the first command can be displayed above other menu items in the second visible menu.

Claims

1. A method, comprising:

at a wearable computing device with a voice-navigable user interface, displaying a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands;

receiving a first utterance at the wearable computing device, the first utterance comprising the speech command corresponding to a first menu item from the one or more menu items;

in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each displayed command comprises a text version of a corresponding speech command from the plurality of speech commands;

receiving a second utterance at the wearable computing device, the second utterance comprising the speech command corresponding to a first command from the one or more commands; and

in response to receiving the second utterance, invoking the first command.

2. The method of claim 1, further comprising:

displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items, wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands; and

wherein the second visible menu displays the first command above the at least one menu item.

3. The method of claim 2, further comprising:

receiving a third utterance at the wearable computing device, the third utterance comprising the speech command corresponding to a second command from the one or more commands, the second command differing from the first command;

after receiving the third utterance, invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

4. The method of claim 3,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

5. The method of claim 3,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a recency of use for the first command and a recency of use for the second command.

6. The method of claim 3,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a combination of a frequency of use and a recency of use for the first command and a combination of a frequency of use and a recency of use for the second command.

7. The method of claim 1, wherein displaying a first visible menu further comprises:

displaying at least a portion of the first visible menu in response to the HMD receiving a hotword utterance comprising a hotword.

8. The method of claim 1, wherein the first menu item comprises an identification of a category.

9. The method of claim 1, wherein at least one menu item of the one or more menu items comprises an identification of a category.

10. The method of claim 1, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises expanding the first visible menu to display the one or more commands in-line with the first menu item.

11. The method of claim 1, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises displaying the one or more commands as a submenu.

12. A wearable computing device comprising:

a processor; and

a non-transitory computer-readable medium configured to store at least program instructions that, when executed by the processor, cause the wearable computing device to carry out functions comprising;

displaying a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands,

receiving a first utterance, the first utterance comprising the speech command corresponding to a first menu item from the one or more menu items,

in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each command comprises a text version of a corresponding speech command from the plurality of speech commands,

receiving a second utterance at the wearable computing device, the second utterance comprising the speech command corresponding to a first command from the one or more commands, and

in response to receiving the second utterance, invoking the first command.

13. The wearable computing device of claim 12, wherein the functions further comprise:

displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items; and

wherein the second visible menu displays the first command above the at least one menu item.

14. The wearable computing device of claim 13, wherein the functions further comprise:

receiving a third utterance, the third utterance comprising a second command, the second command differing from the first command;

after receiving the third utterance, invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

15. The wearable computing device of claim 14, wherein:

the third visible menu further includes the first command; and

an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

16. A non-transitory computer-readable medium configured to store at least program instructions that, when executed by a processor, cause the processor to carry out functions comprising:

displaying, on a display associated with a wearable computing device, a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands;

receiving a first utterance, the first utterance comprising the speech command corresponding to a first menu item from the one or more menu items;

in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each command comprises a text version of a corresponding speech command from the plurality of speech commands;

receiving a second utterance, the second utterance comprising the speech command corresponding to a first command from the one or more commands; and

in response to receiving the second utterance, invoking the first command.

17. The non-transitory computer readable medium of claim 16, wherein the functions further comprise:

displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items; and

wherein the second visible menu displays the first command above the at least one menu item.

18. The non-transitory computer readable medium of claim 17, wherein the functions further comprise:

receiving a third utterance, the third utterance comprising a second command, the second command differing from the first command;

after receiving the third utterance, invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.