Multi-level voice menu

• US 10,319,382 B2
• Filed: 06/06/2018
• Issued: 06/11/2019
• Est. Priority Date: 01/30/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Nass, Clifford Ivar
• Primary Examiner(s): Pullias, Jesse S
• Application Number:
  US16/001,769
• Publication Number:
  US 20180286402A1
• Time in Patent Office: 370 Days
• Field of Search: 704231-257, 704270-275
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04R 1/08 Mouthpieces; Microphones; Attachments therefor

Abstract

Methods, apparatus, and computer-readable media are described herein related to a user interface (UI) that can be implemented on a head-mountable device (HMD). The UI can include a voice-navigable UI. The voice-navigable UI can include a voice navigable menu that includes one or more menu items. The voice-navigable UI can also present a first visible menu that includes at least a portion of the voice navigable menu. In response to a first utterance comprising one of the one or more menu items, the voice-navigable UI can modify the first visible menu to display one or more commands associated with the first menu item. In response to a second utterance comprising a first command, the voice-navigable UI can invoke the first command. In some embodiments, the voice-navigable UI can display a second visible menu, where the first command can be displayed above other menu items in the second visible menu.

Claims

1. A method, comprising:

receiving, by a computing device, a first utterance comprising a hotword;

in response to receiving the first utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising one or more commands associated with the hotword, wherein each displayed command comprises a text version of a corresponding speech command from a plurality of speech commands;

receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and

in response to receiving the second utterance, invoking the first command;

receiving, by the computing device, a third utterance comprising the speech command corresponding to a second command from the one or more commands, the second command differing from the first command; and

after receiving the third utterance;

invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

2. The method of claim 1,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

3. A method, comprising:

receiving, by a computing device, a first utterance comprising a hotword, wherein the first utterance comprising the hotword is received in a first context of the voice navigable interface;

in response to receiving the first utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising one or more commands associated with the hotword, wherein each displayed command comprises a text version of a corresponding speech command from a plurality of speech commands, and wherein the first visible menu is displayed in a second context of the voice navigable interface;

receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and

in response to receiving the second utterance, invoking the first command;

subsequently, in the first context of the voice navigable interface, receiving a third utterance comprising a second hotword,

in response to receiving the third utterance comprising the second hotword, displaying a second visible menu comprising one or more second commands associated with the second hotword, wherein each displayed second command comprises a text version of a corresponding speech command from a second plurality of speech commands;

receiving, by the computing device, a fourth utterance comprising a speech command corresponding to a command from the second plurality of speech commands; and

in response to receiving the fourth utterance, invoking the corresponding command from the second plurality of speech commands.

4. A method, comprising:

receiving, by a computing device, an utterance comprising a hotword;

in response to receiving the utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands;

receiving, by the computing device, a first utterance comprising a speech command corresponding to a first menu item from the one or more menu items;

in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each displayed command comprises a text version of a corresponding speech command from the plurality of speech commands;

receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and

in response to receiving the second utterance, invoking the first command;

displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items, wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands, and wherein the second visible menu displays the first command above the at least one menu item;

receiving a third utterance at the computing device, the third utterance comprising the speech command corresponding to a second command from the one or more commands, the second command differing from the first command;

after receiving the third utterance, invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

5. The method of claim 4, further comprising:

initially, displaying a graphic representation corresponding to the hotword.

6. The method of claim 4,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

7. The method of claim 6,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a recency of use for the first command and a recency of use for the second command.

8. The method of claim 6,

wherein the third visible menu further includes the first command;

and

wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a combination of a frequency of use and a recency of use for the first command and a combination of a frequency of use and a recency of use for the second command.

9. The method of claim 4, wherein the first menu item comprises an identification of a category.

10. The method of claim 4, wherein at least one menu item of the one or more menu items comprises an identification of a category.

11. The method of claim 4, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises expanding the first visible menu to display the one or more commands in-line with the first menu item.

12. The method of claim 4, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises displaying the one or more commands as a submenu.

13. A computing device comprising:

a processor; and

a non-transitory computer-readable medium configured to store at least program instructions that, when executed by the processor, cause the computing device to carry out functions comprising;

displaying a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands,

receiving a first utterance, the first utterance comprising the speech command corresponding to a first menu item from the one or more menu items,

in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each command comprises a text version of a corresponding speech command from the plurality of speech commands,

receiving a second utterance at the computing device, the second utterance comprising the speech command corresponding to a first command from the one or more commands,

in response to receiving the second utterance, invoking the first command;

displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items, and wherein the second visible menu displays the first command above the at least one menu item;

receiving a third utterance, the third utterance comprising a second command, the second command differing from the first command;

after receiving the third utterance, invoking the second command; and

displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

14. The computing device of claim 13, wherein:

the third visible menu further includes the first command; and

an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.