Multi-mode guard for voice commands

• US 10,181,324 B2
• Filed: 12/06/2016
• Issued: 01/15/2019
• Est. Priority Date: 04/09/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: LeBeau, Michael J., Balez, Mat
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US15/371,027
• Publication Number:
  US 20170084276A1
• Time in Patent Office: 770 Days
• Field of Search: 704231, 704235, 704251, 704270, 7042701, 704275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Embodiments may be implemented by a computing device, such as a head-mountable display, in order to use a single guard phrase to enable different voice commands in different interface modes. An example device includes an audio sensor and a computing system configured to analyze audio data captured by the audio sensor to detect speech that includes a predefined guard phrase, and to operate in a plurality of different interface modes comprising at least a first and a second interface mode. During operation in the first interface mode, the computing system may initially disable one or more first-mode speech commands, and respond to detection of the guard phrase by enabling the one or more first-mode speech commands. During operation in the second interface mode, the computing system may initially disable a second-mode speech command, and to respond to the guard phrase by enabling the second-mode speech command.

Claims

1. A device comprising:

at least one audio sensor; and

a computing system configured to;

(a) analyze audio data captured by the at least one audio sensor in order to detect speech that includes a predefined guard phrase and (b) cause the device to operate in a plurality of different interface modes comprising at least a first and a second interface mode;

wherein, during operation in the first interface mode, the computing system is configured to initially disable one or more first-mode speech commands, and to respond to detection of the guard phrase by enabling the one or more first-mode speech commands;

wherein, during operation in the second interface mode, the computing system is configured to initially disable one or more second-mode speech commands, and to respond to detecting the guard phrase by enabling the one or more second-mode speech commands; and

wherein the computing system is further configured to set a duration of a time-out period based at least in part on a current interface mode, wherein the device disables enabled speech commands if at least one enabled speech command is not detected within the time-out period of time after detection of the guard phrase, and wherein the duration is set to a first value during operation in the first interface mode, and is set to a second value during operation in the second interface mode, wherein the second value is different from the first value.

2. The device of claim 1, wherein the first interface mode is a video-recording mode and the second interface mode is a home-screen mode, and wherein first value of the time-out period is less than the second value.

3. The device of claim 1, wherein, during operation in the first interface mode, the computing system is further configured to:

while the one or more first-mode speech commands are disabled, cause a display of the computing device to display a guard-phrase cue corresponding to the guard phrase; and

in further response to detection of the guard phrase, cause the display of the computing device to display one or more speech-command cues corresponding to the one or more first-mode speech commands.

4. The device of claim 1, wherein, during operation in the second interface mode, the computing system is further configured to:

while the one or more second-mode speech commands are disabled, cause a display of the device to display a guard-phrase cue corresponding to the guard phrase; and

in further response to detection of the guard phrase, cause the display of the device to display one or more speech-command cues corresponding to the one or more second-mode speech commands.

5. The device of claim 1, wherein, during operation in both the first and the second interface mode, the computing system loads a hotword process to listen for the guard phrase.

6. The device of claim 1:

wherein the computing system enables the one or more first-mode speech by loading a first hotword process to listen for the one or more first-mode speech commands; and

wherein the computing system enables the one or more second-mode speech by loading a second hotword process to listen for the one or more second-mode speech commands.

7. The device of claim 6:

wherein, during operation in the first interface mode, the computing system is further configured to, in further response to detection of the guard phrase, cause the display of the device to display one or more first speech-command cues corresponding to the one or more first-mode speech commands, and

wherein, during operation in the second interface mode, the computing system is further configured to, in further response to detection of the guard phrase, cause the display of the device to display a second speech-command cue corresponding to the one or more second-mode speech commands, wherein display of the second speech-command cue is less visually obtrusive than display of the one or more first speech-command cues.

8. The device of claim 1, wherein the first interface mode corresponds to a home interface, and wherein the one or more first-mode speech commands correspond to one or more actions that are available via the home interface.

9. The computing device of claim 1, wherein the second interface mode corresponds to an incoming voice call, and wherein the one or more second-mode speech commands comprise at least one of:

(a) an accept-call command, (b) a reject-call command, (c) a send-to-voicemail command, and (d) a send-busy-message command.

10. The device of claim 1, wherein the second interface mode corresponds to an incoming text-based communication, and wherein the one or more second-mode speech commands comprise at least one of:

(a) a view command, (b) a delete command, and (c) a reply command.

11. The device of claim 1, wherein the second interface mode is a video-recording mode, and wherein the one or more second-mode speech commands comprise a stop-recording speech command.

12. The device of claim 11, wherein the computing system is further configured to:

at or near a time when the interface system switches to the video-recording mode, cause an image-capture device to begin recording a video; and

stop recording the video in response to detection of the stop-recording speech command, wherein the video includes an audio portion that captures audio comprising at least one of the guard phrase and the stop-recording command, wherein the computing system is further configured to remove, from the video, at least part of the audio of the at least one of the guard phrase and the stop-recording command.

13. The device of claim 1, wherein, during operation in at least one of the first and the second interface mode, the interface system is configured to implement a time-out process that disables the one or more enabled speech commands if at least one of the enabled speech commands is not detected within a predetermined period of time after detection of the guard phrase.

14. A computer-implemented method comprising:

a computing device operating in a first interface mode, wherein, during operation in the first interface mode, the computing device;

initially disables one or more first-mode speech commands;

responds to detection of a guard phrase by enabling the one or more first-mode speech commands; and

setting a duration of a time-out period to a first value, wherein the one or more first-mode speech commands are disabled when no first-mode speech command is detected within the time-out period after detection of the guard phrase;

a computing device operating in a second interface mode, wherein, during operation in the second interface mode, the computing device;

initially disables one or more second-mode speech commands;

responds to detection of a guard phrase by enabling the one or more second-mode speech commands; and

setting the duration of the time-out period to a second value different from the first value, wherein the one or more second-mode speech commands are disabled when no second-mode speech command is detected within the time-out period after detection of the guard phrase, wherein the second value is different from the first value.

15. The method of claim 14, wherein the first interface mode is a video-recording mode and the second interface mode is a home-screen mode, and wherein first value of the time-out period is less than the second value.

16. A device comprising:

at least one audio sensor; and

a computing system configured to;

(a) analyze audio data captured by the at least one audio sensor in order to detect speech that includes a predefined guard phrase and (b) cause the device to operate in a plurality of different interface modes comprising at least a first and a second interface mode;

wherein, during operation in the first interface mode, the computing system is configured to;

(i) initially disable one or more first-mode speech commands, (ii) analyze first audio data captured by the audio sensor for the predefined guard phrase, and (iii) respond to a first detection of the predefined guard phrase in the first audio data by enabling the one or more first-mode speech commands, wherein the first detection of the predefined guard phrase occurs during the operation in the first interface mode, and (iv) set a duration of a time-out period to a first value, wherein the one or more second-mode speech commands are disabled when no second-mode speech command is detected within the time-out period after detection of the guard phrase; and

wherein, during operation in the second interface mode that is non-contemporaneous to the operation in the first interface mode, the computing system is configured to;

(i) initially disable one or more second-mode speech commands, (ii) analyze second audio data captured by the audio sensor for the predefined guard phrase, (iii) respond to a second detection of the predefined guard phrase in the second audio data by enabling the one or more second-mode speech commands, and (iv) set the duration of the time-out period to a second value different from the first value, wherein the one or more second-mode speech commands are disabled when no second-mode speech command is detected within the time-out period after detection of the guard phrase, wherein the second value is different from the first value.

17. The device of claim 16, wherein the first interface mode is a video-recording mode and the second interface mode is a home-screen mode, and wherein first value of the time-out period is less than the second value.