Multi-mode guard for voice commands

• US 9,530,410 B1
• Filed: 04/09/2013
• Issued: 12/27/2016
• Est. Priority Date: 04/09/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Balez, Mathieu, LeBeau, Michael J.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US13/859,588
• Time in Patent Office: 1,358 Days
• Field of Search: 704/231, 704/235, 704/251, 704/270, 704/270.1, 704/275
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Embodiments may be implemented by a computing device, such as a head-mountable display, in order to use a single guard phrase to enable different voice commands in different interface modes. An example device includes an audio sensor and a computing system configured to analyze audio data captured by the audio sensor to detect speech that includes a predefined guard phrase, and to operate in a plurality of different interface modes comprising at least a first and a second interface mode. During operation in the first interface mode, the computing system may initially disable one or more first-mode speech commands, and respond to detection of the guard phrase by enabling the one or more first-mode speech commands. During operation in the second interface mode, the computing system may initially disable a second-mode speech command, and to respond to the guard phrase by enabling the second-mode speech command.

Claims

1. A device comprising:

at least one audio sensor; and

a computing system configured to;

(a) analyze audio data captured by the at least one audio sensor in order to detect speech that includes a predefined guard phrase and (b) cause the device to operate in a plurality of different interface modes comprising at least a first and a second interface mode;

wherein, during operation in the first interface mode, the computing system is configured to;

(i) initially disable one or more first-mode speech commands, (ii) analyze first audio data captured by the audio sensor for the predefined guard phrase, and (iii) respond to a first detection of the predefined guard phrase in the first audio data by enabling the one or more first-mode speech commands, wherein the first detection of the predefined guard phrase occurs during the operation in the first interface mode; and

wherein, during operation in the second interface mode that is non-contemporaneous to the operation in the first interface mode, the computing system is configured to;

(i) initially disable one or more second-mode speech commands, (ii) analyze second audio data captured by the audio sensor for the predefined guard phrase, and (iii) respond to a second detection of the predefined guard phrase in the second audio data by enabling the one or more second-mode speech commands, wherein the second detection of the predefined guard phrase occurs during the operation in the second interface mode, and wherein at least one speech command differs between the one or more first-mode speech commands and the one or more second-mode speech commands.

2. The device of claim 1, wherein, during operation in the first interface mode, the computing system is further configured to:

while the one or more first-mode speech commands are disabled, cause a display of the computing device to display a guard-phrase cue corresponding to the predefined guard phrase; and

in further response to detection of the guard phrase, cause the display of the computing device to display one or more speech-command cues corresponding to the one or more first-mode speech commands.

3. The device of claim 1, wherein, during operation in the second interface mode, the computing system is further configured to:

while the one or more second-mode speech commands are disabled, cause a display of the device to display a guard-phrase cue corresponding to the predefined guard phrase; and

in further response to detection of the guard phrase, cause the display of the device to display one or more speech-command cues corresponding to the one or more second-mode speech commands.

4. The device of claim 1, wherein the one or more second-mode speech commands comprise only one second-mode speech command.

5. The device of claim 4, wherein the one or more first-mode speech commands comprise a plurality of first-mode speech commands.

6. The device of claim 1, wherein, during operation in both the first and the second interface mode, the computing system loads a hotword process to listen for the predefined guard phrase.

7. The device of claim 1:

wherein the computing system enables the one or more first-mode speech by loading a first hotword process to listen for the one or more first-mode speech commands; and

wherein the computing system enables the one or more second-mode speech by loading a second hotword process to listen for the one or more second-mode speech commands.

8. The device of claim 7:

wherein, during operation in the first interface mode, the computing system is further configured to, in further response to the first detection of the predefined guard phrase, cause the display of the device to display one or more first speech-command cues corresponding to the one or more first-mode speech commands;

wherein, during operation in the second interface mode, the computing system is further configured to, in further response to the second detection of the predefined guard phrase, cause the display of the device to display a second speech-command cue corresponding to the one or more second-mode speech commands, wherein display of the second speech-command cue is less visually obtrusive than display of the one or more first speech-command cues.

9. The device of claim 1, wherein the first interface mode corresponds to a home interface, and wherein the one or more first-mode speech commands comprise a set of one or more speech commands that are pre-defined as corresponding to the home interface.

10. The computing device of claim 1, wherein the second interface mode corresponds to an incoming voice call, and wherein the one or more second-mode speech commands comprise at least one of:

(a) an accept-call command, (b) a reject-call command, (c) a send-to-voicemail command, and (d) a send-busy-message command.

11. The device of claim 1, wherein the second interface mode corresponds to an incoming text-based communication, and wherein the one or more second-mode speech commands comprise at least one of:

(a) a view command, (b) a delete command, and (c) a reply command.

12. The device of claim 1, wherein the second interface mode is a video-recording mode, and wherein the one or more second-mode speech commands comprise a stop-recording speech command.

13. The device of claim 12, wherein the computing system is further configured to:

at or near a time when the interface system switches to the video-recording mode, cause an image-capture device to begin recording a video; and

stop recording the video in response to detection of the stop-recording speech command.

14. The device of claim 12, wherein the video includes an audio portion that comprises at least one of the guard phrase and the stop-recording command, wherein the computing system is further configured to remove, from the video, at least part of the audio of the at least one of the guard phrase and the stop-recording command.

15. The device of claim 1, wherein, during operation in at least one of the first and the second interface mode, the interface system is configured to implement a time-out process that disables the one or more enabled speech commands if at least one of the enabled speech commands is not detected within a predetermined period of time after detection of the predefined guard phrase.

21. The device of claim 1, wherein the device is a wearable device comprising a head-mountable display.

16. A computer-implemented method comprising:

a computing device operating in a first interface mode, wherein, during operation in the first interface mode, the computing device performs functions comprising;

initially disabling one or more first-mode speech commands;

analyzing first audio data captured by an audio sensor for a predefined guard phrase; and

responsive to a first detection of the predefined guard phrase in the first audio data, enabling the one or more first-mode speech commands, wherein the first detection of the predefined guard phrase occurs during the operation in the first interface mode; and

a computing device subsequently operating in a second interface mode, wherein, during operation in the second interface mode, the computing device performs functions comprising;

initially disabling one or more second-mode speech commands;

analyzing second audio data captured by the audio sensor for the predefined guard phrase; and

responsive to a second detection of the predefined guard phrase in the first audio data, enabling the one or more second-mode speech commands, wherein the second detection of the predefined guard phrase occurs during the operation in the second interface mode, and wherein at least one speech command differs between the one or more first-mode speech commands and the one or more second-mode speech commands.

17. The method of claim 16, further comprising:

during operation in the first interface mode, the computing device detecting the predefined guard phrase followed by one of the first-mode speech commands; and

responsively switching from operating in the first interface mode to operating in the second interface mode.

18. The method of claim 16:

wherein the first interface mode corresponds to a home-screen interface, wherein the one or more first-mode speech commands comprise a set of one or more speech commands that are predefined as corresponds to the home interface, and wherein one of the first-mode speech commands comprises a speech command to start recording a video; and

wherein the second interface mode corresponds to a video-recording interface, wherein the one or more second-mode speech commands comprises a speech command to stop recording the video.

19. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

operating in a first interface mode, wherein the functions for operating in the first interface mode comprise;

initially disabling one or more first-mode speech commands;

analyzing first audio data captured by an audio sensor for a predefined guard phrase; and

in response to a first detection of the predefined guard phrase in the first audio data, enabling the one or more first-mode speech commands, wherein the first detection of the predefined guard phrase occurs during the operation in the first interface mode; and

subsequently operating in a second interface mode, wherein the functions for operating in the second interface mode comprise;

initially disabling one or more second-mode speech commands;

analyzing second audio data captured by the audio sensor for the predefined guard phrase; and

in response to a second detection of the predefined guard phrase in the first audio data, enabling the one or more second-mode speech commands, wherein the second detection of the predefined guard phrase occurs during the operation in the second interface mode, and wherein at least one speech command differs between the one or more first-mode speech commands and the one or more second-mode speech commands.

20. The non-transitory computer readable medium of claim 19:

wherein the first interface mode corresponds to a home-screen interface, wherein the one or more first-mode speech commands comprise a set of one or more speech commands that are pre-defined as corresponds to the home interface, and wherein one of the first-mode speech commands comprises a speech command to start recording a video; and

wherein the second interface mode corresponds to a video-recording interface, wherein the one or more second-mode speech commands comprises a speech command to stop recording the video.