Voice control user interface with multiple voice processing modules

• US 10,366,688 B2
• Filed: 03/29/2017
• Issued: 07/30/2019
• Est. Priority Date: 10/30/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Gunn, Michael E., Kamdar, Pratik M.
• Primary Examiner(s): Yehl, Walter
• Application Number:
  US15/473,131
• Publication Number:
  US 20170213553A1
• Time in Patent Office: 853 Days
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3234 Power saving characterised by the action undertaken
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09G 2330/021 Power management, e.g. power saving
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G10K 11/341 Circuits therefor
• CPC Class Codes: G10K 2200/10 Beamforming, e.g. time reversal, phase conjugation or similar
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04M 1/6041 Portable telephones adapted for handsfree use
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/11 : Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 29/005 : Microphone arrays
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A method include placing a first processor in a sleep operating mode and running a second processor that is operative to wake the first processor from the sleep operating mode in response to a speech command phrase. The method includes identifying, by the second processor, a speech command phrase segment and performing a control operation in response to detecting the segment in detected speech. The control operation is performed while the first processor is maintained in the sleep operating mode.

Claims

1. A computer-implemented method comprising:

during operation of a mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced;

receiving, at data processing hardware of the mobile device, an utterance of at least an initial portion of a predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance;

detecting, by the data processing hardware, the initial portion of the predetermined command phrase in the utterance; and

in response to detecting the initial portion of the predetermined command phrase in the utterance;

determining, by the data processing hardware, whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and

activating, by the data processing hardware, a display of the mobile device to present display data for output on the display; and

when the remaining portion of the predetermined command phrase is detected in the utterance within the predetermined period of time, commanding, by the data processing hardware, the mobile device to wake-up from the low-power mode and operate in an awake mode; and

during operation of the mobile device in the awake mode in which full functionality of the speech recognition engine is provided, detecting, by the data processing hardware, a voice command in the received utterance that follows the predetermined command phrase.

2. The method of claim 1, further comprising, when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating, by the data processing hardware, the display of the mobile device a second predetermined period of time after presenting the display data for output on the display.

3. The method of claim 1, wherein the display data includes data specifying a time of day, a battery level, and a notification indicator.

4. The method of claim 1, wherein activating the display of the mobile device is based on a setting input by a user of the mobile device that specifies to activate the display of the mobile device in response to receiving the initial portion of the predetermined command phrase.

5. The method of claim 1, further comprising, after detecting the voice command in the received utterance that follows the predetermined command phrase, preforming, by the data processing hardware, an action that corresponds to the voice command.

6. The method of claim 1, wherein operation of the mobile device in the low-power mode maintains applications running on the mobile device in an inactive state.

7. A system comprising:

one or more computers of a mobile device and one or more storage devices of the mobile device storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

during operation of the mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced;

receiving an utterance of at least an initial portion of a predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance;

detecting the initial portion of the predetermined command phrase in the utterance; and

in response to detecting the initial portion of the predetermined command phrase in the utterance;

determining whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and

activating a display of the mobile device to present display data for output on the display; and

when the remaining portion of the predetermined command phrase is detected in the utterance within the predetermined period of time, commanding the mobile device to wake-up from the low-power mode and operate in an awake mode; and

during operation of the mobile device in the awake mode in which full functionality of the speech recognition engine is provided, detecting a voice command in the received utterance that follows the predetermined command phrase.

8. The system of claim 7, wherein the operations further comprise, when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating, by the data processing hardware, the display of the mobile device a second predetermined period of time after presenting the display data for output on the display.

9. The system of claim 7, wherein the display data includes data specifying a time of day, a battery level, and a notification indicator.

10. The system of claim 7, wherein activating the display of the mobile device is based on a setting input by a user of the mobile device that specifies to activate the display of the mobile device in response to receiving the initial portion of the predetermined command phrase.

11. The system of claim 7, wherein the operations further comprise, after detecting the voice command in the received utterance that follows the predetermined command phrase, preforming an action that corresponds to the voice command.

12. The system of claim 7, wherein operation of the mobile device in the low-power mode maintains applications running on the mobile device in an inactive state.

13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

during operation of the mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced;

receiving an utterance of at least an initial portion of a predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance;

detecting the initial portion of the predetermined command phrase in the utterance; and

in response to detecting the initial portion of the predetermined command phrase in the utterance;

determining whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and

activating a display of the mobile device to present display data for output on the display; and

when the remaining portion of the predetermined command phrase is detected in the utterance within the predetermined period of time, commanding the mobile device to wake-up from the low-power mode and operate in an awake mode; and

during operation of the mobile device in the awake mode in which full functionality of the speech recognition engine is provided, detecting a voice command in the received utterance that follows the predetermined command phrase.

14. The non-transitory computer-readable medium of claim 13, wherein the operations further comprise, when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating, by the data processing hardware, the display of the mobile device a second predetermined period of time after presenting the display data for output on the display.

15. The non-transitory computer-readable medium of claim 13, wherein activating the display of the mobile device is based on a setting input by a user of the mobile device that specifies to activate the display of the mobile device in response to receiving the initial portion of the predetermined command phrase.

16. The non-transitory computer-readable medium of claim 13, wherein the operations further comprise:

in response to the voice command, preforming an action that corresponds to the voice command.

17. The non-transitory computer-readable medium of claim 13, wherein the operations further comprise, after detecting the voice command in the received utterance that follows the predetermined command phrase, preforming an action that corresponds to the voice command.