METHODS AND APPARATUS FOR DETECTING A VOICE COMMAND

• US 20160358605A1
• Filed: 08/16/2016
• Published: 12/08/2016
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ganong, William F. III, Van Mulbregt, Paul Adrian, Sejnoha, Vladimir, Wilson, Glen Edward
• Granted Patent: US 11,087,750 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04W 40/005 Routing actions in the presence of nodes in sleep or doze mode
• CPC Class Codes: H04W 52/0251 using monitoring of local events, e.g. events related to user activity
• CPC Class Codes: H04W 52/0254 detecting a user operation or a tactile contact or a motion of the device
• CPC Class Codes: H04W 52/0261 managing power supply demand, e.g. depending on battery level
• CPC Class Codes: H04W 88/02 Terminal devices
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

According to some aspects, a method of monitoring an acoustic environment of a mobile device, at least one computer readable medium encoded with instructions that, when executed, perform such a method and/or a mobile device configured to perform such a method is provided. The method comprises receiving acoustic input from the environment of the mobile device while the mobile device is operating in the low power mode, detecting whether the acoustic input includes a voice command based on performing a plurality of processing stages on the acoustic input, wherein at least one of the plurality of processing stages is performed while the mobile device is operating in the low power mode, and using at least one contextual cue to assist in detecting whether the acoustic input includes a voice command.

Claims

1. A mobile device comprising:

at least one input configured to receive acoustic input from an environment of the mobile device while the mobile device is operating in a low power mode; and

at least one processor configured to perform, while in the low power mode, at least one processing stage on the acoustic input to evaluate whether the acoustic input includes a voice command, the at least one processing stage including transmitting at least a portion of the acoustic input to at least one server via a network for processing by the at least one server at least to evaluate whether the acoustic input includes a voice command,

wherein the mobile device is configured to initiate responding to the voice command when the at least one processing stage determines that the acoustic input includes a voice command.

2. The mobile device of claim 1, wherein the at least one processor comprises a first processor configured to perform at least one first processing stage to evaluate whether the acoustic input includes a voice command and a second processor configured to perform at least one second processing stage to evaluate whether the acoustic input includes a voice command.

3. The mobile device of claim 2, wherein the first processor is configured to perform at least one first processing stage prior to the second processor being engaged to perform at least one second processing stage.

4. The mobile device of claim 3, wherein the first processor is a digital signal processor configured to operate at a lower power than the second processor.

5. The mobile device of claim 4, wherein the first processor is configured to perform at least one voice activity detection processing stage including spectral analysis on the acoustic input to evaluate whether the spectrum of the acoustic input is indicative of voice activity, periodicity analysis to evaluate whether the signal periodicity is indicative of voice activity and/or phone loop analysis to evaluate whether the acoustic input includes speech.

6. The mobile device of claim 4, wherein the first processor is configured to perform limited vocabulary automatic speech recognition on the acoustic input to detect whether the acoustic input includes an explicit voice trigger.

7. The mobile device of claim 4, wherein the second processor is configured to perform automatic speech recognition on the acoustic input, at least one classification processing stage and/or at least one natural language processing stage to understand the meaning of the voice command.

8. The mobile device of claim 7, wherein the second processor is configured to perform limited vocabulary automatic speech recognition on the acoustic input to detect whether the acoustic input includes an explicit voice trigger.

9. The mobile device of claim 7, wherein the mobile device is further configured to provide, prior to activating the second processor or turning on a display of the mobile device, at least one indication to the user to confirm that the acoustic input was received and is being processed.

10. The mobile device of claim 3, wherein the mobile device is further configured to transition to an active mode to respond to the voice command when either the at least one first processing stage or the at least one second processing stage determines that the acoustic input includes a voice command.

11. The mobile device of claim 1, wherein at least one task specified by the voice command is performed with the mobile device remaining in the idle mode.

12. The mobile device of claim 1, wherein the mobile device is further configured to perform one or more default actions when a specific task could not be ascertained from the acoustic input.

13. The mobile device of claim 3, wherein performing the at least one first processing stage comprises evaluating at least one contextual cue selected from a group consisting of:

time of day, motion of the mobile device, location of the mobile device, an occurrence of an event on the mobile device, recent activity on the mobile device, and state of a battery powering the mobile device.

14. The mobile device of claim 3, wherein performing the at least one second processing stage comprises evaluating at least one contextual cue selected from a group consisting of:

time of day, motion of the mobile device, location of the mobile device, an occurrence of an event on the mobile device, recent activity on the mobile device, and state of a battery powering the mobile device.

15. At least one non-transitory computer readable medium storing instructions that, when executed on at least one processor, perform a method of monitoring an acoustic environment of a mobile device to receive acoustic input from the environment of the mobile device while the mobile device is operating in a low power mode, the method comprising:

performing, while in the low power mode, at least one processing stage on the acoustic input to evaluate whether the acoustic input includes a voice command, the at least one processing stage including transmitting at least a portion of the acoustic input to at least one server via a network for processing by the at least one server at least to evaluate whether the acoustic input includes a voice command; and

initiating responding to the voice command when the at least one processing stage determines that the acoustic input includes a voice command.

17. The at least one non-transitory computer readable medium of claim 15, further comprising performing limited vocabulary automatic speech recognition on the acoustic input to detect whether the acoustic input includes an explicit voice trigger.

18. The at least one non-transitory computer readable medium of claim 15, further comprising providing, while in the low power mode, at least one indication to the user to confirm that the acoustic input was received and is being processed.

19. The at least one non-transitory computer readable medium of claim 15, further comprising performing at least one task specified by the voice command with the mobile device remaining in the idle mode.

20. The at least one non-transitory computer readable medium of claim 15, further comprising evaluating, while in the low power mode, at least one contextual cue selected from a group consisting of:

time of day, motion of the mobile device, location of the mobile device, an occurrence of an event on the mobile device, recent activity on the mobile device, and state of a battery powering the mobile device.

16. The at least one non-transitory computer readable medium of claim 16, wherein the first processor is configured to perform at least one first processing stage prior to the second processor being engaged to perform at least one second processing stage.