Controlling a speech recognition process of a computing device

• US 9,564,128 B2
• Filed: 12/09/2013
• Issued: 02/07/2017
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Park, KeeHyun, Cheng, Hsuan-Hsu, Sankaran, Aravind, Kadadi, Parashuram
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Le, Thuykhanh
• Application Number:
  US14/100,934
• Publication Number:
  US 20150161998A1
• Time in Patent Office: 1,156 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Methods, systems and devices are provided for controlling a speech recognition process on a computing device. A computing device may receive audio signals from a microphone and muscle movement signals from a muscle movement detector, such as an electromyography sensor. The computing device may determine whether the audio signals satisfy an audio characteristic criterion indicative of speech and whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user speaking. The computing device may perform voice recognition processing on the audio signals through a voice recognition algorithm in response to determining that both the audio signals and the head muscle activity signals satisfy their respective criterion indicative of the user speaking, and not perform voice recognition processing of audio signals while either audio signals or head muscle activity signals do not satisfy their respective criterion indicative of the user speaking.

Claims

1. A method of controlling a speech recognition process of a computing device, comprising:

receiving audio signals from a microphone;

receiving head muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device, wherein the audio characteristic criterion comprises a first root mean square (RMS) threshold level indicative of the user directing audio commands to the computing device;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals satisfy the audio characteristic criterion when an RMS level of the audio signals does not exceed the first RMS threshold level; and

not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals do not satisfy the audio characteristic criterion when the RMS level of the audio signals exceeds the first RMS threshold level.

2. The method of claim 1, wherein determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device comprises:

comparing the audio signals to characteristic tonal patterns of speech;

determining whether the audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the audio signals match the characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

3. The method of claim 1, wherein determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

4. The method of claim 1, wherein determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

5. The method of claim 4, wherein the muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the computing device.

6. A computing device, comprising:

a memory;

a microphone;

a muscle movement detector; and

a processor coupled to the memory, the microphone, and the muscle movement detector, wherein the processor is configured with processor-executable instructions to perform operations comprising;

receiving audio signals from the microphone;

receiving head muscle activity signals from the muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device, wherein the audio characteristic criterion comprises a first root mean square (RMS) threshold level indicative of the user directing audio commands to the computing device;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals satisfy the audio characteristic criterion when an RMS level of the audio signals does not exceed the first RMS threshold level; and

not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals do not satisfy the audio characteristic criterion when the RMS level of the audio signals exceeds the first RMS threshold level.

7. The computing device of claim 6, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device comprises:

comparing the audio signals to characteristic tonal patterns of speech;

determining whether the audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the audio signals match the characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

8. The computing device of claim 6, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

9. The computing device of claim 6, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

10. The computing device of claim 9, wherein the processor is configured with processor-executable instructions to perform operations such that the muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in the memory of the computing device.

11. A computing device, comprising:

means for receiving audio signals from a microphone;

means for receiving head muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device, wherein the audio characteristic criterion comprises a first root mean square (RMS) threshold level indicative of the user directing audio commands to the computing device;

means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

means for processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals satisfy the audio characteristic criterion when an RMS level of the audio signals does not exceed the first RMS threshold level; and

means for not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals do not satisfy the audio characteristic criterion when the RMS level of the audio signals exceeds the first RMS threshold level.

12. The computing device of claim 11, wherein means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device comprises:

means for comparing the audio signals to characteristic tonal patterns of speech;

means for determining whether the audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

means for determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the audio signals match the characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

13. The computing device of claim 11, wherein means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises means for determining whether the head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

14. The computing device of claim 11, wherein means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

means for comparing the head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

means for determining whether the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

means for determining that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

15. The computing device of claim 14, wherein the muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the computing device.

16. A non-transitory processor-readable medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations comprising:

receiving audio signals from a microphone;

receiving head muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device, wherein the audio characteristic criterion comprises a first root mean square (RMS) threshold level indicative of the user directing audio commands to the computing device;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user directing audio commands to the computing device, wherein the audio signals satisfy the audio characteristic criterion when an RMS level of the audio signals does not exceed the first RMS threshold level; and

not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking, wherein the audio signals do not satisfy the audio characteristic criterion when the RMS level of the audio signals exceeds the first RMS threshold level.

17. The non-transitory processor-readable medium of claim 16, wherein the stored processor-executable instructions are configured to cause the computing device to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user directing audio commands to the computing device comprises:

comparing the audio signals to characteristic tonal patterns of speech;

determining whether the audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the audio signals match the characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

18. The non-transitory processor-readable medium of claim 16, wherein the stored processor-executable instructions are configured to cause the computing device to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

19. The non-transitory processor-readable medium of claim 16, wherein the stored processor-executable instructions are configured to cause the computing device to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the head muscle activity signals match the muscle movement signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

20. The non-transitory processor-readable medium of claim 19, wherein the stored processor-executable instructions are configured to cause the computing device to perform operations such that the muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the computing device.