Controlling a Speech Recognition Process of a Computing Device

• US 20150161998A1
• Filed: 12/09/2013
• Published: 06/11/2015
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Park, KeeHyun, Cheng, Hsuan-Hsu, Sankaran, Aravind, Kadadi, Parashuram
• Granted Patent: US 9,564,128 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Methods, systems and devices are provided for controlling a speech recognition process on a computing device. A computing device may receive audio signals from a microphone and muscle movement signals from a muscle movement detector, such as an electromyography sensor. The computing device may determine whether the audio signals satisfy an audio characteristic criterion indicative of speech and whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user speaking. The computing device may perform voice recognition processing on the audio signals through a voice recognition algorithm in response to determining that both the audio signals and the head muscle activity signals satisfy their respective criterion indicative of the user speaking, and not perform voice recognition processing of audio signals while either audio signals or head muscle activity signals do not satisfy their respective criterion indicative of the user speaking.

Claims

1. A method of controlling a speech recognition process of a computing device, comprising:

receiving audio signals from a microphone;

receiving muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking; and

not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user of the computing device speaking or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking.

2. The method of claim 1, wherein determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises determining whether an RMS level of the audio signals exceeds an RMS threshold level.

3. The method of claim 1, wherein determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

4. The method of claim 1, wherein determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

determining whether an RMS level of the audio signals exceeds an RMS threshold level indicative of a user speaking;

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

5. The method of claim 1, wherein determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the received head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

6. The method of claim 1, wherein determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the received head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the received head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

7. The method of claim 6, wherein muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the computing device.

8. A computing device, comprising:

a memory;

a microphone;

a muscle movement detector; and

a processor coupled to the memory, the microphone, and the muscle movement detector, wherein the processor is configured with processor-executable instructions to perform operations comprising;

receiving audio signals from the microphone;

receiving muscle activity signals from the muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking; and

not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user of the computing device speaking or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking.

9. The computing device of claim 8, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises determining whether an RMS level of the audio signals exceeds an RMS threshold level.

10. The computing device of claim 8, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

11. The computing device of claim 8, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

determining whether an RMS level of the audio signals exceeds an RMS threshold level indicative of a user speaking;

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

12. The computing device of claim 8, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the received head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

13. The computing device of claim 8, wherein the processor is configured with processor-executable instructions to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the received head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the received head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

14. The computing device of claim 13, wherein the processor is configured with processor-executable instructions to perform operations such that muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the computing device.

15. A wearable computing device, comprising:

means for receiving audio signals from a microphone;

means for receiving muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking;

means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

means for processing the audio signals through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking; and

means for not processing the audio signals through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user of the computing device speaking or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking.

16. The wearable computing device of claim 15, wherein means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises means for determining whether an RMS level of the audio signals exceeds an RMS threshold level.

17. The wearable computing device of claim 15, wherein means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

means for comparing the received audio signals to characteristic tonal patterns of speech;

means for determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

means for determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

18. The wearable computing device of claim 15, wherein means for determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

means for determining whether an RMS level of the audio signals exceeds an RMS threshold level indicative of a user speaking;

means for comparing the received audio signals to characteristic tonal patterns of speech;

means for determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

means for determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

19. The wearable computing device of claim 15, wherein means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises means for determining whether the received head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

20. The wearable computing device of claim 15, wherein means for determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

means for comparing the received head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

means for determining whether the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

means for determining that the received head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

21. The wearable computing device of claim 20, wherein the muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the wearable computing device.

22. A non-transitory processor-readable medium having stored thereon processor-executable instructions configured to cause a processor of a wearable computing device to perform operations comprising:

receiving audio signals from a microphone;

receiving muscle activity signals from a muscle movement detector configured and positioned to measure head muscle activity of a user of the computing device contemporaneously with receiving audio signals from the microphone;

determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking;

determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking;

processing the audio signal through a voice recognition algorithm in response to determining both that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking and that the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking; and

not processing the audio signal through the voice recognition algorithm while either the audio signals do not satisfy an audio characteristic criterion indicative of the user of the computing device speaking or the head muscle activity signals do not satisfy a muscle movement criterion indicative of the user of the computing device speaking.

23. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises determining whether an RMS level of the audio signals exceeds an RMS threshold level.

24. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

25. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that determining whether the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking comprises:

determining whether an RMS level of the audio signals exceeds an RMS threshold level indicative of a user speaking;

comparing the received audio signals to characteristic tonal patterns of speech;

determining whether the received audio signals match the characteristic tonal patterns of speech to a degree that exceeds a tonal-pattern matching threshold; and

determining that the audio signals satisfy an audio characteristic criterion indicative of the user of the computing device speaking in response to determining that the received audio signals match characteristic tonal patterns of speech to a degree that exceeds the tonal-pattern matching threshold.

26. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises determining whether the received head muscle activity signals exceeds a muscle movement signal magnitude threshold indicative of speech.

27. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that determining whether the head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking comprises:

comparing the received head muscle activity signals to muscle movement signal patterns indicative of the user of the computing device speaking;

determining whether the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds a muscle activity-pattern matching threshold; and

determining that the received head muscle activity signals satisfy a muscle movement criterion indicative of the user of the computing device speaking in response to determining that the received head muscle activity signals match the characteristic head muscle activity signal patterns indicative of the user of the computing device speaking to a degree that exceeds the muscle activity-pattern matching threshold.

28. The non-transitory processor-readable medium of claim 27, wherein the stored processor-executable instructions are configured to cause the wearable computing device to perform operations such that muscle movement signal patterns indicative of the user of the computing device speaking comprise head muscle activity signals learned from training by the user and stored in memory of the wearable computing device.