EXTRACTION AND ANALYSIS OF AUDIO FEATURE DATA

• US 20130110521A1
• Filed: 05/30/2012
• Published: 05/02/2013
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hwang, Kyu Woong, You, Kisun, Shah, Peter Jivan, Chan, Kwokleung, Kim, Taesu, Jin, Minho
• Granted Patent: US 9,992,745 B2
• Time in Patent Office: Days
• US Class Current: 704/500
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/18 Vocoders using multiple modes
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: H04W 52/02 Power saving arrangements in wired systems H04L12/12; signaling of mobile application services, e.g. low battery notifications H04W4/20
• CPC Class Codes: H04W 52/028 switching on or off only a part of the equipment circuit blocks
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

A particular method includes transitioning out of a low-power state at a processor. The method also includes retrieving audio feature data from a buffer after transitioning out of the low-power state. The audio feature data indicates features of audio data received during the low-power state of the processor.

Claims

1. A method comprising:

transitioning out of a low-power state at a processor; and

after transitioning out of the low-power state, retrieving audio feature data from a buffer, the audio feature data indicating features of audio data received during the low-power state of the processor.

2. The method of claim 1, wherein the processor comprises a digital signal processor, and further comprising determining whether to activate an application processor based on the retrieved audio feature data.

3. The method of claim 1, wherein the audio feature data is extracted by a coder/decoder (CODEC) that is active during at least a portion of the low-power state of the processor.

4. The method of claim 1, wherein the processor is configured to operate in a plurality of modes, wherein a ratio of CODEC activity to processor activity in a first mode of the plurality of modes is greater than a ratio of CODEC activity to processor activity in a second mode of the plurality of modes, and further comprising determining what mode to operate the processor in based on an application context of the processor.

5. The method of claim 4, wherein during the first mode, the CODEC is always on and the processor is duty-cycled at a first rate, and wherein during the second mode, the CODEC is duty-cycled at a second rate and the processor is duty-cycled at the first rate, wherein the second rate is greater than or equal to the first rate.

6. The method of claim 4, wherein during the first mode, the CODEC is duty-cycled at a first rate and the processor is duty-cycled at a second rate, and wherein during the second mode, the CODEC is duty-cycled at a third rate and the processor is duty-cycled at the second rate, wherein the first rate is greater than the second rate and wherein the third rate is less than or equal to the second rate.

7. The method of claim 4, wherein in a particular mode, the audio feature data corresponds to a plurality of audio frames, and wherein the processor retrieves and processes the audio feature data prior to transitioning back into the low-power state.

8. The method of claim 4, wherein in a particular mode, the audio feature data corresponds to a single audio frame, and wherein the processor retrieves and processes the audio feature data prior to transitioning back into the low-power state.

9. The method of claim 4, wherein:

the processor transitions out of the low-power state in response to expiration of a programmable time period;

in a store-and-forward mode, the programmable time period is less than or equal to a maximum time period that is determined based on a size of the buffer; and

in a direct transfer mode, the programmable time period is independent of the size of the buffer.

10. The method of claim 3, wherein while the processor is in the low-power state, the CODEC:

receives the audio data;

filters the audio data via a plurality of filters;

computes energies of filtered audio data from each of the plurality of filters;

applies a logarithm function to the computed energies to generate the audio feature data;

stores the audio feature data in the buffer; and

discards the audio data.

11. The method of claim 3, wherein the buffer is internal to the CODEC.

12. The method of claim 3, wherein the buffer is external to the CODEC and external to the processor.

13. The method of claim 1, wherein the processor transitions in and out of the low-power state based on a duty cycle.

14. The method of claim 1, further comprising:

transforming the audio feature data;

performing one or more sound recognition operations on the transformed audio feature data; and

transitioning back to the low-power state after performing the one or more sound recognition operations.

15. The method of claim 14, wherein transforming the audio feature data generates a plurality of mel-frequency cepstral coefficients.

16. The method of claim 14, wherein the one or more sound recognition operations include a listen location operation, a keyword detection operation, an audio fingerprinting operation, a target sound detection operation, a novelty detection operation, or any combination thereof.

17. The method of claim 14, wherein the processor is integrated into a wireless device, and further comprising determining whether to activate at least a portion of a mobile station modem of the wireless device based on a result of the one or more sound recognition operations.

18. The method of claim 14, further comprising providing results of the one of more sound recognition operations to each of a plurality of applications.

19. A method comprising:

receiving a frame of audio data at a coder/decoder (CODEC);

extracting audio feature data from the frame of audio data; and

storing the extracted audio feature data in a buffer to be accessible by a duty-cycled processor during an active state of the duty-cycled processor.

20. The method of claim 19, further comprising applying an analog-to-digital converter of the CODEC to the received frame of audio data.

21. The method of claim 19, wherein extracting the audio feature data comprises:

filtering the frame of audio data via a plurality of filters;

computing energies of filtered audio data from each of the plurality of filters; and

applying a logarithm function to the computed energies to generate the audio feature data.

22. The method of claim 21, wherein each of the plurality of filters comprises a mel-band filter that corresponds to a different portion of a human perception frequency scale.

23. The method of claim 19, further comprising discarding the frame of audio data after extracting the audio feature data.

24. An apparatus, comprising:

a processor;

a plurality of filters configured to filter one or more frames of audio data to produce energies of filtered audio data independent of whether the processor is in a low-power state or in an active state;

a converter configured to generate audio feature data based on the energies of the filtered audio data; and

a transformer configured to apply a transform function to the audio feature data to generate transformed data,

wherein, the processor is configured to perform one or more operations on the transformed data after transitioning out of the low-power state to the active state.

25. The apparatus of claim 24, further comprising a coder/decoder (CODEC), wherein the converter comprises a log converter, and wherein the plurality of filters and the log converter are integrated into the CODEC.

26. The apparatus of claim 24, wherein the transformer comprises a discrete cosine transformer that is integrated into the processor.

27. The apparatus of claim 24, further comprising a buffer configured to store the audio feature data.

28. The apparatus of claim 24, wherein the buffer comprises random access memory (RAM).

29. An apparatus comprising:

a processor configured to;

dynamically switch between operating in a first mode and operating in a second mode based on an application context of the processor;

retrieve audio feature data from a buffer after transitioning out of a low-power state, the audio feature data indicating features of audio data received by a coder/decoder (CODEC) while the processor is in the low-power state; and

analyze the audio feature data,

wherein a ratio of CODEC activity to processor activity in the first mode is greater than a ratio of CODEC activity to processor activity in the second mode.

30. The apparatus of claim 29, wherein during the first mode, the CODEC is always on and the processor is duty-cycled at a first rate, and wherein during the second mode, the CODEC is duty-cycled at a second rate and the processor is duty-cycled at the first rate, wherein the second rate is greater than or equal to the first rate.

31. The apparatus of claim 29, wherein during the first mode, the CODEC is duty-cycled at a first rate and the processor is duty-cycled at a second rate, and wherein during the second mode, the CODEC is duty-cycled at a third rate and the processor is duty-cycled at the second rate, wherein the first rate is greater than the second rate and wherein the third rate is greater than or equal to the second rate.

32. The apparatus of claim 29, wherein the application context includes listen location detection, continuous keyword detection, continuous audio fingerprinting, target sound detection, novelty detection, or any combination thereof.

33. The apparatus of claim 29, further comprising the coder/decoder (CODEC), wherein the CODEC is configured to extract the audio feature data from the audio data and to store the extracted audio feature data in the buffer.

34. The apparatus of claim 29, wherein the processor transitions out of the low-power state in response to expiration of a programmable time period.

35. An apparatus comprising:

means for receiving one or more frames of audio data;

means for filtering the one or more frames of audio data to produce filtered audio data independent of whether a processor is in a low-power state or in an active state;

means for generating audio feature data based on the energies of the filtered audio data;

means for transforming the audio feature data to generate transformed data; and

means for performing one or more operations on the transformed data after the processor transitions out of the low-power state to the active state.

36. The apparatus of claim 35, further comprising means for buffering an output of at least one of the means for filtering, the means for generating, and the means for transforming.

37. A non-transitory processor-readable medium comprising instructions that, when executed by a processor, cause the processor to:

dynamically switch between operating in a first mode and operating in a second mode, wherein a ratio of coder/decoder (CODEC) activity to processor activity in the first mode is greater than a ratio of CODEC activity to processor activity in the second mode;

transition out of a low-power state during a duty cycle;

analyze audio feature data that is extracted during the low-power state; and

transition back into the low-power state.

38. The non-transitory processor-readable medium of claim 37, wherein the audio feature data is extracted by a CODEC.

39. A method comprising

receiving sound data at a first component of an electronic device;

performing, at the first component, at least one signal detection operation on the sound data; and

selectively activating a second component of the electronic device based on a result of the at least one signal detection operation,

wherein the second component when active consumes more power at the electronic device than the first component when active.

40. The method of claim 39, wherein the first component comprises a first portion of a coder/decoder (CODEC).

41. The method of claim 40, wherein the second portion comprises a second portion of the CODEC.

42. The method of claim 40, wherein the second component comprises a processor of the electronic device.

43. The method of claim 40, further comprising:

performing, at the second component, at least one second signal detection operation; and

selectively activating a third component of the electronic device based on a result of the at least one second signal detection operation,

wherein the third component when active consumes more power at the electronic device than the second component when active.

44. The method of claim 39, wherein the at least one signal detection operation comprises a root mean square (RMS) classification operation, a band-power classification operation, a time domain operation, a frequency domain operation, a pattern matching operation, a model-based operation, a novelty detection operation, or any combination thereof.