METHODS AND SYSTEM FOR CUE DETECTION FROM AUDIO INPUT, LOW-POWER DATA PROCESSING AND RELATED ARRANGEMENTS

• US 20200133625A1
• Filed: 10/28/2019
• Published: 04/30/2020
• Est. Priority Date: 12/24/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sharma, Ravi K., Thagadur Shivappa, Shankar, Alattar, Osama M., Bradley, Brett A., Long, Scott M., Kamath, Ajith M., Holub, Vojtech, Brunk, Hugh L., Lyons, Robert G., Gurijala, Aparna R.
• Granted Patent: US 11,080,006 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 25/09 the extracted parameters being zero crossing rates
• CPC Class Codes: G10L 25/21 the extracted parameters being power information

Abstract

Methods and arrangements involving electronic devices, such as smartphones, tablet computers, wearable devices, etc., are disclosed. One arrangement involves a low-power processing technique for discerning cues from audio input. Another involves a technique for detecting audio activity based on the Kullback-Liebler divergence (KLD) (or a modified version thereof) of the audio input. Still other arrangements concern techniques for managing the manner in which policies are embodied on an electronic device. Others relate to distributed computing techniques. A great variety of other features are also detailed.

Claims

1. A method, comprising:

obtaining audio input;

at a first processor, processing the audio input to discern a characteristic of the audio input;

wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input,

the processing of the audio input to discern the auxiliary data comprising;

buffering frames of the audio input, transforming the frames into spectral magnitude frames, accumulating spectral magnitude frames into an accumulation buffer, extracting spectral magnitude values corresponding to selected bits of the digital audio watermark signal, and correlating the extracted spectral magnitude values with a predetermined signal to produce a correlation metric;

generating an output based upon the processing to discern the characteristic; and

controlling an operation of a second processor distinct from the first processor based on the generated output.

2. The method of claim 1, wherein the first and second processors are components of an electronic device, the method further comprising generating an audio signal corresponding to sound propagating within an aural environment surrounding the electronic device, wherein the obtained audio input comprises a plurality of samples of the audio signal.

3. The method of claim 2, wherein the second processor is a CPU.

4. The method of claim 3, wherein the first processor is a digital signal processor.

5. The method of claim 3, further comprising processing the audio input while the second processor is in an idle or sleep state.

6. The method of claim 5, wherein controlling an operation of the second processor comprises causing the second processor to enter into a higher power state than the idle or sleep state.

7. The method of claim 1, wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to determine the presence of audio activity within the audio input.

8. The method of claim 7, wherein processing the audio input to determine the presence of audio activity comprises determining zero-crossing or short-term energy metrics from the audio input, determining co-occurrence statistics of the zero-crossing or short term energy metrics, and classifying the audio input based on the co-occurrence statistics.

10. The method of claim 1, wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input.

12. The method of claim 1, wherein accumulating comprises:

accumulating spectral magnitude frames into a first accumulation buffer, the spectral magnitude frames corresponding to shift groups; and

accumulating spectral magnitude frames from the first accumulation buffer according to shift group in a second accumulation buffer.

13. The method of claim 1 comprising:

scaling the spectral magnitude frames in the second accumulation buffer according to plural noise profiles to produce candidate spectral magnitude profiles for each of the noise profiles; and

extracting spectral magnitude values from the candidate spectral magnitude profiles corresponding to selected bits of the digital audio watermark signal.

14. The method of claim 1, comprising:

correlating the extracted spectral magnitude values with predetermined signals to produce correlation metrics for the predetermined signals;

determining a reference spectral magnitude sequence for a predetermined signal detected based on the correlation metrics;

generating a structural strength metric for the reference spectral magnitude sequence;

selecting spectral magnitude sequences from which to decode auxiliary data by identifying spectral magnitude sequences with a structural strength metric that exceeds a threshold decode candidate value.

15. The method of claim 14, comprising:

identifying similar spectral magnitude code sequences based on similarity of time shift or noise profile of the spectral magnitude code sequences to produce sub-sets of similar spectral magnitude code sequences; and

selecting spectral magnitude sequences from which to decode auxiliary data by selecting within a sub-set based on the structural strength metric.

16. The method of claim 1, comprising:

detecting presence of the digital watermark signal from a first sub-band spanning a first frequency range; and

decoding auxiliary data from the digital watermark signal from second sub-bands spanning a frequency range greater than the first frequency range.

17. The method of claim 16, comprising:

transforming the frame with a sparse FFT in a process of detecting the presence of the digital watermark signal from the first sub-band.

18. The method of claim 16, comprising:

transforming a frame with a first FFT for audio input sampled at a first sample rate in a process of detecting the presence of the digital watermark signal from the first sub-band; and

transforming a frame with a second FFT for audio input sampled at a second sample rate higher than the first sample rate, in a process of decoding auxiliary data from the digital watermark signal from the second sub-bands.

22. The method of claim 1, comprising:

obtaining a plurality of samples of audio input;

processing the audio input samples to determine a relative-entropy of the audio input;

estimating the presence of audio activity based on the determined relative-entropy of the audio input.

9. (canceled)

11. (canceled)

19-21. -21. (Canceled)

23. A method, comprising:

receiving a captured audio signal corresponding to sound captured by a microphone of an electronic device comprising a CPU; and

processing the captured audio signal to perform at least one of the following;

estimate a likelihood that the captured sound has encoded therein a watermark signal;

detect a watermark signal encoded within the captured sound; and

decode a watermark signal to extract auxiliary data from the captured sound.

24. The method of claim 23, further comprising processing the captured audio when the CPU is in a sleep or idle state.

25. The method of claim 23, further comprising processing the captured audio when the device is in a global sleep state.