Methods and system for cue detection from audio input, low-power data processing and related arrangements

• US 9,891,883 B2
• Filed: 06/24/2016
• Issued: 02/13/2018
• Est. Priority Date: 12/24/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sharma, Ravi K., Thagadur Shivappa, Shankar, Alattar, Osama M., Bradley, Brett A., Long, Scott M., Kamath, Ajith M., Holub, Vojtech, Brunk, Hugh L., Lyons, Robert G., Gurijala, Aparna R.
• Primary Examiner(s): Saunders, Jr., Joseph
• Application Number:
  US15/192,925
• Publication Number:
  US 20160378427A1
• Time in Patent Office: 599 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 25/09 the extracted parameters being zero crossing rates
• CPC Class Codes: G10L 25/21 the extracted parameters being power information

Abstract

Methods and arrangements involving electronic devices, such as smartphones, tablet computers, wearable devices, etc., are disclosed. One arrangement involves a low-power processing technique for discerning cues from audio input. Another involves a technique for detecting audio activity based on the Kullback-Liebler divergence (KLD) (or a modified version thereof) of the audio input. Still other arrangements concern techniques for managing the manner in which policies are embodied on an electronic device. Others relate to distributed computing techniques. A great variety of other features are also detailed.

Claims

1. A method, comprising:

in an electronic device comprising first and second processors, obtaining audio input corresponding to sound propagating within an aural environment surrounding the electronic device;

at a first processor, processing the audio input to discern a characteristic of the audio input, the processing comprising processing the audio input to determine the presence of audio activity within the audio input and processing the audio input to discern the presence of a digital audio watermark signal within the audio input;

detecting the presence of the digital audio watermark signal from a first sub-band spanning a first frequency range;

decoding auxiliary data from the digital audio watermark signal from second sub-bands spanning a frequency range greater than the first frequency range;

generating an output based upon the processing of the audio to discern the characteristic; and

causing the second processor to enter into a higher power state based on the generated output.

2. The method of claim 1 wherein the first processor is a digital signal processor and the second processor is a CPU, further comprising processing the audio input while the second processor is in an idle or sleep state.

3. The method of claim 2, comprising:

causing the second processor to enter into a higher power state than the idle or sleep state.

4. The method of claim 1, wherein processing the audio input to determine the presence of audio activity comprises determining zero-crossing or short-term energy metrics from the audio input, determining co-occurrence statistics of the zero-crossing or short term energy metrics, and classifying the audio input based on the co-occurrence statistics.

5. The method of claim 1, wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input.

6. The method of claim 1, wherein processing the audio input to discern the presence of a digital audio watermark signal within the audio input comprises:

buffering frames of the audio input;

transforming the frames into spectral magnitude frames;

accumulating spectral magnitude frames into an accumulation buffer;

from the accumulation buffer, extracting spectral magnitude values corresponding to selected bits of the digital audio watermark signal;

correlating the extracted spectral magnitude values with a predetermined signal to produce a correlation metric.

7. The method of claim 6, wherein accumulating comprises:

accumulating spectral magnitude frames into a first accumulation buffer, the spectral magnitude frames corresponding to shift groups; and

accumulating spectral magnitude frames from the first accumulation buffer according to a shift group in a second accumulation buffer.

8. The method of claim 6 comprising:

scaling the spectral magnitude frames in the second accumulation buffer according to plural noise profiles to produce candidate spectral magnitude profiles for each of the noise profiles; and

extracting spectral magnitude values from the candidate spectral magnitude profiles corresponding to selected bits of the digital audio watermark signal.

9. The method of claim 6, comprising:

correlating the extracted spectral magnitude values with predetermined signals to produce correlation metrics for the predetermined signals;

determining a reference spectral magnitude sequence for a predetermined signal detected based on the correlation metrics;

generating a structural strength metric for the reference spectral magnitude sequence;

selecting spectral magnitude sequences from which to decode auxiliary data by identifying spectral magnitude sequences with a structural strength metric that exceeds a threshold decode candidate value.

10. The method of claim 9, comprising:

identifying similar spectral magnitude code sequences based on similarity of time shift or noise profile of the spectral magnitude code sequences to produce sub-sets of similar spectral magnitude code sequences; and

selecting spectral magnitude sequences from which to decode auxiliary data by selecting within a sub-set based on the structural strength metric.

11. The method of claim 1, comprising:

transforming a frame of the audio input with a sparse FFT in a process of detecting the presence of the digital audio watermark signal from the first sub-band.

12. The method of claim 1, comprising:

transforming a frame with a first FFT for audio input sampled at a first sample rate in a process of detecting the presence of the digital audio watermark signal from the first sub-band; and

transforming a frame with a second FFT for audio input sampled at a second sample rate higher than the first sample rate, in a process of decoding auxiliary data from the digital watermark audio signal from the second sub-bands.

13. The method of claim 1 comprising:

processing the input audio in the first processor to derive data therefrom; and

by reference to the derived data, enabling at least one policy embodied on the electronic device.

14. The method of claim 1 comprising:

processing the input audio in the first processor to derive data therefrom; and

by reference to the derived data, disabling at least one policy embodied on the electronic device.

15. The method of claim 1 comprising:

processing the input audio in the first processor to derive a plurality of items of auxiliary data therefrom, wherein at least one of the items of auxiliary data comprises delivery control data; and

by reference to the delivery control data, delivering at least one other of the items of auxiliary data to at least one component of the electronic device.

16. The method of claim 1 comprising:

processing the audio input samples to determine a relative-entropy of the audio input;

estimating the presence of audio activity based on the determined relative-entropy of the audio input.

17. An electronic device comprising:

a microphone;

a first processor; and

a second processor;

the first processor configured with instructions to detect audio activity from audio input corresponding to sound captured by the microphone, and in response, to transform the audio input to detect a digital audio watermark signal within the audio input, the first processor configured with instructions to detect the presence of the digital audio watermark signal from a first sub-band spanning a first frequency range; and

the first processor configured with instructions to decode auxiliary data from the digital audio watermark signal from second sub-bands spanning a frequency range greater than the first frequency range;

the first processor configured to cause the second processor to enter into a higher power state based on detecting the digital audio watermark.

18. The electronic device of claim 17 wherein the first processor is configured to detect audio activity by determining relative entropy of plural frames of audio input.

19. A non-transitory processor readable medium comprising instructions, which when executed by a processor, cause the processor to:

detect audio activity from audio input corresponding to sound captured by a microphone;

in response to detecting the audio activity, detect a digital audio watermark signal within the audio input;

detect the presence of the digital audio watermark signal from a first sub-band spanning a first frequency range;

decode auxiliary data from the digital audio watermark signal from second sub-bands spanning a frequency range greater than the first frequency range; and

cause a second processor to enter a higher power state based on detecting the digital audio watermark.