Methods and system for cue detection from audio input, low-power data processing and related arrangements

• US 10,459,685 B2
• Filed: 02/09/2018
• Issued: 10/29/2019
• Est. Priority Date: 12/24/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sharma, Ravi K., Thagadur Shivappa, Shankar, Alattar, Osama M., Bradley, Brett A., Long, Scott M., Kamath, Ajith M., Holub, Vojtech, Brunk, Hugh L., Lyons, Robert G., Gurijala, Aparna R.
• Primary Examiner(s): Saunders, Jr., Joseph
• Application Number:
  US15/893,127
• Publication Number:
  US 20180246696A1
• Time in Patent Office: 627 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 25/09 the extracted parameters being zero crossing rates
• CPC Class Codes: G10L 25/21 the extracted parameters being power information

Abstract

Methods and arrangements involving electronic devices, such as smartphones, tablet computers, wearable devices, etc., are disclosed. One arrangement involves a low-power processing technique for discerning cues from audio input. Another involves a technique for detecting audio activity based on the Kullback-Liebler divergence (KLD) (or a modified version thereof) of the audio input. Still other arrangements concern techniques for managing the manner in which policies are embodied on an electronic device. Others relate to distributed computing techniques. A great variety of other features are also detailed.

Claims

1. A method, comprising:

in an electronic device comprising first and second processors, obtaining audio input corresponding to sound propagating within an aural environment surrounding the electronic device;

at a first processor, processing the audio input to discern a characteristic of the audio input, the processing comprising processing audio input samples of the audio input to determine a relative-entropy of the audio input, and estimating the presence of audio activity based on the determined relative-entropy of the audio input;

generating an output based upon the processing of the audio input to discern the characteristic; and

causing the second processor to enter into a higher power state based on the generated output.

2. The method of claim 1, wherein processing the audio input to discern the characteristic further comprises determining zero-crossing or short-term energy metrics from the audio input, determining co-occurrence statistics of the zero-crossing or short term energy metrics, and classifying the audio input based on the co-occurrence statistics.

3. The method of claim 1, wherein the output comprises cue detection output, and further comprising:

by reference to the cue detection output, enabling at least one policy embodied on the electronic device.

4. The method of claim 1, wherein the output comprises cue detection output, and further comprising:

by reference to the cue detection output, disabling at least one policy embodied on the electronic device.

5. The method of claim 1, comprising:

processing the received audio to derive a plurality of items of auxiliary data therefrom, wherein at least one of the items of auxiliary data comprises delivery control data; and

by reference to the delivery control data, delivering at least one other of the items of auxiliary data to at least one component of the electronic device.

6. The method of claim 1, wherein the first and second processors are components of an electronic device, the method further comprising generating an audio signal corresponding to sound propagating within an aural environment surrounding the electronic device, wherein the obtained audio input comprises a plurality of samples of the audio signal.

7. The method of claim 6, wherein the second processor is a CPU.

8. The method of claim 7, wherein the first processor is a digital signal processor.

9. The method of claim 7, further comprising processing the audio input while the second processor is in an idle or sleep state.

10. The method of claim 1, wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to discern the presence of a digital audio watermark signal within the audio input.

11. The method of claim 10, wherein processing the audio input to discern the characteristic of the audio input comprises processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input.

12. The method of claim 10, comprising:

detecting presence of the digital watermark signal from a first sub-band spanning a first frequency range; and

decoding auxiliary data from the digital watermark signal from second sub-bands spanning a frequency range greater than the first frequency range.

13. The method of claim 12, comprising:

transforming the frame with a sparse FFT in a process of detecting the presence of the digital watermark signal from the first sub-band.

14. The method of claim 12, comprising:

transforming a frame with a first FFT for audio input sampled at a first sample rate in a process of detecting the presence of the digital watermark signal from the first sub-band; and

transforming a frame with a second FFT for audio input sampled at a second sample rate higher than the first sample rate, in a process of decoding auxiliary data from the digital watermark signal from the second sub-bands.

15. The method of claim 10, wherein processing the audio input to discern the presence of a digital audio watermark signal within the audio input comprises:

buffering frames of the audio input;

transforming the frames into spectral magnitude frames;

accumulating spectral magnitude frames into an accumulation buffer;

from the accumulation buffer, extracting spectral magnitude values corresponding to selected bits of the digital audio watermark signal;

correlating the extracted spectral magnitude values with a predetermined signal to produce a correlation metric.

16. The method of claim 15, wherein accumulating comprises:

accumulating spectral magnitude frames into a first accumulation buffer, the spectral magnitude frames corresponding to shift groups; and

accumulating spectral magnitude frames from the first accumulation buffer according to shift group in a second accumulation buffer.

17. The method of claim 15 comprising:

scaling the spectral magnitude frames in the second accumulation buffer according to plural noise profiles to produce candidate spectral magnitude profiles for each of the noise profiles; and

extracting spectral magnitude values from the candidate spectral magnitude profiles corresponding to selected bits of the digital audio watermark signal.

18. The method of claim 15, comprising:

correlating the extracted spectral magnitude values with predetermined signals to produce correlation metrics for the predetermined signals;

determining a reference spectral magnitude sequence for a predetermined signal detected based on the correlation metrics;

generating a structural strength metric for the reference spectral magnitude sequence;

selecting spectral magnitude sequences from which to decode auxiliary data by identifying spectral magnitude sequences with a structural strength metric that exceeds a threshold decode candidate value.

19. The method of claim 18, comprising:

identifying similar spectral magnitude code sequences based on similarity of time shift or noise profile of the spectral magnitude code sequences to produce sub-sets of similar spectral magnitude code sequences; and

selecting spectral magnitude sequences from which to decode auxiliary data by selecting within a sub-set based on the structural strength metric.

20. A non-transitory computer readable medium, on which is stored instructions, which configure a first processor to:

obtain audio input corresponding to sound propagating within an aural environment surrounding an electronic device;

process the audio input to discern a characteristic of the audio input by determining a relative-entropy of the audio input, and estimating the presence of audio activity based on the determined relative-entropy of the audio input;

generate an output based upon the processing of the audio to discern the characteristic;

and

cause a second processor to enter into a higher power state based on the generated output.

21. The non-transitory computer readable medium of claim 20, on which is stored instructions, which configure the first processor to process the audio input while the second processor is in an idle or sleep state.

22. The non-transitory computer readable medium of claim 20, on which is stored instructions, which configure the first processor to discern the characteristic of the audio input by processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input.

23. The non-transitory computer readable medium of claim 20, on which is stored instructions, which configure the first processor to discern the characteristic of the audio input by processing the audio input to discern presence of a digital audio watermark signal within the audio input.

24. The non-transitory computer readable medium of claim 23, on which is stored instructions to configure the first processor to:

buffer frames of the audio input;

transform the frames into spectral magnitude frames;

accumulate spectral magnitude frames into an accumulation buffer;

from the accumulation buffer, extract spectral magnitude values corresponding to selected bits of the digital audio watermark signal; and

correlate the extracted spectral magnitude values with a predetermined signal to produce a correlation metric.

25. A device comprising:

a first processor coupled to a memory and a second processor;

the first processor configured to obtain from the memory audio input corresponding to sound propagating within an aural environment surrounding an electronic device;

process the audio input to discern a characteristic of the audio input by determining a relative-entropy of the audio input, and estimating presence of audio activity based on the determined relative-entropy of the audio input;

generate an output based upon the processing of the audio to discern the characteristic;

and

cause the second processor to enter into a higher power state based on the generated output.

26. The device of claim 25 wherein the first processor is configured to process the audio input while the second processor is in an idle or sleep state.

27. The device of claim 25 wherein the first processor is configured to discern the characteristic of the audio input by processing the audio input to discern auxiliary data conveyed by a digital audio watermark signal present within the audio input.

28. The device of claim 25 wherein the first processor is configured to discern the characteristic of the audio input by processing the audio input to discern presence of a digital audio watermark signal within the audio input.

29. The device of claim 28 wherein the first processor is configured to:

buffer frames of the audio input;

transform the frames into spectral magnitude frames;

accumulate spectral magnitude frames into an accumulation buffer;

from the accumulation buffer, extract spectral magnitude values corresponding to selected bits of the digital audio watermark signal; and

correlate the extracted spectral magnitude values with a predetermined signal to produce a correlation metric.