Multi-mode audio recognition and auxiliary data encoding and decoding

• US 10,546,590 B2
• Filed: 12/20/2017
• Issued: 01/28/2020
• Est. Priority Date: 10/15/2012
• Status: Expired due to Fees

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sharma, Ravi K., Bradley, Brett A., Bai, Yang, Thagadur Shivappa, Shankar, Gurijala, Aparna
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/848,339
• Publication Number:
  US 20180211673A1
• Time in Patent Office: 769 Days
• Field of Search: 704231
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders

Abstract

Audio signal processing enhances audio watermark embedding and detecting processes. Audio signal processes include audio classification and adapting watermark embedding and detecting based on classification. Advances in audio watermark design include adaptive watermark signal structure data protocols, perceptual models, and insertion methods. Perceptual and robustness evaluation is integrated into audio watermark embedding to optimize audio quality relative the original signal, and to optimize robustness or data capacity. These methods are applied to audio segments in audio embedder and detector configurations to support real time operation. Feature extraction and matching are also used to adapt audio watermark embedding and detecting.

Claims

1. A method of detecting an audio watermark, the method comprising:

receiving an audio signal;

classifying the audio signal;

based on classifying audio, adapting a filter for filtering the audio signal for audio watermark detection;

filtering the audio signal with the adapted filter;

extracting symbol estimates of the audio watermark from the filtered audio; and

decoding a digital payload from the extracted symbol estimates.

2. The method of claim 1 wherein the classifying comprises classifying noise in the audio signal and the adapting comprises adapting the filter to enhance a watermark signal relative to classified noise in the audio signal.

3. The method of claim 1 wherein the classifying comprises speech discrimination.

4. The method of claim 1 wherein the classifying comprises music discrimination.

5. The method of claim 1 wherein the classifying comprises an audio fingerprint classifier, and classifying comprises extracting audio fingerprints from the audio signal, querying a fingerprint database to determine similarity between the audio fingerprints and reference fingerprints in the fingerprint database, and obtaining metadata from the fingerprint database identifying a classification of the audio signal in response to the querying.

6. The method of claim 1 wherein the classifying comprises classifying the audio signal based on environmental information obtained from sensing an ambient environment in which the audio signal is produced or captured.

7. The method of claim 1 comprising measuring perceptual quality of the audio signal and robustness of digital data in the audio signal, and based on the measured perceptual quality and robustness, updating strength parameters used to encode the digital data.

8. The method of claim 1 further comprising:

performing multiple stream analysis to discriminate sounds in the audio signal from different sound sources;

separating a first discriminated sound from the audio signal;

wherein the classifying is performed on the first discriminated sound.

9. The method of claim 1 wherein adapting the filter comprises adapting gain applied to frequency bands of the audio signal.

10. The method of claim 1 wherein adapting the filter comprises selecting a watermark detection filter to enhance a watermark signal relative to another signal in the audio signal.

11. The method of claim 1 wherein the filtering comprises filtering and accumulating portions of the audio signal in which a watermark signal is expected based on the classifying.

12. A method of detecting an audio watermark, the method comprising:

receiving an audio signal;

classifying the audio signal;

based on classifying audio, adapting a filter;

filtering the audio signal with the adapted filter;

extracting symbol estimates from the filtered audio; and

decoding a digital payload from the extracted symbol estimates;

wherein adapting the filter comprises;

determining a masking model applied to embed a watermark in the audio signal based on the classifying; and

obtaining weights to be applied in the filtering based on the masking model.

13. The method of claim 12 wherein the filtering comprises applying the weights to attributes of the audio signal that are expected to have greater signal energy.

14. A non-transitory computer readable medium, on which is stored instructions, which when executed by a processor, configure the processor to:

classify an audio signal to identify a class of audio within the audio signal;

based on the class, adapt a filter for filtering the audio signal for audio watermark detection;

filter the audio signal with the adapted filter;

extract symbol estimates of the audio watermark from the filtered audio; and

decode a digital payload from the extracted symbol estimates.

15. The non-transitory computer readable medium of claim 14 wherein the instructions configure the processor to classify noise in the audio signal and adapt the filter to enhance a watermark signal relative to classified noise in the audio signal.

16. The non-transitory computer readable medium of claim 14 wherein the instructions configure the processor to be an audio fingerprint classifier, in which the instructions configure the audio fingerprint classifier to extract audio fingerprints from the audio signal, query a fingerprint database to determine similarity between the audio fingerprints and reference fingerprints in the fingerprint database, and obtain metadata from the fingerprint database identifying a classification of the audio signal in response to a query.

17. The non-transitory computer readable medium of claim 14 wherein the instructions configure the processor to classify the audio signal based on environmental information obtained from sensing an ambient environment in which the audio signal is produced or captured.

18. The non-transitory computer readable medium of claim 14 wherein the instructions configure the processor to adapt the filter by adapting gain applied to frequency bands of the audio signal.

19. The non-transitory computer readable medium of claim 14 wherein the instructions configure the processor to adapt the filter by selecting a watermark detection filter to enhance a watermark signal relative to another signal in the audio signal.

20. A non-transitory computer readable medium, on which is stored instructions, which when executed by a processor, configure the processor to:

classify an audio signal to identify a class of audio within the audio signal;

based on the class, adapt a filter;

filter the audio signal with the adapted filter;

extract symbol estimates from the filtered audio; and

decode a digital payload from the extracted symbol estimates;

wherein the instructions configure the processor to;

determine a masking model applied to embed a watermark in the audio signal based on output of executing instructions to classify the audio signal;

obtain weights to be applied in the filtering based on the masking model; and

apply the weights to attributes of the audio signal that are expected to have greater signal energy.