Multi-mode audio recognition and auxiliary data encoding and decoding

• US 9,852,736 B2
• Filed: 04/04/2016
• Issued: 12/26/2017
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Sharma, Ravi K., Bradley, Brett A., Bai, Yang, Shivappa, Shankar Thagadur, Kamath, Ajith, Gurijala, Aparna, Cushman, David A.
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US15/090,279
• Publication Number:
  US 20160293172A1
• Time in Patent Office: 631 Days
• Field of Search: 704273
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders

Abstract

Audio signal processing enhances audio watermark embedding and detecting processes. Audio signal processes include audio classification and adapting watermark embedding and detecting based on classification. Advances in audio watermark design include adaptive watermark signal structure data protocols, perceptual models, and insertion methods. Perceptual and robustness evaluation is integrated into audio watermark embedding to optimize audio quality relative the original signal, and to optimize robustness or data capacity. These methods are applied to audio segments in audio embedder and detector configurations to support real time operation. Feature extraction and matching are also used to adapt audio watermark embedding and detecting.

Claims

1. A method of embedding a watermark in an electronic audio signal, the method comprising:

analyzing the audio signal to identify an embedding location that does not have sufficient signal in which to embed a watermark signal element;

boosting the audio signal at the embedding location; and

embedding the watermark signal element at the embedding location, using the boosting to mask audibility of a change in the audio signal made to embed the watermark signal.

2. The method of claim 1 wherein the analyzing comprises analyzing a spectral domain of a segment of the audio signal, and wherein boosting comprises boosting the audio signal at frequency locations where the audio signal has sparse spectral components.

3. The method of claim 2 wherein boosting comprises applying an equalizer function to the segment.

4. The method of claim 3 including controlling the equalizer function based on a measure of correlation of equalized audio segment relative to an original audio segment.

5. The method of claim 4 including varying the equalizer function over time segments, and keeping change due to applying the equalizer from segment to segment within a constraint.

6. A method of embedding a watermark in an electronic audio signal, the method comprising:

determining whether an audio segment of the audio signal is stationary or non-stationary;

adapting resolution of a perceptual model based on whether the audio segment is stationary or non-stationary; and

inserting a watermark into the audio segment using the adapted perceptual model.

7. A method of embedding a watermark in an electronic audio signal, the method comprising:

generating a watermark signal for insertion into the electronic audio signal;

evaluating perceptual audio quality of the electronic audio signal relative to changes of that electronic audio signal corresponding to the watermark signal through automated application of a perceptual audio quality measure that computes audio quality parameters based on a human auditory model, including parameters for estimating quality based on a difference between the audio signal and a watermarked version of the audio signal;

updating a watermark embedding parameter based on the evaluating;

embedding the watermark signal into the electronic audio signal using the updated watermark embedding parameter

analyzing the audio signal for a harmonic; and

for embedding locations corresponding to the harmonic, structuring the watermark signal to be masked by the harmonic.

8. The method of claim 7 including:

detecting a complex tone including harmonics;

generating a watermark signal that exploits a harmonic relationship in the complex tone, including increasing a first harmonic and decreasing a second harmonic in the harmonic relationship.

9. The method of claim 7 wherein generating a watermark signal comprises generating a frequency domain signal with plural elements mapped to corresponding plural frequency locations in an audio frame, with the plural elements being structured having at least partially offsetting values in the first and second harmonics.

10. A method of embedding a watermark in an electronic audio signal, the method comprising:

generating a watermark signal using orthogonal frequency division multiplexing in which auxiliary data is modulated onto OFDM carrier signals;

computing a frequency magnitude envelope for embedding locations in a frequency domain transform of the audio signal;

inserting the watermark signal by replacing audio signal frequency components with modulated OFDM carrier signals at the embedding locations while maintaining the frequency magnitude envelope at the embedding locations, and

weighting the audio signal in a frequency range from 16 to at least 19 Khz, the weighting being selected to counter a drop in frequency response of audio equipment over the frequency range from 16 to at least 19 Khz.

11. The method of claim 10 comprising:

generating a high frequency watermark signal by modulating a carrier signal using a set of frequency shaping patterns at a frequency range of 10 to 22 kHz; and

inserting the watermark signal into carrier signal.

12. The method of claim 11, wherein the high frequency watermark signal is a time-varying signal.

13. The method of claim 11, wherein the high frequency watermark signal is a periodic signal.

14. The method of claim 11, wherein the high frequency watermark signal is a non-periodic signal.

15. A non-transitory computer readable medium, on which is stored instructions, which when executed by a processor perform a method of embedding a watermark in an electronic audio signal, the method comprising:

analyzing the audio signal to identify an embedding location that does not have sufficient signal in which to embed a watermark signal element;

boosting the audio signal at the embedding location; and

embedding the watermark signal element at the embedding location, using the boosting to mask audibility of a change in the audio signal made to embed the watermark signal.

16. The non-transitory computer readable medium of claim 15 wherein the analyzing comprises analyzing a spectral domain of a segment of the audio signal, and wherein boosting comprises boosting the audio signal at frequency locations where the audio signal has sparse spectral components.

17. The non-transitory computer readable medium of claim 16 wherein boosting comprises applying an equalizer function to the segment.

18. The non-transitory computer readable medium of claim 17 including instructions on the non-transitory computer readable medium, which when executed by a processor, perform an act of:

controlling the equalizer function based on a measure of correlation of equalized audio segment relative to an original audio segment.

19. The non-transitory computer readable medium of claim 18 including instructions on the non-transitory computer readable medium, which when executed by a processor, perform acts of:

varying the equalizer function over time segments, and keeping change due to applying the equalizer from segment to segment within a constraint.