Multi-mode audio recognition and auxiliary data encoding and decoding

• US 10,026,410 B2
• Filed: 07/26/2016
• Issued: 07/17/2018
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Gurijala, Aparna R., Bai, Yang, Sharma, Ravi K., Bradley, Brett A.
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US15/220,209
• Publication Number:
  US 20170133022A1
• Time in Patent Office: 721 Days
• Field of Search: 380201
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/028 Noise substitution, i.e. substituting non-tonal spectral components by noisy source comfort noise for discontinuous speech transmission G10L19/012
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

Audio signal processing enhances audio watermark embedding and detecting processes. Audio signal processes include audio classification and adapting watermark embedding and detecting based on classification. Advances in audio watermark design include adaptive watermark signal structure data protocols, perceptual models, and insertion methods. Perceptual and robustness evaluation is integrated into audio watermark embedding to optimize audio quality relative the original signal, and to optimize robustness or data capacity. These methods are applied to audio segments in audio embedder and detector configurations to support real time operation. Feature extraction and matching are also used to adapt audio watermark embedding and detecting.

Claims

1. A method of enhancing a watermark in an electronic audio signal, the method comprising:

generating a watermark signal for insertion into the electronic audio signal;

evaluating perceptual audio quality of the electronic audio signal relative to changes of that electronic audio signal corresponding to the watermark signal through automated application of a perceptual audio quality measure that computes audio quality parameters based on a human auditory model, including parameters for estimating quality based on a difference between the audio signal and a watermarked version of the audio signal;

updating a watermark embedding parameter based on the evaluating; and

embedding the watermark signal into the electronic audio signal using the updated watermark embedding parameter.

2. The method of claim 1 including:

evaluating robustness of a watermarked audio signal using bit error rate or detection rate metrics for the generated watermark signal in the watermarked audio signal; and

based on the robustness, updating the watermark embedding parameter.

3. The method of claim 1 further comprising:

analyzing the audio signal to identify an embedding location that does not have sufficient signal in which to embed a watermark signal element;

boosting the audio signal at the embedding location; and

embedding the watermark signal element at the embedding location, using the boosting to mask audibility of a change in the audio signal made to embed the watermark signal.

4. The method of claim 3 wherein the analyzing comprises analyzing a spectral domain of a segment of the audio signal, and wherein boosting comprises boosting the audio signal at frequency locations where the audio signal has sparse spectral components.

5. The method of claim 4 wherein in boosting comprises applying an equalizer function to the segment.

6. The method of claim 5 including controlling the equalizer function based on a measure of correlation of equalized audio segment relative to an original audio segment.

7. The method of claim 6 including varying the equalizer function over time segments, and keeping change due to applying the equalizer from segment to segment within a constraint.

8. A non-transitory computer readable medium on which is stored instructions, which when executed by one or more processors, perform a method of enhancing a watermark in an electronic audio signal, the method comprising:

obtaining a watermark signal for insertion into the electronic audio signal;

evaluating perceptual audio quality of the electronic audio signal relative to changes of that electronic audio signal corresponding to the watermark signal through automated application of a perceptual audio quality measure that computes audio quality parameters based on a human auditory model, including parameters for estimating quality based on a difference between the audio signal and a watermarked version of the audio signal;

updating a watermark embedding parameter based on the evaluating; and

embedding the watermark signal into the electronic audio signal using the updated watermark embedding parameter.

9. A system for enhancing a watermark signal, the system comprising:

a memory on which is stored instructions;

a processor in communication with the memory, the processor configured with the instructions to evaluate a difference between watermarked audio and an audio signal, determine perceptual masking envelopes for a watermark signal based on the difference, and to apply signal gain to features corresponding to a watermark signal in the watermarked audio based on the perceptual masking envelopes;

the processor configured to apply the signal gain to the features to update the watermark signal based on the difference between the watermarked audio and the audio signal.

10. The system of claim 9 wherein the processor is configured with the instructions to increase signal gain based on a robustness metric obtained from measuring detection of a watermark message.

11. The system of claim 9 wherein the processor is configured with the instructions to apply the signal gain to frequency bands where the perceptual masking envelopes indicate that the watermark signal is below a threshold.

12. The system of claim 9 wherein the processor is configured with the instructions to encode a watermark message in the audio signal to produce the watermarked audio, to evaluate the difference between watermarked audio and an audio signal;

and to provide the signal gain to the features to update insertion of the watermark message in the audio signal.

13. The system of claim 9 wherein the processor is configured with the instructions to evaluate the difference between watermarked audio and an audio signal;

and to provide the signal gain to the features to update insertion of a watermark message in the audio signal.

14. The system of claim 13 wherein the features correspond to frequency components of the audio signal where the difference indicates that the watermark signal is encoded below a perceptual masking threshold.

15. The system of claim 14 wherein the gain is provided by increasing masking thresholds in the perceptual masking envelopes.

16. The system of claim 9 wherein the gain is increased for features where the watermark signal is determined to be below a perceptual masking threshold.

17. A system for enhancing a watermark signal, the system comprising:

a memory on which is stored instructions;

a processor in communication with the memory, the processor configured with the instructions to evaluate a difference between watermarked audio and an audio signal, determine perceptual masking envelopes for a watermark signal based on the difference, and to apply signal gain to features corresponding to a watermark signal in the watermarked audio based on the perceptual masking envelopes;

wherein the gain is adjusted to adapt to a perceptual quality metric and a robustness metric, the robustness metric being derived by assessing detection of a watermark message encoded in the watermark signal.

18. The system of claim 17 wherein the processor is configured with the instructions to encode a watermark message in the audio signal to produce the watermarked audio, to evaluate the difference between watermarked audio and an audio signal, and to provide the signal gain to the features to update insertion of the watermark message in the audio signal.

19. The system of claim 17 wherein the processor is configured with the instructions to evaluate the difference between watermarked audio and an audio signal, and to provide the signal gain to the features to update insertion of a watermark message in the audio signal.

20. The system of claim 19 wherein the features correspond to frequency components of the audio signal where the difference indicates that the watermark signal is encoded below a perceptual masking threshold.