Coordinated watermarking

• US 9,547,753 B2
• Filed: 12/13/2011
• Issued: 01/17/2017
• Est. Priority Date: 12/13/2011
• Status: Active Grant

• Current Assignee: IP Acquisitions, LLC
• Original Assignee: Verance Corporation
• Inventors: Petrovic, Rade, Zhao, Jian, Winograd, Joseph M.
• Primary Examiner(s): LANIER, BENJAMIN E
• Application Number:
  US13/324,276
• Publication Number:
  US 20130152210A1
• Time in Patent Office: 1,862 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 1/32149 Methods relating to embedding, encoding, decoding, detection or retrieval operations

Abstract

Methods, devices and computer program products enable embedding and extraction of multiple watermark messages in a coordinated fashion to improve watermark transparency, robustness against impairments, security, and computational complexity of embedding and extraction of watermarks. Coordinated watermark embedding and extraction operations take advantage of a pre-defined relationship between the symbols of two watermark messages. The improved aspects of the watermarking system provide significant value to content owners, content distributors, and consumers at a small incremental cost.

Claims

1. A method of embedding watermarks into a host content, comprising:

obtaining a payload of a first watermark message for embedding into the host content;

obtaining a payload of a second watermark message that is different from the payload of the first watermark message for embedding into the host content, the second watermark message having a different number of symbols than the first watermark message; and

embedding the first and the second watermark messages into the host content in a coordinated fashion such that the embedded first watermark message has a pre-defined relationship with the embedded second watermark message in a particular domain, wherein the symbols of the first watermark message are embedded in interleaved embedding opportunities with the symbols of the second watermark message in the particular domain, and wherein the pre-defined relationship comprises a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message, the ratio selected to achieve a particular watermark extraction reliability of the first watermark message.

2. The method of claim 1, wherein the particular domain is at least one of a spatial, temporal and frequency domains.

3. The method of claim 1, wherein the embedding of the first and the second watermark messages is conducted in a single watermark embedding session.

4. The method of claim 1, wherein the embedding of the first watermark message is conducted in a different embedding session than the embedding of the second watermark message.

5. The method of claim 1, wherein different watermarking technologies are used for embedding each of the first and the second watermark messages.

6. The method of claim 1, wherein the same watermarking technologies are used for embedding each of the first and the second watermark messages.

7. The method of claim 1, wherein embedding of one or more symbols of the first watermark message is skipped, and a space vacated by skipping the one or more symbols of the first watermark message is used for embedding at least one symbol of the second watermark message.

8. The method of claim 1, further comprising performing a single watermark masking operation.

9. The method of claim 1, wherein the first watermark message is embedded into a first component of the host content and the second watermark message is embedded into a second component of the host content.

10. The method of claim 9, wherein the first component is an audio component and the second component is a video component.

11. The method of claim 1, wherein the first watermark message is a copy control information watermark and the second watermark message is an extended payload watermark.

12. The method of claim 1, wherein embedding the first and the second watermark messages comprises:

identifying all embedding opportunities within the host content based on a content perceptual criterion;

allocating a first subset of identified embedding opportunities for embedding of the first watermark message; and

allocating a second subset of identified embedding opportunities for embedding of the second watermark message.

44. The method of claim 1, wherein the ratio is selected to further achieve at least one of:

a desired security of the first or the second watermark messages, transparency of the embedded first and second watermark messages, computational complexity of extraction of the first or the second watermark messages, size of the first or the second watermark messages, or application of the first or the second watermark messages.

45. The method of claim 1, wherein the pre-defined relationship defines a relative embedding location of a first symbol of the first watermark message with respect to an embedding location of a first symbol of the second watermark message.

13. A method, comprising:

extracting a first watermark message from a host content;

obtaining a predefined relationship between symbols of the extracted first watermark message and symbols of a second watermark message in a particular domain, the second watermark message having a different payload and a different number of symbols than the first watermark message, the symbols of the first watermark message having been embedded in interleaved watermark embedding opportunities with the symbols of the second watermark message in the particular domain, and the pre-defined relationship comprising a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message so as to achieve a particular watermark extraction reliability of the first watermark message; and

identifying locations of the symbols of the second watermark message based on the predefined relationship; and

extracting the second watermark message from the host content from the identified locations.

14. The method of claim 13, wherein the particular domain is at least one of a spatial, temporal and frequency domains.

15. The method of claim 13, wherein the first watermark message is extracted from a first component of the host content and the second watermark message is extracted from a second component of the host content.

16. The method of claim 15, wherein the first component is an audio component and the second component is a video component.

17. The method of claim 13, wherein the second watermark message comprises a time-code and extracting the second watermark message comprises:

extracting the time-code from the second watermark message;

determining content timing information from the extracted time code; and

determining accuracy of the content timing information.

18. The method of claim 13, wherein the second watermark message comprises a time-code and extracting the second watermark message further comprises:

determining content timing information based on time-codes in a plurality of extracted second watermark messages;

determining an error value associated with the content timing information; and

setting a new reference timing information if the determined error value is at or below a target error value.

19. The method of claim 18, further comprising using the time-code to detect content manipulations.

46. The method of claim 13, wherein the ratio is selected to further achieve at least one of:

a desired security of the first or the second watermark messages, transparency of the embedded first and second watermark messages, computational complexity of extraction of the first or the second watermark messages, size of the first or the second watermark messages, or application of the first or the second watermark messages.

47. The method of claim 13, wherein the pre-defined relationship defines a relative embedding location of a first symbol of the first watermark message with respect to an embedding location of a first symbol of the second watermark message.

20. A device for embedding watermarks, comprising:

a watermark packet generator implemented on a processor that is implemented at least partially is hardware and is configured to, upon execution of processor executable instructions embodied on a non-transitory computer readable medium, obtain a payload of a first watermark message and a payload of a second watermark message that is different from the payload of the first watermark message for embedding into a host content; and

a watermark embedder configured to embed the first and the second watermark messages into the host content in a coordinated fashion such that the embedded first watermark message has a pre-defined relationship with the embedded second watermark message in a particular domain, wherein embedded symbols of the first watermark message are substantially non-overlapping with embedded symbols of the second watermark message in the particular domain, wherein the symbols of the first watermark message are embedded in interleaved embedding opportunities with respect to the symbols of the second watermark message in the particular domain, and wherein the pre-defined relationship comprises a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message with respect to the symbols of the second watermark message, the ratio selected to achieve a particular watermark extraction reliability of the first watermark message.

21. The device of claim 20, wherein the particular domain is at least one of a spatial, temporal and frequency domains.

22. The device of claim 20, wherein the watermark embedder is configured to embed the first and the second watermark messages in a single watermark embedding session.

23. The device of claim 20, wherein the watermark embedder is configured to embed the first and the second watermark messages in different embedding sessions.

24. The device of claim 20, wherein different watermarking technologies are used for embedding each of the first and the second watermark messages.

25. The method of claim 20, wherein the watermark embedder is configured to use the same watermarking technologies for embedding each of the first and the second watermark messages.

26. The device of claim 20, wherein the watermark embedder is configured to skip the embedding of one or more symbols of the first watermark message, and use a space vacated by skipping the one or more symbols of the first watermark message for embedding at least one symbol of the second watermark message.

27. The device of claim 20, further configured to perform a single watermark masking operation.

28. The device of claim 20, wherein the watermark embedder is configured to embed the first watermark message into a first component of the host content and the second watermark message into a second component of the host content.

29. The device of claim 28, wherein the first component is an audio component and the second component is a video component.

30. The device of claim 20, wherein the first watermark message is a copy control information watermark and the second watermark message is an extended payload watermark.

31. The device of claim 20, wherein the watermark embedder is configured to:

identify all embedding opportunities within the host content based on a content perceptual criterion;

allocate a first subset of identified embedding opportunities for embedding of the first watermark message; and

allocate a second subset of identified embedding opportunities for embedding of the second watermark message.

48. The device of claim 20, wherein the ratio is selected to further achieve at least one of:

a desired security of the first or the second watermark messages, transparency of the embedded first and second watermark messages, computational complexity of extraction of the first or the second watermark messages, size of the first or the second watermark messages, or application of the first or the second watermark messages.

49. The device of claim 20, wherein the pre-defined relationship defines a relative embedding location of a first symbol of the first watermark message with respect to an embedding location of a first symbol of the second watermark message.

32. A device, comprising:

a processor; and

a memory, comprising processor executable code, the processor executable code when executed by the processor configures the device to;

obtain a payload of a first watermark message for embedding into the host content;

obtain a payload of a second watermark message that is different from the payload of the first watermark message for embedding into the host content, the second watermark message having a different number of symbols than the first watermark message; and

embed the first and the second watermark messages into the host content in a coordinated fashion such that the embedded first watermark message has a pre-defined relationship with the embedded second watermark message in a particular domain, wherein the symbols of the first watermark message are embedded in interleaved embedding opportunities with the symbols of the second watermark message in the particular domain, and wherein the pre-defined relationship comprises a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message, the ratio selected to achieve a particular watermark extraction reliability of the first watermark message.

33. A computer program product, embodied on a non-transitory computer readable medium, comprising:

program code for obtaining a payload of a first watermark message for embedding into the host content;

program code for obtaining a payload of a second watermark message that is different from the payload of the first watermark message for embedding into the host content, the second watermark message having a different number of symbols than the first watermark message; and

program code for embedding the first and the second watermark messages into the host content in a coordinated fashion such that the embedded first watermark message has a pre-defined relationship with the embedded second watermark message in a particular domain, wherein the symbols of the first watermark message are embedded in interleaved embedding opportunities with the symbols of the second watermark message in the particular domain, and wherein the pre-defined relationship comprises a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message, the ratio selected to achieve a particular watermark extraction reliability of the first watermark messages.

34. A device, comprising:

a watermark extractor configured to extract a first watermark message from a host content; and

a data processing component implemented on a processor that is implemented at least partially in hardware and is configured to, upon execution of processor executable instructions embodied on a non-transitory computer readable medium, obtain a predefined relationship between the symbols of the extracted first watermark message and symbols of a second watermark message in a particular domain, the second watermark message having a different payload than the first watermark message, the symbols of the first watermark message having been embedded in interleaved watermark embedding opportunities with respect to the symbols of the second watermark message in the particular domain, and the pre-defined relationship comprising a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message with respect to the symbols of the second watermark message so as to achieve a particular watermark extraction reliability of the first watermark message, wherein the watermark extractor is further configured to extract the second watermark message based on the predefined relationship, and wherein embedded symbols of the first watermark message are substantially non-overlapping with embedded symbols of the second watermark message in the particular domain.

35. The device of claim 34, wherein the particular domain is at least one of a spatial, temporal and frequency domains.

36. The device of claim 34, wherein the watermark extractor is configured to extract the first watermark message from a first component of the host content and the second watermark message from a second component of the host content.

37. The device of claim 36, wherein the first component is an audio component and the second component is a video component.

38. The device of claim 34, wherein the second watermark message comprises a time-code and the watermark extractor is configured to extract the second watermark message, at least in-part, by:

extracting the time-code from the second watermark message;

determining content timing information from the extracted time code; and

determining accuracy of the content timing information.

39. The device of claim 34, wherein the second watermark message comprises a time-code and the watermark extractor is configured to extract the second watermark message, at least in-part, by:

determining content timing information based on time-codes in a plurality of extracted second watermark messages;

determining an error value associated with the content timing information; and

setting a new reference timing information if the determined error value is at or below a target error value.

40. The device of claim 39, the watermark extractor is further configured to use the time-code to detect content manipulations.

50. The device of claim 34, wherein the ratio is selected to further achieve at least one of:

a desired security of the first or the second watermark messages, transparency of the embedded first and second watermark messages, computational complexity of extraction of the first or the second watermark messages, size of the first or the second watermark messages, or application of the first or the second watermark messages.

51. The device of claim 34, wherein the pre-defined relationship defines a relative embedding location of a first symbol of the first watermark message with respect to an embedding location of a first symbol of the second watermark message.

41. A non-transitory computer-readable storage medium with a host content embodied thereupon, the host content comprising:

one or more watermarks that are imperceptibly embedded in the host content, wherein reception of the host content by a content handling device equipped with a watermark extractor triggers the watermark extractor to;

extract a first watermark message from the host content;

obtain a predefined relationship between symbols of the extracted first watermark message and symbols of a second watermark message in a particular domain, the second watermark message having a different payload and a different number of symbols than the first watermark message, the symbols of the first watermark message having been embedded in interleaved watermark embedding opportunities with the symbols of the second watermark message in the particular domain, and the pre-defined relationship comprising a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message so as to achieve a particular watermark extraction reliability of the first watermark message; and

identify locations of symbols of the second watermark message based on the predefined relationship; and

extract the second watermark message from the identified locations of the host content.

42. A device, comprising:

a processor; and

a memory, comprising processor executable code, the processor executable code when executed by the processor configures the device to;

extract a first watermark message from a host content;

obtain a predefined relationship between symbols of the extracted first watermark message and symbols of a second watermark message in a particular domain, the second watermark message having a different payload and a different number of symbols than the first watermark message, the symbols of the first watermark message having been embedded in interleaved watermark embedding opportunities with the symbols of the second watermark message in the particular domain, and the pre-defined relationship comprising a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message so as to achieve a particular watermark extraction reliability of the first watermark message;

identify locations of symbols of the second watermark message based on the predefined relationship; and

extract the second watermark message from the identified locations of the host content.

43. A computer program product, embodied on a non-transitory computer readable medium, comprising:

program code for extracting a first watermark message from a host content;

program code for obtaining a predefined relationship between symbols of the extracted first watermark message and symbols of a second watermark message in a particular domain, the second watermark message having a different payload and a different number of symbols than the first watermark message, the symbols of the first watermark message having been embedded in interleaved watermark embedding opportunities with the symbols of the second watermark message in the particular domain, and the pre-defined relationship comprising a ratio of the interleaved embedding opportunities allocated to the symbols of the first watermark message to the symbols of the second watermark message so as to achieve a particular watermark extraction reliability of the first watermark message;

program code for identifying locations of symbols the second watermark message based on the predefined relationship; and

program code for extracting the second watermark message from the identified locations of the host content.