Signal continuity assessment using embedded watermarks

• US 9,990,688 B2
• Filed: 07/10/2017
• Issued: 06/05/2018
• Est. Priority Date: 10/08/2003
• Status: Expired due to Term

• Current Assignee: IP Acquisitions, LLC
• Original Assignee: Verance Corporation
• Inventors: Tehranchi, Babak, Petrovic, Rade, Winograd, Joseph M., Angelico, Dean Anthony
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US15/645,865
• Publication Number:
  US 20180089790A1
• Time in Patent Office: 330 Days
• Field of Search: None
• CPC Class Codes: G06F 21/1063 Personalisation
• CPC Class Codes: G06F 21/16 Program or content traceability, e.g. by watermarking
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: G06T 1/0071 using multiple or alternating watermarks
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04L 2463/103 applying security measure for protecting copyright
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 1/32288 Multiple embedding, e.g. cocktail embedding, or redundant embedding, e.g. repeating the additional information at a plurality of locations in the image
• CPC Class Codes: H04N 1/32315 Selecting a particular method from amongst a plurality of methods
• CPC Class Codes: H04N 1/32352 Controlling detectability or arrangements to facilitate detection or retrieval of the embedded information, e.g. using markers for decoding, detection or retrieval operations H04N1/32149 and subgroups take precedence
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 2201/3233 of authentication information, e.g. digital signature, watermark
• CPC Class Codes: H04N 2201/3269 of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

Methods, apparatus, and systems use embedded watermarks to determine whether the segments of a multimedia content has been rearranged. The embedded watermarks are recovered from the content and one or more attributes associated with the recovered watermarks are identified. A continuity of the content can then be assessed in accordance with the one or more attributes. The continuity assessment may be based on a variety of factors, including but not limited to a determined heartbeat of the recovered watermarks, a density, separation, location, or extent, of the recovered watermarks, as well as information associated with the watermarks, such as a stego key, channel bits, packet numbers, a fingerprint, or the like.

Claims

1. A method for detecting a change in arrangement of segments of a multimedia content, comprising:

receiving a multimedia content at a multimedia reception device equipped with a watermark extractor, the multimedia content including a plurality of watermark messages embedded therein;

extracting the plurality of watermark messages using the watermark extractor that is implemented at least partially in electronic circuits, wherein;

each embedded watermark message spans a predetermined length or duration of the multimedia content,

each of the plurality of watermark messages, in absence of content manipulations, is nominally separated from another embedded watermark message by a predetermined separation value that is either equal to the predetermined length or duration, or is an integer multiple of the predetermined length or duration,

the received multimedia content includes at least one content manipulation that causes an actual separation of a first one of the plurality of watermark messages from a second one of the plurality of watermark messages to deviate from the predetermined separation value by an offset value; and

determining that the at least one content manipulation has occurred by comparing the offset value to a threshold offset value.

2. The method of claim 1, wherein the threshold offset value is determined at least in-part from a difference in detected position of the second one of the plurality of watermark messages and the first one of the plurality of watermark messages.

3. The method of claim 1, wherein the at least one content manipulation includes an interruption in playback of the multimedia content indicative of a user action.

4. The method of claim 3, wherein the user action includes skipping at least a portion of the multimedia content.

5. The method of claim 1, wherein the at least one content manipulation includes one or more of a cut, an insertion or a re-ordering of segments of the multimedia content.

6. The method of claim 1, wherein the threshold offset value is determined at least in-part based on a measurement error in determining a location of the first one of the plurality of watermark messages or the second one of the plurality of watermark messages.

7. The method of claim 1, wherein the threshold offset value is determined at least in-part based on a predetermined level of allowable content scaling that causes a temporal, a spatial or a frequency variation in the multimedia content.

8. The method of claim 7, wherein the content scaling includes one of or more of:

a linear time scaling of an audio component of the multimedia content,

a spatial scaling of a video component of the multimedia content,

a temporal scaling of a video component of the multimedia content, or

a pitch-invariant time scaling of an audio component of the multimedia content.

9. The method of claim 1, wherein the threshold offset value comprises the following components:

a predetermined allowable content scaling value, a predetermined measurement error associated with determining a location of the first one of the plurality of watermark messages or the second one of the plurality of watermark messages, and a temporal or spatial difference in locations of the second one of the plurality of watermark messages and the first one of the plurality of watermark messages.

10. The method of claim 1, further comprising determining an extent of the multimedia content that has been viewed by a consumer of the multimedia content.

11. The method of claim 10, further comprising determining the multimedia content has been viewed or broadcast in its entirety.

12. A device, comprising:

a processor; and

a memory including processor executable code, the processor executable code, when executed by the processor, configures the device to;

receive a multimedia content, the multimedia content including a plurality of watermark messages embedded therein;

extract the plurality of watermark messages using a watermark extractor, wherein;

each embedded watermark message spans a predetermined length or duration of the multimedia content,

each of the plurality of watermark messages, in absence of content manipulations, is nominally separated from another embedded watermark message by a predetermined separation value that is either equal to the predetermined length or duration, or is an integer multiple of the predetermined length or duration,

the received multimedia content includes at least one content manipulation that causes an actual separation of a first one of the plurality of watermark messages from a second one of the plurality of watermark messages to deviate from the predetermined separation value by an offset value; and

determine that the at least one content manipulation has occurred by comparing the offset value to a threshold offset value.

13. The device of claim 12, wherein the threshold offset value is determined at least in-part from a difference in detected position of the second one of the plurality of watermark messages and the first one of the plurality of watermark messages.

14. The device of claim 12, wherein the at least one content manipulation includes an interruption in playback of the multimedia content indicative of a user action.

15. The device of claim 14, wherein the user action includes skipping at least a portion of the multimedia content.

16. The device of claim 12, wherein the at least one content manipulation includes one or more of a cut, an insertion or a re-ordering of segments of the multimedia content.

17. The device of claim 12, wherein the threshold offset value is determined at least in-part based on a measurement error in determining a location of the first one of the plurality of watermark messages or the second one of the plurality of watermark messages.

18. The device of claim 12, wherein the threshold offset value is determined at least in-part based on a predetermined level of allowable content scaling that causes a temporal, a spatial or a frequency variation in the multimedia content.

19. The device of claim 18, wherein the content scaling includes one of or more of:

a linear time scaling of an audio component of the multimedia content,

a spatial scaling of a video component of the multimedia content,

a temporal scaling of a video component of the multimedia content, or

a pitch-invariant time scaling of an audio component of the multimedia content.

20. The device of claim 12, wherein the threshold offset value comprises the following components:

a predetermined allowable content scaling value, a predetermined measurement error associated with determining a location of the first one of the plurality of watermark messages or the second one of the plurality of watermark messages, and a temporal or spatial difference in locations of the second one of the plurality of watermark messages and the first one of the plurality of watermark messages.

21. The device of claim 12, the processor executable code, when executed by the processor, further configures the device determine a duration of the multimedia content that has been viewed by a consumer of the multimedia content.

22. The device of claim 21, the processor executable code, when executed by the processor, further configures the device to further to determine whether the multimedia content has been viewed or broadcast in its entirety.

23. A computer program product embodied on one or more non-transitory computer readable media, comprising:

program code for receiving a multimedia content at a multimedia reception device equipped with a watermark extractor, the multimedia content including a plurality of watermark messages embedded therein;

program code for extracting the plurality of watermark messages using the watermark extractor, wherein;

each embedded watermark message spans a predetermined length or duration of the multimedia content,

each of the plurality of watermark messages, in absence of content manipulations, is nominally separated from another embedded watermark message by a predetermined separation value that is either equal to the predetermined length or duration, or is an integer multiple of the predetermined length or duration,

the received multimedia content includes at least one content manipulation that causes an actual separation of a first one of the plurality of watermark messages from a second one of the plurality of watermark messages to deviate from the predetermined separation value by an offset value; and

program code for determining that the at least one content manipulation has occurred by comparing the offset value to a threshold offset value.