Video fingerprinting using complexity-regularized video watermarking by statistics quantization

• US 7,702,127 B2
• Filed: 10/21/2005
• Issued: 04/20/2010
• Est. Priority Date: 10/21/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mihcak, Mehmet Kivanc, Harmanci, Oztan
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): Fitzpatrick; Atiba O.
• Application Number:
  US11/256,642
• Publication Number:
  US 20070092103A1
• Time in Patent Office: 1,642 Days
• Field of Search: 382/100, 382232-253, 380200-242, 35842601-42616, 354/555, 3483841-4401, 375/122, 37524001-24029
• US Class Current: 382/100
• CPC Class Codes: G06T 1/0028 Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
• CPC Class Codes: G06T 2201/0202 whereby the quality of watermarked images is measured; Measuring quality or performance of watermarking methods; Balancing between quality and robustness
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

Video fingerprinting using watermarks is described herein. A watermark encoder embeds a watermark into an input video to produce a watermarked video. The watermark encoder can choose particular regions of the video that are sufficiently complex for watermarking. Spatial and temporal complexity analyses can be performed on a per-pixel basis to locate suitable regions of the video. The watermarks can be embedded using a unique codeword associated with a given user. A watermark decoder can receive the watermarked video. In some instances, the watermarked video may have been attacked in some way after being output by the watermark encoder. In these instances, the watermark decoder can extract the watermark from the watermarked video. In turn, the watermark decoder can extract the codeword from the watermark, and compare the extracted codeword to a plurality of known codewords associated with respective users.

Claims

1. A system comprising:

a memory;

a processor coupled to the memory;

a watermark encoder executed by the processor and adapted to receive as input at least one video and at least one watermark, and to embed the watermark into the video, wherein the watermark encoder is adapted to perform a complexity analysis on a per-pixel basis on at least one region of the video, the complexity analysis based on a statistical video model that provides entropy rates in a direct current sub-band portion of the video;

the watermark encoder adapted to partition the video into one or more scenes having one or more corresponding scene-cuts;

the watermark encoder adapted to select one or more regions of the video to embed the watermark, wherein a region is randomly selected in time without the region overlapping any scene-cuts in the video;

the watermark encoder adapted to embed the watermark into the video by using a two-dimensional discrete wavelet transform along spatial axes;

the watermark encoder adapted to populate side information related to a region of the video, wherein the side information comprises a two-dimensional hash of a randomly selected frame within the video and data representing a motion field of a pixel in the video; and

a watermark decoder executed by the processor and adapted to receive a watermarked video and to extract the watermark therefrom, wherein the watermark decoder locates the one or more regions of the video selected by the watermark encoder.

2. The system of claim 1, wherein the watermark encoder is adapted to perform a pixel-wise complexity analysis on at least one region of the video.

3. The system of claim 1, wherein the watermark encoder is adapted to compare at least one region of the video to at least one complexity threshold to determine whether the region is sufficiently complex to embed the watermark therein.

4. The system of claim 1, wherein the watermark encoder is adapted to perform a spatial complexity analysis on at least one region of the video.

5. The system of claim 1, wherein the watermark encoder is adapted to perform a temporal complexity analysis on at least one region of the video.

6. The system of claim 1, wherein the watermark encoder is adapted to combine at least one temporal complexity measure and at least one spatial complexity measure for at least one region of the video into a final per-pixel complexity measure for the region.

7. The system of claim 1, wherein the watermark encoder is adapted to define at least one motion vector associated with a pixel in a frame of the video, wherein the motion vector represents an estimate of the movement of the pixel as the video is played either forward or backward at least one further frame.

8. The system of claim 1, wherein the watermark encoder is adapted to define at least one cumulative motion vector associated with a pixel in a frame of the video, wherein the motion vector represents an estimate of the movement of the pixel as the video is played either forward or backward a plurality of further frames.

9. The system of claim 1, wherein the watermark encoder is adapted to partition the video into respective scenes, and to define at least one candidate region for embedding the watermark based on the respective scenes of the video.

10. The system of claim 1, wherein the watermark encoder is adapted to pass the side information to the watermark decoder.

11. The system of claim 1, wherein the watermark encoder is adapted to define a hash value for at least one frame of the video, and to pass this hash value to the watermark decoder as side information.

12. The system of claim 11, wherein the frame for which the hash values is defined is chosen based at least in part on how distinct the frame is relative to at least one neighboring frame.

13. The system of claim 1, wherein the video decoder is adapted to traverse the watermarked video, to extract hash values for a plurality of frames of the watermarked video, and to match the hash values with further hash values received as side information from the watermark encoder.

14. The system of claim 1, wherein the watermark decoder is adapted to extract a codeword from the watermark, to compare the codeword to a plurality of known codewords associated with respective users, and to estimate which user is associated with the extracted codeword based on the comparison.

15. A computer implemented method having instructions executable by a processor, comprising:

receiving, by a watermark encoder executed by the processor coupled to a memory, at least one video and at least one watermark as input to embed the watermark into the video, wherein the watermark encoder is adapted to perform a complexity analysis on a per-pixel basis on at least one region of the video;

partitioning the video into one or more scenes having one or more corresponding scene-cuts;

selecting one or more regions of the video to embed the watermark, wherein a region is randomly selected in time without the region overlapping any scene-cuts in the video;

embedding the watermark into the video by using a two-dimensional wavelet transform along spatial axes;

populating side information related to a region of the video, wherein the side information comprises a two-dimensional hash of a randomly selected frame within the video and data representing a motion field of a pixel in the video; and

receiving, by a watermark decoder executed by the processor coupled to the memory, a watermarked video to extract the watermark from the video, wherein the watermark decoder locates the one or more regions of the video selected by the watermark encoder.

16. The computer implemented method of claim 15, wherein the watermark encoder is adapted to combine at least one temporal complexity measure and at least one spatial complexity measure for at least one region of the video into a final per-pixel complexity measure for the region.

17. The computer implemented method of claim 15, wherein the watermark encoder is adapted to define at least one cumulative motion vector associated with a pixel in a frame of the video, wherein the motion vector represents an estimate of the movement of the pixel as the video is played either forward or backward a plurality of further frames.

18. The computer implemented method of claim 15, wherein the watermark encoder is adapted to define a hash value for at least one frame of the video, and to pass this hash value to the watermark decoder as side information.

19. One or more computer readable storage media storing computer executable instructions that, when executed by a processor of a computer, cause the processor to perform acts comprising:

receiving, by a watermark encoder executed by the processor, at least one video and at least one watermark as input to embed the watermark into the video, wherein the watermark encoder is adapted to perform a complexity analysis on a per-pixel basis on at least one region of the video, the complexity analysis based on a statistical video model that provides entropy rates in a direct current sub-band portion of the video;

partitioning the video into one or more scenes having one or more corresponding scene-cuts;

selecting one or more regions of the video to embed the watermark, wherein a region is randomly selected in time without the region overlapping any scene-cuts in the video;

embedding the watermark into the video by using a two-dimensional wavelet transform along spatial axes; and

receiving, by a watermark decoder executed by the processor, a watermarked video to extract the watermark from the video, wherein the watermark decoder locates the one or more regions of the video selected by the watermark encoder.

20. The one or more computer readable storage media as recited in claim 19, wherein the watermark encoder is adapted to combine at least one temporal complexity measure and at least one spatial complexity measure for at least one region of the video into a final per-pixel complexity measure for the region.