Spatio-temporal channel for images employing a watermark and its complement

• US 6,563,936 B2
• Filed: 03/09/2001
• Issued: 05/13/2003
• Est. Priority Date: 09/07/2000
• Status: Expired due to Term

• Current Assignee: Transpacific Information, LLC
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Brill, Michael H., Pica, Albert P., Isnardi, Michael A.
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US09/803,392
• Publication Number:
  US 20020027612A1
• Time in Patent Office: 795 Days
• Field of Search: 382/100, 382/232, 713/176, 380/210, 380/252, 380/287, 380/54, 348/461, 348/463, 348/465, 348/467
• US Class Current: 382/100
• CPC Class Codes: G06T 1/0071 using multiple or alternating watermarks
• CPC Class Codes: G06T 1/0085 Time domain based watermarking, e.g. watermarks spread over several images
• CPC Class Codes: G06T 2201/0081 whereby both original and watermarked images are required at decoder, e.g. destination-based, non-blind, non-oblivious
• CPC Class Codes: G06T 2201/0083 whereby only watermarked image required at decoder, e.g. source-based, blind, oblivious
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

A channel is inserted into a sequence of frames for an image, by varying one or more display characteristics across a display in accordance with a spatio-temporal pattern applied to, for example, successive lines of each modified frame. The channel may be employed to watermark the image, and the watermark may be defined as W(x) and its complement [1−W(x)] in counter-phase rotations of successive lines in pairs of frames. The watermark information is applied in relatively still portions of the image with sufficient lightness level. A receiver includes a watermark detector that has a priori information about the watermark. The watermark detector examines specific regions of the displayed image over time, and tests the regions for the watermark during time intervals when the image remains relatively still.

Claims

1. A method for adding a watermark to a video signal representing an image, the method comprising the steps of:

(a) applying a first watermark function to a first set of pixels in a first frame;

(b) applying a complement of the first watermark function to a second set of pixels in the first frame.

(c) applying the complement of the first watermark function to a first set of pixels in a second frame corresponding to the first set of pixels of the first frame; and

(d) applying the first watermark function to a second set of pixels in the second frame corresponding to the second set of pixels of the first frame,

wherein the application of the first watermark function and the complement of the first watermark function in steps(a) through (d) tends to mask visibility of the watermark between the first and second frames.

2. The invention of claim 1, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

3. The invention of claim 1, wherein the watermark is added to only B frames.

4. The invention of claim 1, wherein the watermark is added to only B frames.

5. The invention of claim 1, wherein the first and second sets of pixels are adjacent to one another in the first frame.

6. The invention of claim 1, wherein the first watermark function is applied to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding color component of each pixel in the second set of pixels.

7. The invention of claim 1, wherein the first watermark function is applied to a transform coefficient corresponding to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding transform coefficient corresponding to a corresponding color component of each pixel in the second set of pixels.

8. The invention of claim 1, further comprising the steps of:

(c) applying a second watermark function to a third set of pixels in the first frame; and

(b) applying a complement of the second watermark function to a fourth set of pixels in the first frame,

wherein relative positioning of the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

9. The invention of claim 8, wherein the first watermark function is the same as the second watermark function.

10. The invention of claim 1, wherein the watermark is added only to frames determined to be still when one or more regions in the frame are determined to be still.

11. The invention of claim 1, wherein the watermark is added only to frames determined to be bright when one or more regions in the frame are determined to be bright.

12. The invention of claim 11, wherein the watermark is added only to pixels that are bright in bright frames.

13. The invention of claim 1, wherein the first watermark function and the complement of the first watermark function represent data of a data stream encoded into the watermark.

14. The invention of claim 1, further comprising the steps of repeating steps (a) and (b) for at least one other watermark function in different sets of pixels.

15. The invention of claim 1, wherein application of the first watermark function varies a display characteristic over the first set of pixels and application of the complement of the first watermark function varies the display characteristic over the second set of pixels, wherein the variations of the display characteristic over the first and second sets of pixels tend to mask the visibility of the watermark in the first frame.

16. The invention of claim 1, wherein step (a) applies the first watermark function and step (b) applies the complement of the first watermark function to pixel components conveying blue color information.

17. The invention of claim wherein 16 pixel components conveying blue color information are at least one of B, V, Cb, or Pb components.

18. A method of detecting a watermark in a video signal representing an image, the method comprising the steps of:

(a) identifying first and second sets of pixels in a first frame;

(b) calculating a first difference value for the first and second sets of pixels; and

(c) determining whether the watermark is present based on the first difference value, wherein, when the watermark is present, the first difference value is related to prior application of a first watermark function to the first set of pixels and prior application of a complement of the first watermark function to the second set of pixels, and wherein;

step (a) further comprises the step of identifying corresponding first and second sets of pixels in a second frame;

step (b) further comprises the step of calculating a second difference value for the corresponding first and second sets of pixels in the second frame; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of the complement of the first watermark function to the first set of pixels in the second frame and prior application of the first watermark function to the second set of pixels in the second frame,

wherein the prior application of the first watermark function and the complement of the first watermark function to the first and second sets of pixels in the first and second frames tends to mask visibility of the watermark between the first and second frames.

19. The invention of claim 18, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

20. The invention of claim 18, wherein the first and second frames are successive frames.

21. The invention of claim 18, wherein the watermark is added to only B frames.

22. The invention of claim 18, wherein the first and second sets of pixels are adjacent to one another in the first frame.

23. The invention of claim 18, wherein the first watermark function is applied to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding color component of each pixel in the second set of pixels.

24. The invention of claim 23, wherein the color component conveys blue color information.

25. The invention of claim 24, wherein the color component is at least one of B, V, Cb, or Pb components.

26. The invention of claim 18, wherein the first watermark function is applied to a transform coefficient corresponding to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding transform coefficient corresponding to a corresponding color component of each pixel in the second set of pixels.

27. The invention of claim 18, wherein:

step (a) further comprises the step of identifying third and fourth sets of pixels in the first frame;

step (b) further comprises the step of calculating a second difference value for the third and fourth sets of pixels; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of a second watermark function to the third set of pixels and prior application of a complement of the second watermark function to the fourth set of pixels,

wherein relative positioning of the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

28. The invention of claim 27, wherein the first watermark function is the same as the second watermark function.

29. The invention of claim 27, wherein relative positioning between the first and second sets of pixels and the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

30. The invention of claim 27, wherein the first, second, third, and fourth sets of pixels are adjacent to one another.

31. The invention of claim 18, wherein the watermark is detected only in frames determined to be still, wherein a frame is still when one or more regions in the frame are determined to be still.

32. The invention of claim 18, wherein step (c) further comprises the step of interpreting data of a data stream encoded in the watermark based on the first difference value.

33. The invention of claim 18, wherein step (b) comprises the step of calculating the first difference value based on the first and second sets of pixels in the first frame and corresponding first and second sets of pixels in a reference frame corresponding to the first frame before application of the first watermark function and its complement.

34. Apparatus for adding a watermark to a video signal representing an image, the apparatus comprising:

a watermark inserter generating a watermark and identifying first and second sets of pixels in a first frame for the watermark;

a video processor, based on the watermark and the identified first and second sets of pixels, applying;

(1) a first watermark function to a first set of pixels in a first frame;

(2) a complement of the first watermark function to a second set of pixels in the first frame,

(3) the complement of the first watermark function to a first set of pixels in a second frame corresponding to the first set of pixels of the first frame; and

(4) the first watermark function to a second set of pixels in the second frame corresponding to the second set of pixels of the first frame,

wherein the application of the first watermark function and the complement of the first watermark function in steps (a) through (d) tends to mask visibility of the watermark between the first and second frames.

35. The invention of claim 34, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

36. Apparatus for detecting a watermark in a video signal representing an image, the apparatus comprising:

a video processor identifying first and second sets of pixels in a first frame;

a watermark detector;

(1) calculating a first difference value for the first and second sets of pixels; and

(2) determining whether the watermark is present based on the first difference value, wherein, when the watermark is present, the first difference value is related to prior application of a first watermark function to the first set of pixels and prior application of a complement of the first watermark function to the second set of pixels, and wherein;

step (a) further comprises the step of identifying corresponding first and second sets of pixels in a second frame;

step (b) further comprises the step of calculating a second difference value for the corresponding first and second sets of pixels in the second frame; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of the complement of the first watermark function to the first set of pixels in the second frame and prior application of the first watermark function to the second set of pixels in the second frame,

wherein the prior application of the first watermark function and the complement of the first watermark function to the first and second sets of pixels in the first and second frames tends to mask visibility of the watermark between the first and second frames.

37. The invention of claim 36, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

38. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to implement a method for watermarking video signal representing an image, the method comprising the steps of:

(a) applying a watermark function to a corresponding component of a first set pixels in a first frame;

(b) applying a complement of the watermark function to a corresponding component of a second set of pixels in the first frame to vary a display characteristic over a frame,

(c) applying the complement of the first watermark function to a first set of pixels in a second frame corresponding to the first set of pixels of the first frame; and

(d) applying the first watermark function to a second set of pixels in the second frame corresponding to the second set of pixels of the first frame,

wherein the application of the first watermark function and the complement of the first watermark function in steps (a) through (d) tends to mask visibility of the watermark between the first and second frames.

39. The invention of claim 38, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

40. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to implement a method for detecting a watermark in a video signal representing an image, the method comprising the steps of:

(a) determining whether one or more regions of the image are still;

(b) calculating, when one or more regions of the image are still, a difference value for corresponding components of a first set of pixels and a second set of pixels in a first frame; and

(c) determining whether the watermark is present based on the difference value, wherein the difference value is related to a watermark function applied to the first set of pixels and a complement of the watermark function applied to the second set of pixels, and wherein;

step (a) further comprises the step of identifying corresponding first and second sets of pixels in a second frame;

step (b) further comprises the step of calculating a second difference value for the corresponding first and second sets of pixels in the second frame; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of the complement of the first watermark function to the first set of pixels in the second frame and prior application of the first watermark function to the second set of pixels in the second frame,

wherein the prior application of the first watermark function and the complement of the first watermark function to the first and second sets of pixels in the first and second frames tends to mask visibility of the watermark between the first and second frames.