Spatio-temporal channel for images

• US 20020027612A1
• Filed: 03/09/2001
• Published: 03/07/2002
• Est. Priority Date: 09/07/2000
• Status: Active Grant

• Current Assignee: Transpacific Information, LLC
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Brill, Michael H., Pica, Albert P., Isnardi, Michael A.
• Granted Patent: US 6,563,936 B2
• Time in Patent Office: Days
• US Class Current: 348/473
• CPC Class Codes: G06T 1/0071 using multiple or alternating watermarks
• CPC Class Codes: G06T 1/0085 Time domain based watermarking, e.g. watermarks spread over several images
• CPC Class Codes: G06T 2201/0081 whereby both original and watermarked images are required at decoder, e.g. destination-based, non-blind, non-oblivious
• CPC Class Codes: G06T 2201/0083 whereby only watermarked image required at decoder, e.g. source-based, blind, oblivious
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

A channel is inserted into a sequence of frames for an image, by varying one or more display characteristics of the resulting image across the display in accordance with a spatio-temporal pattern applied to, for example, successive lines of each modified frame. In some cases, a reverse of the pattern is applied to a successive frame, and the frames modified are preferably B-frames if the image may be encoded in accordance with an MPEG standard. The channel may be employed to watermark the image, and the watermark may correspond to the presence of the channel or may be data in accordance with watermark information carried within the channel. For most display devices, display variations are minimized during the design process, but human viewers may still tolerate and accept subtle variations in a displayed image. Slightly changing one or more display characteristics in accordance with watermark information allows for watermarking of the image since viewers may not be aware that display characteristics are changing. For example, the blue color component of the video signal may be modified in accordance with watermark information defined as W(x) and its complement [1−W(x)] in counter-phase rotations of successive lines in pairs of frames. The blue color component of the video signal may be modified in those regions of the image declared to be relatively still and with sufficient level of blue-lightness. If the video signal is not encoded, the watermark information may alter each pre-selected pixel value'"'"'s least significant bit. If the video signal is encoded, such as into the frequency domain as specified in the MPEG standard, the corresponding (2,2) DCT coefficient in pairs of B-frames may be modified in accordance with watermark information. The watermark information is applied in relatively still portions of the image with sufficient blue-lightness level. A receiver includes a watermark detector that has a priori information about the watermark. The watermark detector examines specific regions of the displayed image over time, and tests the regions for the watermark during time intervals when the image remains relatively still. The receiver calculates the difference in pixel values in still regions between the received test frame and either a) corresponding pixel values in the non-modified image frame or b) the corresponding pixel values in the successive frame of a pair.

Claims

1. A method for adding a watermark to a video signal representing an image, the method comprising the steps of:

(a) applying a first watermark function to a first set of pixels in a first frame; and

(b) applying a complement of the first watermark function to a second set of pixels in the first frame.

2. The invention of claim 1, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

3. The invention of claim 1, further comprising the steps of:

(c) applying the complement of the first watermark function to a first set of pixels in a second frame corresponding to the first set of pixels of the first frame; and

(d) applying the first watermark function to a second set of pixels in the second frame corresponding to the second set of pixels of the first frame,

wherein the application of the first watermark function and the complement of the first watermark function in steps (a) through (d) tends to mask visibility of the watermark between the first and second frames.

4. The invention of claim 3, wherein the first and second frames are successive frames.

5. The invention of claim 1, wherein the watermark is added to only B frames.

6. The invention of claim 1, wherein the first and second sets of pixels are adjacent to one another in the first frame.

7. The invention of claim 1, wherein the first watermark function is applied to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding color component of each pixel in the second set of pixels.

8. The invention of claim 1, wherein the first watermark function is applied to a transform coefficient corresponding to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding transform coefficient corresponding to a corresponding color component of each pixel in the second set of pixels.

9. The invention of claim 1, further comprising the steps of:

(c) applying a second watermark function to a third set of pixels in the first frame; and

(b) applying a complement of the second watermark function to a fourth set of pixels in the first frame,

wherein relative positioning of the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

10. The invention of claim 9, wherein the first watermark function is the same as the second watermark function.

11. The invention of claim 1, wherein the watermark is added only to frames determined to be still when one or more regions in the frame are determined to be still.

12. The invention of claim 1, wherein the watermark is added only to frames determined to be bright when one or more regions in the frame are determined to be bright.

13. The invention of claim 12, wherein the watermark is added only to pixels that are bright in bright frames.

14. The invention of claim 1, wherein the first watermark function and the complement of the first watermark function represent data of a data stream encoded into the watermark.

15. The invention of claim 1, further comprising the steps of repeating steps (a) and (b) for at least one other watermark function in different sets of pixels.

16. The invention of claim 1, wherein application of the first watermark function varies a display characteristic over the first set of pixels and application of the complement of the first watermark function varies the display characteristic over the second set of pixels, wherein the variations of the display characteristic over the first and second sets of pixels tend to mask the visibility of the watermark in the first frame.

17. The invention of claim 1, wherein step (a) applies the first watermark function and step (b) applies the complement of the first watermark function to pixel components conveying blue color information.

18. The invention of claim 17 wherein pixel components conveying blue color information are at least one of B, V, Cb, or Pb components.

20. The invention of claim 19, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

21. The invention of claim 19, wherein:

step (a) further comprises the step of identifying corresponding first and second sets of pixels in a second frame;

step (b) further comprises the step of calculating a second difference value for the corresponding first and second sets of pixels in the second frame; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of the complement of the first watermark function to the first set of pixels in the second frame and prior application of the first watermark function to the second set of pixels in the second frame,

wherein the prior application of the first watermark function and the complement of the first watermark function to the first and second sets of pixels in the first and second frames tends to mask visibility of the watermark between the first and second frames.

22. The invention of claim 21, wherein the first and second frames are successive frames.

23. The invention of claim 19, wherein the watermark is added to only B frames.

24. The invention of claim 19, wherein the first and second sets of pixels are adjacent to one another in the first frame.

25. The invention of claim 19, wherein the first watermark function is applied to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding color component of each pixel in the second set of pixels.

26. The invention of claim 25, wherein the color component conveys blue color information.

27. The invention of claim 26, wherein the color component is at least one of B, V, Cb, or Pb components.

28. The invention of claim 19, wherein the first watermark function is applied to a transform coefficient corresponding to a color component of each pixel in the first set of pixels and the complement of the first watermark function is applied to a corresponding transform coefficient corresponding to a corresponding color component of each pixel in the second set of pixels.

29. The invention of claim 19, wherein:

step (a) further comprises the step of identifying third and fourth sets of pixels in the first frame;

step (b) further comprises the step of calculating a second difference value for the third and fourth sets of pixels; and

step (c) further comprises the step of determining whether the watermark is present based on the second difference value, wherein, when the watermark is present, the second difference value is related to prior application of a second watermark function to the third set of pixels and prior application of a complement of the second watermark function to the fourth set of pixels,

wherein relative positioning of the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

30. The invention of claim 29, wherein the first watermark function is the same as the second watermark function.

31. The invention of claim 29, wherein relative positioning between the first and second sets of pixels and the third and fourth sets of pixels tends to mask visibility of the watermark in the first frame.

32. The invention of claim 29, wherein the first, second, third, and fourth sets of pixels are adjacent to one another.

33. The invention of claim 19, wherein the watermark is detected only in frames determined to be still, wherein a frame is still when one or more regions in the frame are determined to be still.

34. The invention of claim 19, wherein step (c) further comprises the step of interpreting data of a data stream encoded in the watermark based on the first difference value.

35. The invention of claim 19, wherein step (b) comprises the step of calculating the first difference value based on the first and second sets of pixels in the first frame and corresponding first and second sets of pixels in a reference frame corresponding to the first frame before application of the first watermark function and its complement.

37. The invention of claim 36, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

39. The invention of claim 38, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

19. A method of detecting a watermark in a video signal representing an image, the method comprising the steps of:

(a) identifying first and second sets of pixels in a first frame;

(b) calculating a first difference value for the first and second sets of pixels; and

(c) determining whether the watermark is present based on the first difference value, wherein, when the watermark is present, the first difference value is related to prior application of a first watermark function to the first set of pixels and prior application of a complement of the first watermark function to the second set of pixels.

36. Apparatus for adding a watermark to a video signal representing an image, the apparatus comprising:

a watermark inserter generating a watermark and identifying first and second sets of pixels in a first frame for the watermark;

a video processor, based on the watermark and the identified first and second sets of pixels, applying;

(1) a first watermark function to a first set of pixels in a first frame; and

(2) a complement of the first watermark function to a second set of pixels in the first frame.

38. Apparatus for detecting a watermark in a video signal representing an image, the apparatus comprising:

a video processor identifying first and second sets of pixels in a first frame;

a watermark detector;

(1) calculating a first difference value for the first and second sets of pixels; and

(2) determining whether the watermark is present based on the first difference value, wherein, when the watermark is present, the first difference value is related to prior application of a first watermark function to the first set of pixels and prior application of a complement of the first watermark function to the second set of pixels.

40. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to implement a method for watermarking video signal representing an image, the method comprising the steps of:

(a) applying a watermark function to a corresponding component of a first set pixels in a first frame; and

(b) applying a complement of the watermark function to a corresponding component of a second set of pixels in the first frame to vary a display characteristic over a frame.

41. The invention of claim 40, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

42. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to implement a method for detecting a watermark in a video signal representing an image, the method comprising the steps of:

(a) determining whether one or more regions of the image are still;

(b) calculating, when one or more regions of the image are still, a difference value for corresponding components of a first set of pixels and a second set of pixels in a first frame; and

(c) determining whether the watermark is present based on the difference value, wherein the difference value is related to a watermark function applied to the first set of pixels and a complement of the watermark function applied to the second set of pixels.

44. The invention of claim 42, wherein relative positioning of the first and second sets of pixels tends to mask visibility of the watermark in the first frame.

43. A medium comprising a watermarked video bit-stream representing an image, wherein the bit-stream is watermarked by:

(a) applying a first watermark function to a first set of pixels in a first frame; and

(b) applying a complement of the first watermark function to a second set of pixels in the first frame.