Modifying a coded bitstream

• US 9,299,120 B2
• Filed: 11/02/2007
• Issued: 03/29/2016
• Est. Priority Date: 03/23/2007
• Status: Active Grant

• Current Assignee: ContentArmor
• Original Assignee: Thomson Licensing LLC (The Woodbridge Co. Ltd.)
• Inventors: Zou, Dekun, Bloom, Jeffrey Adam, Yin, Peng, Divorra Escoda, Oscar
• Primary Examiner(s): Rostami, Mohammad S
• Assistant Examiner(s): ABRAHAM, AHMED M
• Application Number:
  US12/450,343
• Publication Number:
  US 20100169349A1
• Time in Patent Office: 3,070 Days
• Field of Search: 382/100, 707/100, 713/176
• US Class Current: 1/1
• CPC Class Codes: G06T 1/0035 Output size adaptive watermarking
• CPC Class Codes: G06T 2201/0053 Embedding of the watermark in the coding stream, possibly without decoding; Embedding of the watermark in the compressed domain
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Implementations may relate to various aspects of modifying, or using, a coded bitstream, or to recovering watermarking information from data. In one implementation, a coded bitstream is modified to allow for the application of a watermark to the coded bitstream without changing the coding of any other syntax elements in the bitstream. This is performed by identifying conditions necessary to change a coded syntax element without disrupting the interpretation of subsequent coded elements. The coded syntax element is replaced with another coded value if the modified value generates the same decoding variables as the original value would have generated. Particular implementations focus on syntax elements coded using entropy coding, including, for example, Context-based Adaptive Binary Arithmetic Coding (CABAC). Regardless of the type of coding or watermarking, various implementations may provide for recovering watermarking information. Particular implementations recover watermarking information using detection data and feature information.

Claims

1. A method, for use with a processor, the method comprising:

accessing information for use in determining payload information conveyed in a watermarked image, the accessed information having been produced prior to watermarking an image to produce the watermarked image, said payload information being associated with a watermark included in said watermarked image, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, and the accessed information including;

location information that identifies a location of a set of pixels in the watermarked image, wherein the set of pixels conveys the payload information in a pixel-domain representation of the set of pixels of the watermarked image,

one or more values for a feature in the pixel-domain representation of the set of pixels, wherein the one or more values for the feature indicate whether first payload information or second payload information is conveyed in the pixel-domain representation by the set of pixels, and wherein the one or more values comprise;

a first value for the feature in the pixel-domain representation of the set of pixels, wherein the first value for the feature indicates that the first payload information is conveyed in the pixel-domain representation by the set of pixels; and

a second value for the feature in the pixel-domain representation of the set of pixels, wherein the second value for the feature indicates that the second payload information is conveyed in the pixel-domain representation by the set of pixels;

accessing the watermarked image said watermarked image having a test value for the feature based on said modified at least one syntax element;

determining the a-test value for the feature of the set of pixels in the accessed watermarked image;

comparing the test value with at least one of the first value and second values for the feature; and

determining the payload information based on a result of the comparing of the test value.

2. The method of claim 1 wherein the feature comprises an average luminance of the set of pixels, or a variance of the set of pixels.

3. The method of claim 1 wherein the location of the set of pixels that conveys the payload information, as well as the set of pixels, have been selected based on content present at the location prior to watermarking an image to produce the watermarked image.

4. The method of claim 1 further comprising using the determined payload information to identify a device that applied the payload information and created the watermarked image.

5. The method of claim 1 wherein the location information comprises:

a frame identifier that identifies the watermarked image;

and a block identifier that identifies the set of pixels.

6. The method of claim 1 wherein:

the accessed information further comprises a base value for the feature based on a second location that is different from the location of the set of pixels and that does not convey payload information; and

determining the test value for the feature comprises;

determining an actual value for the feature for pixels at the second location,

comparing the base value with the determined actual value for the feature for pixels at the second location,

determining an actual value for the feature of the set of pixels in the accessed watermarked image, and

adjusting the determined actual value for the feature of the set of pixels to produce the test value, the adjusting being based on a result of the comparing of the base value with the determined actual value for the feature for pixels at the second location.

7. The method of claim 6 wherein:

the result of the comparing of the base value comprises a difference between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises adding or subtracting a number to the determined actual value for the feature of the set of pixels, the number being based on the difference.

8. The method of claim 6 wherein:

the result of the comparing of the base value comprises a ratio between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises multiplying or dividing a number by the determined actual value for the feature of the set of pixels, the number being based on the ratio.

9. The method of claim 1 wherein determining the payload information comprises determining a value for a bit of a payload.

10. The method of claim 9 wherein the payload is a one-bit payload.

11. The method of claim 1 further comprising:

repeating the operations of accessing information, determining the test value, and comparing the test value, the repeating being done one or more times for additional locations in the watermarked image.

12. The method of claim 1 further comprising:

repeating the operations of accessing information, determining the test value, comparing the test value, and determining the payload information, the repeating being done one or more times for additional locations in the watermarked image.

13. The method of claim 1 further comprising:

repeating the operations of accessing information, accessing a watermarked image, determining the test value, comparing the test value, and determining the payload information, the repeating being done for one or more additional watermarked images.

39. The method of claim 1 further comprising:

storing the accessed information.

14. An Apparatus comprising a processing device configured to perform at least the following:

accessing information for use in determining payload information conveyed in a watermarked image, the accessed information having been produced prior to watermarking an image to produce the watermarked image, said payload information being associated with a watermark included in said watermarked image, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, and the accessed information including;

location information that identifies a location of a set of pixels in the watermarked image, wherein the set of pixels conveys the payload information in a pixel-domain representation of the set of pixels of the watermarked image,

one or more values for a feature in the pixel-domain representation of the set of pixels, wherein the one or more values for the feature indicate whether first payload information or second payload information is conveyed in the pixel-domain representation by the set of pixels, and wherein the one or more values comprise;

a first value for the feature in the pixel-domain representation of the set of pixels, wherein the first value for the feature indicates that the first payload information is conveyed in the pixel-domain representation by the set of pixels; and

a second value for the feature in the pixel-domain representation of the set of pixels, wherein the second value for the feature indicates that the second payload information is conveyed in the pixel-domain representation by the set of pixels;

accessing the watermarked image said watermarked image having a test value for the feature based on said modified at least one syntax element;

determining the a-test value for the feature of the set of pixels in the accessed watermarked image;

comparing the test value with at least one of the first value and second values for the feature; and

determining the payload information based on a result of the comparing of the test value.

15. The Apparatus of claim 14 wherein the feature comprises an average luminance of the set of pixels, or a variance of the set of pixels.

16. The Apparatus of claim 14 wherein the location of the set of pixels that conveys the payload information, as well as the set of pixels, have been selected based on content present at the location prior to watermarking an image to produce the watermarked image.

17. The Apparatus of claim 14 wherein the processing device is further configured to perform at least the following:

using the determined payload information to identify a device that applied the payload information and created the watermarked image.

18. The Apparatus of claim 14 wherein the location information comprises:

a frame identifier that identifies the watermarked image;

and a block identifier that identifies the set of pixels.

19. The Apparatus of claim 14 wherein:

the accessed information further comprises a base value for the feature based on a second location that is different from the location of the set of pixels and that does not convey payload information; and

determining the test value for the feature comprises;

determining an actual value for the feature for pixels at the second location,

comparing the base value with the determined actual value for the feature for pixels at the second location,

determining an actual value for the feature of the set of pixels in the accessed watermarked image, and

adjusting the determined actual value for the feature of the set of pixels to produce the test value, the adjusting being based on a result of the comparing of the base value with the determined actual value for the feature for pixels at the second location.

20. The Apparatus of claim 14 wherein determining the payload information comprises determining a value for a bit of a payload.

21. The Apparatus of claim 14 wherein the processing device is further configured to perform the following repeating operations:

repeating the operations of accessing information, determining the test value, and comparing the test value, the repeating being done one or more times for additional locations in the watermarked image.

22. The Apparatus of claim 14 further comprising a memory for storing the accessed information.

40. The apparatus of claim 19 wherein:

the result of the comparing of the base value comprises a difference between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises adding or subtracting a number to the determined actual value for the feature of the set of pixels, the number being based on the difference.

41. The apparatus of claim 19 wherein:

the result of the comparing of the base value comprises a ratio between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises multiplying or dividing a number by the determined actual value for the feature of the set of pixels, the number being based on the ratio.

42. The apparatus of claim 14 wherein the payload is a one-bit payload.

43. The apparatus of claim 14 wherein the processing device is further configured to perform:

repeating the operations of accessing information, determining the test value, comparing the test value, and determining the payload information, the repeating being done one or more times for additional locations in the watermarked image.

44. The apparatus of claim 14 wherein the processing device is further configured to perform:

repeating the operations of accessing information, accessing a watermarked image, determining the test value, comparing the test value, and determining the payload information, the repeating being done for one or more additional watermarked images.

23. An Apparatus comprising:

means for performing at least the following;

accessing information for use in determining payload information conveyed in a watermarked image, the accessed information having been produced prior to watermarking an image to produce the watermarked image, said payload information being associated with a watermark included in said watermarked image, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, and the accessed information including;

location information that identifies a location of a set of pixels in the watermarked image, wherein the set of pixels conveys the payload information in a pixel-domain representation of the set of pixels of the watermarked image,

one or more values for a feature in the pixel-domain representation of the set of pixels, wherein the one or more values for the feature indicate whether first payload information or second payload information is conveyed in the pixel-domain representation by the set of pixels, and wherein the one or more values comprise;

a first value for the feature in the pixel-domain representation of the set of pixels, wherein the first value for the feature indicates that the first payload information is conveyed in the pixel-domain representation by the set of pixels; and

a second value for the feature in the pixel-domain representation of the set of pixels, wherein the second value for the feature indicates that the second payload information is conveyed in the pixel-domain representation by the set of pixels;

accessing the watermarked image said watermarked image having a test value for the feature based on said modified at least one syntax element;

determining the a-test value for the feature of the set of pixels in the accessed watermarked image;

comparing the test value with at least one of the first value and second value for the feature; and

determining the payload information based on a result of the comparing of the test value; and

a processor readable medium for storing the accessed information.

24. The Apparatus of claim 23 wherein the feature comprises an average luminance of the set of pixels, or a variance of the set of pixels.

25. The Apparatus of claim 23 wherein the location of the set of pixels that conveys the payload information, as well as the set of pixels, have been selected based on content present at the location prior to watermarking an image to produce the watermarked image.

26. The Apparatus of claim 23 wherein the processing device is further configured to perform at least the following:

using the determined payload information to identify a device that applied the payload information and created the watermarked image.

27. The Apparatus of claim 23 wherein the location information comprises:

a frame identifier that identifies the watermarked image;

and a block identifier that identifies the set of pixels.

28. The Apparatus of claim 23 wherein:

the accessed information further comprises a base value for the feature based on a second location that is different from the location of the set of pixels and that does not convey payload information; and

determining the test value for the feature comprises;

determining an actual value for the feature for pixels at the second location,

comparing the base value with the determined actual value for the feature for pixels at the second location,

determining an actual value for the feature of the set of pixels in the accessed watermarked image, and

adjusting the determined actual value for the feature of the set of pixels to produce the test value, the adjusting being based on a result of the comparing of the base value with the determined actual value for the feature for pixels at the second location.

29. The Apparatus of claim 23 wherein determining the payload information comprises determining a value for a bit of a payload.

30. The Apparatus of claim 23 wherein the processing device is further configured to perform:

repeating the operations of accessing information, determining the test value, and comparing the test value, the repeating being done one or more times for additional locations in the watermarked image.

45. The apparatus of claim 28 wherein:

the result of the comparing of the base value comprises a difference between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises adding or subtracting a number to the determined actual value for the feature of the set of pixels, the number being based on the difference.

46. The apparatus of claim 28 wherein:

the result of the comparing of the base value comprises a ratio between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises multiplying or dividing a number by the determined actual value for the feature of the set of pixels, the number being based on the ratio.

47. The apparatus of claim 23 wherein the payload is a one-bit payload.

48. The apparatus of claim 23 wherein the processing device is further configured to perform:

repeating the operations of accessing information, determining the test value, comparing the test value, and determining the payload information, the repeating being done one or more times for additional locations in the watermarked image.

49. The apparatus of claim 23 wherein the processing device is further configured to perform:

repeating the operations of accessing information, accessing a watermarked image, determining the test value, comparing the test value, and determining the payload information, the repeating being done for one or more additional watermarked images.

31. An Apparatus comprising a non-transitory processor-readable medium including instructions stored on the processor-readable medium for performing at least the following:

accessing information for use in determining payload information conveyed in a watermarked image, the accessed information having been produced prior to watermarking an image to produce the watermarked image, said payload information being associated with a watermark included in said watermarked image, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, and the accessed information including;

location information that identifies a location of a set of pixels in the watermarked image, wherein the set of pixels conveys the payload information in a pixel-domain representation of the set of pixels of the watermarked image,

one or more values for a feature in the pixel-domain representation of the set of pixels, wherein the one or more values for the feature indicate whether first payload information or second payload information is conveyed in the pixel-domain representation by the set of pixels, and wherein the one or more values comprise;

a first value for the feature in the pixel-domain representation of the set of pixels, wherein the first value for the feature indicates that the first payload information is conveyed in the pixel-domain representation by the set of pixels; and

a second value for the feature in the pixel-domain representation of the set of pixels, wherein the second value for the feature indicates that the second payload information is conveyed in the pixel-domain representation by the set of pixels;

accessing the watermarked image said watermarked image having a test value for the feature based on said modified at least one syntax element;

determining the a-test value for the feature of the set of pixels in the accessed watermarked image;

comparing the test value with at least one of the first value and second values for the feature; and

determining the payload information based on a result of the comparing of the test value.

32. The Apparatus of claim 31 wherein the feature comprises an average luminance of the set of pixels, or a variance of the set of pixels.

33. The Apparatus of claim 31 wherein the location of the set of pixels that conveys the payload information, as well as the set of pixels, have been selected based on content present at the location prior to watermarking an image to produce the watermarked image.

34. The Apparatus of claim 31 wherein the non-transitory processor-readable medium further includes instructions stored on the processor-readable medium for performing at least the following:

using the determined payload information to identify a device that applied the payload information and created the watermarked image.

35. The Apparatus of claim 31 wherein the location information comprises:

a frame identifier that identifies the watermarked image;

and a block identifier that identifies the set of pixels.

36. The Apparatus of claim 31 wherein:

the accessed information further comprises a base value for the feature based on a second location that is different from the location of the set of pixels and that does not convey payload information; and

determining the test value for the feature comprises;

determining an actual value for the feature for pixels at the second location,

comparing the base value with the determined actual value for the feature for pixels at the second location,

determining an actual value for the feature of the set of pixels in the accessed watermarked image, and

adjusting the determined actual value for the feature of the set of pixels to produce the test value, the adjusting being based on a result of the comparing of the base value with the determined actual value for the feature for pixels at the second location.

37. The Apparatus of claim 31 wherein determining the payload information comprises determining a value for a bit of a payload.

38. The Apparatus of claim 31 wherein the non-transitory processor-readable medium further includes instructions stored on the processor-readable medium for performing:

repeating the operations of accessing information, determining the test value, and comparing the test value, the repeating being done one or more times for additional locations in the watermarked image.

50. The apparatus of claim 36 wherein:

the result of the comparing of the base value comprises a difference between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises adding or subtracting a number to the determined actual value for the feature of the set of pixels, the number being based on the difference.

51. The apparatus of claim 36 wherein:

the result of the comparing of the base value comprises a ratio between the base value and the determined actual value for the feature for pixels at the second location, and

adjusting the determined actual value for the feature of the set of pixels comprises multiplying or dividing a number by the determined actual value for the feature of the set of pixels, the number being based on the ratio.

52. The apparatus of claim 31 wherein the payload is a one-bit payload.

53. The apparatus of claim 31 wherein the non-transitory processor-readable medium further includes instructions stored on the processor-readable medium for performing:

repeating the operations of accessing information, determining the test value, comparing the test value, and determining the payload information, the repeating being done one or more times for additional locations in the watermarked image.

54. The apparatus of claim 31 wherein the non-transitory processor-readable medium further includes instructions stored on the processor-readable medium for performing:

repeating the operations of accessing information, accessing a watermarked image, determining the test value, comparing the test value, and determining the payload information, the repeating being done for one or more additional watermarked images.

55. The apparatus of claim 31 wherein the non-transitory processor-readable medium further includes instructions stored on the processor-readable medium for performing:

storing the accessed information.

56. A method comprising:

watermarking an image with a watermark, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, said watermarked image having a value for a feature of said image based on said modified at least one syntax element.

57. An Apparatus comprising a processing device configured to perform at least the following:

watermark an image with a watermark, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, said watermarked image having a value for a feature of said image based on said modified at least one syntax element.

58. An Apparatus comprising a non-transitory processor-readable medium including instructions stored on the processor-readable medium for performing at least the following:

watermark an image with a watermark, said watermarked image and said image being arithmetically encoded by an arithmetic code, said watermark modifying at least one syntax element of said image and having watermarked said image by replacing arithmetically coded bits directly without prior arithmetic decoding and re-encoding of said image, said watermarked image having a value for a feature of said image based on said modified at least one syntax element.