Modifying a coded bitstream

• US 8,588,459 B2
• Filed: 06/13/2008
• Issued: 11/19/2013
• Est. Priority Date: 06/14/2007
• Status: Active Grant

• Current Assignee: ContentArmor
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Bloom, Jeffrey Adam, Zou, Dekun
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US12/451,551
• Publication Number:
  US 20100150394A1
• Time in Patent Office: 1,985 Days
• Field of Search: 382/100, 382/135, 382/218, 382/232, 382/238, 382/244, 382/240, 382/246, 380/28, 380/201, 341/50, 703/22, 715/723, 713/150, 713/168, 713/164, 713181-194, 726 26- 30, 348/509, 348/589, 358/3.28, 358/500, 358/426.04, 358/400, 358/539
• US Class Current: 382/100
• CPC Class Codes: G06T 1/0035 Output size adaptive watermarking
• CPC Class Codes: G06T 2201/0053 Embedding of the watermark in the coding stream, possibly without decoding; Embedding of the watermark in the compressed domain
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/93 Run-length coding
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

Various implementations are provided for, for example, determining watermarking data, performing the watermarking, and processing watermarked data. At least one implementation watermarks AVC bitstream that uses CAVLC entropy coding. The watermarking data may be determined in a pre-processing stage and stored for later use in watermark a bitstream. One method accesses variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image. The encoding of the information has a particular value. The method determines an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference. The alternative value is for use in watermarking the variable-length encoded data.

Claims

1. A method for determining a watermark, the method comprising:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference; and

determining an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, wherein the alternative value is for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

2. The method of claim 1 further comprising designating the alternative value as a substitute for the particular value for use in watermarking the variable-length encoded data.

3. The method of claim 2 further comprising evaluating effectiveness of using the alternative value, in place of the current value, as a watermark, and

wherein designating the alternative value as a substitute is based on a result of evaluating effectiveness.

4. The method of claim 3 wherein evaluating comprises determining a cost for the alternative value based on fidelity of the watermark and robustness of the watermark.

5. The method of claim 3 further comprising:

evaluating effectiveness of an additional alternative value; and

determining, based on results from evaluating the alternative value and evaluating the additional alternative value, that the alternative value is more effective, and

wherein designating the alternative value as a substitute is based on determining that the alternative value is more effective.

6. The method of claim 5 wherein:

the particular value identifies a first value for a macroblock type field for an intra-predicted 16×

16 macroblock in H.264/AVC or a related standard, the first value indicating a first intra-prediction mode,

the alternative value identifies a second value for a macroblock type field for an intra-predicted 16×

16 macroblock in H.264/AVC or a related standard, the second value indicating a second intra-prediction mode, and

the additional alternative value identifies a third value for a macroblock type field for an intra-predicted 16×

16 macroblock in H.264/AVC or a related standard, the third value indicating a third intra-prediction mode.

7. The method of claim 6 wherein a CAVLC encoding of each of the first value, the second value, and the third value uses the same number of bits.

8. The method of claim 1 further comprising:

inserting into a table the alternative value;

determining location of the particular value in the variable-length encoded data; and

inserting into the table information identifying the location in the variable-length encoded data at which the particular value is located, the location being where the alternative value would be inserted into the variable-length encoded data to replace the particular value.

9. The method of claim 1 further comprising replacing the particular value with the alternative value in the variable-length encoded data.

10. The method of claim 1 wherein the information identifying the actual reference identifies pixel locations of blocks that are adjacent to a current block, wherein the values of the pixel locations are used to predict at least a portion of the current block.

11. The method of claim 1 wherein the information identifying the actual reference identifies a reference block or a reference picture.

12. The method of claim 1 wherein the information identifying the actual reference comprises a macroblock type field in H.264/AVC or a related standard.

13. The method of claim 1 wherein the information identifying the actual reference comprises a macroblock type field for an intra-predicted 16×

16 macroblock in H.264/AVC or a related standard.

14. The method of claim 1 wherein the information identifying the actual reference comprises a prediction mode in H.264/AVC or a related standard.

15. The method of claim 1 wherein the information identifying the actual reference comprises a motion vector.

16. The method of claim 1 wherein the variable-length encoded data further includes an encoding of information identifying a current residue resulting from predicting the image with the current reference.

17. The method of claim 1 wherein:

the variable-length encoded data has been encoded using a particular variable-length code, and

the alternative value encodes the alternative information using the particular variable-length code.

18. An apparatus for determining a watermark, the apparatus comprising:

an entropy decoder to decode variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference; and

a watermark generator, coupled to the entropy decoder, to determine an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, wherein the alternative value is for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

19. The apparatus of claim 18 wherein the watermark generator is configured to create metadata for use in watermarking the variable-length encoded data, the metadata including (1) the alternative value and (2) a location of the particular value in the variable-length encoded data, and the metadata associating the alternative value and the location.

20. An apparatus for determining a watermark, the apparatus comprising:

means for accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference; and

means for determining an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, wherein the alternative value is for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

21. A processor configured to perform at least the following:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference; and

determining an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, wherein the alternative value is for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

22. A non-transitory processor-readable medium having stored thereon instructions for performing at least the following:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference; and

determining an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, wherein the alternative value is for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

23. A non-transitory processor-readable medium having watermark data stored thereon,

the watermark data for use in watermarking variable-length encoded data that includes an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference, and

the watermark data including an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference, the alternative value being a substitute for the particular value for use in watermarking the variable-length encoded data by inserting the alternative value into the variable-length encoded data as a substitute for the particular value and by also retaining in the variable-length encoded data the residue produced from predictively encoding the image based on the actual reference.

24. The processor-readable medium of claim 23 wherein the processor-readable medium has stored thereon the variable-length encoded data that includes the encoding of information identifying the actual reference used to predictively encode the image, the encoding of the information having the particular value.

25. A method for watermarking, the method comprising:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference;

accessing an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference; and

inserting the alternative value into the variable-length encoded data as a substitute for the particular value to watermark the image, wherein the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference and also includes the residue produced from predictively encoding the image based on the actual reference.

26. The method of claim 25 wherein:

accessing the variable-length encoded data occurs in response to a request for presentation of the variable-length encoded data, and

processing operations to determine the alternative value have been performed prior to the accessing of the variable-length encoded data, such that upon receiving the request for presentation of the variable-length encoded data and subsequently accessing the variable-length encoded data, the alternative value may be inserted without needing to determine the alternative value.

27. The method of claim 25 further comprising performing processing operations to determine the alternative value, prior to accessing the alternative value.

53. The method of claim 25 wherein the predictive encoding of the image is intra-prediction, such that the actual reference is an intra-prediction reference.

54. The method of claim 25 wherein the different reference is an intra-prediction reference.

55. The method of claim 25 wherein the information identifying the actual reference includes a macroblock type field as used in H.264 or a related standard.

56. The method of claim 25 wherein the information identifying the actual reference includes an intra-prediction mode indicator, as used in H.264 or a related standard.

28. An apparatus comprising a stream modification unit configured to perform at least the following:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference;

accessing an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference; and

inserting the alternative value into the variable-length encoded data as a substitute for the particular value to watermark the image, wherein the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference also includes the residue produced from predictively encoding the image based on the actual reference.

29. The apparatus of claim 28 further comprising a DVD reader, and wherein the apparatus is a DVD player.

30. An apparatus for watermarking, the apparatus comprising:

means for accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference;

means for accessing an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference; and

means for inserting the alternative value into the variable-length encoded data as a substitute for the particular value to watermark the image wherein the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference and also includes the residue produced from predictively encoding the image based on the actual reference.

31. A non-transitory processor configured to perform at least the following:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference;

accessing an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference; and

inserting the alternative value into the variable-length encoded data as a substitute for the particular value to watermark the image, wherein the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference and also includes the residue produced from predictively encoding the image based on the actual reference.

32. A non-transitory processor-readable medium having stored thereon instructions for performing at least the following:

accessing variable-length encoded data including an encoding of information identifying an actual reference used to predictively encode an image, the encoding of the information having a particular value, and the variable-length encoded data also including an encoding of a residue produced from predictively encoding the image based on the actual reference;

accessing an alternative value that is different from the particular value, such that the alternative value is an encoding of alternative information that identifies a different reference from the actual reference; and

inserting the alternative value into the variable-length encoded data as a substitute for the particular value to watermark the image, wherein the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference and also includes the residue produced from predictively encoding the image based on the actual reference.

33. A non-transitory processor-readable medium having stored thereon variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from the actual reference used to predictively encode the image, wherein:

the information identifying the alternative reference provides a watermark for the image,

the variable-length encoded data stored on the medium further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference, and

the variable-length encoded data includes the alternative value encoding the alternative information identifying the different reference and also includes the residue produced from predictively encoding the image based on the actual reference.

34. A method for decoding, the method comprising:

decoding variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provides a watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

35. The method of claim 34 wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

36. The method of claim 34 wherein decoding comprises decoding the variable-length code and the predictive code.

37. The method of claim 34 wherein decoding produces a baseband reconstruction of the image that varies from the original image by including a detectable feature arising from the use of the information identifying the alternative reference as a watermark.

38. An apparatus comprising a decoder configured to decode variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provides a watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

39. An apparatus comprising:

means for decoding variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provides a watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

40. The apparatus of claim 39 further comprising memory means for storing the variable-length encoded data.

41. A processor configured to decode variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provides a watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

42. A processor-readable medium having stored thereon instructions for decoding variable-length encoded data for an encoding of an image, the variable-length encoded data comprising an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provides a watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

43. A non-transitory processor-readable medium having stored thereon a pixel-domain image that includes a detectable feature arising from a watermark of a variable-length encoding of one or more syntax elements from a predictive encoding of the image, the variable-length encoding having included an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provided the watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

44. A method comprising providing for display a pixel-domain image that includes a detectable feature arising from a watermark of a variable-length encoding of one or more syntax elements from a predictive encoding of the image, the variable-length encoding having included an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image,

wherein the information identifying the alternative reference provided the watermark for the image,

wherein the variable-length encoded data further comprises an encoding of information identifying a residue from predictively encoding the image using the actual reference.

45. A method comprising:

variable-length encoding a predictive encoding of a watermarked version of an image, the variable-length encoded data comprising an indicator of a current reference used in encoding the watermarked version of the image and an indicator of a resulting residue, the watermarked version of the image having a detectable characteristic that provides a watermark,

wherein the detectable characteristic results from (1) modifying a previous variable-length encoding of a previous predictive-encoding of the image, the previous predictive-encoding being based on a previous reference and including information identifying the previous reference, wherein the modifying includes distorting the information in order to indicate an alternative reference and to provide the detectable characteristic for the image, and (2) decoding the previous variable-length encoding and the previous predictive-encoding of the image to produce the watermarked version of the image in which the detectable characteristic is present.

46. A non-transitory processor-readable medium having stored thereon:

variable-length encoded data from a variable-length encoding of a predictive encoding of a watermarked version of an image, the variable-length encoded data comprising an indicator of a current reference used in encoding the watermarked version of the image and an indicator of a resulting residue, the watermarked version of the image having a detectable characteristic that provides a watermark,

wherein the detectable characteristic results from (1) modifying a previous variable-length encoding of a previous predictive-encoding of the image, the previous predictive-encoding being based on a previous reference and including information identifying the previous reference, wherein the modifying includes distorting the information in order to indicate an alternative reference and to provide the detectable characteristic for the image, and (2) decoding the previous variable-length encoding and the previous predictive-encoding of the image to produce the watermarked version of the image in which the detectable characteristic is present.

47. A method comprising decoding variable-length encoded data from a variable-length encoding of a predictive encoding of a watermarked version of an image, the variable-length encoded data comprising an indicator of a current reference used in encoding the watermarked version of the image and an indicator of a resulting residue, the watermarked version of the image having a detectable characteristic that provides a watermark,

wherein the detectable characteristic results from (1) modifying a previous variable-length encoding of a previous predictive-encoding of the image, the previous predictive-encoding being based on a previous reference and including information identifying the previous reference, wherein the modifying includes distorting the information in order to indicate an alternative reference and to provide the detectable characteristic for the image, and (2) decoding the previous variable-length encoding and the previous predictive-encoding of the image to produce the watermarked version of the image in which the detectable characteristic is present.

48. A non-transitory processor-readable medium having stored thereon pixel-domain image data resulting from decoding variable-length encoded data from a variable-length encoding of a predictive encoding of a watermarked version of an image, the variable-length encoded data comprising an indicator of a current reference used in encoding the watermarked version of the image and an indicator of a resulting residue, the watermarked version of the image having a detectable characteristic that provides a watermark,

wherein the detectable characteristic results from (1) modifying a previous variable-length encoding of a previous predictive-encoding of the image, the previous predictive-encoding being based on a previous reference and including information identifying the previous reference, wherein the modifying includes distorting the information in order to indicate an alternative reference and to provide the detectable characteristic for the image, and (2) decoding the previous variable-length encoding and the previous predictive-encoding of the image to produce the watermarked version of the image in which the detectable characteristic is present.

49. A method comprising providing for display pixel-domain image data resulting from decoding variable-length encoded data from a variable-length encoding of a predictive encoding of a watermarked version of an image, the variable-length encoded data comprising an indicator of a current reference used in encoding the watermarked version of the image and an indicator of a resulting residue, the watermarked version of the image having a detectable characteristic that provides a watermark,

wherein the detectable characteristic results from (1) modifying a previous variable-length encoding of a previous predictive-encoding of the image, the previous predictive-encoding being based on a previous reference and including information identifying the previous reference, wherein the modifying includes distorting the information in order to indicate an alternative reference and to provide the detectable characteristic for the image, and (2) decoding the previous variable-length encoding and the previous predictive-encoding of the image to produce the watermarked version of the image in which the detectable characteristic is present.

50. A method for determining payload information, the method comprising:

accessing information that identifies a location in variable-length encoded data at which a variable-length encoding of information is located, the information identifying one of at least two references associated with a predictive encoding of an image;

accessing the encoding of the information at the location in the variable-length encoded data, wherein the information also identifies a residue resulting from predictively encoding the image using a reference other than the identified reference;

determining, based on the encoding of the information, which of the at least two references is identified; and

determining payload information based on a result of determining which of the at least two references is identified.

51. The method of claim 50 wherein the identified reference is different from an actual reference used in the predictive encoding of the image, and variable-length encoding the identified reference rather than the actual reference provided a watermark for the image.

52. A method for determining payload information, the method comprising:

accessing a pixel-domain image that includes a detectable feature arising from a watermark of a variable-length encoding of one or more syntax elements from a predictive encoding of the image, the variable-length encoding including an encoding of information identifying an alternative reference that is different from an actual reference used in predictively encoding the image, wherein the information identifying the alternative reference provided the watermark for the image;

accessing detection metadata indicating a location in the pixel-domain image at which the detectable feature is located;

analyzing the pixel-domain image at the location to determine a value for the detectable feature;

accessing additional detection metadata indicating a comparison value for the detectable feature;

comparing the determined value for the detectable feature with the comparison value; and

determining payload information based on a result of the comparing.