Efficient watermarking approaches of compressed media

• US 8,515,123 B2
• Filed: 07/06/2009
• Issued: 08/20/2013
• Est. Priority Date: 07/03/2008
• Status: Active Grant

• Current Assignee: Verimatrix Incorporated (Verimatrix SA)
• Original Assignee: Verimatrix Incorporated (Verimatrix SA)
• Inventors: Thorwirth, Niels J.
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US13/002,280
• Publication Number:
  US 20110129116A1
• Time in Patent Office: 1,506 Days
• Field of Search: 382/100, 382118-120, 382/212, 382/232, 382248-250, 382/276, 382/280, 382/284, 382/293, 382/296, 380/54, 380/210, 380/252, 380/287, 713/176
• US Class Current: 382/100
• CPC Class Codes: G06T 1/0092 Payload characteristic determination in a watermarking scheme, e.g. number of bits to be embedded
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking

Abstract

Systems and methods are described for imperceptibly embedding information by identifying locations that can be imperceptibly modified by replacing original content using replacement content derived from elsewhere in the compressed bitstream to remove at least a portion of the original content, generating at least one piece of replacement data for each identified location, where overwriting a portion of the compressed bitstream with replacement data prevents a decoder from decoding original content due to decoding replacement content derived from elsewhere in the compressed bitstream, selecting pieces of replacement data using an embedding device, where overwriting portions of the compressed bitstream using pieces of replacement data encodes payload information in accordance with a predetermined coding scheme, and embedding the payload information in the compressed bitstream by using the embedding device to overwrite at least a portion of the compressed bitstream with the selected replacement data.

Claims

1. A method of imperceptibly embedding information in a compressed video bitstream, comprising:

analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified, where the imperceptible modification includes replacing original content using replacement content derived from elsewhere in the compressed bitstream to remove at least a portion of the original content;

generating at least one piece of replacement data for each identified location using the pre-processor, where overwriting a portion of the compressed bitstream with any of the pieces of replacement data prevents a decoder from decoding original content and causes a decoder to decode replacement content derived from elsewhere in the compressed bitstream;

selecting pieces of replacement data using an embedding device, wherein overwriting portions of the compressed bitstream using the selected pieces of replacement data encodes payload information in accordance with a predetermined coding scheme; and

embedding the payload information in the compressed bitstream by using the embedding device to overwrite at least a portion of the compressed bitstream with the selected replacement data.

2. The method of claim 1, wherein the replacement data is stored on a physical media with the compressed video stream.

3. The method of claim 1, wherein the embedding device is a media server configured to transmit the compressed bitstream to a decoder, and the method further comprises:

generating payload information using the media server, where the payload information is indicative of a decoder that requests the transmission of the compressed bitstream; and

transmitting the compressed bitstream in which the payload information is embedded to the decoder using the media server.

4. The method of claim 1, wherein the embedding device is a decoder configured to receive the compressed bitstream and the replacement data, and the method further comprises:

decoding the video in the compressed bitstream after the decoder has embedded the payload information; and

outputting the decoded video to a rendering device using the decoder.

5. The method of claim 4, wherein the compressed bitstream received by the decoder includes the replacement data as part of the compressed bitstream.

6. The method of claim 4, wherein the replacement data is transmitted in a separate stream.

7. The method of claim 1, wherein the embedding device comprises a media server and a decoder, and the decoder selects the pieces of replacement data used to embed the payload information, and embedding the payload information in the compressed bitstream further comprises:

overwriting portions of the compressed bitstream with portions from a plurality of the generated pieces of replacement data so that a decoder cannot decode the original content in the overwritten locations without perceptible errors;

transmitting from the media server to the decoder the compressed bitstream, the original overwritten portions of the compressed bitstream and the portions of the generated pieces of replacement data that were not used by the media server in the overwriting of the compressed bitstream; and

embedding the payload information in accordance with the predetermined coding scheme by;

overwriting portions of the compressed bitstream using portions of the generated pieces of replacement data that were not used by the media server in the overwriting of the compressed bitstream, where the portions the decoder uses to overwrite the compressed bitstream are from the generated pieces of replacement data selected by the decoder; and

overwriting portions of replacement data within the compressed bitstream using the original overwritten portions of the compressed bitstream, where the portions of replacement data within the compressed bitstream that are overwritten by the decoder are from the generated pieces of replacement data that were not selected by the decoder.

8. The method of claim 1, wherein analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified further comprises identifying portions of video frames from within the compressed bitstream that can be imperceptibly modified, where the imperceptible modification includes replacing a portion of the original content of the video frame using replacement content derived from other frames in the compressed bitstream.

9. The method of claim 1, wherein analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified further comprises identifying video frames that can be imperceptibly modified, where the imperceptible modification includes replacing the original content of the video frame using replacement content derived from at least one other frame within the compressed bitstream.

10. The method of claim 9, wherein the replacement content is derived from the content of the previous video frame within the compressed bitstream.

11. The method of claim 10, wherein the replacement content is the same as the content of the previous video frame.

12. The method of claim 10, wherein the replacement content is derived by adding delta additions to the content of the previous video frame.

13. The method of claim 9, wherein the replacement content is derived from the content of the subsequent video frame from within the compressed bitstream.

14. The method of claim 13, wherein the replacement content is the same as the content of the subsequent video frame.

15. The method of claim 13, wherein the replacement content is derived by adding delta additions to the content of the subsequent video frame.

16. The method of claim 9, wherein the replacement content is derived from the content of the previous and subsequent video frames from within the compressed stream.

17. The method of claim 16, wherein the replacement content is the average of the content of the previous and subsequent video frames.

18. The method of claim 16, wherein the replacement content is derived by adding delta additions to the average of the content of the previous and subsequent video frames.

19. The method of claim 1 where the replacement data and at least a portion of the compressed bitstream are encrypted using at least one common encryption key so that the encrypted compressed bitstream can be overwritten with replacement data and the compressed bitstream decrypted.

20. The method of claim 1, wherein the replacement data used to overwrite at least a portion of the compressed bitstream indicates how to derive the replacement content, and the method further comprises:

decoding the compressed bitstream in which the payload information is embedded using a decoder;

wherein decoding the compressed bitstream comprises deriving replacement content from elsewhere in the compressed bitstream in accordance with each of the pieces of replacement data encoding payload information in the compressed bitstream.

21. The method of claim 20, wherein the replacement data indicates how to derive a replacement video frame using at least one other video frame within the compressed bitstream.

22. The method of claim 1, wherein:

the compressed bitstream incorporates video frames that are not referenced by other video frames in the compressed bitstream; and

analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified further comprises;

locating the video frames that are not referenced by other video frames in the compressed bitstream using the pre-processor; and

identifying the video frames that are not referenced by other video frames in the compressed bitstream that can be imperceptibly modified using the pre-processor, where the imperceptible modification includes replacing at least a portion of the original content of the video frame using replacement content derived from at least one other frame within the compressed bitstream.

23. The method of claim 22, wherein identifying the video frames that are not referenced by other video frames in the compressed bitstream that can be imperceptibly modified further comprises calculating the difference between each decoded video frame that is not referenced by other video frames in the compressed bitstream and at least one decoded neighboring video frame within the compressed bitstream.

24. The method of claim 1, wherein analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified further comprises analyzing the compressed bitstream using a pre-processor to identify locations that can be imperceptibly modified without propagating errors to other locations in the compressed bitstream.

25. A method of imperceptibly embedding information in a compressed video bitstream, comprising:

analyzing the compressed bitstream using a pre-processor to identifying video frames that are not referenced by other video frames in the compressed bitstream and which can be imperceptibly modified without propagating errors to other locations in the compressed bitstream, where the imperceptible modification includes replacing the original content of the video frame using replacement content derived from at least one other frame within the compressed bitstream;

generating at least one piece of replacement data for each identified video frame using the pre-processor, where overwriting a portion of the compressed bitstream with any of the pieces of replacement data prevents a decoder from decoding the original content of the identified video frame and causes a decoder to derive a replacement frame from at least one other frame within the compressed bitstream in a manner indicated by the replacement data;

selecting pieces of replacement data using an embedding device, wherein overwriting the compressed bitstream using each of the selected pieces of replacement data replaces an entire frame and encodes payload information in accordance with a predetermined coding scheme; and

embedding the payload information in the compressed bitstream by using the embedding device to overwrite at least a portion of the compressed bitstream with the selected replacement data.

26. The method of claim 25, wherein identifying the video frames that are not referenced by other video frames in the compressed bitstream that can be imperceptibly modified without propagating errors to other locations in the compressed bitstream further comprises calculating the difference between each decoded video frame that is not referenced by other video frames in the compressed bitstream and at least one decoded neighboring video frame within the compressed bitstream.

27. The method of claim 25, wherein the replacement data indicates that the replacement frame is derived by averaging the content of the previous and subsequent video frames from within the compressed stream.

28. A method of extracting payload information embedded in a copy of a marked bitstream, where the marked bitstream is derived by overwriting portions of the original bitstream using pieces of replacement data selected in accordance with a predetermined coding scheme to imperceptibly embed the payload information, and the selected replacement data removes at least a portion of the original content and causes a decoder to decode replacement content derived from elsewhere in the marked bitstream, the method comprising:

inspecting the copy of the marked bitstream to estimate the locations within the original bitstream that were overwritten using an extraction device; and

extracting the payload information embedded in the copy of the compressed bitstream based upon the predetermined coding scheme, and the estimates of the locations within the original bitstream that were overwritten using the extraction device.

29. The method of claim 28, wherein extracting the payload information embedded in the copy of the compressed bitstream based upon the predetermined coding scheme, and the estimates of the locations within the original bitstream that were overwritten using the extraction device further comprises comparing the content of the original compressed bitstream to the content of the copy of the compressed bitstream using the extraction device.

30. The method of claim 28, wherein:

each piece of replacement data replaces the original content of a video frame using replacement content derived from at least one other frame within the marked bitstream; and

inspecting the copy of the marked bitstream to estimate the locations within the original bitstream that were overwritten using an extraction device further comprises;

comparing each of a plurality of video frames in the copy of the marked bitstream to at least one other frame in the copy of the marked bitstream to determine the probability that each of the plurality of video frames is derived from at least one other video frame within the marked bitstream; and

estimating the locations within the original bitstream that were overwritten based upon the probabilities.

31. The method of claim 30, wherein:

the replacement content is derived from the previous frame from the original bitstream; and

each of the plurality of video frames is compared to its previous frame by the extraction device.

32. The method of claim 30, wherein:

the replacement content is derived from the subsequent frame from the original bitstream; and

each of the plurality of video frames is compared to its subsequent frame by the extraction device.

33. The method of claim 30, wherein:

the replacement content is derived from a combination of the previous and a subsequent frames from the original bitstream; and

each of the plurality of video frames is compared to the combination of its previous and a subsequent frames by the extraction device.

34. A system configured to pre-process a compressed video bitstream to identify frames that can be imperceptibly modified to embed information in the compressed bitstream, comprising:

a pre-processor configured to communicate with a storage device;

wherein the pre-processor is configured to;

receive the compressed bitstream;

analyze the compressed bitstream to identify locations that can be imperceptibly modified, where the imperceptible modification includes replacing original content using replacement content derived from elsewhere in the compressed bitstream to remove at least a portion of the content;

generate at least one piece of replacement data for each identified location, where overwriting a portion of the compressed bitstream with any of the pieces of replacement data prevents a decoder from decoding original content and causes a decoder to decode replacement content derived from elsewhere in the compressed bitstream; and

store the replacement data corresponding to each identified location on the storage device.

35. A system configured to imperceptibly embed payload information in a compressed bitstream using pre-processed replacement data associated with identified locations within the compressed bitstream, where overwriting a portion of the compressed bitstream with any of the pieces of replacement data prevents a decoder from decoding original content and causes a decoder to decode replacement content derived from elsewhere in the compressed bitstream, the system comprising:

an embedding device configured to receive the compressed bitstream and the pre-processed replacement data;

wherein the embedding device is configured to select pieces of pre-processed replacement data, wherein overwriting portions of the compressed bitstream using the selected pieces of replacement data encodes payload information in accordance with a predetermined coding scheme; and

wherein the embedding device is configured to embed the payload information in the compressed bitstream by using the embedding device to overwrite at least a portion of the compressed bitstream with the selected replacement data.