Efficient application of video marking technologies

• US 7,738,677 B2
• Filed: 05/16/2007
• Issued: 06/15/2010
• Est. Priority Date: 05/17/2006
• Status: Active Grant

• Current Assignee: Verimatrix Incorporated (Verimatrix SA)
• Original Assignee: Verimatrix Incorporated (Verimatrix SA)
• Inventors: Thorwirth, Niels
• Primary Examiner(s): Lu; Tom Y
• Application Number:
  US11/749,589
• Publication Number:
  US 20070291979A1
• Time in Patent Office: 1,126 Days
• Field of Search: 382/100
• US Class Current: 382/100
• CPC Class Codes: G11B 20/00086 Circuits for prevention of unauthorised reproduction or copying, e.g. piracy indicating unauthorised use of record carriers in general G11B23/28; scrambling for television signal recording H04N5/913; network architectures or network protocols for network security H04L63/00; cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
• CPC Class Codes: G11B 20/00144 involving a user identifier, e.g. a unique customer ID
• CPC Class Codes: G11B 20/00884 involving a watermark, i.e. a barely perceptible transformation of the original data which can nevertheless be recognised by an algorithm
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: H04N 2005/91335 the copy protection signal being a watermark
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/913 for scrambling ; for copy protection
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal

Abstract

Systems and methods are described for rendering information to be embedded in media content at a first location and for embedding the rendered information into the media content at a second location. In many embodiments, the embedding process is less processor intensive than the rendering process and can be performed on a consumer electronics device such as a set top box, using existing processing mechanisms. One embodiment of the invention includes rendering the information into an image at a first location and embedding the image in the media at a second location in order to achieve efficient marking of the media content.

Claims

1. A method of embedding information in media content during a process for decoding the media content, comprising:

delivering media content to a decode device configured to decode the media content and output frames of video for display on an output device, where the decode device includes an alpha blending function configured to combine an on screen display and media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present and the decode device is configured to output the combined frame for display on the output device;

decoding a frame of the media content;

rendering the information as an overlay image and alpha values according to perceptual characteristics of the media content so that the modifications to the decoded frame are imperceptible when the overlay image is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device;

at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, modifying the decoded frame by using the alpha blending function of the decode device to combine the entire rendered overlay image with the decoded frame of the media content in accordance with the alpha values, and thereby imperceptibly producing a marked frame in which the rendered overlay image is present and imperceptible; and

outputting the marked frame from the decode device to the output device for display.

2. The method of claim 1 wherein the information embedded in media content is transformed from a frequency domain to render the image.

3. The method of claim 1 wherein:

the media content is delivered as a compressed file; and

the perceptual characteristics of the media content are determined from the compressed electronic file.

4. The method of claim 1 wherein the information comprises metadata related to the media content.

5. The method of claim 1 wherein the information identifies a time and location of media content playback.

6. The method of claim 1 wherein the information identifies a copyright owner or recipient of the media content.

7. The method of claim 1 wherein the information represents a database index.

8. The method of claim 1 wherein the rendered image is stored for repeated or subsequent use.

14. The method of claim 1, wherein the device renders the image from the information.

15. The method of claim 1, wherein the device determines the perceptual characteristics of the media content.

16. The method of claim 1, further comprising determining the perceptual characteristics of at least one frame of the video sequence and rendered the overlay image and alpha values in accordance with the perceptual characteristics of the at least one frame of the video sequence.

17. The method of claim 1, wherein the media content is delivered over a network.

18. The method of claim 1, wherein the media content is delivered via a fixed media.

21. The method of claim 1, wherein the alpha blending function is implemented as follows:

S_x,y,f=I_x,y,f* α

_x,y+O_x,y,f* (1−

α

_x,y)

where S_x,y,f is the pixel at position x,y in frame number f of the marked video;

I_x,y,f is the pixel at position x,y of the overlay image;

O_x,y,f is the pixel at position x,y in frame number f of the original video; and

α

_x,y is the alpha value indicating the strength of the overlay image at position x,y.

23. The method of claim 1, wherein the overlay image and the alpha values are generated by the decode device.

24. The method of claim 1, wherein the overlay image and the alpha values are received by the decode device.

9. A device for decoding media content and embedding information in the media content during decoding, comprising:

decode circuitry configured to decode the media content for display on a display device;

alpha blending circuitry configured to overlay an on screen display onto media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present for display on an output device;

rendering circuitry configured to render the information as an overlay image and alpha values according to perceptual characteristics of the media content so that the modifications to the decoded frame are imperceptible when the overlay image is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device; and

wherein the device is configured to imperceptibly embed the information in the media content at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, the embedding being achieved by overlaying the entire overlay image onto the decoded frame using the alpha blending circuitry in accordance with the alpha values, thereby imperceptibly producing a marked frame in which the rendered overlay image is present and imperceptible.

10. The device of claim 9, wherein the device is a digital set top box.

11. The device of claim 9, wherein the device is configured to transform the information that is to be embedded in the media content from the frequency domain to create the rendered image.

12. The device of claim 9, wherein

the device is configured to determine the perceptual characteristics of the media content.

13. The device of claim 12, wherein:

the media content is stored as a compressed electronic file;

the device is configured to determine the perceptual characteristics of the media from the compressed electronic file.

19. The device of claim 9, further comprising a microprocessor connected to a memory, wherein the microprocessor is configured to render the information as an overlay image and store the overlay image in memory.

20. The device of claim 19, wherein the microprocessor and memory is the circuitry configured to decode the media content for display on a display device and the circuitry configured to overlay an on screen display onto decoded media content for display on an output device.

22. The system of claim 9, wherein the alpha blending circuitry is configured to perform the following operation:

S_x,y,f=I_x,y,f* α

_x,y+O_x,y,f* (1 −

α

_x,y)

where S_x,y,f is the pixel at position x,y in frame number f of the marked video;

I_x,y,f is the pixel at position x,y of the overlay image;

O_x,y,f is the pixel at position x,y in frame number f of the original video; and

α

_x,y is the alpha value indicating the strength of the overlay image at position x,y.

25. A method of embedding information in media content during a process for decoding the media content, comprising:

delivering media content to a decode device configured to decode the media content and output frames of video for display on an output device, where the decode device includes an alpha blending function configured to combine an on screen display and media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present and the decode device is configured to output the combined frame for display on the output device;

decoding a frame of the media content;

rendering the information as an overlay image frame and alpha values so that the modifications to the decoded frame are imperceptible when the overlay image frame is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device;

at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, modifying the decoded frame by using the alpha blending function of the decode device to combine the entire rendered overlay image frame with the decoded frame of the media content in accordance with the alpha values, and thereby imperceptibly producing a marked frame in which the rendered overlay image frame is present and imperceptible; and

outputting the marked frame from the decode device to the output device for display.

26. The method of claim 25 wherein the information embedded in media content is transformed from a frequency domain to render the image.

27. The method of claim 25 wherein the information comprises metadata related to the media content.

28. The method of claim 25 wherein the rendered overlay image frame is stored for repeated or subsequent use.

29. The method of claim 25, wherein the device renders the image from the information.

30. The method of claim 25, wherein the media content is delivered over a network.

31. The method of claim 25, wherein the alpha blending function is implemented as follows:

S_x,y,f=I_x,y,f* α

_x,y+O_x,y,f* (1−

α

_x,y)

where S_x,y,f is the pixel at position x,y in frame number f of the marked video;

I_x,y,f is the pixel at position x,y of the overlay image;

O_x,y,f is the pixel at position x,y in frame number f of the original video; and

α

_x,y is the alpha value indicating the strength of the overlay image at position x,y.

32. The method of claim 25, wherein the overlay image and the alpha values are generated by the decode device.

33. A device for decoding media content and embedding information in the media content during decoding, comprising:

decode circuitry configured to decode the media content for display on a display device;

alpha blending circuitry configured to overlay an on screen display onto media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present for display on an output device;

rendering circuitry configured to render the information as an overlay image frame and alpha values so that the modifications to the decoded frame are imperceptible when the overlay image frame is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device; and

wherein the device is configured to imperceptibly embed the information in the media content at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, the embedding being achieved by overlaying the entire overlay image frame onto the decoded frame using the alpha blending circuitry in accordance with the alpha values, thereby imperceptibly producing a marked frame in which the rendered overlay image frame is present and imperceptible.

34. The device of claim 33, wherein the device is a digital set top box.

35. The device of claim 33, wherein the device is configured to transform the information that is to be embedded in the media content from the frequency domain to create the rendered overlay image frame.

36. The device of claim 33, wherein:

the device is configured to determine the perceptual characteristics of the media content; and

the device is configured to modify the overlay image frame generated from the information according to the perceptual characteristics of the media content.

37. The device of claim 36, wherein:

the media content is stored as a compressed electronic file; and

the device is configured to determine the perceptual characteristics of the media from the compressed electronic file.

38. The device of claim 33, further comprising a microprocessor connected to a memory, wherein the microprocessor is configured to render the information as an overlay image frame and store the overlay image frame in memory.

39. The device of claim 38, wherein the microprocessor and memory is the circuitry configured to decode the media content for display on a display device and the circuitry configured to overlay an on screen display onto decoded media content for display on an output device.