Fingerprinting digital video for rights management in networks

• US 20050213826A1
• Filed: 03/25/2004
• Published: 09/29/2005
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Neogi, Raja
• Granted Patent: US 7,336,841 B2
• Time in Patent Office: Days
• US Class Current: 382/232
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking

Abstract

Systems and methods generate video fingerprints for video data streams. A collection of sub-fingerprints constitute a fingerprint block. The size and temporal position of the block is chosen to represent the video that is being served. The video server has prior information about the video semantics. Based on this information, the server generates control codes that identify which blocks, based on time/byte offsets, in the sequence best represent the video object. The video receiver is equipped with facilities to generate these fingerprint blocks. The block(s) are sent back to the server. The server is equipped with facilities to verify and record the mismatches with respect to references. The mismatch statistics may be archived for use in a number of applications.

Claims

1. A method comprising:

receiving a video stream comprising a plurality of image frames, each image frame comprising a matrix of pixels;

selecting a subset of the image frames;

for each image frame in the subset determining a sub-fingerprint for the image frame; and

assembling the sub-fingerprints into a fingerprint for the video stream.

2. The method of claim 1, further comprising:

transmitting the fingerprint to a fingerprint verification system; and

comparing the fingerprint to a predetermined fingerprint for the video stream.

3. The method of claim 1, wherein selecting the subset of the image frames includes reading control codes from the video stream, said control codes identifying the subset of the image frames.

4. The method of claim 1 wherein determining a sub-fingerprint for the image frame comprises:

computing a discrete cosine transformation (DCT) block for a pixel block surrounding a pixel, said DCT block having coefficients;

computing an estimation of a variance of the coefficients;

setting a variance value in a variance matrix with the estimation of the variance, wherein the value is set at a position in the variance matrix corresponding to the pixel position in the image frame matrix;

determining a minimum variance value in a signature window of the variance matrix enclosing the pixel position; and

setting a first predetermined value representing the minimum variance in a constellation matrix at a position corresponding to the minimum variance value and setting all other positions in the signature window to a different predetermined value.

5. The method of claim 4, further comprising encoding the sub-fingerprint.

6. The method of claim 5, wherein the encoding comprises a run-length encoding.

7. The method of claim 4, wherein the DCT block has a size of eight by eight.

8. The method of claim 4, wherein the signature window has a size of eight by eight.

9. A device comprising:

a processor;

a network interface module operable to receive video data, the video data comprising a plurality of frames, each frame comprising a pixel matrix;

a memory coupled to the processor for storing the pixel matrix, a variance matrix, and a constellation matrix; and

a fingerprint generation module executing on the processor and operable to;

determine a discrete cosine transformation (DCT) block for a pixel block surrounding a pixel in the pixel matrix, said DCT block having coefficients;

compute an estimation of a variance of the coefficients;

set a variance value in the variance matrix with the estimation of the variance, wherein the value is set at a position in the variance matrix corresponding to the pixel position in the image frame matrix;

determine a minimum variance value in a signature window of the variance matrix enclosing the pixel position; and

set a first predetermined value representing the minimum variance in the constellation matrix at a position corresponding to the minimum variance value and setting all other positions in the signature window to a different predetermined value.

10. The device of claim 9, wherein the fingerprint generation module is further operable to run-length encode the sub-fingerprint.

11. The device of claim 9, wherein the DCT block has a size of eight by eight.

12. The device of claim 9, wherein the signature window has a size of eight by eight.

13. The device of claim 9 further comprising a DCT accelerator operable to calculate the DCT coefficients.

14. The device of claim 9, wherein the processor and memory are housed in a set-top box.

15. The device of claim 9, wherein the processor and memory are housed in a personal computer.

16. A system comprising

a video server communicably coupled to a communication channel and operable to transmit a video data stream through the communication channel;

a video receiver communicably coupled to the communication channel and operable to;

receive the video data stream;

determine a subset of images in the video data stream;

calculate a sub-fingerprint for each of the subset of images;

assemble the sub-fingerprint for each of the subset of images into a fingerprint; and

transmit the fingerprint to a fingerprint verification module.

17. The system of claim 16, further comprising a fingerprint mismatch database operable to store a reference fingerprint for the video data stream and wherein the fingerprint verification module is operable to compare the fingerprint to the reference fingerprint.

18. The system of claim 16, wherein the fingerprint verification module is located with the video server.

19. A machine-readable medium having machine executable instructions for performing a method, the method comprising:

receiving a video stream comprising a plurality of image frames, each image frame comprising a matrix of pixels;

selecting a subset of the image frames;

for each image frame in the subset determining a sub-fingerprint for the image frame; and

assembling the sub-fingerprints into a fingerprint for the video stream.

20. The machine-readable medium of claim 19, wherein the method further comprising:

transmitting the fingerprint to a fingerprint verification system; and

comparing the fingerprint to a predetermined fingerprint for the video stream.

21. The machine-readable medium of claim 19, wherein selecting the subset of the image frames includes reading control codes from the video stream, said control codes identifying the subset of the image frames.

22. The machine-readable medium of claim 19 wherein determining a sub-fingerprint for the image frame comprises:

computing a discrete cosine transformation (DCT) block for a pixel block surrounding a pixel, said DCT block having coefficients;

computing an estimation of a variance of the coefficients;

setting a variance value in a variance matrix with the estimation of the variance, wherein the value is set at a position in the variance matrix corresponding to the pixel position in the image frame matrix;

determining a minimum variance value in a signature window of the variance matrix enclosing the pixel position; and

setting a first predetermined value representing the minimum variance in a constellation matrix at a position corresponding to the minimum variance value and setting all other positions in the signature window to a different predetermined value.

23. The machine-readable medium of claim 22, wherein the method further comprises encoding the sub-fingerprint.

24. The machine-readable medium of claim 23, wherein the encoding comprises a run-length encoding.

25. The machine-readable medium of claim 22, wherein the DCT block has a size of eight by eight.

26. The machine-readable medium of claim 22, wherein the signature window has a size of eight by eight.