IDENTIFYING VIDEO CONTENT VIA COLOR-BASED FINGERPRINT MATCHING

• US 20150110340A1
• Filed: 10/23/2013
• Published: 04/23/2015
• Est. Priority Date: 10/23/2013
• Status: Active Grant

• Current Assignee: Roku, Inc.
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: Harron, Wilson, Cremer, Markus K.
• Granted Patent: US 9,465,995 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G06V 40/1365 Matching; Classification
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof

Abstract

Systems and methods of identifying unknown video content are described. An example method may include receiving a first fingerprint and a second fingerprint. The first fingerprint may be a color-based fingerprint derived from colors in a portion of the unknown video content, and the second fingerprint may be at least partially based on a feature other than the colors of the unknown video content. A reference database of reference fingerprints may then be queried using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints. The candidate group of fingerprints may then be queried using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint. The unknown video content may then be identified using the at least one query fingerprint. For example, the second fingerprint may be a luminance-based fingerprint derived from luminance in the unknown video content.

Claims

1. A method of identifying unknown video content, the method comprising:

receiving a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the unknown video content;

querying a reference database of reference fingerprints using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints;

querying the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

identifying the unknown video content using the at least one query fingerprint.

2. The method of claim 1, wherein the second fingerprint is a luminance-based fingerprint derived from luminance in the unknown video content.

3. The method of claim 1, wherein the reference database is queried with the first fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the second fingerprint to identify the at least one query fingerprint.

4. The method of claim 1, wherein the reference database is queried with the second fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the first fingerprint to identify the at least one query fingerprint.

5. The method of claim 1, wherein the query fingerprint is based on a value calculated for multiple patches of a frame of video content using an integral image technique and includes a value based on multiple colors within the frame of video content.

6. The method of claim 1, wherein the query fingerprint is based on a value calculated for multiple Haar-like features of the frame of video content and includes a value based on multiple colors within the frame of video content.

7. The method of claim 1, wherein the query fingerprint is based on:

a value calculated for multiple Haar-like features of the frame of video content, wherein each of the Haar-like features is associated with a unique location of a unique scale within the frame of video content; and

a value based on multiple colors within the frame of video content.

8. The method of claim 1, wherein the color-based fingerprint is based on a quantization of eight dominant colors within at least a portion of a frame of the unknown video content.

9. The method of claim 1, wherein the color-based fingerprint is based on a quantization of sixteen dominant colors within at least a portion of a frame of the unknown video content.

10. The method of claim 1, wherein querying a reference database using one of the first fingerprint or the second fingerprint uses a fingerprint defined by a lower number of bits than that used to query the candidate group.

11. The method of claim 1, wherein querying the reference database using the first fingerprint comprises querying an index of 8-bit numbers that are associated with colors within frames of known video content.

12. The method of claim 1, wherein querying the reference database of reference fingerprints using color-based fingerprint includes querying an index of 16-bit numbers that are associated with colors within the frames of known video content.

13. The method of claim 1, wherein

querying the reference database of reference fingerprints using color-based fingerprint includes querying an index of 8-bit numbers that are associated with colors within the frames of known video content;

and

querying the candidate group of reference fingerprints includes querying an index of 24-bit numbers that are associated with single frames of the reference fingerprints.

14. The method of claim 1, wherein the querying the database of reference fingerprints using the first fingerprint includes querying an index of 16-bit numbers that are associated with colors within the frames of known video content, and wherein querying the group of candidate reference fingerprints using the second fingerprint includes querying an index of 24-bit numbers that are associated with single frames of the reference fingerprints.

15. The method of claim 1, wherein querying the reference database with the color-based fingerprint comprises determining that a similarity between the color-based fingerprint and the reference fingerprints satisfies a predetermined threshold associated with a Tanimoto distance measurement.

16. The method of claim 1, wherein querying the reference database with the color-based fingerprint comprises determining that a similarity between the color-based fingerprint and the reference fingerprints satisfies a predetermined threshold associated with a Manhattan distance measurement.

17. The method of claim 1, wherein the first and the second fingerprint correspond to one of a plurality of patches, the patches being defined by regions of a frame of the unknown video content.

18. A system, comprising:

a fingerprint module configured to receive a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the unknown video content;

a processor-based query module configured to;

query a reference database of reference fingerprints using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints;

query the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

an identification module configured to identify the unknown video content using the at least one query fingerprint.

19. The system of claim 18, wherein the second fingerprint is a luminance-based fingerprint derived from luminance in the unknown video content.

20. A non-transitory computer-readable medium embodying instructions which, when executed by a machine, cause the machine to perform operations comprising:

receive a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the unknown video content;

query a reference database of reference fingerprints using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints;

query the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

identify the unknown video content using the at least one query fingerprint.