VIDEO DNA (VDNA) METHOD AND SYSTEM FOR MULTI-DIMENSIONAL CONTENT MATCHING

• US 20150254343A1
• Filed: 05/27/2015
• Published: 09/10/2015
• Est. Priority Date: 05/30/2011
• Status: Abandoned Application

• Current Assignee: Vobile Incorporated (Vobile Group Ltd.)
• Original Assignee: Vobile Incorporated (Vobile Group Ltd.)
• Inventors: Yu, Lei, Wang, Yangbin, Liu, Xiaozhi
• Application Number:
  US14/722,694
• Publication Number:
  US 20150254343A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content

Abstract

A method and system of identifying and matching content characteristics comprises the steps of ingesting VDNA (Video DNA) fingerprints from input media contents, quick hash-based query across the VDNA registered indexer servers, and performing multi-dimensional content identification in query engines to obtain best matched results of the input media content.

Claims

1. A method of progressive strategies for index search to enhance quality of master VDNA (video DNA) fingerprint candidates before proceeding in query engines and increase overall matching probabilities, said method comprising:

a) multi-layered sifting on candidates,

b) extended preprocessing of sample contents based on predefined or adaptive transformation pattern set before index searching,

c) extended preprocessing of master contents before indexing,

d) recursive index search,

e) heuristic index-key screening,

f) adaptive splitting on video sample contents, and

g) master fingerprint manipulation.

2. The method as recited in claim 1, wherein said progressive strategies include applying said multi-layered sifting on said candidates generated from said index search, so as to improve quality of said candidates.

3. The method as recited in claim 2, wherein said multi-layers consist of various categories of information obtained or inferred from samples and their metadata, including title and release date.

4. The method as recited in claim 3, wherein said categories of information is rated by machine learning and various granted weight value, wherein, if video uploader of said sample content has been tagged as an uploader who owns a certain amount of infringing contents by automatic data learning based on previous query results, said weight value of metadata information will increase in subsequent processes of content sifting, index search as well as query.

5. The method as recited in claim 1, wherein said progressive strategies include extracting multiple instances of fingerprints from same sample content or fragments of said sample content based on a set of predefined parameters, and applying index search on each one from said fingerprints, combining and generating a list of candidates with broader coverage so as to resolve situation where master content contains subset of data from said sample content.

6. The method as recited in claim 5, wherein said fragments of a sample content refer to various shapes of areas sliced from an image sample or an image frame from a video sample clip, using said predefined parameters.

7. The method as recited in claim 5, wherein said predefined parameters consist of shape, size, density, rotation and scale of said image or sliced image fragment.

8. The method as recited in claim 5, wherein said predefined parameters are a pattern set containing manually defined transformation patterns, wherein images transformed after applying said pattern set can produce better quality of query candidates in said index search.

9. The method as recited in claim 8, wherein said pattern set is adaptively generated by analyzing feedback of short-term or long-term query results, wherein, by learning output of image query, if image frames of a sample video transformed by certain pattern are proven to improve query success rate, such pattern is added in said pattern set, so as to improve quality and performance of said index search on related images.

10. The method as recited in claim 5, wherein applying all said predefined parameters in said pattern set on all image frames extracted from a video is a time and resource consuming process, wherein both sets of said predefined parameters and said image frames is reduced by performing transformation and query on random sampling of said image frames from said video so as to learn the most effective subset of said predefined parameters and said image frames to proceed.

11. The method as recited in claim 1, wherein said progressive strategies include extracting multiple instances of fingerprints from same master content or fragments of said master content based on a set of said predefined parameters, and using said fingerprints as master fingerprints in said index search to increase probability of matches so as to resolve situation where said sample content contains subset of data from said master content.

12. The method as recited in claim 1, wherein said progressive strategies include recursive index search wherein matching probability will be increased by achieving broader coverage of said candidates combined from results of said recursive index search.

13. The method as recited in claim 12, wherein said recursive index search is performed using an entry sample or a list of result candidates from previous round of said index search as sample input(s) for next round of said index search, and said index search will terminate if a predefined threshold is reached.

14. The method as recited in claim 1, wherein said progressive strategies include heuristic index-key screening which is performed by learning and analyzing distribution of index-keys in master index to determine weight of index-keys, and top ranked index-keys are prioritized to be applied in said index search.

15. The method as recited in claim 14, wherein said weight of index-keys is defined by popularity of said index-keys, wherein the more frequent said index-key is discovered to be appeared in said master fingerprints, the lower weight said index-key is valued, and the more unique said index-key is tagged, the higher rank it is.

16. The method as recited in claim 1, wherein said progressive strategies include adaptively splitting video sample into timely equal, various or overlapping clips, applying said index search on each of said clips and combining all search results into a candidate list with broader coverage to increase probabilities of matches so as to resolve situation where said video sample is concatenated by several master videos.

17. The method as recited in claim 1, wherein said progressive strategies include master fingerprint manipulation on using new fingerprints output from manipulating existing said master fingerprints based on predefined parameters if said master content is unavailable or regenerating fingerprint is difficult, to increase said matching probabilities, especially for those sample images or videos which have been altered.

18. The method as recited in claim 17, wherein said predefined parameters consist of shape, size, density, rotation and scale of fingerprint.

19. The method as recited in claim 17, wherein said fingerprint manipulation is feasible due to VDNA fingerprint extraction rules, and because VDNA fingerprints are representation of characteristics of source image, VDNA fingerprint that is manipulated after applying certain predefined parameter is considered similar to the original VDNA fingerprint.

20. The method as recited in claim 17, after transforming said fingerprint manipulation on said VDNA fingerprint with certain predefined parameters, bit interpolation is required due to data loss in certain kinds of transformations.