Indexing media content library using audio track fingerprinting

• US 20190311746A1
• Filed: 04/08/2019
• Published: 10/10/2019
• Est. Priority Date: 04/06/2018
• Status: Abandoned Application

• Current Assignee: Deluxe 3D LLC (Deluxe Corp.), Deluxe Digital Distribution Inc. (Deluxe Corp.), Deluxe Digital Studios, Inc. (Deluxe Corp.), Deluxe Entertainment Services Inc., Deluxe Laboratories LLC, Deluxe Media Inc., Deluxe One, LLC, Softitler Net, Inc., Deluxe Creative Services Inc. (The Framestore Ltd.)
• Original Assignee: Deluxe One, LLC
• Inventors: Shenkler, Andrew
• Application Number:
  US16/378,539
• Publication Number:
  US 20190311746A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/32 on separate auxiliary tracks of the same or an auxiliary record carrier

Abstract

A computer-implemented video classification system includes a storage device configured to ingest and store video files having respective audio tracks. Processors in the system are configured with instructions to perform a perceptual hash algorithm on each of the audio tracks to generate one or more audio fingerprints for each audio track. The audio fingerprints are associated with the respective video files. The video files are indexed based upon the audio fingerprints. Audio tracks from unknown video files are hashed and compared to audio fingerprints of known video files in a repository to identify the unknown video files.

Claims

1. A computer-implemented video classification system comprising

a storage device configured to ingest and store one or more video files thereon, wherein the one or more video files comprise one or more respective audio tracks;

one or more processors configured with instructions to

perform a perceptual hash algorithm on each of the one or more audio tracks;

generate one or more audio fingerprints for the respective one or more audio tracks;

associate the one or more audio fingerprints with the respective one or more video files; and

index the one or more video files based upon the one or more respective audio fingerprints.

2. The computer-implemented video classification system of claim 1, wherein performing the perceptual hash algorithm and generating one or more audio fingerprints comprises

dividing the one or more audio tracks into a plurality of sampling segments along the entire length of each of the one or more audio tracks, wherein the plurality of sampling segments are generated at a sampling rate that is the same for each of the one or more audio tracks;

performing the perceptual hash algorithm on each of the plurality of sampling segments to produce a hash code for each of the plurality of sampling segments; and

stringing together the hash code of each of the plurality of sampling segments on an audio track for the one or more audio tracks to produce the one or more audio fingerprints.

3. The computer-implemented video classification system of claim 1, wherein the one or more processors are further configured with instructions to

compare two or more audio fingerprints to determine a match level between the two or more audio fingerprints;

determine whether the match level is within a matching threshold;

if the one or more processors determines the match level is within the matching threshold, identify the associated one or more video files as one or more new versions; and

if the one or more processors determines the match level is not within the matching threshold, identify the associated one or more video files as one or more new video files or copies.

4. The computer-implemented video classification system of claim 3, wherein the matching threshold is within a range of less than about 95% and greater than about 50%.

5. The computer-implemented video classification system of claim 3, wherein the one or more video files are identified as one or more new video files when the match level is equal to less than about 50%.

6. The computer-implemented video classification system of claim 3, wherein the one or more video files are identified as one or more copies when the match level is equal to greater than about 95%.

7. The computer-implemented video classification system of claim 3, wherein the one or more video files are identified as one or more new versions when the matching level is about 85%.

8. The computer-implemented video classification system of claim 1, wherein the one or more processors are further configured with instructions to store the one or more audio fingerprints associated with each of the one or more video files in a database on the storage device.

9. A method implemented in a computer system for identifying video files in a media library, wherein one or more processors in the computer system is particularly configured to perform a number of processing steps comprising

performing an audio perceptual hash of a plurality of audio tracks associated with a plurality of video files in a media library to produce a plurality of audio fingerprints;

associating each audio fingerprint with a respective video file in the media library;

creating a media library of identifiable video files based upon the respective audio fingerprint;

comparing the plurality of audio fingerprints to determine whether any of the plurality of audio fingerprints match within a threshold matching rate; and

if there is a match between two or more audio fingerprints within the threshold matching rate, identifying the respective video files associated with the two or more audio fingerprints as different versions of the same video file.

10. The method of claim 10, wherein the threshold match level is within a range of less than about 95% and greater than about 50%.

11. A non-transitory computer readable storage medium containing instructions for instantiating a special purpose computer to index an unknown video file in a media library of known video files, wherein the instructions implement a computer process comprising the steps of

ingesting and storing one or more known video files on a storage device within a computer system, wherein the one or more known video files are associated with one or more hashed audio tracks with one or more associated audio fingerprints;

receiving one or more unknown video files;

performing audio perceptual hashing of one or more audio tracks associated with the one or more unknown video files to produce one or more associated audio fingerprints of the one or more unknown video files;

comparing the one or more audio fingerprints of the one or more unknown video files to the one or more audio fingerprints of the one or more known video files; and

determining a match level for each of the one or more audio fingerprints of the one or more unknown video files to each of the one or more audio fingerprints of the one or more known video files; and

identifying and indexing the one or more unknown video files based upon the match level to each known video file.

12. The non-transitory computer readable storage medium of claim 12, wherein the identifying and indexing step further comprises determining whether the match level is less than about 50%, is within a threshold matching range of greater than about 50% and less than about 95%, or is greater than about 95%.

13. The non-transitory computer readable storage medium of claim 13, wherein

if the match level is 0%, determining that the one or more unknown video files are not associated with one or more known video files and identifying and indexing the one or more unknown video files as new video files;

if the match level is within the threshold matching range of greater than about 50% to less than about 95%, determining the one or more unknown video files are associated with one or more known video files and identifying and indexing the one or more unknown video files as new versions of the one or more known video files; and

if the match level is greater than about 95%, determining the one or more unknown video files are associated with the one or more known video files and identifying and indexing the one or more unknown video files as copies of the one or more known video files.