Generating and matching hashes of multimedia content

• US 20020178410A1
• Filed: 02/11/2002
• Published: 11/28/2002
• Est. Priority Date: 02/12/2001
• Status: Active Grant

• Current Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: Oostveen, Job Cornelis, Baggen, Constant Paul Marie Jozef, Haitsma, Jaap Andre, Kalker, Antonius Adrianus Cornelis Maria
• Granted Patent: US 7,549,052 B2
• Time in Patent Office: Days
• US Class Current: 714/709
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06T 1/0028 Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
• CPC Class Codes: G06T 1/005 Robust watermarking, e.g. average attack or collusion attack resistant
• CPC Class Codes: G06T 2201/0051 Embedding of the watermark in the spatial domain
• CPC Class Codes: G06T 2201/0061 Embedding of the watermark in each block of the image, e.g. segmented watermarking
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 2210/295 Spatial effects, musical uses of multiple audio channels, e.g. stereo Helmholtz resonance effects G10H2210/275; reverberation or echo G10H2210/281
• CPC Class Codes: G10H 2240/061 MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking

Abstract

Hashes are short summaries or signatures of data files which can be used to identify the file. Hashing multimedia content (audio, video, images) is difficult because the hash of original content and processed (e.g. compressed) content may differ significantly.

The disclosed method generates robust hashes for multimedia content, for example, audio clips. The audio clip is divided (12) into successive (preferably overlapping) frames. For each frame, the frequency spectrum is divided (15) into bands. A robust property of each band (e.g. energy) is computed (16) and represented (17) by a respective hash bit. An audio clip is thus represented by a concatenation of binary hash words, one for each frame. To identify a possibly compressed audio signal, a block of hash words derived therefrom is matched by a computer (20) with a large database (21). Such matching strategies are also disclosed. In an advantageous embodiment, the extraction process also provides information (19) as to which of the hash bits are the least reliable. Flipping these bits considerably improves the speed and performance of the matching process.

Claims

1. A method of generating a hash signal identifying an information signal, the method comprising the steps of:

dividing the information signal into frames,

computing a hash word for each frame, and

concatenating successive hash words to constitute the hash signal.

2. The method of claim 1, wherein said computing step comprises the steps of:

dividing each frame of the information signal into disjoint bands or blocks;

calculating a property of the signal in each of said bands or blocks;

comparing the properties in the bands or blocks with respective thresholds;

representing the results of said comparisons by respective bits of the hash word.

3. The method as claimed in claim 2, wherein the property of a neighboring band or block constitutes said threshold.

4. The method as claimed in claim 2, wherein the property of a corresponding band or block in a previous frame constitutes said threshold.

5. The method as claimed in claim 2, wherein the bands or blocks are frequency bands of the frequency spectrum of the respective frame of the information signal.

6. The method as claimed in claim 5, wherein the frequency bands have an increasing bandwidth as a function of the frequency.

7. The method as claimed in claim 5, wherein said property is the energy of a frequency band.

8. The method as claimed in claim 5, wherein said property is the tonality of a frequency band.

9. The method of claim 1, wherein said information signal is divided into overlapping frames.

10. The method as claimed in claim 2, wherein the information signal is a video signal, the frames of which are divided into blocks, the mean luminance of a block constituting the property of said block.

11. The method of claim 2, further comprising the step of using the inputs of said comparing steps to generate information which is indicative of the reliability of the bits of the hash word.

13. An arrangement for generating a hash signal identifying an information signal in accordance with the method as claimed in any one of claims 1 to 12.

15. The method of claim 14, wherein the further selected hash word is another hash word of the input block of hash words.

16. The method of claim 14, wherein the further selected hash word is obtained by reversing a bit of the previously selected hash word.

17. The method of claim 16, further comprising the steps of receiving information which is indicative of the reliability of the bits of the selected hash word, and using said information to determine the bit to be reversed.

20. The method as claimed in claim 19, further comprising the steps of repeating steps (b)-(f) for other hash values of the unidentified information signal until a series of stored hash values is found for which the bit error rate is minimal and sufficiently small.

12. A method of generating a hash signal to identify an information signal, comprising the steps of:

dividing the information signal into blocks;

extracting for each block a feature of the information signal within said block;

comparing the value of the extracted feature with a threshold;

generating for each block a hash bit indicating whether the value of the extracted feature is larger or smaller than said threshold;

determining for each block reliability information indicating whether the value of the extracted feature differs substantially from said threshold;

combining said hash bits and said reliability information of the blocks into a hash value having reliable hash bits for which the extracted feature differs substantially from said threshold, and unreliable bits for which the extracted feature does not differ substantially from said threshold.

14. A method of matching an input block of hash words representing at least a part of an information signal with hash signals identifying respective information signals stored in a database, the method comprising the steps of:

(a) selecting a hash word of said input block of hash words;

(b) searching said hash word in the database;

(c) calculating a difference between the input block of hash words and a stored block of hash words in which the hash word found in step (b) has the same position as the selected hash word in the input block;

(d) repeating steps (a) to (c) for a further selected hash word until said difference is lower than a predetermined threshold.

21. An arrangement for matching an input block of hash words representing at least a part of an information signal with hash signals identifying respective information signals stored in a database in accordance with the method as claimed in any one of claims 14 to 20.

18. A method of matching a hash value representing an unidentified information signal with a plurality of hash values stored in a database and identifying a respective plurality of information signals, the method comprising the steps of:

(a) receiving said hash value in the form of a plurality of reliable hash bits and unreliable hash bits;

(b) searching in the database the stored hash values for which holds that the reliable bits of the applied hash value match the corresponding bits of the stored hash value;

(c) for each stored hash value found in step (b), calculating the bit error rate between the reliable bits of the hash value representing the unidentified information signal and the corresponding bits of the stored hash value; and

(d) determining for which stored hash values the bit error rate is minimal and sufficiently small.

19. A method of matching a hash signal representing an unidentified information signal with a plurality of hash signals stored in a database and identifying a respective plurality of information signals, the method comprising the steps of:

(a) receiving said hash signal in the form of a series of hash values, each hash value having reliable hash bits and unreliable hash bits;

(b) applying one of the hash values of said series to the database;

(c) searching in the database the stored hash values for which holds that the reliable bits of the applied hash value match the corresponding bits of the stored hash value;

(d) for each stored hash value found in step (c);

selecting in the database the corresponding series of stored hash values;

calculating the bit error rate between the reliable bits of the series of hash values representing the unidentified information signal and the corresponding bits of the selected series of hash values in the database; and

(f) determining for which series of stored hash values the bit error rate is minimal and sufficiently small.

22. A method of redirecting a receiver of an information signal to an Internet website, the method comprising the steps of deriving a hash signal from said information signal, and matching said hash signal with hash signals identifying Internet websites stored in a database.

25. The method of claim 24, wherein said steps of receiving and/or recording the multimedia signal, deriving and sending the hash signal, and receiving the identifier are performed by a mobile telephone device.

23. A method of measuring the quality of an information signal, the method comprising the steps of deriving a hash signal from said information signal, matching said hash signal with a hash signal identifying said information signal stored in a database, and calculating the difference between the derived hash signal and the stored hash signal.

24. A method of identifying a multimedia signal, the method comprising the steps of receiving and/or recording at least a part of said multimedia signal, deriving a hash signal from said multimedia signal, sending said hash signal to a database for matching it with hash signals stored in said database, and receiving from said database an identifier of the multimedia signal.