Context based video finder

• US 8,346,801 B2
• Filed: 03/04/2010
• Issued: 01/01/2013
• Est. Priority Date: 03/24/2009
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hagg, Wilhelm, Eggink, Jana, Kemp, Thomas, Giron, Franck, Garcia, Javier Alonso
• Primary Examiner(s): GORTAYO, DANGELINO N
• Application Number:
  US12/717,415
• Publication Number:
  US 20100250585A1
• Time in Patent Office: 1,034 Days
• Field of Search: 707/1, 707/100, 707/104, 707/769, 707/772, 707/766, 707/804, 707/913, 707/916, 707/728, 707/732, 707/802
• US Class Current: 707/772
• CPC Class Codes: G06F 16/70 of video data

Abstract

Embodiments of the invention relate to a method for video retrieval by providing a first audio video file, determining a first identifier of a first piece of music in the first audio video file, looking up for the first identifier first meta data in a music database, in which said first meta data are associated with said first identifier, providing second meta data of a second piece of music from the music database. The second piece of music is included in a second audio video file. Further the method is conducted by determining a similarity measure by comparing the first and second meta data, and providing the second audio video file or an identifier thereof depending on the similarity measure. Further embodiments relate to a server, a user device and a system and a computer program product for video retrieval.

Claims

1. A method for video retrieval, comprising:

providing a first audio video file;

determining a first identifier of a first piece of music in the first audio video file, wherein the first piece of music can appear at any position within the first audio video file;

looking up, using said first identifier, first meta data in a music database, in which said first meta data are associated with said first identifier, wherein the first meta data are descriptive of an entire piece of music identified based on the first piece of music;

providing second meta data of a second piece of music from said music database, wherein the second piece of music is included in a second audio video file, wherein the second meta data are descriptive of an entire piece of music identified based on the second piece of music, and the second piece of music can appear at any position in the second audio video file;

determining, using a microprocessor, a similarity measure by comparing the first and second meta data; and

providing the second audio video file or an identifier thereof, when the similarity measure meets or exceeds a predetermined threshold.

2. The method of claim 1, wherein at least one of the first and second meta data are descriptive of at least one of a genre, a mood, a title and an artist.

3. The method of claim 1, wherein at least one of the first and second meta data are descriptive of at least one of beats per minute (BPM) and of a further low level descriptor.

4. The method of claim 1, further comprising:

looking up, using said first identifier, third meta data in the music database, in which said third meta data are associated with said first identifier,

wherein at least one of the third and second meta data are descriptive of collaborative filtering information; and

the similarity measure is determined by comparing the first and third meta data with the second meta data.

5. The method of claim 1, further comprising:

providing a confidence measure indicating a probability that the second piece of music is correctly identified in the second audio video file, wherein the similarity measure depends on the confidence measure.

6. The method of claim 1, wherein the similarity measure depends on a similarity of the user profile to a collaborative filtering information.

7. The method of claim 1, wherein at least one of the first and second meta data are descriptive of a first and a second temporal position of the first and second pieces of music in the first and second audio video file, respectively.

8. The method of claim 1, further comprising:

looking up, using said first identifier, third meta data in the music database, in which said third meta data are associated with said first identifier,

wherein at least one of the third and second meta data are descriptive of an temporal order of pieces of music included in the first and second audio video files, respectively; and

the similarity measure is determined by comparing the first and third meta data with the second meta data.

9. The method of claim 1, further comprising:

looking up, using said first identifier, third meta data in the music database, in which said third meta data are associated with said first identifier,

wherein at least one of the third and second meta data are descriptive of a first and second length of the first and second pieces of music in the first and second audio video file, respectively; and

the similarity measure is determined by comparing the first and third meta data with the second meta data.

10. The method of claim 1, wherein the second audio video file is included in or associated with a web page and the web page is provided to the user.

11. The method of claim 10, wherein the first audio video file is included in or associated with a further web page and the similarity measure is determined by comparing a design of the first and second web pages.

12. The method of claim 1, wherein at least one of said first and said second identifier is a fingerprint.

13. The method of claim 1, wherein said first meta data are determined independently from said first audio video file based on at least one of a music file and an audio video file, which is obtained independently from said audio video file provided by said user.

14. The method of claim 1, wherein the similarity measure is provided in a similarity database comprising a similarity matrix of pieces of music.

15. A server, comprising:

a communication unit configured to receive a first audio video file;

a storage configured to store a music database;

a data processor configured to determine a first identifier of a first piece of music in the first audio video file, wherein the first piece of music can appear at any position within the first audio video file, and to look up, using said first identifier, first meta data in the music database, in which said first meta data are associated with said first identifier, wherein the first meta data are descriptive of an entire piece of music identified based on the first piece of music, and to look up second meta data of a second piece of music from said music database, wherein the second piece of music is included in a second audio video file, wherein the second meta data are descriptive of an entire piece of music identified based on the second piece of music, and the second piece of music can appear at any position in the second audio video file, the data processor being further configured to determine a similarity measure by comparing the first and second meta data, and to provide, to the communication unit, the second audio video file or an identifier thereof when the similarity measure meets or exceeds a predetermined threshold,

wherein the communication unit is further configured to provide the second audio video file or the identifier thereof.

16. A method for video retrieval, comprising:

providing a first audio video file;

determining a first identifier of a first piece of music in the first audio video file, wherein the first piece of music can appear at any position within the first audio video file;

looking up, using said first identifier, first meta data in a music database, in which said first meta data are associated with said first identifier, wherein the first meta data are descriptive of an entire piece of music identified based on the first piece of music;

providing second meta data of a second piece of music from said music database, wherein the second piece of music is included in a second audio video file, wherein the second meta data are descriptive of an entire piece of music identified based on the second piece of music, and the second piece of music can appear at any position in the second audio video file, and wherein at least one of the first and second meta data are descriptive of a first and a second temporal position of the first and second pieces of music in the first and second audio video files, respectively, and/or a temporal position and length of a found music segment within said first or second piece of music;

determining, using a microprocessor, a similarity measure by comparing the first and second meta data; and

providing the second audio video file or an identifier thereof, when the similarity measure meets or exceeds a predetermined threshold.

17. A method for video retrieval, comprising:

providing a first audio video file;

determining a first identifier of a first piece of music in the first audio video file, wherein the first piece of music can appear at any position within the first audio video file;

looking up, using said first identifier, first meta data in a music database, in which said first meta data are associated with said first identifier, wherein the first meta data are descriptive of an entire piece of music identified based on the first piece of music;

providing second meta data of a second piece of music from said music database, wherein the second piece of music is included in a second audio video file, wherein the second meta data are descriptive of an entire piece of music identified based on the second piece of music, and the second piece of music can appear at any position in the second audio video file;

determining, using a microprocessor, a similarity measure by comparing the first and second meta data; and

providing the second audio video file or an identifier thereof, when the similarity measure meets or exceeds a predetermined threshold;

wherein at least one of the first and second meta data are descriptive of a temporal order of pieces of music included in the first and second audio video files, respectively.