Method and apparatus for automatically creating database for use in automated media recognition system

• US 7,359,889 B2
• Filed: 03/01/2002
• Issued: 04/15/2008
• Est. Priority Date: 03/02/2001
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Landmark Digital Services LLC
• Inventors: Inghelbrecht, Philip, Mukherjee, Dheeraj Shankar, Barton, Christopher Jacques Penrose, Wang, Avery Li-Chun
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US10/087,204
• Publication Number:
  US 20020161741A1
• Time in Patent Office: 2,237 Days
• Field of Search: 707/3, 707/10, 707/104.1, 707/2, 707/205, 707/1, 709/231, 709/219, 704/270, 715/763
• US Class Current: 707/661
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A method for automatically creating a database of parameters (e.g., fingerprints/landmarks) used in a media sample recognition system. The user places a media sample in his media player, e.g., a CD or DVD player in his personal computer. A software process installed on the user'"'"'s personal computer, to which is connected the CD/DVD player, extracts the constellation or “raw parameters” from the CD tracks as the CD plays, or under control of the software process. This extraction process may operate in the background. These raw parameters are then transmitted via the Internet (or other computer network) to a server coupled to a recognition database. The raw parameters are stored in the database. At some point in time, the raw parameters are processed into fingerprint/landmarks (e.g., using linkage) using another software process and then stored either back in the database or in random access memory (RAM) for use during a subsequent recognition process. The media sample can also be simultaneously identified during playback, which identity information is then sent to the server coupled to the recognition database.

Claims

1. A method for creating a database comprising fingerprint and landmark pairs associated with content samples, the method comprising the steps of:

extracting a plurality of characteristics from a content sample;

transferring said plurality of characteristics to a server coupled with a database;

deriving fingerprint and landmark pairs from said characteristics, wherein landmarks from said fingerprint and landmark pairs are reproducible selected points in a segment of the content, and wherein fingerprints are values calculable from said characteristics of said content sample at their associated landmarks; and

storing said fingerprint and landmark pairs in said database.

2. The method of claim 1 further comprising the step of:

simultaneously transferring metadata used to identify said content sample when transferring said plurality of characteristics.

3. The method of claim 2 further comprising the step of:

associating said metadata used to identify said content sample with said stored fingerprint and landmark pairs.

4. The method of claim 1 wherein said processing is executed by an updateable or replaceable algorithm.

5. The method of claim 1 further comprising the step of:

prior to extracting said characteristics, determining whether or not said database already contains landmark and fingerprint pairs associated with said content sample.

6. The method of claim 1 where said content comprises content stored on a compact disk or digital video disk.

7. The method of claim 1 where said content comprises a streaming content.

8. The method of claim 1 where said content comprises a file stored on a user'"'"'s computer.

9. The method of claim 1 wherein said processing said characteristics into fingerprint and landmark pairs comprises the steps of:

compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated; and

calculating one or more fingerprints at one or more said distinctive and reproducible points in time.

10. The method of claim 1 wherein said characteristics are a member of the group comprising:

a frequency value of the strongest spectral peak in proximity to a landmark,

a plurality of frequency values of the strongest spectral peaks in proximity to a landmark,

LPC coefficients,

Cepstral Coefficients,

a single value that is a hashed function of multiple characteristics.

11. The method of claim 9 wherein said compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated comprises calculating the instantaneous power at said points in rime and selecting a power maxima within those points in time.

12. The method of claim 9 wherein said compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated comprises calculating the absolute value of one or more spectral components at said points in time and finding the local maxima of said absolute value.

13. The method of claim 1 wherein said extracting is accomplished using a data archiving service.

14. The method of claim 1 wherein said extracted characteristics are associated with metadata from a content file.

15. The method of claim 14 wherein

said extracted characteristics are used to look up said metadata in a metadata database.

16. The method of claim 1 further comprising:

creating an index of fingerprint and landmark pairs associated with said content samples and sorting said index according to fingerprint.

17. An apparatus comprising a database of fingerprint and landmark pairs associated with content samples, said database comprising:

a plurality of stored fingerprint and landmark pairs, where said fingerprint and landmark pairs are constructed by;

extracting a plurality of characteristics from a content sample;

transferring said plurality of characteristics to a server coupled with a database; and

deriving fingerprint and landmark pairs from said characteristics, wherein landmarks from said fingerprint and landmark pairs are reproducible selected points in a segment of the content, and wherein fingerprints are values calculable from said characteristics of said content sample at their associated landmarks.

18. The apparatus of claim 17 where said database further comprises:

metadata used to identify said content sample associated with said stored fingerprint and landmark pairs.

19. The apparatus of claim 17 wherein said processing is executed by an updateable or replaceable algorithm.

20. The apparatus of claim 17 where said fingerprint and landmark pairs are further constructed by:

prior to extracting said characteristics, determining whether or not said database already contains landmark and fingerprint pairs associated with said content sample.

21. The apparatus of claim 17 where said content comprises content stored on a compact disk or digital video disk.

22. The apparatus of claim 17 where said content comprises a streaming content.

23. The apparatus of claim 17 where said content comprises a tile stored on a user'"'"'s computer.

24. The apparatus of claim 17 wherein said processing said characteristics into fingerprint and landmark pairs comprises the steps of:

compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated; and

calculating one or more fingerprint at one or more said distinctive and reproducible points in time.

25. The apparatus of claim 17 wherein said characteristics are a member of the group comprising:

a frequency value of the strongest spectral peak in proximity to a landmark,

a plurality of frequency values of the strongest spectral peaks in proximity to a landmark,

LPC Coefficients,

Cepstral Coefficients,

a single value that is a hashed function of multiple characteristics.

26. The apparatus of claim 24 wherein said compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated comprises calculating the instantaneous power at said points in time and selecting a power maxima within those points in time.

27. The apparatus of claim 24 wherein said compiling a list of distinctive and reproducible points in time at which fingerprints should be calculated comprises calculating the absolute value of one or more spectral components at said points in time and finding the local maxima of said absolute value.

28. The apparatus of claim 17 wherein said extracting is accomplished using a data archiving service.

29. The apparatus of claim 17 wherein said extracted characteristics are associated with metadata from a content file.

30. The apparatus of claim 29 wherein

said extracted characteristics are used to look up said metadata in a metadata database.

31. The apparatus of claim 17 where said database further comprises:

an index of fingerprint and landmark pairs associated with said content samples, said index sorted according to fingerprint.

32. A method of creating a database of fingerprint and landmark pairs associated with an music sample comprising the steps of:

playing a music sample to be identified;

determining that the music sample is missing from said database;

extracting one or more of characteristics from the music sample, where said characteristic, are common to distinctive and reproducible points in time within said music sample;

transferring said plurality of characteristics to a server coupled with a database;

deriving fingerprint and landmark pairs from said characteristics using an updateable or replaceable algorithm, wherein landmarks from said fingerprint and landmark pairs are reproducible selected points in a segment of the content, and wherein fingerprints are values calculable from said characteristics of said content sample at their associated landmarks;

storing said fingerprint and landmark pairs in said database; and

indexing said fingerprint and landmark pairs associated with said music samples, according to fingerprint.

33. The method of claim 32 further comprising, when fingerprint and landmark pairs associated with said music sample are already contained in the database;

determining whether or not said characteristics were processed with a most recent version of said updateable or replaceable algorithm;

updating said version of said updateable or replaceable algorithm, when it is determined that said characteristics were processed with an old version, and processing said characteristics with said most recent version of said algorithm.