Multiple step identification of recordings

• US 20030028796A1
• Filed: 07/31/2002
• Published: 02/06/2003
• Est. Priority Date: 07/31/2001
• Status: Abandoned Application

• Current Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: White, Stephen, Roberts, Dale T., Hyman, David
• Application Number:
  US10/208,189
• Publication Number:
  US 20030028796A1
• Time in Patent Office: Days
• US Class Current: 713/193
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data

Abstract

Multiple information is extracted from an unknown recording and information associated therewith. Associated information includes the filename, if the recording is a computer file in, e.g., MP3 format, or table of contents (TOC) data, if the recording is on a removable medium, such as a compact disc. At least one and preferably several algorithmically determined fingerprints are extracted from the recording using one or more fingerprint extraction methods. The information extracted is compared with corresponding information in a database maintained for reference recordings. Identification starts with the most accurate and efficient method available, e.g., using a hash ID, a unique ID or text. Fingerprint matching is used to confirm other matches and validation is performed by comparing the duration of the unknown and a possibly matching reference recording.

Claims

1. A method of identifying recordings, comprising:

extracting information about an unknown recording stored in media possessed by a user and at least one algorithmically determined fingerprint from at least one portion of the unknown recording;

determining a possible identification of the unknown recording using at least one piece of the information extracted from the unknown recording and an identification database of corresponding information for reference recordings; and

identifying the unknown recording when the possible identification based on each of the at least one piece of the information in combination with the at least one algorithmically determined fingerprint identifies a single reference recording with respective confidence levels.

2. A method as recited in claim 1,

wherein the identification database is maintained by a provider of identification services, and

wherein said determining uses a unique identifier from the provider of identification services when the unique identifier is associated with the unknown recording and otherwise uses text associated with the unknown recording when text is associated with the unknown recording.

3. A method as recited in claim 2, further comprising validating said identifying by comparing an extracted length of the unknown recording with a stored length of the single reference recording.

4. A method as recited in claim 3, wherein the text associated with the unknown recording includes a filename of the recording.

5. A method as recited in claim 3, wherein the text associated with the unknown recording includes an ID3 tag for the recording.

6. A method as recited in claim 1, wherein the at least one algorithmically determined fingerprint is extracted from at least one of audio and video information in the at least one portion of the unknown recording.

7. A method as recited in claim 1,

wherein the at least one algorithmically determined fingerprint includes at least two fingerprints, and

wherein said identifying requires each of the at least two fingerprints to identify the single reference recording with respective confidence levels.

8. A method as recited in claim 1, further comprising validating said identifying by comparing an extracted length of the unknown recording with a stored length of the single reference recording.

9. A method as recited in claim 8, further comprising:

repeating said extracting, determining and identifying for a plurality unknown recordings from a plurality users;

monitoring unsuccessful identifications of the unknown recordings; and

detecting a possible error in the identification database from a pattern of errors.

10. A method as recited in claim 9, wherein said monitoring includes receiving information from the users indicating that said identifying was incorrect.

11. A method as recited in claim 9,

wherein said monitoring includes storing the at least one algorithmically determined fingerprint and an identifier of the single reference recording when said validating is not successful, and

wherein said method further comprises indicating the possible error in the identification database when substantially different fingerprints are stored for a single identifier.

12. A method as recited in claim 9, wherein said method further comprises indicating the possible error when the at least one algorithmically determined fingerprint matches one of the reference recordings, but the unknown recording is associated with ID3 Tag information different from that of the one of the reference recordings.

13. A method as recited in claim 9, further comprising correcting the possible error based on the information extracted from the unknown recordings.

14. A method as recited in claim 8, further comprising indicating a possible error in the identification database when the at least one algorithmically determined fingerprint is substantially similar to one of the reference recordings, but substantially different information is extracted from the unknown recording.

15. A method as recited in claim 1, further comprising delivering related data for the unknown recording from a supplemental database to supplement data embedded within the unknown recording for display and user manipulation.

16. A method of identifying recordings, comprising:

extracting fingerprints from at least one portion of an unknown recording using a plurality of algorithms;

determining a possible identification of the unknown recording using at least two of the fingerprints extracted from the unknown recording and at least one database of correspondingly generated fingerprints for reference recordings; and

identifying the unknown recording when the possible identification based on each of the fingerprints identifies a single reference recording with respective confidence levels.

17. A method as recited in claim 16, wherein each fingerprint is extracted from at least one of audio and video information in the at least one portion of the unknown recording.

18. A method as recited in claim 16, further comprising validating said identifying by comparing a length of the unknown recording with a stored length of the single reference recording.

19. A method as recited in claim 18,

wherein said extracting is performed by client equipment possessed by a user,

wherein said determining, identifying and validating are performed by at least one server under control of a provider of identification services, and

wherein said method further comprises:

transmitting a unique identifier associated with the single reference recording from the at least one server to the client equipment after said validating is successful; and

associating the unique identifier with the unknown recording in the client equipment.

20. A method as recited in claim 19, further comprising:

comparing the unique identifier with a permission list of stored identifiers in the at least one database;

indicating that the recording may be shared if there is a match for the unique identifier in the permission list.

21. A method as recited in claim 19, further comprising:

comparing the unique identifier with a block list of stored identifiers in the at least one database;

indicating that the recording may not be shared if there is a match for the unique identifier in the block list.

22. A method of obtaining reference information stored in a database used to identify unknown recordings, comprising:

obtaining non-waveform data associated with a recording possessed by a user of the database for identification of recordings possessed by the user;

extracting at least one fingerprint from at least one portion of the recording; and

storing the at least one fingerprint as identifying information for the recording, when a match is found in the database for the non-waveform data.

23. A method as recited in claim 22, wherein the non-waveform data indicates the length of the recording.

24. A method as recited in claim 23, wherein the recording is permanently stored on a removable medium and the non-waveform data is derived from table of contents data for the recording.

25. A method as recited in claim 22, wherein the non-waveform data includes text associated with the recording.

26. A method as recited in claim 25, wherein the non-waveform data includes and ID3 tag.

27. A method as recited in claim 26,

wherein the ID3 tag includes encoded information,

wherein the database is maintained by a provider of identification services and the encoded information is generated under control of the provider of identification services, and

wherein said method further comprises validating the non-waveform data by decoding the encoded information prior to said storing of the identifying information in the database.

28. A method as recited in claim 25, wherein the non-waveform data includes a watermark.

29. A method as recited in claim 25, wherein the non-waveform data includes media information regarding source media type.

30. A method as recited in claim 29, wherein the media information identifies the source media type as CD-R.

31. A method as recited in claim 29, wherein the media information identifies the source media type as CD-DA.

32. A method as recited in claim 29, wherein the media information identifies the source media type as a digital file.

33. A method as recited in claim 29, wherein the media information identifies the source media type as a digital versatile disc.

34. A method as recited in claim 25, wherein the text includes a filename of the recording.

35. A method as recited in claim 25, wherein the text includes a title of the recording.

36. A method as recited in claim 25, wherein the text includes an artist name of a participant in creation of the recording.

37. A method as recited in claim 25, wherein the text includes an album name associated with the recording.

38. A method as recited in claim 22, wherein said obtaining and extracting are performed by client equipment possessed by a plurality users for different copies of the recording and different users extract different fingerprints from the recording.

39. A method as recited in claim 38, further comprising:

maintaining the database on at least one server under control of a provider of identification services, and

transmitting from the at least one server to the client equipment, extraction instructions on which of the different fingerprints each of the client equipment extracts.

40. A method as recited in claim 39, further comprising:

transmitting the non-waveform data from the client equipment to the at least one server; and

selecting the extraction instructions by the at least one server for said transmitting to the client equipment based on the non-waveform data.

41. A method as recited in claim 40, further comprising updating the extraction instructions based at least in part on frequency of receipt of the non-waveform data for the recording.

42. A method as recited in claim 40, wherein said selecting of the extraction instructions is based at least in part on type of the client equipment receiving the extraction instructions.

43. A method as recited in claim 40, wherein said selecting of the extraction instructions is based at least in part on geographical location of the client equipment receiving the extraction instructions.

44. A method as recited in claim 40, wherein said selecting of the extraction instructions is based at least in part on software operating on the client equipment receiving the extraction instructions.

45. A method as recited in claim 40, further comprising updating the extraction instructions based at least in part on number of users who have supplied the identifying information.

46. A method as recited in claim 40, wherein said selecting of the extraction instructions is based at least in part on quality of the copies of the recording.

47. A method as recited in claim 40, further comprising transmitting the at least one fingerprint from the client equipment to the at least one server at a time specified by the extraction instructions.

48. A method as recited in claim 47, further comprising storing the at least one fingerprint at the client equipment until a specified number of fingerprints are ready for said transmitting.

49. A method as recited in claim 47, wherein said transmitting of the at least one fingerprint occurs when a communication channel with the at least server is available.

50. A method as recited in claim 47, wherein said transmitting of the at least one fingerprint for a first recording accessed by a piece of client equipment occurs with said transmitting of the non-waveform data for a second recording accessed by the piece of client equipment.

51. A method as recited in claim 47,

wherein the recording is permanently stored on a removable medium and the client equipment generates at least one encoded file from the recording, and

wherein said transmitting transmits the at least one fingerprint before encoding the recording.

52. A method as recited in claim 47,

wherein the recording is permanently stored on a removable medium and the client equipment generates at least one encoded file from the recording, and

wherein said transmitting transmits the at least one fingerprint after encoding one track of the removable medium.

53. A method as recited in claim 47,

wherein the recording is permanently stored on a removable medium and the client equipment generates at least one encoded file from the recording, and

wherein said transmitting transmits the at least one fingerprint after receiving an indication that encoding of the removable medium has been completed.

54. A method as recited in claim 22, wherein the database includes the identifying information for musical recordings.

55. A method as recited in claim 22, wherein the database includes the identifying information for video recordings.

56. A method as recited in claim 22, further comprising:

detecting a quality of the at least one fingerprint;

identifying another copy of the recording using the at least one fingerprint; and

replacing the at least one fingerprint with a higher quality fingerprint when the other copy of the recording produces the higher quality fingerprint.

57. A method as recited in claim 56, wherein said detecting of the quality is based on an encoding technique used for the recording.

58. A method as recited in claim 56, wherein said detecting of the quality is based on a media type used to store the recording.

59. A method as recited in claim 56, wherein said detecting of the quality is based on error correction capability of user equipment accessing the recording.

60. A method as recited in claim 59, wherein said detecting of the quality assigns higher quality when hardware error correction is used than when software error correction is by the user equipment.

61. A method as recited in claim 56, wherein said detecting of the quality is based on number of errors detected during said extracting of the fingerprint.

62. A method as recited in claim 22,

wherein said obtaining and extracting are performed by client equipment possessed by a plurality users for different copies of the recording, and

wherein said method further comprises:

comparing the at least one fingerprint obtained from one of the users with the at least one fingerprint extracted from at least one other user; and

updating the at least one fingerprint in the database based on said comparing.

63. A method as recited in claim 62, wherein said updating is performed after said comparing determines that fingerprints from different users have a predetermined correlation.

64. A method as recited in claim 62, wherein said updating combines fingerprints from different users for storage in the database.

65. A system for identifying recordings, comprising:

an extraction unit to extract information about an unknown recording stored in media possessed by a user and at least one algorithmically determined fingerprint from at least one portion of the unknown recording; and

an identification unit, coupled to said extraction unit, to make a possible identification of the unknown recording using at least one piece of the information extracted from the unknown recording and an identification database of corresponding information for reference recordings, and to identify the unknown recording when the possible identification based on each of the at least one piece of the information in combination with the at least one algorithmically determined fingerprint identifies a single reference recording with respective confidence levels.

66. A system for identifying recordings, comprising:

an extraction unit to extract fingerprints from at least one portion of an unknown recording using a plurality of algorithms; and

an identification unit, coupled to said extraction unit, to make a possible identification of the unknown recording using at least two of the fingerprints extracted from the unknown recording and at least one database of correspondingly generated fingerprints for reference recordings, and to identify the unknown recording when the possible identification based on each of the fingerprints identifies a single reference recording with respective confidence levels.

67. A system for obtaining reference information stored in a database used to identify unknown recordings, comprising:

a receiving unit to obtain non-waveform data associated with a recording possessed by a user of the database for identification of recordings possessed by the user;

an extraction unit to extract at least one fingerprint from at least one portion of the recording; and

a storage unit, coupled to said receiving unit and said extraction unit, to store the at least one fingerprint as identifying information for the recording, when a match is found in the database for the non-waveform data.