MEDIA RIGHTS MANAGEMENT USING MELODY IDENTIFICATION

• US 20120123831A1
• Filed: 11/12/2010
• Published: 05/17/2012
• Est. Priority Date: 11/12/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, David G., Salem, George, Wang, Yiling Su, Wiseman, Matthew
• Granted Patent: US 8,584,197 B2
• Time in Patent Office: Days
• US Class Current: 705/14.7
• CPC Class Codes: A63H 5/00 Musical or noise- producing devices for additional toy effects other than acoustical musical tops A63H1/28; sound-producing dolls A63H3/28; musical toy vehicles A63H17/268; acoustical vehicle-signalling devices A63H17/32; whistling toy locomotive arrangements A63H19/14; music, sound or noise production in general G10
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06Q 30/0274 Split fees
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 50/184 Intellectual property management
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2240/141 Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
• CPC Class Codes: G10H 2240/181 Billing, i.e. purchasing of data contents for use with electrophonic musical instruments; Protocols therefor; Management of transmission or connection time therefor
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

A content recognition system operates in conjunction with a media hosting service to identify hosted media content and ownership rights associated with the hosted content. By applying melody recognition, the content recognition system can identify compositions embodied in hosted media content even when these compositions do not precisely match any known sound recording. Thus, the content recognition system is beneficially able to detect, for example, recorded cover performances and recorded live performances embodied in hosted media content. Once identified, ownership information is determined and the media hosting service can carry out appropriate rights management policies associated with the content such as monetizing or blocking the protected content.

Claims

1. A computer-implemented method for determining ownership rights to a composition, the method comprising:

receiving an input media file, the input media file comprising an audio clip;

generating an input melody fingerprint for the audio clip, the input melody fingerprint representing a melody of a first composition that is embodied in the audio clip, the melody fingerprint being invariant to parameters used to perform or record the performance of the composition;

querying a reference database for a reference melody fingerprint that matches the input melody fingerprint, each reference melody fingerprint in the reference database representing a melody of a composition that is embodied in a reference sound recording; and

if a matching reference melody fingerprint for the input melody fingerprint is found, determining ownership information for the first composition from ownership information that is associated with the matching reference melody fingerprint.

2. The method of claim 1, wherein the parameters used to perform or record the performance of the composition include a key in which the composition was performed, an instrument used to perform the composition, or instrumentation used to make the audio clip.

3. The method of claim 1, further comprising:

determining an ownership policy associated with the composition that is associated with the matching melody fingerprint.

4. The method of claim 3, further comprising providing the ownership policy to a media hosting service hosting the input media file.

5. The method of claim 3, further comprising blocking access to the input media file when the ownership policy indicates that at least one entity having an ownership interest in the composition wishes to block access to the composition.

6. The method of claim 3, further comprising monetizing the input media file when the ownership policy indicates that all entities having an ownership interest in the composition wish to monetize the composition.

7. The method of claim 6, wherein monetizing the input media file comprises:

receiving a request for the input media file from a user of the media hosting service;

selecting an advertisement for placement with the input media file;

providing the input media file to the user together with the selected advertisement; and

sharing the revenue generated from the advertisement with a content owner of the composition.

8. The method of claim 3, further comprising tracking the input media file when the ownership policy indicates that no entities having an ownership interest in the composition wish to block access to the composition and at least one entity having an ownership interest in the composition does not wish to monetize the composition.

9. The method of claim 1, wherein the input media file comprises a video clip, and wherein the audio clip comprises an audio track of the video clip.

10. A computer-implemented method for determining ownership rights to a sound recording, comprising:

receiving an input media file, the input media file comprising an audio clip;

generating an input audio fingerprint for the audio clip, the audio fingerprint representing a first sound recording of a first composition that is embodied in the audio clip;

querying a reference database for a reference audio fingerprint that matches the input audio fingerprint, each reference audio fingerprint in the reference database representing a sound recording that embodies a composition; and

if a matching reference audio fingerprint for the input audio fingerprint in the reference database is found, determining the ownership information for the first sound recording from ownership information that is associated with the matching reference audio fingerprint;

if a matching reference audio fingerprint for the input audio fingerprint is not found;

generating an input melody fingerprint for the audio clip, the input melody fingerprint representing a melody of the first composition, the melody fingerprint being invariant to parameters used to perform or record the performance of the first composition;

querying a reference database for a reference melody fingerprint that matches the input melody fingerprint, each reference melody fingerprint in the reference database representing a melody of a composition that is embodied in a reference sound recording; and

if a matching reference melody fingerprint for the input melody fingerprint is found, determining ownership information for the first composition from ownership information that is associated with the matching reference melody fingerprint.

11. The method of claim 10, wherein a matching reference audio fingerprint for the input audio fingerprint is found, further comprising determining ownership information for the first composition from ownership information that is associated with the matching reference audio fingerprint.

12. The method of claim 10, wherein the matching reference melody fingerprint is derived from a first recorded performance of the composition, and wherein the matching audio fingerprint is derived from a second recorded performance of the composition that is different from the first recorded performance.

13. A computer-implemented method for generating a searchable reference database of ownership rights for audio compositions, the method comprising:

receiving a reference sound recording embodying a composition and composition ownership information indicating one or more entities having ownership rights to the composition;

generating a melody fingerprint from the reference sound recording, the melody fingerprint representing a melody of the composition embodied in the reference sound recording in a manner that is invariant to parameters used to perform or record the performance of the composition; and

storing the melody fingerprint and the associated composition ownership information in the searchable reference database.

14. The method of claim 13, wherein the melody fingerprint is invariant to the key in which the composition is performed, the instrumentation used to make the reference sound recording, or the instrumentation used to record the reference sound recording.

15. The method of claim 13, further comprising:

generating an audio fingerprint representing the reference sound recording;

receiving reference sound recording ownership information indicating one or more entities having ownership rights to the reference sound recording; and

storing the audio fingerprint and the associated reference sound recording ownership information in the searchable reference database.

16. The method of claim 13, wherein the composition ownership information comprises an identifying song title and songwriter of the composition embodied in the reference sound recording.

17. The method of claim 13, wherein the composition ownership information further comprises an ownership policy indicating a method for enforcing ownership rights pertaining to user-uploaded media having a melody fingerprint matching the reference melody fingerprint.

18. A system for determining ownership rights to a composition, the system comprising:

a processor; and

a memory coupled to the processor, wherein the memory stores instructions operable to cause the processor to;

receive an input media file, the input media file comprising an audio clip;

generate an input melody fingerprint for the audio clip, the input melody fingerprint representing a melody of a first composition that is embodied in the audio clip, the melody fingerprint being invariant to parameters used to perform or record the performance of the composition;

query a reference database for a reference melody fingerprint that matches the input melody fingerprint, each reference melody fingerprint in the reference database representing a melody of a composition that is embodied in a reference sound recording; and

if a matching reference melody fingerprint for the input melody fingerprint is found, determine ownership information for the first composition from ownership information that is associated with the matching reference melody fingerprint.

19. The system of claim 18, wherein the parameters used to perform or record the performance of the composition include a key in which the composition was performed, an instrument used to perform the composition, or instrumentation used to make the audio clip.

20. The system of claim 18, wherein the memory further stores instructions operable to cause the processor to determine an ownership policy associated with the composition that is associated with the matching melody fingerprint.

21. The system of claim 20, wherein the memory further stores instructions operable to cause the processor to provide the ownership policy to a media hosting service hosting the input media file.

22. The system of claim 20, wherein the memory further stores instructions operable to cause the processor to block access to the input media file when the ownership policy indicates that at least one entity having an ownership interest in the composition wishes to block access to the composition.

23. The system of claim 20, wherein the memory further stores instructions operable to cause the processor to monetize the input media file when the ownership policy indicates that all entities having an ownership interest in the composition wish to monetize the composition.

24. The system of claim 23, wherein the instructions stored in the memory operable to cause the processor to monetize the input media file further comprises instructions stored in the memory operable to cause the processor to:

receive a request for the input media file from a user of the media hosting service;

select an advertisement for placement with the input media file;

provide the input media file to the user together with the selected advertisement; and

share of revenue generated from the advertisement with a content owner of the composition.

25. The system of claim 20, wherein the memory further stores instructions operable to cause the processor to track the input media file when the ownership policy indicates that no entities having an ownership interest in the composition wish to block access to the composition and at least one entity having an ownership interest in the composition does not wish to monetize the composition.

26. The system of claim 18, wherein the input media file comprises a video clip, and wherein the audio clip comprises an audio track of the video clip.

27. A system for determining ownership rights to a sound recording, the system comprising:

a processor; and

a memory coupled to the processor, wherein the memory stores instructions operable to cause the processor to;

receive an input media file, the input media file comprising an audio clip;

generating an input audio fingerprint for the audio clip, the audio fingerprint representing a first sound recording of a first composition that is embodied in the audio clip;

query a reference database for a reference audio fingerprint that matches the input audio fingerprint, each reference audio fingerprint in the reference database representing a sound recording that embodies a composition; and

if a matching reference audio fingerprint for the input audio fingerprint in the reference database is found, determine the ownership information for the first sound recording from ownership information that is associated with the matching reference audio fingerprint;

if a matching reference audio fingerprint for the input audio fingerprint is not found;

generate an input melody fingerprint for the audio clip, the input melody fingerprint representing a melody of the first composition, the melody fingerprint being invariant to parameters used to perform or record the performance of the first composition;

query a reference database for a reference melody fingerprint that matches the input melody fingerprint, each reference melody fingerprint in the reference database representing a melody of a composition that is embodied in a reference sound recording; and

if a matching reference melody fingerprint for the input melody fingerprint is found, determine ownership information for the first composition from ownership information that is associated with the matching reference melody fingerprint.

28. The system of claim 27, wherein a matching reference audio fingerprint for the input audio fingerprint is found, further comprising determining ownership information for the first composition from ownership information that is associated with the matching reference audio fingerprint.

29. The system of claim 27, wherein the matching reference melody fingerprint is derived from a first recorded performance of the composition, and wherein the matching audio fingerprint is derived from a second recorded performance of the composition that is different from the first recorded performance.

30. A system for generating a searchable reference database of ownership rights for audio compositions, the system comprising:

a processor; and

a memory coupled to the processor, wherein the memory stores instructions operable to cause the processor to;

receive a reference sound recording embodying a composition and composition ownership information indicating one or more entities having ownership rights to the composition;

generate a melody fingerprint from the reference sound recording, the melody fingerprint representing a melody of the composition embodied in the reference sound recording in a manner that is invariant to parameters used to perform or record the performance of the composition; and

store the melody fingerprint and the associated composition ownership information in the searchable reference database.

31. The system of claim 30, wherein the melody fingerprint is invariant to the key in which the composition is performed, the instrumentation used to make the reference sound recording, or the instrumentation used to record the reference sound recording.

32. The system of claim 30, wherein the memory further stores instructions operable to cause the processor to:

generate an audio fingerprint representing the reference sound recording;

receive reference sound recording ownership information indicating one or more entities having ownership rights to the reference sound recording; and

store the audio fingerprint and the associated reference sound recording ownership information in the searchable reference database.

33. The system of claim 30, wherein the composition ownership information comprises an identifying song title and songwriter of the composition embodied in the reference sound recording

34. The system of claim 30, wherein the composition ownership information further comprises an ownership policy indicating a method for enforcing ownership rights pertaining to user-uploaded media having a melody fingerprint matching the reference melody fingerprint.