Method and system for determining content treatment

• US 9,179,200 B2
• Filed: 03/13/2008
• Issued: 11/03/2015
• Est. Priority Date: 03/14/2007
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Conwell, William Y.
• Primary Examiner(s): Reagan, James A
• Application Number:
  US12/048,072
• Publication Number:
  US 20080228733A1
• Time in Patent Office: 2,791 Days
• Field of Search: 705/59
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/26606 for generating or managing entitlement messages, e.g. Entitlement Control Message [ECM] or Entitlement Management Message [EMM] arrangements for conditional access to broadcast information or to broadcast-related services H04H60/14
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/8355 involving usage data, e.g. number of copies or viewings allowed
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/8405 represented by keywords

Abstract

Metadata determines treatment of content by automated systems, including “user generated content” web sites. The metadata may be conveyed with the content or may be determined by other techniques, including systems based on digital watermarking or content fingerprinting. In some arrangements, treatment depends on the temporal length of a content excerpt that matches a pre-existing work. In others, treatment depends on popularity—either of the content, or a pre-existing work from which it was derived. A great variety of other factors and contexts can also be considered. Automated tools to preliminarily identify possible “fair use” can be realized; further determination may be made by human evaluators (including crowd-source approaches).

Claims

1. A method comprising, for each of plural items of audio-visual content uploaded to a web site that distributes user-generated content for possible distribution of the uploaded content therefrom, each of said plural items of audio-visual content including both audio and video:

at a time such audio-visual content is ingested by the web site, conducting a fingerprint-based analysis of the audio of the uploaded content item, to identify one or more portions of said content item that have been derived from one or more pre-existing works, said analysis being performed by a software-configured processor, said analysis including transforming plural partially overlapping temporal excerpts of the audio into spectral band data to determine, for each such excerpt, relative energy within plural different frequency bands;

repeating said transforming plural times for each second of the audio;

forming fingerprints using the spectral band data for said plural partially overlapping temporal excerpts; and

matching said fingerprints with reference data to identify one or more portions of the content;

wherein said fingerprints enable identification of content portions with a granularity of three seconds in length;

determining a length of an identified content portion, based on the matched fingerprints;

consulting stored rules to determine how to treat said uploaded content item, wherein treatment of the content item indicated by at least one of said stored rules depends on said length of the identified content portion; and

controlling distribution of the uploaded content item in accordance with a rule that depends on said length of the identified content portion;

wherein for a first content item, for which the identified portion is of a first length, said controlling comprises treating the uploaded first content item in accordance with a first stored rule that prohibits distribution of the uploaded content, and

wherein for a second content item, for which the identified portion is of a second, different, length, said controlling comprises treating the uploaded second content item in accordance with a second, different, stored rule that permits distribution of the uploaded content and provides a majority share of ad revenue associated therewith to a rights-holder of a pre-existing work from which at least a portion of the uploaded second content item was derived.

2. The method of claim 1 wherein, for a third content item, for which the identified portion is of a third length different than the first and second lengths, said treating comprises the uploaded third content item in accordance with a third stored rule that is different than the first and second stored rules.

3. The method of claim 1 wherein, for a third content item, for which the identified portion is of a third length different than the first and second lengths, said treating comprises permitting distribution of the uploaded content without sharing of ad revenue with a rights-holder of a pre-existing work from which at least a portion of the third content item was derived.

4. The method of claim 1 that further includes treating the uploaded content item in accordance with a rule that depends on country, so that distribution of the uploaded content item in a first country is treated differently than distribution of the uploaded content item in a second country.

5. The method of claim 4 wherein distribution of the uploaded content item in the first country is prohibited.

6. The method of claim 4 wherein the uploaded content item is refused distribution in the first country, but is distributed in the second country with advertising, revenue for said advertising being shared with a rights-holder of a pre-existing work from which at least a portion of the uploaded content was derived.

7. The method of claim 1 wherein said temporal excerpts overlap by 11.6 milliseconds.

8. The method of claim 1 wherein each of said temporal excerpts is 0.37 seconds in length.

9. The method of claim 1 that includes determining, for each excerpt, relative energy within 32 different frequency bands.

10. The method of claim 1 in which said transforming includes performing a Fast Fourier Transform on each of said plural temporal excerpts of audio.

11. The method of claim 1 that includes controlling distribution of the uploaded content in accordance with a rule that depends on a length of the identified content portion, wherein said length is not an absolute length, but rather is a fraction of a pre-existing work that is included in the uploaded content.

12. The method of claim 1 that includes repeating said analysis 31034 times for a single item of audio-visual content.

13. A system comprising:

a server that receives items of audio-video content uploaded by users, for possible distribution;

a fingerprint processor that identifies, at a time of ingest—

for each of plural items of said audio-video content, and by reference to audio fingerprint data derived from the uploaded audio-content item—

one or more portions of said uploaded audio-video content item that have been derived from one or more pre-existing works, said processor performing said identification by analysis acts including;

transforming plural partially overlapping temporal excerpts of the audio into spectral band data to determine, for each such excerpt, relative energy within plural frequency bands;

repeating such transforming plural times for each second of audio;

forming fingerprints using the spectral band data for said plural partially overlapping temporal excerpts; and

matching the fingerprints with reference data to identify portions of the content, said fingerprints enabling identification of content portions with a granularity of three seconds in length;

a database, the database containing rules specifying how uploaded content items should be treated; and

a processor that controls distribution of uploaded content items in accordance with said rules;

wherein at least one of said rules specifies treatment of content items based on a length of a portion identified as having been derived from a pre-existing work,

wherein for a first item of content, for which the identified portion is of a first length, said rules cause the processor to prohibit distribution of said first item of content; and

for a second item of content, for which the identified portion is of a second, different, length, said rules cause the processor to permit distribution of said second content item, with a majority of ad revenue associated therewith shared with a rights-holder of a pre-existing work from which at least a portion of said second item of content was derived.

14. The system of claim 13, wherein at least one of said rules specifies treatment of content items based on a country of distribution.

15. The system of claim 13 wherein for a third item of content, for which the identified portion is of a third length different than the first and second lengths, said rules cause the processor to permit distribution of said third item of content without said sharing of ad revenue.

16. The system of claim 13 in which the database rules comprise a rule that depends on country, so that distribution of a fourth item of content in a first country is treated differently than distribution of said fourth item of content in a second country.

17. The system of claim 16 wherein said fourth item of content is refused distribution in the first country, but is distributed in the second country with advertising, revenue for the advertising being shared with a rights-holder of a pre-existing work from which at least a portion of said fourth item of content was derived.

18. The system of claim 13 wherein said temporal excerpts overlap by 11.6 milliseconds.

19. The system of claim 13 wherein each of said temporal excerpts is 0.37 seconds in length.

20. The system of claim 13 wherein said analysis determines, for each excerpt, relative energy within 32 different frequency bands.

21. The system of claim 13 wherein said transforming comprises performing a Fast Fourier Transform on each of said plural temporal excerpts of audio.

22. The system of claim 13 wherein at least one of said rules specifies treatment of content items based on a length of a portion identified as having been derived from a pre-existing work, wherein said length is not an absolute length, but rather is a fraction of said pre-existing work.

23. The system of claim 13 wherein said repeating comprises repeating said analysis 31034 times for a single item of audio-visual content.

24. A non-transitory computer readable medium comprising instructions stored thereon to cause one or more processors to perform the following:

conducting an audio fingerprint-based analysis of each of plural items of content, to identify one or more portions of said content item that have been derived from one or more pre-existing works, said analysis including transforming plural partially overlapping temporal excerpts of the audio into spectral band data to determine, for each such excerpt, relative energy within plural different frequency bands;

repeating said transforming plural times for each second of the audio;

forming fingerprints using the spectral band data for said plural partially overlapping temporal excerpts; and

matching said fingerprints with reference data to identify portions of the content;

wherein the fingerprints enable identification of content portions with a granularity of three seconds in length;

determining a length of an identified content portion, based on the matched fingerprints;

determining at least one rule that governs treatment of the uploaded content item, based at least in part on said length of the identified content portion; and

controlling distribution of the uploaded content item in accordance with said determined rule;

wherein a first rule prohibits distribution of the uploaded content; and

a second rule permits distribution of the uploaded content, with a majority of ad revenue associated therewith shared with a rights-holder of a pre-existing work from which at least a portion of the uploaded content item was derived.

25. The non-transitory computer readable medium of claim 24 in which a third rule specifies handling of the content in accordance with a third, different, way.

26. The non-transitory computer readable medium of claim 24 in which a third rule permits distribution of the uploaded content without said sharing of ad revenue.

27. The non-transitory computer readable medium of claim 24 that further includes instructions for treating the uploaded content in accordance with a rule that depends on country, so that distribution of the uploaded content in a first country is treated differently than distribution of the uploaded content in a second country.

28. The non-transitory computer readable medium of claim 27 that further includes instructions such that distribution of the uploaded content in the first country is prohibited.

29. The non-transitory computer readable medium of claim 27 that further includes instructions such that the uploaded content is refused distribution in the first country, but is distributed in the second country with advertising, revenue for said advertising being shared with a rights-holder of a pre-existing work from which at least a portion of the uploaded content was derived.

30. The medium of claim 24 wherein said temporal excerpts overlap by 11.6 milliseconds.

31. The medium of claim 24 wherein each of said temporal excerpts is 0.37 seconds in length.

32. The medium of claim 24 that includes determining, for each excerpt, relative energy within 32 different frequency bands.

33. The medium of claim 24 wherein said transforming includes performing a Fast Fourier Transform on each of said plural temporal excerpts of audio.

34. The medium of claim 24 in which said determining comprises determining a rule that governs treatment of the uploaded content item, based at least in part on said length of the identified content portion, wherein said length is not an absolute length, but rather is a fraction of a pre-existing work from which said identified portion was derived.