Systems and methods for performing semantic analysis of media objects

• US 9,226,047 B2
• Filed: 12/08/2008
• Issued: 12/29/2015
• Est. Priority Date: 12/07/2007
• Status: Active Grant

• Current Assignee: Verimatrix Incorporated (Verimatrix SA)
• Original Assignee: Verimatrix Incorporated (Verimatrix SA)
• Inventors: Thorwirth, Niels J., Munro, Thomas, Kulakowski, Robert
• Primary Examiner(s): Lin, Jason K
• Application Number:
  US13/266,761
• Publication Number:
  US 20130097625A1
• Time in Patent Office: 2,577 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4627 Rights management associated to the content security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 21/8355 involving usage data, e.g. number of copies or viewings allowed
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 5/913 for scrambling ; for copy protection

Abstract

System and method for identification of ownership to media objects through analysis of semantic information in the media objects are disclosed Semantic information is a term that can be used to describe aspects of the content of a media object that are apparent to a human viewing or listening to the content One embodiment of the invention includes extracting semantic information from a specified media object, matching said semantic information of the specified media object against known semantic information of known media objects, and taking action based on the result of the companson.

Claims

1. A method for restricting access to digital video content based on recognition of a TV station logo embedded within the video content that varies according to the video usage rights, the method comprising:

extracting the TV station logo from the video content by combining several frames of video content;

identifying the TV station logo by comparing the TV station logo to known TV station logos that most closely matches said TV station logo extracted from the video content, where the known TV station logos are associated with user profiles of copyright rights holders, and wherein at least some of the known TV station logos are supplied by rights holders for content analysis to identify their owned content;

estimating a first match probability of the likely copyright rights holder based on the identified TV station logo;

identifying a second media object within at least one frame of the video content different from the TV station logo;

extracting at least a second semantic property from the second specified media object within at least one frame of the video content;

matching said at least second semantic property of the second specified media object against the known semantic properties of known media objects associated with user profiles of copyright rights holders;

comparing matches between said at least second semantic property and the known semantic properties to a second threshold value;

estimating a second match probability of the likely copyright rights holder based on the second compared matches;

combining the first match probability and the second match probability into a combined match probability;

identifying a copyright rights holder based on at least the combined match probability;

retrieving rules governing distribution rights of the content based at least on identification of the copyright rights holder; and

restricting access to the digital video content in accordance with the retrieved rules.

2. The method of claim 1, wherein restricting access comprises combination of advertisement with said video content.

3. The method of claim 1, wherein said TV station logo is visibly embedded within the video content.

4. The method of claim 1, wherein said TV station logo is invisibly embedded within the video content.

15. The method of claim 1 where the copyright rights holders are copyright owners.

16. The method of claim 15 further comprising locating the TV station logo within the video content through scanning online sites;

and

wherein restricting access to the digital video content in accordance with the retrieved rules comprises notifying a content distributor of the video content that the content is copyright protected and who the identified content owner is.

19. The method of claim 1 wherein identifying a copyright rights holder based on at least the estimated match probability comprises identifying a broadcast source.

21. The method of claim 1 further comprising:

identifying keywords in the identified content using a reference database of semantic properties; and

assigning advertising commercials based upon the identified keywords advertising into the video content; and

wherein restricting access to the digital video content in accordance with the retrieved rules comprises combining the advertising commercials into the video content.

5. A method for restricting access to a media object based on automatic recognition of human recognizable semantic properties evident during playback of the media object, the method comprising:

extracting at least one semantic property from a first specified media object within at least one frame of video content;

matching said at least one semantic property of the first specified media object against known semantic properties of known media objects associated with user profiles of rights holders, wherein at least some of the known semantic properties are generated from material supplied by rights holders for content analysis to identify their owned content;

comparing matches between said at least one semantic property and known semantic properties to a threshold value;

estimating a first match probability of the likely copyright rights holder based on the compared matches;

identifying a second media object within at least one frame of the video content different from the first media object;

extracting at least a second semantic property from the second specified media object within at least one frame of the video content;

matching said at least second semantic property of the second specified media object against the known semantic properties of known media objects associated with user profiles of copyright rights holders;

comparing matches between said at least second semantic property and the known semantic properties to a second threshold value;

estimating a second match probability of the likely copyright rights holder based on the second compared matches;

combining the first match probability and the second match probability into a combined match probability;

identifying a copyright rights holder based on at least the combined match probability;

retrieving rules governing distribution rights of the video content based at least on identification of the copyright rights holder; and

determining allowed actions that can be performed with respect to the media objects based upon the retrieved rules;

restricting access to playback of video content containing the media objects in accordance with the allowed actions.

6. The method of claim 5, wherein said specified media object is present on at least one of:

a website, a file transport protocol site, a peer to peer network, a disc drive, a hard drive, a computer, a compact disc, and a digital video disc.

7. The method of claim 5, wherein the known semantic properties are derived from previously identified semantic properties.

8. The method of claim 5, wherein the semantic property is identification of a TV station logo on at least one frame of a video track within the media object.

9. The method of claim 5, wherein the semantic information is the result of optical character recognition on at least one frame of a video track within the media object.

10. The method of claim 5, wherein the semantic property is the result of performing speaker recognition on an audio track within the media object.

11. The method of claim 5, wherein the semantic property is the result of performing automated speech recognition on an audio track within the media object.

12. The method of claim 5, wherein restricting access to playback of video content containing the media object in accordance with the allowed actions comprises restriction to playback in combination of advertisement with said video content.

13. The method of claim 5, wherein said restriction of access includes notification of the content distributor of the access rights.

14. The method of claim 5, further comprising:

identifying a second media object within the video content different from the first media object;

extracting at least a second semantic property from the second specified media object within the video content;

matching said at least second semantic property of the second specified media object against the known semantic properties of known media objects associated with user profiles of copyright rights holders;

comparing matches between said at least second semantic property and the known semantic properties to a second threshold value;

estimating a second match probability of the likely copyright rights holder based on the second compared matches;

combining the first match probability and the second match probability into a combined match probability; and

identifying a copyright rights holder based on at least the combined match probability.

17. The method of claim 5 where the copyright rights holders are copyright owners.

18. The method of claim 17 further comprising locating the specified media object within video content through scanning an online site containing user-generated content;

and

wherein restricting access to playback of video content containing the media object in accordance with the allowed actions comprises notifying a content distributor of the video content that the content is copyright protected and who the identified content owner is.

20. The method of claim 8 wherein identifying a copyright rights holder based on at least the estimated match probability comprises identifying a broadcast source.

22. The method of claim 5 further comprising:

identifying keywords in the identified content using a reference database of semantic properties; and

assigning advertising commercials based upon the identified keywords advertising into the video content; and

wherein restricting access to playback of video content containing the media object in accordance with the allowed actions comprises combining the advertising commercials into the video content.

23. The method of claim 18, wherein notifying a content distributor of the video content that the content is copyright protected and who the identified content owner is comprises issuing a take down notice to stop distribution of the video content.