Detecting repeating content in broadcast media

• US 8,700,641 B2
• Filed: 08/01/2011
• Issued: 04/15/2014
• Est. Priority Date: 11/29/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Covell, Michele, Baluja, Shumeet, Fink, Michael
• Primary Examiner(s): Uddin, Md I
• Application Number:
  US13/195,330
• Publication Number:
  US 20120059845A1
• Time in Patent Office: 988 Days
• Field of Search: 707/748, 707/749, 707/758, 707/722, 707/751, 707/600, 707/603
• US Class Current: 707/749
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 20/14 for monitoring programmes
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/06 Arrangements for scheduling broadcast services or broadcast-related services
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/372 Programme
• CPC Class Codes: H04H 60/375 Commercial
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04H 60/64 for providing detail information
• CPC Class Codes: H04H 60/65 for using the result on users' side
• CPC Class Codes: H04H 60/66 for using the result on distributors' side
• CPC Class Codes: H04H 60/80 characterised by transmission among terminal devices
• CPC Class Codes: H04H 60/87 accessed over computer networks
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/26603 : for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/41407 : embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4143 : embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42203 : sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4394 : involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44016 : involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/440236 : by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/44213 : Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4542 : Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/4622 : Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4756 : for rating content, e.g. scoring a recommended movie
• CPC Class Codes: H04N 21/4782 : Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8106 : involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8133 : specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/445 : for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 7/17336 : Handling of requests in head-ends

Abstract

Systems, methods, devices, and computer program products provide social and interactive applications for detecting repeating content in broadcast media. In some implementations, a method includes: generating a database of audio statistics from content; generating a query from the database of audio statistics; running the query against the database of audio statistics to determine a non-identity match; if a non-identity match exists, identifying the content corresponding to the matched query as repeating content.

Claims

1. A method, comprising:

determining, by a system including a processor, a first match M_h between a first audio descriptor representing a first recording at a first time step in an environment and a first reference descriptor, the first match M_h having-a first confidence score C_h indicative of a confidence of the first match, the first time step having a time step length l;

determining, by the system, a second match M₀ between a second audio descriptor representing a second recording at a second time step in the environment and a second reference descriptor, the second match M₀ having a second confidence score C₀ indicative of a confidence of the second match, the second time step having the time step length l, wherein the first time step is prior to the second time step, and the first match M_h and the second match M₀ are non-identity matches determined using a direct or locality sensitive hashing function and a validation process to select a most accurate match out of a plurality of candidate matches, wherein the first confidence score C_h and the second confidence score C₀ are based upon a log-likelihood function given by an audio fingerprinting process;

discounting, by the system, the first confidence score C_h by a discount value l/L to generate a discounted first confidence score C_h−

l/L, where L is an expected dwell time between a channel change;

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, employing, by the system, the first reference descriptor associated with the first match M_h for selecting related content; and

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, employing, by the system, the second reference descriptor associated with the second match M₀ for selecting the related content.

2. The method of claim 1, further comprising:

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, designating, by the system, the first reference descriptor as a best match to the second audio descriptor of the second time step.

3. The method of claim 1, further comprising:

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, designating, by the system, the second reference descriptor as a best match to the second audio descriptor of the second time step.

4. The method of claim 1, wherein the second reference descriptor has a highest confidence score of a plurality of reference descriptors that match the second audio descriptor.

5. The method of claim 1, wherein the first reference descriptor has a highest confidence score of a plurality of reference descriptors that match the first audio descriptor.

6. A non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations comprising:

determining a first match M_h between a first audio descriptor representing a first recording at a first time step in an environment and a first reference descriptor, the first match M_h having-a first confidence score C_h indicative of a confidence of the first match, the first time step having a time step length l;

determining a second match M₀ between a second audio descriptor representing a second recording at a second time step in the environment and a second reference descriptor, the second match M₀ having a second confidence score C₀ indicative of a confidence of the second match, the second time step having the time step length l, wherein the first time step is temporally prior to the second time step, and the first match M_h and the second match M₀ are non-identity matches determined using a direct or locality sensitive hashing function and a validation process to select a most accurate match out of a plurality of candidate matches, wherein the first confidence score C_h and the second confidence score C₀ are based upon a log-likelihood function given by an audio fingerprinting process;

discounting the first confidence score C_h by a discount value l/L to generate a discounted first confidence score C_h−

l/L, where L is an expected dwell time between a channel change;

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, employing the first reference descriptor associated with the first match M_h for selecting related content; and

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, employing the second reference descriptor associated with the second match M₀ for selecting the related content.

7. The non-transitory computer-readable medium of claim 6, the operations further comprising:

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, designating the first reference descriptor as a best match to the second audio descriptor of the second time step.

8. The non-transitory computer-readable medium of claim 6, the operations further comprising:

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, designating the second reference descriptor as a best match of the second time step.

9. The non-transitory computer-readable medium of claim 6, wherein the second reference descriptor has a highest confidence score of a plurality of reference descriptors that match the second audio descriptor.

10. The non-transitory computer-readable medium of claim 6, wherein the first reference descriptor has a highest confidence score of a plurality of reference descriptors that match the first audio descriptor.

11. A system, comprising:

a processor, communicatively coupled to a memory that stores computer-executable instructions, that executes or facilitates execution of the computer-executable instructions to perform operations comprising;

a social application server;

determine a first match M_h between a first audio descriptor representing a first recording at a first time step in an environment and a first reference descriptor, the first match M_h having-a first confidence score C_h indicative of a confidence of the first match, the first time step having a time step length l;

determine a second match M₀ between a second audio descriptor representing a second recording at a second time step in the environment and a second reference descriptor, the second match M₀ having a second confidence score C₀ indicative of a confidence of the second match, the second time step having the time step length l, wherein the first time step is temporally prior to the second time step, and the first match M_h and the second match M₀ are non-identity matches determined using a direct or locality sensitive hashing function and a validation process to select a most accurate match out of a plurality of candidate matches, wherein the first confidence score C_h and the second confidence score C₀ are based upon a log-likelihood function given by an audio fingerprinting process;

discount the first confidence score C_h by a discount value l/L to generate a discounted first confidence score C_h−

l/L, where L is an expected dwell time between a channel change;

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, employ the first reference descriptor associated with the first match M_h for selecting related content; and

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, employ the second reference descriptor associated with the second match M₀ for selecting the related content.

12. The system of claim 11, wherein the social application server further determine:

in response to the discounted first confidence score C_h−

l/L being greater than the second confidence score C₀, designate the first reference descriptor as a best match to the second audio descriptor of the second time step.

13. The system of claim 11, wherein the social application server further determine:

in response to the discounted first confidence score C_h−

l/L not being greater than the second confidence score C₀, designate the second reference descriptor as a best match to the second audio descriptor of the second time step.

14. The system of claim 11, wherein the second reference descriptor has a highest confidence score of a plurality of reference descriptors that match the second audio descriptor.

15. The system of claim 11, wherein the first reference descriptor has a highest confidence score of a plurality of reference descriptors that match the first audio descriptor.