Detecting repeating content in broadcast media

• US 7,991,770 B2
• Filed: 11/27/2006
• Issued: 08/02/2011
• Est. Priority Date: 11/29/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Fink, Michael, Baluja, Shumeet, Covell, Michele
• Primary Examiner(s): Truong; Cam Y
• Assistant Examiner(s): UDDIN, MD I
• Application Number:
  US11/563,665
• Publication Number:
  US 20070124756A1
• Time in Patent Office: 1,709 Days
• Field of Search: 707/1, 707/3, 707/706, 707/713, 707/769, 707/770, 725/18, 725/19, 725/22, 381/56, 381/57
• US Class Current: 707/722
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 20/14 for monitoring programmes
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/06 Arrangements for scheduling broadcast services or broadcast-related services
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/372 Programme
• CPC Class Codes: H04H 60/375 Commercial
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04H 60/64 for providing detail information
• CPC Class Codes: H04H 60/65 for using the result on users' side
• CPC Class Codes: H04H 60/66 for using the result on distributors' side
• CPC Class Codes: H04H 60/80 characterised by transmission among terminal devices
• CPC Class Codes: H04H 60/87 accessed over computer networks
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/26603 : for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/41407 : embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4143 : embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42203 : sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4394 : involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44016 : involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/440236 : by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/44213 : Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4542 : Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/4622 : Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4756 : for rating content, e.g. scoring a recommended movie
• CPC Class Codes: H04N 21/4782 : Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8106 : involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8133 : specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/445 : for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 7/17336 : Handling of requests in head-ends

Abstract

Systems, methods, devices, and computer program products provide social and interactive applications for detecting repeating content in broadcast media. In some implementations, a method includes: generating a database of audio statistics from content; generating a query from the database of audio statistics; running the query against the database of audio statistics to determine a non-identity match; if a non-identity match exists, identifying the content corresponding to the matched query as repeating content.

Claims

1. A method executed by one or more computing devices, the method comprising:

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

performing post-match processing, including;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length; and

applying the following formula, upon condition that C_h−

1/L>

C₀, to the previous match and the previous confidence score;

{M₀,C₀}={M_h,C_h−

l/L}

where;

M₀ is a match for selecting content related to the audio descriptor;

C₀ is the current confidence score;

M_h is the previous match occurring at the previous time step;

C_h is the previous confidence score associated with the previous match;

l is a time step length; and

L is an expected dwell time;

selecting content related to the audio descriptor based on the match M₀; and

removing the selected content from a presentation to a user.

2. The method of claim 1, where the first reference descriptor and second reference descriptor are frame descriptors.

3. The method of claim 1, where the current match is a non-identity match.

4. The method of claim 1, where determining the current match includes:

generating a short list of candidate auditory matches using hashing techniques; and

validating the candidate auditory matches using a validation procedure.

5. The method of claim 1, where at least one of the previous confidence score or the current confidence score is a log likelihood confidence score given by an audio fingerprinting process.

6. A system, comprising:

a memory;

one or more processors configured to perform operations comprising;

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

performing post-match processing, including;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length; and

applying the following formula, upon condition that C_h−

1/L>

C₀, to the previous match and the previous confidence score;

{M₀,C₀}={M_h,C_h−

l/L}

where;

M₀ is a match for selecting content related to the audio descriptor;

C₀ is a the current confidence score;

M_h is the previous match occurring at the previous time step;

C_h is the previous confidence score associated with the previous match;

l is a time step length; and

L is an expected dwell time;

selecting content related to the audio descriptor based on the match M₀; and

removing the selected content from a presentation to a user.

7. The system of claim 6, the operations further comprising:

validating the current match using non-auditory information.

8. The system of claim 6, the operations further comprising:

determining end points of the repeating content.

9. The system of claim 8, where the end points are determined using dynamic programming techniques.

10. The system of claim 6, the operations further comprising:

applying metrics to the selected content to determine if the repeating content is an advertisement.

11. The system of claim 10, where the metrics are from a group of metrics consisting of time duration, volume, visual activity, and blank frame bracketing.

12. The system of claim 6, where the first reference descriptor and second reference descriptor are generated from ambient audio snippets of a media broadcast.

13. The system of claim 6, where the first reference descriptor and second reference descriptor are frame descriptors.

14. The system of claim 6, where the current match is a non-identity match.

15. The system of claim 14, where the non-identity match is determined using hashing techniques.

16. The system of claim 6, where at least one of the previous confidence score or the current confidence score is a log likelihood confidence score given by an audio fingerprinting process.

17. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by a processor, causes the processor to perform operations comprising:

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

performing post-match processing, including;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length; and

applying the following formula, upon condition that C_h−

1/L>

C₀, to the previous match and the previous confidence score;

{M₀,C₀}={M_h,C_h−

l/L}

where;

M₀ is a match for selecting content related to the audio descriptor;

C₀ is the current confidence score;

M_h is the previous match occurring at the previous time step;

C_h is the previous confidence score associated with the previous match;

l is a time step length; and

L is an expected dwell time;

selecting content related to the audio descriptor based on the match M₀; and

providing the selected content for a presentation to a user.

18. The medium of claim 17, the operations further comprising:

applying at least one metric to the selected content to determine if the repeating content is an advertisement.

19. The medium of claim 18, wherein the metric is associated with one of a length of a media broadcast and a volume of a media broadcast.

20. The medium of claim 17, where at least one of the previous confidence score or the current confidence score is a log likelihood confidence score given by an audio fingerprinting process.

21. A method executed by a computer, the method comprising:

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length;

discounting the previous confidence score by a discount value, the discount value being calculated based on a ratio between the time step length and an expected dwell time;

determining that the discounted previous confidence score is greater than the current confidence score;

selecting content related to the audio descriptor based on the previous match; and

removing the selected content from a presentation to a user.

22. The method of claim 21, where the current match is a non-identity match.

23. The method of claim 21, where the expected dwell time indicates an expected time between channel changes.

24. The method of claim 21, wherein the previous match is a best match of the previous time step.

25. The method of claim 24, wherein each of the current confidence score and the previous confidence score includes a log-likelihood confidence score given by an audio fingerprinting process.

26. A system, comprising:

a processor;

a memory;

one or more computers configured to perform operations comprising;

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length;

discounting the previous confidence score by a discount value, the discount value being calculated based on a ratio between the time step length and an expected dwell time;

determining that the discounted previous confidence score is greater than the current confidence score;

selecting content related to the audio descriptor based on the previous match; and

removing the selected content from a presentation to a user.

27. The system of claim 26, wherein the current match is a non-identity match.

28. The system of claim 26, wherein the expected dwell time indicates an expected time between channel changes.

29. The system of claim 26, wherein the previous match is a best match of the previous time step.

30. The system of claim 29, wherein each of the current confidence score and the previous confidence score includes a log-likelihood confidence score given by an audio fingerprinting process.

31. A computer program product tangibly stored on a non-transitory computer-readable storage medium and operable to cause one or more computers to perform operations comprising:

determining a current match between an audio descriptor and a first reference descriptor, the current match occurring at a current time step and being associated with a current confidence score, the current time step having a time step length;

determining a previous match between the audio descriptor and a second reference descriptor, the previous match occurring at a previous time step and being associated with a previous confidence score, the previous time step having the time step length;

discounting the previous confidence score by a discount value, the discount value being calculated based on a ratio between the time step length and an expected dwell time;

determining that the discounted previous confidence score is greater than the current confidence score;

selecting content related to the audio descriptor based on the previous match; and

removing the selected content from a presentation to a user.

32. The product of claim 31, wherein the current match is a non-identity match.

33. The product of claim 31, wherein the expected dwell time indicates an expected time between channel changes.

34. The product of claim 31, wherein the previous match is a best match of the previous time step.

35. The product of claim 34, wherein each of the current confidence score and the previous confidence score includes a log-likelihood confidence score given by an audio fingerprinting process.