Detection of creative works on broadcast media

• US 8,433,577 B2
• Filed: 09/27/2011
• Issued: 04/30/2013
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Oztaskent, Ant, Volovich, Yaroslav
• Primary Examiner(s): WOZNIAK, JAMES S
• Application Number:
  US13/246,558
• Publication Number:
  US 20130080159A1
• Time in Patent Office: 581 Days
• Field of Search: 704/230, 704/243, 704/245, 704/270, 707/706, 707/913, 705/58
• US Class Current: 704/270
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/81 for discriminating voice from music
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/39 for identifying broadcast space-time use of Electronic Programme Guides H04H60/72
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/65 for using the result on users' side
• CPC Class Codes: H04H 60/74 using programme related information, e.g. title, composer or interpreter
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04N 21/442 Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00

Abstract

This disclosure relates to systems and methods for proactively determining identification information for a plurality of audio segments within a plurality of broadcast media streams, and providing identification information associated with specific audio portions of a broadcast media stream automatically or upon request.

Claims

1. A device, comprising:

at least one processor;

at least one non-transitory computer readable medium communicatively coupled to the at least one processor, the at least one non-transitory computer readable medium having stored therein computer-executable components, comprising;

a broadcast media reception component that receives a set of media streams over a set of channels;

an identification component that determines respective identification information and respective audio segment location information for a plurality of audio segments within the set of media streams concurrently with the broadcast media component receiving the set of media streams, wherein an audio segment location information includes, for an associated audio segment, an identifier of a media stream in which the audio segment is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio segment; and

a client interface component that;

receives a request from a client device to identify an audio portion of a media stream of the set of media streams, wherein the request comprises audio portion location information of the audio portion, wherein the audio portion location information includes an identifier of a media stream in which the audio portion is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio portion, wherein the request does not include an audio snippet associated with the audio portion;

employs the audio portion location information to identify at least one audio segment having respective audio segment location information within a predetermined range of the audio portion location information; and

responds to the client device with identification information corresponding to the at least one audio segment.

2. The device of claim 1, wherein the identification component comprises an audio tagging component that associates metadata with a subset of the media streams, the metadata identifying the respective audio segment location information for the plurality of audio segments within the set of media streams and the identification information for the plurality of audio segments.

3. The device of claim 1, wherein the identification component comprises an audio segment categorizing component that categorizes an audio segment of the plurality of audio segments as comprising at least one of music or dialogue.

4. The device of claim 1, wherein the identification component comprises a fingerprint component that generates a fingerprint for an audio segment of the plurality of audio segments.

5. The device of claim 4, wherein the identification component comprises a matching component that employs an algorithm to match the fingerprint to a data record in an audio library.

6. The device of claim 4, wherein the fingerprint component comprises a noise filtering component that removes portions of the audio segment not relevant to determining identification information for the audio segment.

7. The device of claim 1, wherein the identification information comprises an identification of at least one of a musical recording or a literary work.

8. The device of claim 1, wherein the client interface component adjusts the predetermined range based upon the audio portion location information.

9. The device of claim 1, wherein the identification component generates a hierarchical organization of the plurality of audio segments, wherein at least two audio segments have respective audio segment location information that at least partially overlap.

10. The device of claim 1, wherein the identification component accepts a user preference setting input that limits determination of identification information by the identification component to a selected one or more types of creative works.

11. The device of claim 1, wherein the predetermined range includes timestamp or counter values prior to the at least one of the timestamp or counter value of the audio portion location information.

12. The device of claim 1, wherein the identification component associates purchase information with the identification information, wherein the purchase information relates to a creative work identified in the identification information.

13. A method, comprising:

employing at least one processor executing computer executable instructions embodied on at least one non-transitory computer readable medium to perform operations comprising;

receiving a plurality of media streams over a set of channels;

determining, concurrently with the receiving the plurality of media streams, respective identification information and respective audio segment location information for a plurality of audio segments within the plurality of media streams, wherein an audio segment location information includes, for an associated audio segment, an identifier of the media stream in which the audio segment is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio segment;

receiving a request from a client device to identify an audio portion of a media stream of the plurality of media streams, wherein the request comprises audio portion location information of the audio portion, wherein the audio portion location information includes an identifier of a media stream in which the audio portion is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio portion, wherein the request does not include an audio snippet associated with the audio portion;

employing the audio portion location information to identify at least one audio segment having respective audio segment location information within a predetermined range of the audio portion location information; and

responding to the client device with identification information corresponding to the at least one audio segment.

14. The method of claim 13, wherein the determining identification information comprises:

generating a fingerprint for an audio segment of the plurality of audio segments; and

matching the fingerprint to a data record in an audio library.

15. The method of claim 13, further comprising removing portions of an audio segment of the plurality of audio segments that are not relevant to determining identification information for the audio segment.

16. The method of claim 13, further comprising:

generating a hierarchical organization of the plurality audio segments, wherein at least two audio segments have respective audio segment location information that at least partially overlap.

17. The method of claim 13, further comprising adjusting the predetermined range based upon the audio portion location information.

18. The method of claim 13, further comprising:

receiving a user preference setting input that limits determination of identification information to a selected one or more types of creative works; and

limiting determination of identification information to the selected one or more types of creative works.

19. A device, comprising:

at least one processor;

at least one non-transitory computer readable medium communicatively coupled to the at least one processor, the at least one non-transitory computer readable medium having stored therein computer-executable components, comprising;

a broadcast media reception component that receives at least one media stream on at least one channel; and

a query component that;

sends a query to an audio identification server to determine identification information associated with an audio portion of the at least one media stream, wherein the query comprises audio portion location information of the audio portion, wherein the audio portion location information includes an identifier of the media stream in which the audio portion is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio portion, wherein the request does not include an audio snippet associated with the audio portion; and

receives a response from the audio identification server comprising identification information corresponding to at least one audio segment having respective audio segment location information within a predetermined range of the audio portion location information, wherein an audio segment location information includes, for an associated audio segment, an identifier of the media stream in which the audio segment is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio segment.

20. The device of claim 19, wherein the query component receives a request to provide the identification information associated with the audio portion of the at least one media stream.

21. The device of claim 19, wherein the identification information comprises an indication of how closely the audio portion matches the at least one audio segment associated with the identification information.

22. The device of claim 19, wherein the query is automatically generated based upon a preference setting of the device.

23. The device of claim 22, wherein the preference setting specifies an interval at which the query is automatically generated.

24. The device of claim 19, further comprising a display configured to display purchase information received with the identification information, wherein the purchase information is related to a creative work identified in the identification information.

25. A method, comprising:

transmitting, by a client device, a query to an audio identification server to determine identification information associated with an audio portion of at least one media stream, wherein the query comprises audio portion location information of the audio portion, wherein the audio portion location information includes an identifier of a media stream in which the audio portion is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio portion, wherein the request does not include an audio snippet associated with the audio portion; and

receiving, by the client device, a response from the audio identification server comprising identification information corresponding to at least one audio segment having respective audio segment location information within a predetermined range of the audio portion location information, wherein an audio segment location information includes, for an associated audio segment, an identifier of the media stream in which the audio segment is located and at least one of a timestamp or counter value within the media stream associated with the location of the audio segment.

26. The method of claim 25, wherein the identification information comprises an indication of how closely the audio portion matches the at least one audio segment associated with the identification information.

27. The method of claim 26, further comprising displaying, by the client device, the identification information is ranked in order according to the indication of how closely the audio portion matches the at least one audio segment associated with the identification information.