System and method for matching a query against a broadcast stream

• US 9,047,371 B2
• Filed: 02/21/2012
• Issued: 06/02/2015
• Est. Priority Date: 07/29/2010
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Mohajer, Keyvan, Mont-Reynaud, Bernard, Aung, Joe
• Primary Examiner(s): ELBIN, JESSE A
• Application Number:
  US13/401,728
• Publication Number:
  US 20120239175A1
• Time in Patent Office: 1,197 Days
• Field of Search: 340/4.37, 340/4.4, 381/56, 700/94, 707/746, 707/747, 716/727, 725/18, 725/19
• US Class Current: 1/1
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/686 using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings
• CPC Class Codes: G06F 16/687 using geographical or spatial information, e.g. location
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: H04H 20/14 for monitoring programmes
• CPC Class Codes: H04H 60/13 Arrangements for device control affected by the broadcast information
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/35 Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
• CPC Class Codes: H04H 60/372 Programme
• CPC Class Codes: H04H 60/43 for identifying broadcast channels
• CPC Class Codes: H04H 60/44 for identifying broadcast stations
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00

Abstract

A method for identifying with a broadcast stream. That method includes receiving one or more broadcast streams, from which it generates and stores an audio fingerprint of a selected portion of each received broadcast stream. A query is received, and the method generates an audio fingerprint of the query. From that point, the method continues by identifying audio content from the query, using the query audio fingerprint and a database of indexed audio content. The method concludes by identifying the source of the query using the query audio fingerprint and the stored audio fingerprints. Embodiments of the method further include predictively caching audio fingerprint sequences and corresponding audio item identifiers from a server after storing audio fingerprints extracted from the broadcast stream; and using the predictively cached audio fingerprint sequences to identify an audio item within the audio signal based on at least some additional audio fingerprints of the audio signal.

Claims

1. A method of identifying an audio content sample and determining a broadcast station source for a broadcast, including:

monitoring a plurality of broadcast stations and automatically identifying broadcast audio content from the broadcast stations as it is received;

caching to memory fingerprints of the identified broadcast audio content from the plurality of broadcast stations;

receiving a sampled audio content;

matching fingerprints of the sampled audio content against;

the cache of fingerprints from the identified broadcast audio content; and

a database of audio content;

identifying the sampled audio content and, when the sampled audio content originates from a broadcast, a broadcast station source that was sampled;

receiving a location from a portable device; and

using at least the location and the identified sampled audio content to target the portable device for advertising or offers.

2. The method of claim 1, wherein the sampled audio content is received from the portable device.

3. The method of claim 1, further including matching of the fingerprints of the sampled audio content against the cache and bypassing matching against the database if the matching against the cache succeeds.

4. The method of claim 1,

wherein the sampled audio content is received from the portable device and further including:

using the location to validate an identity of the broadcast station source.

5. The method of claim 1,

wherein the sampled audio content is received from the portable device and further including:

receiving a location from the portable device; and

using at least the location and the identified broadcast station source to target the portable device for advertising or offers.

6. The method of claim 1, wherein:

the identified sampled audio content is an advertisement;

further including using at least the identified sampled audio content and the identified broadcast station source to target the portable device for advertising or offers.

7. The method of claim 1, wherein receiving the sampled audio content includes receiving fingerprints from the portable device.

8. The method of claim 1, further including creating fingerprints from the received sampled audio content.

9. The method of claim 1, further including sending at least some data regarding the identified sampled audio content to the portable device.

10. A method of identifying an audio content sample from a real time broadcast, including:

monitoring a plurality of broadcast stations and identifying broadcast audio content from the broadcast stations as it is received;

wherein monitoring at least one broadcast station of the plurality of broadcast stations relies on a web-based feed that is delayed from an over-the-air feed from the same broadcast station;

receiving sampled audio content from a portable device;

receiving a location from the portable device;

identifying the sampled audio content by matching fingerprints of the sampled audio content against a database of audio broadcast content;

wherein matching the fingerprints of the sampled audio content against the database of broadcast content is limited by a predetermined delay bound; and

combining the location and the identified sample audio content to identify a broadcast station source which the portable device sampled.

11. The method of claim 10, further including caching fingerprints of the identified broadcast audio content from the plurality of broadcast stations and matching the fingerprints of the sampled audio against the cache of fingerprints.

12. The method of claim 10, wherein:

the identified sampled audio content is an advertisement;

further including using at least the identified sampled audio content and the identified broadcast station source to target the portable device for advertising or offers.

13. The method of claim 10, further including:

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending the portable device at least a recent portion of the playlist that identifies a plurality of items broadcast by the identified broadcast station source.

14. A method of identifying an audio content sample that originates from an actual broadcast, including:

monitoring a plurality of broadcast stations, fingerprinting and saving the fingerprints of broadcast audio content in a database of broadcast content as it is received;

identifying at least some of the broadcast audio content from the broadcast stations against a database of pre-indexed audio content and caching fingerprints of the identified broadcast audio content as it is received;

receiving sampled audio content from a portable device;

matching the fingerprints of the sampled audio content against;

the cache of fingerprints for the identified broadcast audio content;

the database of broadcast content; and

the database of pre-indexed audio content; and

identifying a broadcast station source of the sampled audio content that originates from an actual broadcast and, when the sampled audio content can be found in the database of pre-indexed audio content, identifying the sampled audio content.

15. The method of claim 14, further including:

receiving a location from the portable device; and

using the location to validate an identity of the broadcast station source.

16. The method of claim 14, further including:

receiving a location from the portable device; and

using at least the location and the identified sampled audio content to target the portable device for advertising or offers.

17. The method of claim 14, wherein:

the monitoring of at least one broadcast station relies on a web-based feed that is delayed from an over-the-air feed from the same broadcast station; and

matching the fingerprints of the audio content sample against the cache is limited by a predetermined delay bound.

18. The method of claim 14, further including:

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending the portable device at least a recent portion of the playlist that identifies a plurality of recent items broadcast by the identified broadcast station source.

19. A method of identifying an audio content sample and determining a broadcast station:

monitoring a plurality of broadcast stations and automatically identifying broadcast audio content from the broadcast stations as it is received;

wherein monitoring at least one broadcast station of the plurality of broadcast stations relies on a web-based feed that is delayed from an over-the-air feed from the same broadcast station;

caching to memory fingerprints of the identified broadcast audio content from the plurality of broadcast stations;

receiving a sampled audio content;

matching fingerprints of the sampled audio content against;

the cache of fingerprints from the identified broadcast audio content; and

a database of audio content;

wherein matching the fingerprints of the sampled audio content against the database of broadcast audio content is limited by a predetermined delay bound; and

identifying the sampled audio content and, when the sampled audio content originates from a broadcast, a broadcast station source that was sampled.

20. The method of claim 19, further including:

receiving a location from the portable device; and

using the location to validate an identity of the broadcast station source.

21. The method of claim 19, wherein:

the monitoring of at least one broadcast station relies on a web-based feed that is delayed from an over-the-air feed from the same broadcast station; and

matching the fingerprints of the audio content sample against the cache is limited by a predetermined delay bound.

22. The method of claim 19, further including:

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending the portable device at least a recent portion of the playlist that identifies a plurality of recent items broadcast by the identified broadcast station source.

23. A method of identifying an audio content sample and determining a broadcast station source for a broadcast, including:

monitoring a plurality of broadcast stations and automatically identifying broadcast audio content from the broadcast stations as it is received;

caching to memory fingerprints of the identified broadcast audio content from the plurality of broadcast stations;

receiving a sampled audio content from a portable device;

matching fingerprints of the sampled audio content against;

the cache of fingerprints from the identified broadcast audio content;

and a database of audio content; and

identifying the sampled audio content and, when the sampled audio content originates from a broadcast, a broadcast station source that was sampled;

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending the portable device at least a recent portion of the playlist that identifies a plurality of recent items broadcast by the identified broadcast station source.

24. The method of claim 23,

wherein the sampled audio content is received from a portable device and further including:

receiving a location from the portable device; and

using the location to validate an identity of the broadcast station source.

25. The method of claim 23, further including:

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending a portable device at least a recent portion of the playlist that identifies a plurality of recent items broadcast by the identified broadcast station source.

26. A method of identifying an audio content sample and determining a broadcast station source for a broadcast, including:

monitoring a plurality of broadcast stations and automatically identifying broadcast audio content from the broadcast stations as it is received;

caching to memory fingerprints of the identified broadcast audio content from the plurality of broadcast stations;

receiving a sampled audio content;

concurrently matching fingerprints of the sampled audio content against the database of broadcast audio content and a database of audio content; and

identifying the sampled audio content and, when the sampled audio content originates from a broadcast, a broadcast station source that was sampled.

27. The method of claim 26, further including:

receiving a location from a portable device; and

using the location to validate an identity of the broadcast station source.

28. A method of identifying an audio content sample from a real time broadcast, including:

monitoring a plurality of broadcast stations and identifying broadcast audio content from the broadcast stations as it is received;

receiving sampled audio content from a portable device;

receiving a location from the portable device;

identifying the sampled audio content by matching fingerprints of the sampled audio content against a database of audio content; and

combining the location and the identified sample audio content to identify a broadcast station source which the portable device sampled; and

using at least the identified broadcast station source to target the portable device for advertising or offers.

29. The method of claim 28, wherein:

the identified sampled audio content is an advertisement;

further including using at least the identified sampled audio content and the identified broadcast station source to target the portable device for advertising or offers.

30. The method of claim 28, further including:

generating and saving playlists of the broadcast audio content from the broadcast stations monitored; and

sending the portable device at least a recent portion of the playlist that identifies a plurality of items broadcast by the identified broadcast station source.