SYSTEM AND METHODS FOR CONTINUOUS AUDIO MATCHING

• US 20120029670A1
• Filed: 07/28/2011
• Published: 02/02/2012
• Est. Priority Date: 07/29/2010
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Mont-Reynaud, Bernard, Master, Aaron, Stonehocker, Timothy P., Mohajer, Keyvan
• Granted Patent: US 9,390,167 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content

Abstract

The present invention relates to the continuous monitoring of an audio signal and identification of audio items within an audio signal. The technology disclosed utilizes predictive caching of fingerprints to improve efficiency. Fingerprints are cached for tracking an audio signal with known alignment and for watching an audio signal without known alignment, based on already identified fingerprints extracted from the audio signal. Software running on a smart phone or other battery-powered device cooperates with software running on an audio identification server.

Claims

1. A method of using a battery powered device to continuously monitor an audio signal and identify audio items within the audio signal, the method including:

predictively caching audio fingerprint sequences and corresponding audio item identifiers from a server after the device sends initial audio fingerprints extracted from the audio signal by the device; and

using the predictively cached audio fingerprint sequences to identify an audio item within the audio signal based on at least some additional audio fingerprints of the audio signal.

2. The method of claim 1, wherein the predictively cached audio fingerprint sequences include audio fingerprint sequences of future audio items predicted to follow the initial fingerprints.

3. The method of claim 1, wherein the predictively cached audio fingerprint sequences include audio fingerprint sequences of potentially identified songs that share the initial fingerprints.

4. The method of claim 1, wherein the identified audio item is a song.

5. The method of claim 1, wherein the cached fingerprint sequences are stored in a local cache on the device.

6. The method of claim 1, wherein the cached fingerprint sequences are stored in a local cache on the server.

7. The method of claim 1, including receiving additional predictively cached audio fingerprint sequences upon to identification of the audio item within the audio signal.

8. The method of claim 1, including using the predictively cached audio fingerprint sequences to identify another audio item within the audio signal without intervention by a user.

9. The method of claim 1, including maintaining a timed log of identified audio items within the audio signal.

10. A method of efficiently using a battery powered device to continuously monitor an audio signal and identify audio items within the audio signal, the method including:

receiving into a local cache on the device predictive audio fingerprints and corresponding audio item identifiers appropriate to a watching mode and a tracking mode as the device switches between the watching and tracking modes; and

switching between the watching mode in which a transition has occurred between a known audio item and a new unknown audio item, and the tracking mode in which a plurality of candidates for a current audio item have been identified, but not resolved to a single current audio item.

11. The method of claim 10, wherein the switching includes switching between the watching mode, the tracking mode and an identifying mode in which the device relies upon a sever to act upon fingerprints sent from the device which cannot be resolved using the predictively cached audio fingerprints.

12. The method of claim 11, including receiving into the local cache updated predictive audio fingerprints and corresponding updated audio item identifiers upon a successful identification of one or more audio items for the fingerprints which were not resolved using the predictively cached audio fingerprints.

14. The method of claim 10, wherein the tracking mode includes resolving the single current audio item from among the plurality of candidates.

15. The method of claim 10, wherein the switching between the watching mode and the tracking mode occurs without intervention by a user.

16. The method of claim 10, including switching from the watching mode to the tracking mode upon a determination one or more of the plurality of candidates match the current audio item.

17. A method of managing resources in a server to continuously monitor an audio signal and identify audio items within the audio signal, the method including:

receiving into a local cache on the server predictive audio fingerprints and corresponding audio item identifiers appropriate to a watching mode and a tracking mode as the server switches between the watching and tracking modes; and

switching between the watching mode in which a transition has occurred between a known audio item and a new unknown audio item, and the tracking mode in which a plurality of candidates for a current audio item have been identified, but not resolved to a single current audio item.

18. The method of claim 17, wherein the switching includes switching between the watching mode, the tracking mode and an identifying mode in which the server acts upon additional fingerprints of the audio signal which cannot be resolved using the predictively cached audio fingerprints.

19. The method of claim 18, including receiving into the local cache updated predictive audio fingerprints and corresponding updated audio item identifiers upon a successful identification of one or more audio items for the fingerprints which were not resolved using the predictively cached audio fingerprints.

20. The method of claim 17, wherein the tracking mode includes resolving the single current audio item from among the plurality of candidates.

21. The method of claim 17, wherein the switching between the watching mode and the tracking mode occurs without intervention by a user.

22. The method of claim 17, including switching from the watching mode to the tracking mode upon a determination that one or more of the plurality of candidates match the current audio item.