System and methods for offline audio recognition

• US 9,196,242 B1
• Filed: 05/29/2012
• Issued: 11/24/2015
• Est. Priority Date: 05/29/2012
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Master, Aaron, Mont-Reynaud, Bernard, Mohajer, Keyvan
• Primary Examiner(s): Fleurantin, Jean B
• Assistant Examiner(s): ASPINWALL, EVAN S
• Application Number:
  US13/482,792
• Time in Patent Office: 1,274 Days
• Field of Search: 707/769, 707/768
• US Class Current: 1/1
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 25/54 for retrieval

Abstract

In one implementation, a method is described of retrying matching of an audio query against audio references. The method includes receiving a follow-up query that requests a retry at matching a previously submitted audio query. In some implementations, this follow-up query is received without any recognition hint that suggests how to retry matching. The follow-up query includes the audio query or a reference to the audio query to be used in the retry. The method further includes retrying matching the audio query using retry matching resources that include an expanded group of audio references, identifying at least one match and transmitting a report of the match. Optionally, the method includes storing data that correlates the follow-up query, the audio query or the reference to the audio query, and the match after retrying.

Claims

1. A method of retrying matching of an initial audio query against audio references, the method including:

receiving a follow-up query from a user that requests a retry at matching an initial audio query, the initial audio query comprising a recorded audio sample or audio fingerprints derived from the recorded audio sample, the follow-up query including;

the initial audio query previously submitted by the user or a reference to the initial audio query previously submitted by the user; and

a recognition hint from the user that suggests how to retry matching;

queuing a retry request comprising the follow-up query after an initial audio query is not matched;

processing the queued retry request from the queue based upon one of;

a period of time after which the initial audio query is not matched;

or

the completion of an update of the expanded group of audio references;

wherein processing the queued retry request includes retrying matching the initial audio query using the recognition hint and retrying matching resources that include an expanded group of audio references;

identifying at least one match for the retry request and transmitting a report of the match; and

storing data that correlates at least the follow-up query, the initial audio query or the reference to the initial audio query, and the match after retrying.

2. The method of claim 1, further including, before receiving the follow-up query:

receiving an initial query including the audio query;

attempting to match the audio query using primary matching resources; and

transmitting a report of no match found.

3. The method of claim 1, further including, before receiving the follow-up query:

receiving the audio query;

attempting to match the audio query using primary matching resources; and

transmitting a report of one or more candidate matches, wherein the candidate matches do not correctly identify the audio query;

wherein the recognition hint includes rejection of the candidate matches.

4. The method of claim 1, further including, responsive to the recognition hint, increasing a popularity score of the match, whereby the popularity score indicates that the match should be part of a core group of audio references.

5. The method of claim 1, wherein the recognition hint is an indication that the audio query is from an obscure reference.

6. The method of claim 5, further including, responsive to the recognition hint, matching the audio query against the expanded group of audio references that includes obscure references.

7. The method of claim 1, wherein:

the recognition hint is a partial indication of what audio reference the audio query should match, including one or more of artist, composer, title, lyrics, or genre;

further including using the partial indication of what reference the audio query should match when the retrying of the matching.

8. The method of claim 1, wherein the recognition hint is a sample of background noise.

9. The method of claim 1, wherein the recognition hint is an indication that the matching should be performed against a private group of audio references.

10. The method of claim 9, further including, responsive to the recognition hint, matching the audio query against a private group of audio references.

11. An article of manufacture including a non-transitory computer readable storage medium loaded with instructions that, when executed on a device, cause the device to carry out the actions of:

receiving a follow-up query from a user that requests a retry at matching an initial audio query, the initial audio query comprising a recorded audio sample or audio fingerprints derived from the recorded audio sample, the follow-up query including;

the initial audio query previously submitted by the user or a reference to the initial audio query previously submitted by the user; and

a recognition hint from the user that suggests how to retry matching,

queuing a retry request comprising the follow-up query after an initial audio query is not matched;

processing the queued retry request from the queue based upon one of;

a period of time after which the initial audio query is not matched;

or

the completion of an update of the expanded group of audio references;

wherein processing the queued retry request includes retrying matching the initial audio query using the recognition hint and retrying matching resources that include an expanded group of audio references;

identifying at least one match for the retry request and transmitting a report of the match; and

storing data that correlates at least the follow-up query, the initial audio query or the reference to the initial audio query, and the match after retrying.

12. A device including a processor and memory, the memory loaded with instructions that, when executed on the device, cause the device to carry out the actions of:

receiving a follow-up query from a user that requests a retry at matching an initial audio query, the initial audio query comprising a recorded audio sample or audio fingerprints derived from the recorded audio sample, the follow-up query including;

the initial audio query previously submitted by the user or a reference to the initial audio query previously submitted by the user; and

a recognition hint from the user that suggests how to retry matching,

queuing a retry request comprising the follow-up query after an initial audio query is not matched;

processing the queued retry request from the queue based upon one of;

a period of time after which the initial audio query is not matched;

or

the completion of an update of the expanded group of audio references;

wherein processing the queued retry request includes retrying matching the initial audio query using the recognition hint and retrying matching resources that include an expanded group of audio references;

identifying at least one match for the retry request and transmitting a report of the match; and

storing data that correlates at least the follow-up query, the initial audio query or the reference to the initial audio query, and the match after retrying.