Systems and methods for sound recognition

• US 8,688,253 B2
• Filed: 05/24/2012
• Issued: 04/01/2014
• Est. Priority Date: 05/04/2010
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Master, Aaron Steven, Stonehocker, Timothy P., Levitt, Benjamin John, Huang, Jun, Mohajer, Keyvan
• Primary Examiner(s): Flanders, Andrew C
• Application Number:
  US13/480,445
• Publication Number:
  US 20120232683A1
• Time in Patent Office: 677 Days
• Field of Search: 700/94, 704/231, 704/247, 704/270, 707913-916, 707/E17.101, 707/E17.102, 707/E17.103, 707/E17.009, 707/999.107, 707/999.003, 707/758, 707/769, 707/771, 707/705
• US Class Current: 700/94
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Systems and methods for recognizing sounds are provided herein. User input relating to a sound is received from a computing device. Instructions, which are stored in memory, are executed by a processor to discriminate the sound, analyze one or more databases using data included in the discriminated sound, the one or more databases being selected based upon the discrimination of the sound. Additionally, the processor may be executed to obtain information regarding the discriminated sound, from the one or more databases, based on the analysis, and in response to the search query, transmitting for display the information regarding the discriminated sound to the computing device.

Claims

1. A method for recognizing a sound on a server computer, the method comprising:

receiving on the server computer a user input from a computing device, the user input providing a search query comprising the sound;

executing instructions stored in memory on the server computer, wherein execution of the instructions by a processor on the server computer;

provides an available set of sound recognition applications, the available set of sound recognition applications comprising at least two sound recognition applications selected from a group of sound recognition applications at least comprising a first sound recognition application for singing or humming sounds, a second sound recognition application for recorded music, and a third sound recognition application for speech,

provides an available set of databases, each database comprising a sound type and associated data, the available set of databases comprising at least two databases selected from a group of databases at least comprising a first database for singing or humming sounds, a second database for recorded music, and a third database for speech,

discriminates the sound by classifying the sound according to a sound type and routing the sound to a corresponding sound recognition applications for processing based on the sound type, the corresponding sound recognition application being selected from the available set of sound recognition applications,

analyzes a database for searching based on the sound type using data included in the discriminated sound, the database being selected from of the available set of databases, and

obtains information regarding the discriminated sound based on the analysis; and

in response to the search query, transmitting for display information regarding the discriminated sound to a display of the computing device.

16. The method of claim 1, the group of sound recognition applications further comprising a fourth sound recognition application for recorded television programs, the group of databases further comprising a fourth database for recorded television programs.

17. The method of claim 1, the group of sound recognition applications further comprising a fourth sound recognition application for recorded audio, the group of databases further comprising a fourth database for recorded audio.

2. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for recognizing a sound, the method comprising:

receiving on a server computer a user input from a computing device, the user input providing a search query comprising the sound;

providing an available set of sound recognition applications on the server computer, the available set of sound recognition applications comprising at least two sound recognition applications selected from a group of sound recognition applications at least comprising a first sound recognition application for singing or humming sounds, a second sound recognition application for recorded music, and a third sound recognition application for speech;

providing an available set of databases on the server computer, each database comprising a sound type and associated data, the available set of databases comprising at least two databases selected from a group of databases at least comprising a first database for singing or humming sounds, a second database for recorded music, and a third database for speech;

discriminating the sound on the server computer by classifying the sound according to a sound type and routing the sound to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from the available set of sound recognition applications;

analyzing a database on the server computer based on the sound type using data included in the discriminated sound, the database being selected from the available set of databases;

obtaining information regarding the discriminated sound, from the one or more databases, based on the analysis; and

in response to the search query, transmitting for display the information regarding the discriminated sound to the computing device.

3. The non-transitory computer readable storage medium of claim 2, wherein the method further comprises utilizing load balancing routers to distribute workload to one or more computing resources.

4. The non-transitory computer readable storage medium of claim 2, wherein information regarding the discriminated sound comprises a song title, a name of an artist, an artist'"'"'s biographical information, identification of similar artists, a link to download a song, a link to download a video related to the song, or any combination thereof.

5. The non-transitory computer readable storage medium of claim 2, wherein the method further comprises executing instructions stored in memory, wherein execution of the instructions by the processor separates sound artifacts from the sound by noise cancellation.

6. The non-transitory computer readable storage medium of claim 2, wherein the method further comprises:

executing instructions stored in memory, wherein execution of the instructions by the processor determines whether the information regarding the discriminated sound is already included in a local music database of the computing device; and

transmitting to display a flag on the display of the computing device, the flag depicting the determination of whether the information regarding the discriminated sound is already included in the local music database of the computing device.

7. The non-transitory computer readable storage medium of claim 2, wherein the method further comprises:

executing instructions stored in memory, wherein execution of the instructions by a processor provides relevant content relating to the information regarding the discriminated sound;

and transmitting to display a premium row on the display of the computing device, the premium row depicting the relevant content relating to the information regarding the discriminated sound.

8. A system for recognizing a sound, comprising:

a computing device having a display;

one or more databases coupled to the computing device;

a server having a music processor, the server being coupled to the computing device and the one or more databases, the music processor configured to execute instructions stored in memory to;

receive a user input from the computing device, the user input providing a search query comprising the sound;

provide an available set of sound recognition applications on a server, the available set of sound recognition applications comprising at least two sound recognition applications selected from a group of sound recognition applications at least comprising a first sound recognition application for singing or humming sounds, a second sound recognition application for recorded music, and a third sound recognition application for speech;

provide an available set of databases on the server, each database comprising a sound type and associated data, the available set of databases comprising at least two databases selected from a group of databases at least comprising a first database for singing or humming sounds, a second database for recorded music, and a third database for speech;

discriminate the sound by classifying the sound according to a sound type and routing the sound to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from the available set of sound recognition applications;

analyze a database based on the sound type using data included in the discriminated sound, the database being selected from the available set of databases;

obtain information regarding the discriminated sound, from the one or more databases, based on the analysis; and

in response to the search query, transmit for display the information regarding the discriminated sound to the computing device.

9. The system of claim 8, wherein the processor is further configured to classify the sound of the search query as a singing or humming sound, recorded music, or speech.

10. The system of claim 8, wherein the sound includes a song, a song clip, a song snippet, a humming sound, voice, or any combination thereof.

11. The system of claim 8, wherein the processor is further configured to utilize load balancing routers to distribute workload to one or more computing resources.

12. The system of claim 8, wherein information regarding the discriminated sound comprises a song title, a name of an artist, an artist'"'"'s biographical information, identification of similar artists, a link to download or purchase a song, a link to download or purchase a video related to the song, or any combination thereof.

13. The system of claim 8, wherein the processor is further configured to execute instructions stored in memory to separate sound artifacts from the sound by noise cancellation.

14. The system of claim 8, wherein the processor is further configured to execute instructions stored in memory to determine whether the information regarding the discriminated sound is already included in a local music database of the computing device, and display a flag on the display of the computing device, the flag depicting the determination of whether the information regarding the discriminated sound is already included in a local music database of the computing device.

15. The system of claim 8, wherein the processor is further configured to execute instructions stored in memory to provide relevant content relating to the information regarding the discriminated sound;

and display a premium row on the display of the computing device, the premium row depicting the relevant content relating to the information regarding the discriminated sound.

18. The system of claim 8, the group of sound recognition applications further comprising a fourth sound recognition application for recorded television programs, the group of databases further comprising a fourth database for recorded television programs.

19. The system of claim 8, the group of sound recognition applications further comprising a fourth sound recognition application for recorded audio, the group of databases further comprising a fourth database for recorded audio.