Systems and methods for sound recognition

• US 9,280,598 B2
• Filed: 05/04/2010
• Issued: 03/08/2016
• Est. Priority Date: 05/04/2010
• Status: Active Grant

• Current Assignee: Soundhound AI IP LLC
• Original Assignee: SoundHound Incorporated
• Inventors: Master, Aaron Steven, Stonehocker, Timothy P., Levitt, Benjamin John, Huang, Jun, Mohajer, Keyvan
• Primary Examiner(s): Flanders, Andrew C
• Application Number:
  US12/773,753
• Publication Number:
  US 20100211693A1
• Time in Patent Office: 2,135 Days
• Field of Search: 700/94, 704/231, 704/247, 704/270, 707913-916, 707/758, 707/769, 707/771, 707/705, 707/999.107, 707/999.003, 707/E17.101, 707/E17.102, 707/E17.103, 707/E17.009
• US Class Current: 1/1
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Systems and methods for recognizing sounds are provided herein. User input relating to one or more sounds is received from a computing device. Instructions, which are stored in memory, are executed by a processor to discriminate the one or more sounds, extract music features from the one or more sounds, analyze the music features using one or more databases, and obtain information regarding the music features based on the analysis. Further, information regarding the music features of the one or more sounds may be transmitted to display on the computing device.

Claims

1. A method for recognizing one or more sounds on a server computer, the method comprising:

receiving on the server computer a user input from a computing device, the user input providing a search query comprising the one or more sounds, the one or more sounds having been recorded on the computing device using at least one microphone;

executing instructions stored in memory on the server computer, wherein execution of the instructions by a processor on the server computer;

provides an available set of databases, each database comprising a sound type and associated data, the available set of databases comprising a first database for recorded audio and second database for speech,

discriminates the one or more sounds by classifying the one or more sounds according to a sound type and routing the one or more sounds to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from a first sound recognition application for recorded audio and second sound recognition application for speech,

extracts features from the one or more sounds,

analyzes the extracted features using a database selected from the available set of databases based on the sound type, and

obtains information regarding the extracted features based on the analysis; and

in response to the search query, transmitting for display the information regarding the extracted features of the one or more sounds to the computing device.

2. The method of claim 1, wherein the method discriminates the one or more sounds by classifying the one or more sounds of the search query as at least one of a singing or humming sound, a recorded music sound, and a speech sound.

3. The method of claim 1, wherein the one or more sounds includes any of a song, a song clip, a song snippet, a singing or humming sound, voice, spoken words or any combination thereof.

4. The method of claim 1, wherein information regarding the extracted features of the one or more sounds comprises a song title, a name of an artist, an artist'"'"'s biographical information, identification of similar artists, a link to download a song, a link to download a video related to the song, or any combination thereof.

5. The method of claim 1, further comprising executing instructions stored in memory, wherein execution of the instructions by a processor separates sound artifacts from the sound by noise cancellation.

6. The method of claim 1, further comprising:

executing instructions stored in memory, wherein execution of the instructions by a processor determines whether the extracted features of the one or more sounds are already included in a local music database of the computing device; and

transmitting to display a flag on the display of the computing device, the flag depicting the determination of whether the extracted features of the one or more sounds are already included in a local music database of the computing device.

7. The method of claim 1, further comprising:

executing instructions stored in memory, wherein execution of the instructions by a processor provides relevant content relating to the extracted features; and

transmitting to display a premium row on the display of the computing device, the premium row depicting the relevant content relating to the extracted features.

8. A method for recognizing one or more sounds on a server computer, the method comprising:

receiving on the server computer a user input from a computing device, the user input providing a search query comprising the one or more sounds, the one or more sounds having been recorded on the computing device using at least one microphone;

executing instructions stored in memory on the server computer, wherein execution of the instructions by a processor on the server computer;

provides an available set of sound recognition applications, the available set of sound recognition applications comprising a first sound recognition application for recorded audio and second sound recognition application for speech;

provides an available set of databases, each database comprising a sound type and associated data, the available set of databases comprising a first database for recorded audio and second database for speech;

discriminates the one or more sounds by classifying the one or more sounds according to a sound type and routing the one or more sounds to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from the available set of sound recognition applications,

analyzes a databases for searching based on the sound type using data included in the discriminated one or more sounds, the database being selected from the available set of databases, and

obtains information regarding the discriminated one or more sounds based on the analysis; and

in response to the search query, transmitting for display the information regarding the discriminated one or more sounds to the display of the computing device.

26. The method of claim 8, the available set of sound recognition applications further comprising a third sound recognition application for recorded television programs, the group of databases further comprising a third database for recorded television programs.

27. The method of claim 8, the available set of sound recognition applications further comprising a third sound recognition application for recorded audio, the group of databases further comprising a third database for recorded audio.

9. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for recognizing one or more sounds, the method comprising:

receiving on a server computer a user input from a computing device, the user input providing a search query comprising the one or more sounds, the one or more sounds having been recorded on the computing device using at least one microphone;

providing an available set of sound recognition applications on the server computer, the available set of sound recognition applications comprising a first sound recognition application for recorded audio and second sound recognition application for speech;

providing an available set of databases on the server computer, each database comprising a sound type and associated data, the available set of databases comprising a first database for recorded audio and second database for speech;

discriminating the one or more sounds on the server computer by classifying the sound according to a sound type and routing the sound to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from of the available set of sound recognition applications;

analyzing a database on the server computer based on the sound type using data included in the discriminated sound, the database being selected from the available set of databases;

obtaining information regarding the discriminated one or more sounds, from the database, based on the analysis; and

in response to the search query, transmitting for display the information regarding the discriminated one or more sounds to the computing device.

10. The non-transitory computer readable storage medium of claim 9, wherein discriminating the one or more sounds further comprises classifying the one or more sounds of the search query as at least one of a singing or humming sound, a recorded music sound, and a speech sound.

11. The non-transitory computer readable storage medium of claim 9, wherein the one or more sounds includes any of a song, a song clip, a song snippet, a humming sound, voice, or any combination thereof.

12. The non-transitory computer readable storage medium of claim 9, wherein information regarding the discriminated one or more sounds comprises a song title, a name of an artist, an artist'"'"'s biographical information, identification of similar artists, a link to download a song, a link to download a video related to the song, or any combination thereof.

13. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises executing instructions stored in memory, wherein execution of the instructions by a processor separates sound artifacts from the one or more sounds by noise cancellation.

14. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises:

executing instructions stored in memory, wherein execution of the instructions by a processor determines whether the information regarding the discriminated one or more sounds is already included in a local music database of the computing device; and

transmitting to display a flag on the display of the computing device, the flag depicting the determination of whether the information regarding the discriminated one or more sounds is already included in a local music database of the computing device.

15. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises:

executing instructions stored in memory, wherein execution of the instructions by a processor provides relevant content relating to the discriminated one or more sounds; and

transmitting to display a premium row on the display of the computing device, the premium row depicting the relevant content relating to the discriminated one or more sounds.

16. A system for recognizing one or more sounds, comprising:

a computing device having a display;

one or more databases coupled to the computing device;

a server having a music processor, the server being coupled to the computing device and the one or more databases, the music processor configured to execute instructions stored in memory to;

receive a user input from the computing device, the user input providing a search query comprising the one or more sounds, the one or more sounds having been recorded on the computing device using at least one microphone,

provide an available set of sound recognition applications, the available set of sound recognition applications comprising a first sound recognition application for recorded audio and second sound recognition application for speech;

provide an available set of databases, each database comprising a sound type and associated data, the available set of databases comprising a first database for recorded audio and second database for speech;

discriminate the one or more sounds by classifying the one or more sounds according to a sound type and routing the one or more sounds to a corresponding sound recognition application for processing based on the sound type, the corresponding sound recognition application being selected from the available set of sound recognition applications,

analyze a database based on the sound type using data included in the discriminated one or more sounds, the database being selected from the available set of databases, and

obtain information regarding the discriminated one or more sounds, from the database, based on the analysis; and

in response to the search query, transmit for display the information regarding the discriminated one or more sounds to the computing device.

17. The system of claim 16, wherein the processor is further configured to classify the one or more sounds of the search query as at least one of a singing or humming sound, a recorded music sound, and a speech sound.

18. The system of claim 16, wherein the one or more sounds includes a song, a song clip, a song snippet, a humming sound, voice, or any combination thereof.

19. The system of claim 16, wherein information regarding the discriminated one or more sounds comprises a song title, a name of an artist, an artist'"'"'s biographical information, identification of similar artists, a link to download or purchase a song, a link to download or purchase a video related to the song, or any combination thereof.

20. The system of claim 16, wherein the processor is further configured to execute instructions stored in memory to separate sound artifacts from the one or more sounds by noise cancellation.

21. The system of claim 16, wherein the processor is further configured to execute instructions stored in memory to determine whether the information regarding the discriminated one or more sounds is already included in a local music database of the computing device, and the method further comprises transmitting to display a flag on the display of the computing device, the flag depicting the determination of whether the information regarding the discriminated one or more sounds is already included in a local music database of the computing device.

22. The system of claim 16, wherein the processor is further configured to execute instructions stored in memory to provide relevant content relating the discriminated one or more sounds;

and the method further comprises transmitting to display a premium row on the display of the computing device, the premium row depicting the relevant content relating to the discriminated one or more sounds.

23. A system for recognizing one or more sounds, comprising:

a computing device having a display;

three databases coupled to a server computer,

the server computer having a music processor, the music processor having three sound recognition applications coupled to the three databases, the server computer being coupled to the computing device and the three databases, the music processor configured to execute instructions stored in memory to;

receive a user input from the computing device, the user input providing a search query comprising the one or more sounds, the one or more sounds having been recorded on the computing device using at least one microphone,

discriminates the one or more sounds by classifying the one or more sounds according to a sound type and routing the one or more sounds to a corresponding sound recognition application for processing based on the sound type, corresponding sound recognition application being selected from a group of sound recognition applications comprising at least two of a first sound recognition application for singing or humming sounds, a second sound recognition application for recorded music, and a third sound recognition application for speech,

extract music features from the one or more sounds,

analyzes and searches the music features using a database for searching based on the sound type using data included in the discriminated one or more sounds, the database being selected from a group of databases comprising at least two of a first database for singing or humming sounds, a second database for recorded music, and a third database for speech, and

obtains information regarding the music features, from the one or more databases, based on the analysis, searching and extraction; and

in response to the search query transmitting for display the information regarding the music features of the one or more sounds to the display of the computing device.

28. The system of claim 23, the group of sound recognition applications further comprising a fourth sound recognition application for recorded television programs, the group of databases further comprising a fourth database for recorded television programs.

24. A computing device comprising:

a display displaying a flag and a local music database for storing one or more music features, the flag generated by a server coupled to the computing device, the flag visually depicting a determination of whether music features of one or more sounds are already stored in the local music database of the computing device, the flag being transmitted by the server to display on the display of the computing device, the one or more sounds having been;

recorded on the computing device using at least one microphone;

classified according to a sound type, the sound types comprising both a recorded audio sound type and a speech sound type; and

routed to a corresponding sound recognition application for processing based on the sound type, the recorded audio sound type being routed to a first sound recognition application for recorded audio and the speech sound type being routed to a second sound recognition application for speech.

25. A computing device comprising:

a display displaying a premium row, the premium row generated by a server coupled to the computing device, the premium row depicting relevant content relating to music features of one or more sounds, the premium row being transmitted by the server to display on the display of the computing device, the one or more sounds having been;

recorded on the computing device using at least one microphone;

classified according to a sound type the sound types comprising both a recorded audio sound type and a speech sound type; and

routed to a corresponding sound recognition application for processing based on the sound type, the recorded audio sound type being routed to a first sound recognition application for recorded audio and the speech sound type being routed to a second sound recognition application for speech.