Identifying media content

• US 9,031,840 B2
• Filed: 12/27/2013
• Issued: 05/12/2015
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Postelnicu, Gheorghe
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US14/142,042
• Publication Number:
  US 20140114659A1
• Time in Patent Office: 501 Days
• Field of Search: 704/231, 704/270, 704/275, 704/277, 704/257, 704/251, 704/240, 704/235, 704/233, 704/226, 704/222, 704/205, 725/133, 705/14.73, 379/114.14, 715/206
• US Class Current: 704/235
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10H 2210/031 Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal neural networks for electrophonic musical instruments or musical processing G10H2250/311
• CPC Class Codes: G10H 2240/141 Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 25/54 for retrieval

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving (i) audio data that encodes a spoken natural language query, and (ii) environmental audio data, obtaining a transcription of the spoken natural language query, determining a particular content type associated with one or more keywords in the transcription, providing at least a portion of the environmental audio data to a content recognition engine, and identifying a content item that has been output by the content recognition engine, and that matches the particular content type.

Claims

1. A computer-implemented method comprising:

receiving, by one or more processors, audio data that encodes (i) a spoken natural language query, and (ii) music;

determining, by the one or more processors, that one or more keywords in a transcription of the spoken natural language query are associated with a movie content type; and

based on determining that the one or more keywords in the transcription of the spoken natural query are associated with the movie content type, identifying, by the one or more processors, a movie content item that is recognized using the music.

2. The computer-implemented method of claim 1, wherein receiving the audio data further comprises receiving the audio data from a mobile computing device.

3. The computer-implemented method of claim 2, wherein receiving the audio data further comprises receiving environmental audio data associated with the mobile computing device.

4. The computer-implemented method of claim 1, wherein the audio data that encodes the music is generated within a predetermined period of time before receiving the audio data that encodes the spoken natural language query.

5. The computer-implemented method of claim 1, wherein determining that the one or more keywords in the transcription of the spoken natural language query are associated with the movie content type further includes identifying the one or more keywords using one or more databases that map at least one of the keywords to the movie content type.

6. A computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by one or more processors, (i) an image or a video, and (ii) audio data that encodes a spoken natural language query;

determining, by the one or more processors, that one or more keywords in a transcription of the spoken natural language query are associated with a music content type; and

based on determining that the one or more keywords in the transcription of the spoken natural query are associated with the music content type, identifying, by the one or more processors, a music content item that is recognized using the image or the video.

7. The computer-readable medium of claim 6, wherein receiving (i) the image or the video and (ii) the data that encodes the spoken natural language query further comprises receiving the (i) the image or the video and the (ii) the audio data that encodes the spoken natural language query from a mobile computing device.

8. The computer-readable medium of claim 6, wherein the image or the video is generated within a predetermined period of time before receiving the audio data that encodes the spoken natural language query.

9. The computer-readable medium of claim 6, wherein determining that the one or more keywords in the transcription of the spoken natural language query are associated with the music content type further includes identifying the one or more keywords using one or more databases that map at least one of the keywords to the music content type.

10. The computer-readable medium of claim 7, wherein receiving the image or the video further comprises receiving an environmental image or video associated with the mobile computing device.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by one or more processors, audio data that encodes (i) a spoken natural language query and (ii) music;

determining, by the one or more processors, that one or more keywords in a transcription of the spoken natural language query are associated with a movie content type; and

based on determining that the one or more keywords in the transcription of the spoken natural query are associated with the movie content type, identifying, by the one or more processors, a movie content item that is recognized using the music.

12. The system of claim 11, wherein receiving the audio data further comprises receiving the audio data from a mobile computing device.

13. The system of claim 11, wherein the audio data that encodes the music is generated within a predetermined period of time before receiving the audio data that encodes the spoken natural language query.

14. The system of claim 11, wherein determining that the one or more keywords in the transcription of the spoken natural language query are associated with the movie content type further includes identifying the one or more keywords using one or more databases that map at least one of the keywords to the movie content type.

15. The system of claim 12, wherein receiving the audio data further comprises receiving environmental audio data associated with the mobile computing device.