Methods and devices for clarifying audible video content

• US 10,638,203 B2
• Filed: 01/04/2019
• Issued: 04/28/2020
• Est. Priority Date: 06/20/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Trollope, Ingrid McAulay, Oztaskent, Ant, Volovich, Yaroslav
• Primary Examiner(s): Phantana-angkool, David
• Application Number:
  US16/240,608
• Publication Number:
  US 20190141413A1
• Time in Patent Office: 480 Days
• Field of Search: 715719
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

The various implementations described herein include methods, devices, and systems for clarifying media content. In one aspect, a method is performed at a client device that includes a microphone, memory, and one or more processors. The method includes: (1) receiving, via the microphone, audio content of a media content item playing on a second client device in proximity to the client device; (2) receiving, via the microphone, a verbal query from a user of the client device to clarify a portion of the media content item; (3) sending a request to a remote server system; (4) receiving from the remote server system information responsive to the verbal user query for the portion of the media content item; and (5) presenting the information to the user.

Claims

1. A method, comprising:

at a client device having a microphone, one or more processors, and memory;

receiving, via the microphone, audio content of a media content item playing on a second client device in proximity to the client device;

receiving, via the microphone, a verbal query from a user of the client device to clarify a portion of the media content item;

in response to receiving the verbal query sending a request to a remote server system, the request including at least a portion of the audio content and the verbal query;

in response to sending the request, receiving from the remote server system information responsive to the verbal query for the portion of the media content item; and

presenting the information to the user.

2. The method of claim 1, wherein the information comprises song lyrics.

3. The method of claim 1, wherein the information comprises a transcription of speech.

4. The method of claim 1, wherein the portion of the media content item corresponds to a plurality of persons speaking;

and

wherein the verbal query comprises a request to clarify the speech of a particular person of the plurality of persons.

5. The method of claim 1, wherein the information comprises subtitle data associated with the media content item.

6. The method of claim 1, wherein the information comprises information obtained from an online document.

7. The method of claim 1, wherein the portion of the media content item includes speech in a first language;

and

wherein the information comprises a translation of the speech to a second language.

8. The method of claim 1, wherein the verbal query comprises a request to translate content within the portion of the media content;

and

wherein the information comprises a translation of the content.

9. The method of claim 1, wherein the media content item is live television content.

10. The method of claim 1, wherein the media content item is previously broadcast television content.

11. The method of claim 1, wherein the media content item is recorded content.

12. The method of claim 1, wherein the media content item is streaming content.

13. An electronic device, comprising:

a microphone;

memory;

one or more processors; and

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for;

receiving, via the microphone, audio content of a media content item playing on a second client device in proximity to the electronic device;

receiving, via the microphone, a verbal query from a user of the electronic device to clarify a portion of the media content item;

in response to receiving the verbal query sending a request to a remote server system, the request including at least a portion of the audio content and the verbal query;

in response to sending the request, receiving from the remote server system information responsive to the verbal query for the portion of the media content item; and

presenting the information to the user.

14. The electronic device of claim 13, further comprising a display, wherein the information is presented to the user via the display.

15. The electronic device of claim 13, further comprising a speaker, wherein the information is presented to the user via the speaker.

16. The electronic device of claim 13, wherein the portion of the media content item corresponds to a plurality of persons speaking;

and

wherein the verbal query comprises a request to clarify the speech of a particular person of the plurality of persons.

17. The electronic device of claim 13, wherein the verbal query comprises a request to translate content within the portion of the media content;

and

wherein the information comprises a translation of the content.

18. A non-transitory computer-readable storage medium storing one or more programs to be executed by an electronic device with a microphone, memory, and one or more processors, the one or more programs comprising instructions for:

receiving, via the microphone, audio content of a media content item playing on a second client device in proximity to the electronic device;

receiving, via the microphone, a verbal query from a user of the electronic device to clarify a portion of the media content item;

in response to receiving the verbal query sending a request to a remote server system, the request including at least a portion of the audio content and the verbal query;

in response to sending the request, receiving from the remote server system information responsive to the verbal query for the portion of the media content item; and

presenting the information to the user.

19. The non-transitory computer-readable storage medium of claim 18, wherein the portion of the media content item corresponds to a plurality of persons speaking;

and

wherein the verbal query comprises a request to clarify the speech of a particular person of the plurality of persons.

20. The non-transitory computer-readable storage medium of claim 18, wherein the verbal query comprises a request to translate content within the portion of the media content;

and

wherein the information comprises a translation of the content.