Displaying information related to spoken dialogue in content playing on a device

• US 9,946,769 B2
• Filed: 06/20/2014
• Issued: 04/17/2018
• Est. Priority Date: 06/20/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Oztaskent, Ant, Volovich, Yaroslav, Trollope, Ingrid McAulay
• Primary Examiner(s): Lee, Ting
• Application Number:
  US14/311,204
• Publication Number:
  US 20150370864A1
• Time in Patent Office: 1,397 Days
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/60 of audio data
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program

Abstract

A method at a server includes identifying a media content item currently being presented in proximity to a first user; identifying, in an entities repository, one or more first quotations associated with the media content item, where the first quotations are determined to be popular in accordance with one or more popularity criteria; sending, to a client device associated with the first user, the first quotations and one or more affordances associated with the first quotations; receiving selection of a first affordance of the affordances, the first affordance associated with a respective quotation of the first quotations; and in accordance with the selection of the first affordance, performing an operation associated with the respective quotation.

Claims

1. A method, comprising:

at a server;

detecting presentation of a media content item being played at a first device in proximity to a second device associated with a first user, including receiving from the second device content information derived from sound output from the presentation of the media content item at the first device captured at the second device;

identifying, based on the received content information, the media content item being played at the first device;

maintaining an entities repository of entities associated with a plurality of media content items, wherein each of the entities corresponds to a respective distinct noun associated with one or more of the media content items, the entities including specific quotations, and the entities repository includes references between related entities;

identifying, in the entities repository, a plurality of first quotations associated with the media content item, wherein the first quotations are determined to be popular in an aggregate, in accordance with a plurality of popularity criteria;

sending to the second device the first quotations and, for each of the first quotations, a menu affordance and a plurality of quotation operation affordances included within the menu affordance, wherein each quotation operation affordance corresponds to an operation with respect to the quotation, and wherein the quotation operation affordances for each of the first quotations are displayed in response to user activation of the menu affordance for each of the first quotations;

receiving selection by the first user of a first affordance of the quotation operation affordances, for the first one of the first quotations; and

in accordance with the selection of the first affordance, performing a first operation, corresponding to the first affordance, with respect to the first one of the first quotations.

2. The method of claim 1, wherein the quotation operation affordances for a quotation provide a plurality of options for interacting with the quotation.

3. The method of claim 1, wherein performing an operation, corresponding to the first affordance, with respect to the first one of the first quotations comprises any of:

sending to the second device information related to the first one of the first quotations for display at the second device;

sharing the first one of the first quotations;

sending to the second device a media snippet that includes the first one of the first quotations for display at the second device;

initiating a search having the first one of the first quotations as a search query;

initiating a search for an entity related to the first one of the first quotations;

providing to the second device a text entry interface configured to receive input of a comment on the first one of the first quotations;

or sharing an indication of interest in the media content item, the indication of interest including the first one of the first quotations as a caption.

4. The method of claim 1, wherein the popularity criteria comprise one or more of:

a search query volume of a respective quotation by the first user, an aggregate search query volume of the respective quotation, a number of mentions of the respective quotation in social networks, and a number of documents of predefined categories that include the respective quotation.

5. The method of claim 1, wherein the popularity criteria comprise one or more realtime criteria.

6. The method of claim 5, wherein the first quotations are determined to be popular in realtime in accordance with the popularity criteria.

7. The method of claim 1, wherein identifying the media content item being played at the first device in proximity to the second device comprises determining a portion of the media content item being played at the first device.

8. The method of claim 7, wherein the first quotations are, within the media content item, proximate to the portion of the media content item being played at the first device.

9. The method of claim 7, further comprising:

identifying one or more second quotations associated with a portion of the media content item succeeding the portion being played at the first device;

detecting playing of the succeeding portion at the first device; and

in accordance with the detection of the playing of the succeeding portion, sending to the second device the second quotations and, for each of the second quotations, a menu affordance and a plurality of quotation operation affordances.

10. A method, comprising:

at a client device;

detecting presentation of a media content item being played at a second client device in proximity to the client device, including receiving content information derived from sound output from the presentation of the media content item at the second client device captured at the client device;

transmitting to a server the content information;

receiving from the server and displaying a plurality of popular quotations associated with the media content item and, for each of the popular quotations, a menu affordance, wherein the popular quotations are identified from an entities repository of entities associated with a plurality of media content items, each of the entities corresponds to a respective distinct noun associated with one or more of the media content items, the entities include specific quotations and the entities repository includes references between related entities, the popular quotations are determined to be popular in an aggregate, and wherein the menu affordance provides a plurality of options for interacting with a displayed quotation;

receiving user activation of the menu affordance for a first one of the displayed quotations;

in accordance with activation of the menu affordance for the first one of the displayed quotations, displaying a plurality of quotation operation affordances corresponding to the plurality of options for interacting with the first one of the displayed quotations;

receiving user activation of a first one of the quotation operation affordances; and

in accordance with the activation of the first one of the quotation operation affordances, performing a first operation with respect to the first one of the displayed quotations.

11. The method of claim 10, wherein performing an operation with respect to the first one of the displayed quotations comprises any of:

displaying information related to the first one of the displayed quotations;

sharing the first one of the displayed quotations;

displaying a media snippet that includes the with respect to the first one of the displayed quotations;

initiating a search having the with respect to the first one of the displayed quotations as a search query;

initiating a search for an entity related to the with respect to the first one of the displayed quotations;

displaying a text entry interface configured to receive input of a comment on the with respect to the first one of the displayed quotations;

or sharing an indication of interest in the media content item, the indication of interest including the with respect to the first one of the displayed quotations as a caption.