Displaying information related to content playing on a device

• US 9,838,759 B2
• Filed: 06/20/2014
• Issued: 12/05/2017
• Est. Priority Date: 06/20/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kirmse, Andrew, Maarse, Jan-Willem, Volovich, Yaroslav, Oztaskent, Ant, Trollope, Ingrid McAulay, Foryciarz, Jurek
• Primary Examiner(s): Lee, Ting
• Application Number:
  US14/311,218
• Publication Number:
  US 20150370435A1
• Time in Patent Office: 1,264 Days
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/4387 by the use of playlists
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/60 of audio data
• CPC Class Codes: G06F 16/63 Querying
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/25866 Management of end-user data customer care in data networks H04L41/5077
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/25891 : being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2665 : Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/42203 : sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/4884 : for displaying subtitles
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/6582 : Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/8133 : specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8352 : involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A computer-implemented method includes: detecting whether a user is watching media content; after detecting that the user is watching media, presenting on a user device a first affordance providing a first user-selectable election to receive information on entities relevant to the media content; in response to user selection of the election: sampling at the user device program information from the media content including one or more of audio signals and subtitles, and sending the program information to a server, which identifies the media content and generates one or more second user-selectable user elections for the identified media content and sends to the user device one or more second affordances providing the second user-selectable elections; displaying the second affordances on the user device; and in response to user selection of one of the second affordances, displaying on the user device information on a respective entity relevant to the media content.

Claims

1. A method comprising:

at an electronic device with a display;

obtaining one or more media consumption status signals with respect to a second electronic device in proximity to the electronic device;

based on the obtained media consumption status signals, determining whether a user is likely to he consuming media playing at the second electronic device;

in accordance with a determination that the user is likely to be consuming media playing at the second electronic device, displaying on the display at the electronic device a first affordance providina the user a first user-selectable election, wherein the first affordance includes a prompt that invites the user to receive information on entities relevant to media playing at the second electronic device;

receiving a selection of the first, user-selectable election by the user;

after and in response to the user selection of the first user-selectable election;

sampling at the electronic device program information from a media content item of the media playing at the second electronic device, the program information including one or more of audio signals and subtitles from the media content item, and

sending the program information to a server, wherein in accordance with the program information the server;

identifies the media content item,

generates one or more second user-selectable user elections for the identified media content item, and

sends to the electronic device one or more second affordances providing the second user-selectable elections;

displaying the second affordances providing the second user-selectable elections on the display at the electronic device, wherein the second affordances providing the second user-selectable elections indicates entities relevant to the identified media content item;

receiving a user selection of a first one of the second user-selectable elections; and

in response to the user selection of the first one of the second user-selectable elections, displaying on the display at the electronic device information on a respective entity relevant to the identified media content item.

2. The method of claim 1, wherein the media content item is selected from:

a live media content item or a time shifted media content item.

3. The method of claim 1, wherein the server identifies a genre of the identified media content item, and generates the second user-selectable elections based on the genre of the identified media content item.

4. The method of claim 1, wherein the second user-selectable elections correspond to respective entities relevant to the identified media content item, and displaying on the display at the electronic device information on a respective entity relevant to the identified media content item comprises displaying information on a respective entity corresponding to the first one of the second user-selectable elections.

5. The method of claim 4, wherein the information on the respective entity comprises factual information related to the respective entity.

6. The method of claim 4, wherein the information on the respective entity comprises search results for a search having the respective entity as the query.

7. The method of claim 1, wherein the electronic device is selected from the group consisting of:

a computer, a mobile phone, a smartphone, a tablet device, a multimedia player device, and a game player device.

8. The method of claim 1, wherein determining whether a user is likely to be consuming media playing at the second electronic device comprises:

determining a power status of the second electronic device; and

in accordance with a determination that the power status of the second electronic device is a powered-on status, determining that the user is likely to be consuming media playing at the second electronic device.

9. The method of claim 8, wherein obtaining one or more media consumption status signals with respect to a second electronic device in proximity to the electronic device comprises:

obtaining a presence message from the second electronic device.

10. The method of claim 8, wherein determining a power status of the second electronic device further comprises:

determining whether the second electronic device is in an active status.

11. The method of claim 8, wherein obtaining one or more media consumption status signals with respect to a second electronic device in proximity to the electronic device comprises:

obtaining one or more of;

a current location of the electronic device, a current time, and user history data with respect to media interest.

12. An electronic device, comprising:

a display;

memory;

one or more processors; and

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for;

obtaining, one or more media consumption status signals with respect to a second electronic device in proximity to the electronic device;

based on the obtained media consumption status signals, determining whether a user is likely to be consuming media playing at the second electronic device;

in accordance with a determination that the user is likely to be consuming media playing at the second electronic device, displaying on the display at the electronic device a first affordance providing the user a first user-selectable election, wherein the first affordance includes a prompt that invites the user to receive information on entities relevant to media playing at the second electronic device;

receiving a selection of the first user-selectable election by the user;

after and in response to the user selection of the first user-selectable election;

sampling at the electronic device program information from a media content item of the media playing at the second electronic device, the program information including one or more of audio signals and subtitles from the media content item, and

sending the program information to a server, wherein in accordance with the program information the server;

identifies the media content item,

generates one or more second user-selectable user elections for the identified media content item, and

sends to the electronic device one or more second affordances providing the second user-selectable elections;

displaying the second affordances providing the second user-selectable elections on the display at the electronic device, wherein the second affordances providing the second user-selectable elections indicates entities relevant to the identified media content item;

receiving a user selection of a first one of the second user-selectable elections; and

in response to the user selection of the first one of the second user-selectable elections, displaying on the display at the electronic device information on a respective entity relevant to the identified media content item.

13. The device of claim 12, wherein the server identifies a genre of the identified media content item, and generates the second user-selectable elections based on the genre of the identified media content item.

14. The device of claim 12, wherein the second user-selectable elections correspond to respective entities relevant to the identified media content item, and

the device comprises instructions for displaying information on a respective entity corresponding to the first one of the second user-selectable elections.

15. The device of claim 12, comprising instructions for:

determining a power status of the second electronic device; and

in accordance with a determination that the power status of the second electronic device is a powered-on status, determining that the user is likely to be consuming media playing at the second electronic device.

16. The device of claim 15, comprising instructions for:

obtaining a presence message from the second electronic device.

17. A non-transitory computer readable storage medium storing one or more programs to be executed by an electronic device with a display, memory and one or more processors, the one or more programs comprising:

instructions for obtaining one or more media consumption status signals with respect to a second electronic device in proximity to the electronic device;

instructions for determining, based on the obtained media consumption status signals, whether a user is likely to be consuming media playing at the second electronic device;

instructions for, in accordance with a determination that the user is likely to be consuming media playing at the second electronic device, displaying on the display at the electronic device a first affordance providing the user a first user-selectable election, wherein the first affordance includes a prompt that invites the user to receive information on entities relevant to media playing at the second electronic device;

instructions for receiving a selection of the first user-selectable election by the user;

instruction, responsive to the user selection of the first user-selectable election, for, after the user selection of the first user-selectable election;

sampling at the electronic device program information from a media content item of the media playing at the second electronic device, the program information including one or more of audio signals and subtitles from the media content item, and

sending the program information to a server, wherein in accordance with the program information the server;

identifies the media content item,

generates one or more second user-selectable user elections for the identified media content item, and

sends to the electronic device one or more second affordances providing the second user-selectable elections;

instructions for displaying the second affordances providing the second user-selectable elections on the display at the electronic device, wherein the second affordances providing the second user-selectable elections indicates entities relevant to the identified media content item;

instructions for receiving a user selection of a first one of the second user-selectable elections; and

instructions, responsive to the user selection of the first one of the second user-selectable elections, for displaying on the display at the electronic device information on a respective entity relevant to the identified media content item.

18. The computer readable storage medium of claim 17, wherein the server identifies a genre of the identified media content item, and generates the second user-selectable elections based on the genre of the identified media content item.

19. The computer readable storage medium of claim 17, wherein the second user-selectable elections correspond to respective entities relevant to the identified media content item, and

the computer readable storage medium comprises instructions for displaying information on a respective entity corresponding to the first one of the second user-selectable elections.

20. The computer readable storage medium of claim 17, comprising instructions for:

determining a power status of the second electronic device; and

in accordance with a determination that the power status of the second electronic device is a powered-on status, determining that the user is likely to be consuming media playing at the second electronic device.