Contextual, two way remote control

• US 10,129,324 B2
• Filed: 12/27/2012
• Issued: 11/13/2018
• Est. Priority Date: 07/03/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Lu, Jeff Ting Yann
• Primary Examiner(s): McCormack, Thomas
• Application Number:
  US13/728,980
• Publication Number:
  US 20150195474A1
• Time in Patent Office: 2,147 Days
• Field of Search: 348734, 340 1226, 709223-226
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 23/04 using light waves, e.g. infrared
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Methods and systems for allowing interaction between devices are described. At a first device, an application is executed. The application operates interactively with a remote control application executing on a second device. One or more information items are transmitted to the second device for display as one or more selectable display items. Each respective information item is associated with an action definition. The action definition includes a command to be sent by the second device to the first device in response to a user selection of the respective information item. A command is received from the second device in response to a selection of a first one of the selectable display items by the user. A predefined action is performed in response to receiving the command.

Claims

1. A computer-implemented method for allowing interaction between devices, the method comprising:

at a mobile device including a processor, memory, a touch display, and an input device;

communicatively coupling to a computer system that executes one or more media applications to control a plurality of media content sources including at least one Internet based media source and access media content from the plurality of media content sources for display on a display device coupled to the computer system, the one or more media applications including a first media application for accessing and controlling a first content source selected from the plurality of media content sources;

receiving from the computer system a control specification that includes an information item associated with the first media application and an action definition associated with the information item, wherein the action definition includes a command to be sent by the mobile device to the computer system in response to a user selection of the information item;

displaying on the mobile device a remote control user interface associated with the first media application according to the control specification, including concurrently (1) displaying a plurality of predefined playback controls on a control region of the remote control user interface according to a template and (2) displaying the information item on a content region of the remote control user interface according to a display instruction customized by the first media application, wherein the predefined playback controls are consistently displayed for the one or more media applications according to the template when the content region is dynamically customized based on the plurality of media content sources corresponding to the one or more media applications;

displaying the information item on the remote control user interface;

receiving a selection of the information item from a user via the input device; and

in accordance with the selection of the information item, transmitting the command of the corresponding action definition from the mobile device to the computer system, including instructing the computer system to perform the command in the first media application for controlling the first media content source, accessing respective media content from the first media content source and providing the respective media content to the display device for display.

2. The computer-implemented method of claim 1, wherein the control specification further includes a display instruction associated with the information item, and displaying the information item on the user interface further comprises:

displaying media content associated with the information item according to the display instruction.

3. The computer-implemented method of claim 2, wherein the media content is one or more of:

a video, a television station, an audio track, an audio station, and a picture.

4. The computer-implemented method of claim 1, wherein the action definition is one or more of:

changing a channel, changing volume level, changing a device setting, initiating a record function, adjusting a playback speed, and changing an operating state of a device.

5. The computer-implemented method of claim 1, wherein the information item is a label.

6. The computer-implemented method of claim 5, wherein the label includes at least one of a graphic and text.

7. The computer-implemented method of claim 1, wherein the plurality of predefined playback controls are represented by a subset of a plurality of icons for rewind, stop, play, pause, and fast-forward of the respective media content associated with the information item.

8. The computer-implemented method of claim 1, wherein the information item is displayed in a predefined format based at least in part on a property of the information item.

9. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

communicatively coupling to a computer system that executes one or more media applications to control a plurality of media content sources including at least one Internet based media source and access media content from the plurality of media content sources for display on a display device coupled to the computer system, the one or more media applications including a first media application for accessing and controlling a first content source selected from the plurality of media content sources;

receiving from the computer system a control specification that includes an information item associated with the first media application and an action definition associated with the information item, wherein the action definition includes a command to be sent by the electronic device to the computer system in response to a user selection of the information item;

displaying on the electronic device a remote control user interface associated with the first media application according to the control specification, including concurrently (1) displaying a plurality of predefined playback controls on a control region of the remote control user interface according to a template and (2) displaying the information item on a content region of the remote control user interface according to a display instruction customized by the first media application, wherein the predefined playback controls are consistently displayed for the one or more media applications according to the template when the content region is dynamically customized based on the plurality of media content sources corresponding to the one or more media applications;

displaying the information item on the remote control user interface;

receiving a selection of the information item from a user via the input device; and

in accordance with the selection of the information item, transmitting the command of the corresponding action definition from the electronic device to the computer system, including instructing the computer system to perform the command in the first media application for controlling the first media content source, accessing respective media content from the first media content source and providing the respective media content to the display device for display.

10. The electronic device of claim 9, wherein the information item is one of a collection of information items each representing a media content item, and the collection of information items is associated with one of the plurality of media content sources, and displayed in a predefined format based at least in part on a property of the information items in the collection.

11. The electronic device of claim 9, wherein the predefined format is a list format, a tree format, a gallery format, or a hierarchical folder format.

12. The electronic device of claim 9, wherein the information item is displayed according to a first display instruction, and the one or more programs further comprise instructions for:

receiving, from the computer system, a collection of information items; and

receiving, from the computer system, a second display instruction associated with the collection of information items.

13. The electronic device of claim 9, wherein the electronic device is a mobile telephone.

14. The electronic device of claim 9, wherein the plurality of media content sources include two or more providers with different system types selected from:

broadcast, cable, satellite television sources, Internet-based streaming movie services, and Internet-based music services.

15. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a mobile device including a processor, memory, a display, and an input device, cause the mobile device to:

at the mobile device;

communicatively couple to a computer system that executes one or more media applications to control a plurality of media content sources including at least one Internet based media source and access media content from the plurality of media content sources for display on a display device coupled to the computer system, the one or more media applications including a first media application for accessing and controlling a first content source selected from the plurality of media content sources;

receive from the computer system a control specification that includes an information item associated with the first media application and an action definition associated with the information item, wherein the action definition includes a command to be sent by the mobile device to the computer system in response to a user selection of the information item;

display on the mobile device a remote control user interface associated with the first media application according to the control specification, including concurrently (1) displaying a plurality of predefined playback controls on a control region of the remote control user interface according to a template and (2) displaying the information item on a content region of the remote control user interface according to a display instruction customized by the first media application, wherein the predefined playback controls are consistently displayed for the one or more media applications according to the template when the content region is dynamically customized based on the plurality of media content sources corresponding to the one or more media applications;

display the information item on the remote control user interface;

receive a selection of the information item from a user via the input device; and

in accordance with the selection of the information item, transmit the command of the corresponding action definition from the mobile device to the computer system, including instructing the computer system to perform the command in the first media application for controlling the first media content source, accessing respective media content from the first media content source and providing the respective media content to the display device for display.

16. The non-transitory computer-readable storage medium of claim 15, wherein the information item is displayed according to a first display instruction, and the one or more programs further comprise instructions for:

receiving, from the computer system, a group of collections; and

receiving, from the computer system, a second display instruction associated with the group of collections.

17. The non-transitory computer-readable storage medium of claim 16, wherein the second display instruction indicates that a respective collection of the group of collections is to be displayed in a separate tab of the remote control user interface on the mobile device.

18. The non-transitory computer-readable storage medium of claim 16, wherein the second display instruction indicates that a respective collection of the group of collections is to be displayed in a separate folder of the remote control user interface of the mobile device.

19. The non-transitory computer-readable storage medium of claim 15, wherein the one or more media applications are one or more of a web browser, a media browser, and a media player.

20. The non-transitory computer-readable storage medium of claim 15, wherein the one or more programs further comprise instructions for:

receiving from the computer system a plurality of display instructions in association with the information item and the action definition, wherein each of the plurality of display instructions is tailored for a respective computer device based on one or more of a type, form factors and functionality of the respective computer device; and

selecting and implementing by the mobile device one of the plurality of display instructions for displaying the information item on the remote control user interface in accordance with the type, form factors and functionality of the mobile device.