Contextual, two way remote control

• US 10,212,212 B2
• Filed: 03/22/2018
• Issued: 02/19/2019
• Est. Priority Date: 07/03/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Lu, Jeff Ting Yann
• Primary Examiner(s): Aziz, Adnan
• Application Number:
  US15/928,933
• Publication Number:
  US 20180213022A1
• Time in Patent Office: 334 Days
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 23/04 using light waves, e.g. infrared
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

A mobile device is coupled to a computer system configured to access media content sources and play media content items provided thereby on a media device. The mobile device receives a first control specification that includes a plurality of first information items each associated with one of the media content sources and a first action definition. The first information items are concurrently displayed on a remote control user interface of the mobile device. In accordance with a selection of one of the first information items corresponding to a first content source, the mobile device transmits a command of a first action definition corresponding to the selected first information item to the computer system, and receives a second control specification that includes a plurality of second information items. Each second information item is associated with a respective content item provided by the first content source and a second action definition.

Claims

1. A computer-implemented method for allowing interaction between devices, the method comprising:

at a mobile device including a processor, memory and a touch display;

communicatively coupling to a computer system to control a plurality of media content sources and access media content from the plurality of media content sources for play on a media device;

receiving a first control specification that includes a plurality of first information items, wherein each of the first information items is associated with one of the plurality of media content sources and a first action definition, and the first action definition includes a first command to be sent by the mobile device to the computer system to access media content from the one of the plurality of media content sources in response to a user selection of the respective first information item;

in accordance with the first control specification, displaying on the mobile device a remote control user interface associated with a media application, including concurrently displaying the plurality of first information items;

receiving a selection of one of the plurality of first information items from a user via the touch display, the selected one of the plurality of first information items corresponding to a first content source; and

in accordance with the selection of the one of the plurality of first information items that corresponds to the first content source;

transmitting the first command of the first action definition that is associated with the selected one of the plurality of first information items corresponding to the first content source from the mobile device to the computer system;

receiving a second control specification that includes a plurality of second information items, wherein each of the second information items is associated with a respective media content item provided by the first content source and a second action definition, and the second action definition is associated with a second command to be sent by the mobile device to access the respective media content item from the first content source in response to a user selection of the respective second information item; and

in accordance with the second control specification, updating on the mobile device display of the remote control user interface associated with the media application, including concurrently displaying the plurality of second information items on the remote control user interface.

2. The computer-implemented method of claim 1, wherein the plurality of second information items are displayed in place of the plurality of first information items.

3. The computer-implemented method of claim 1, further comprising:

receiving a selection of one of the plurality of second information items from the user via the touch display, the selected one of the plurality of second information items corresponding a first media content item; and

in accordance with the selection of the one of the plurality of second information items, transmitting the second command of the corresponding second action definition from the mobile device to the computer system, including instructing the computer system to perform the command for controlling the first content source, accessing respective media content from the first content source and providing the respective media content for play on the media device.

4. The computer-implemented method of claim 2, wherein the plurality of first information items are concurrently displayed in a first list in the remote control user interface, and the plurality of second information items are concurrently displayed in a second list in the remote control user interface in place of the first list.

5. The computer-implemented method of claim 1, further comprising:

receiving by the mobile device a plurality of display instructions in association with the plurality of first information items, wherein each of the plurality of display instructions is tailored for a client device based on one or more of a type, form factors and functionality of the client device, and the mobile device is configured to select and implement one of the plurality of display instructions tailored for the mobile device for displaying selectable display elements corresponding to the plurality of first information items.

6. The computer-implemented method of claim 1, further comprising:

receiving by the mobile device a plurality of display instructions in association with the plurality of second information items, wherein each of the plurality of display instructions is tailored for a client device based on one or more of a type, form factors and functionality of the client device, and the mobile device is configured to select and implement one of the plurality of display instructions tailored for the mobile device for displaying selectable display elements corresponding to the plurality of second information items.

7. The computer-implemented method of claim 1, wherein each of the first information items corresponds to one of an audio/radio station and an Internet-based music service, and each of the second information items corresponds to an audio track.

8. The computer-implemented method of claim 1, wherein each of the second information items includes information describing the respective media content item provided by the first content source, and the information describing the respective media content item is selected from a group consisting of a title, a file name, a thumbnail, and description of the respective media content item.

9. The computer-implemented method of claim 8, wherein for each of the second information items, the respective media content item provided by the first content source is selected from a group consisting of a video clip, a television station, an audio track, an audio/radio station, and a picture.

10. The computer-implemented method of claim 1, further comprising:

receiving a display instruction associated with the first information items, wherein the display instruction indicates that the first information items is to be displayed in list format, a tree format, a gallery format, or a hierarchical folder format.

11. The computer-implemented method of claim 1, wherein the computer system includes the media device.

12. The computer-implemented method of claim 11, wherein the media device is a speaker device.

13. A mobile device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for, at the mobile device;

communicatively coupling to a computer system to control a plurality of media content sources and access media content from the plurality of media content sources for play on a media device;

receiving a first control specification that includes a plurality of first information items, wherein each of the first information items is associated with one of the plurality of media content sources and a first action definition, and the first action definition includes a first command to be sent by the mobile device to the computer system to access media content from the one of the plurality of media content sources in response to a user selection of the respective first information item;

in accordance with the first control specification, displaying on the mobile device a remote control user interface associated with a media application, including concurrently displaying the plurality of first information items;

receiving a selection of one of the plurality of first information items from a user via the touch display, the selected one of the plurality of first information items corresponding a first content source; and

in accordance with the selection of the one of the plurality of first information items that corresponds to the first content source;

transmitting the first command of the first action definition that is associated with the selected one of the plurality of first information items corresponding to the first content source from the mobile device to the computer system;

receiving a second control specification that includes a plurality of second information items, wherein each of the second information items is associated with a respective media content item provided by the first content source and a second action definition, and the second action definition is associated with a second command to be sent by the mobile device to access the respective media content item from the first content source in response to a user selection of the respective second information item; and

in accordance with the second control specification, updating on the mobile device display of the remote control user interface associated with the media application, including concurrently displaying the plurality of second information items on the remote control user interface.

14. The mobile device of claim 13, wherein the one or more programs further comprise instructions for:

receiving a display instruction associated with the second information items, wherein the display instruction indicates that the second information items is to be displayed in list format, a tree format, a gallery format, or a hierarchical folder format.

15. The mobile device of claim 13, wherein the first control specification is incorporated in a HTML file.

16. The mobile device of claim 13, wherein the second control specification is incorporated in a HTML file.

17. The mobile device of claim 13, wherein the remote control user interface includes a content region and a control region that are distinct from each other, and when the plurality of first information items and the plurality of second information items are displayed in the content region, one or more predefined playback controls are displayed in the control region.

18. The mobile device of claim 17, wherein the one or more predefined playback controls are represented by a subset of a plurality of icons for rewinding, stopping, playing, pausing, and fast-forwarding the media content provided by the plurality of media content sources.

19. The mobile device of claim 13, wherein for each of the second information items, the respective second media content item provided by the first content source represents a single media content item provided by the first content source or a folder containing multiple media content items provided by the first content source.

20. The mobile device of claim 13, wherein the first content source includes an Internet-based audio channel, and the respective media content item associated with each of the second information items includes an audio track.

21. A computer-implemented method for allowing interaction between devices, the method comprising:

at a mobile device including a processor, memory and a touch display;

communicatively coupling to a computer system to control a plurality of media content sources and access media content from the plurality of media content sources for play on a media device;

receiving a control specification that includes a plurality of information items associated with one of the plurality of media content sources, wherein each of the information items is associated with an action definition including a command to be sent by the mobile device to the computer system to access media content from the one of the plurality of media content sources in response to a user selection of the respective information item;

in accordance with the control specification, displaying on the mobile device a remote control user interface associated with a media application, including concurrently (1) displaying one or more predefined playback controls on a control region of the remote control user interface according to a template and (2) displaying the plurality of information items on a content region of the remote control user interface according to a display rule customized for the one of the plurality of media content sources, wherein the predefined playback controls are consistently displayed on the control region according to the template when the content region is dynamically customized based on the plurality of media content sources;

receiving a selection of one of the plurality of information items from a user via the touch display; and

in accordance with the selection of the one of the plurality of information items, transmitting the command of the corresponding action definition from the mobile device to the computer system, including instructing the computer system to perform the command for controlling the one of the plurality of media content sources, accessing respective media content from the one of the plurality of media content sources and providing the respective media content for play on the media device.

22. The computer-implemented method of claim 21, wherein each of the one or more playback controls is one or more of:

rewinding, stopping, playing, pausing, and fast-forwarding.

23. The computer-implemented method of claim 21, wherein the display rule includes a plurality of display instructions, further comprising:

receiving by the mobile device the plurality of display instructions in association with the plurality of information items, wherein each of the plurality of display instructions is tailored for a client device based on one or more of a type, form factors and functionality of the client device, and the mobile device is configured to select and implement one of the plurality of display instructions tailored for the mobile device for displaying selectable display elements corresponding to the plurality of information items.

24. The computer-implemented method of claim 21, wherein the plurality of information items correspond to one of an audio/radio station and an Internet-based music service, and each information item represents an audio track.

25. The computer-implemented method of claim 21, wherein each of the plurality of information items includes information describing corresponding media content provided by the one of the plurality of media content sources, and the information describing the corresponding media content is selected from a group consisting of a title, a file name, a thumbnail, and description of the media content.

26. The computer-implemented method of claim 25, wherein for each of the plurality of information items, the corresponding media content provided by the one of the plurality of media content sources is selected from a group consisting of a video clip, a television station, an audio track, an audio/radio station, and a picture.

27. The computer-implemented method of claim 21, wherein the one or more predefined playback controls include all playback controls that are displayed on the control region of the remote control user interface according to the template.

28. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a processor of a mobile device, cause the mobile device to perform operations for:

communicatively coupling to a computer system to control a plurality of media content sources and access media content from the plurality of media content sources for play on a media device;

receiving a control specification that includes a plurality of information items associated with one of the plurality of media content sources, wherein each of the information items is associated with an action definition including a command to be sent by the mobile device to the computer system to access media content from the one of the plurality of media content sources in response to a user selection of the respective information item;

in accordance with the control specification, displaying on the mobile device a remote control user interface associated with a media application, including concurrently (1) displaying one or more predefined playback controls on a control region of the remote control user interface according to a template and (2) displaying the plurality of information items on a content region of the remote control user interface according to a display rule customized for the one of the plurality of media content sources, wherein the predefined playback controls are consistently displayed on the control region according to the template when the content region is dynamically customized based on the plurality of media content sources;

receiving a selection of one of the plurality of information items from a user via the touch display; and

in accordance with the selection of the one of the plurality of information items, transmitting the command of the corresponding action definition from the mobile device to the computer system, including instructing the computer system to perform the command for controlling the one of the plurality of media content sources, accessing respective media content from the one of the plurality of media content sources and providing the respective media content for play on the media device.

29. The non-transitory computer-readable storage medium of claim 28, wherein the one or more programs further comprise instructions for:

receiving the display rule associated with the plurality of information items, wherein the display rule indicates that the plurality of information items is to be displayed in list format, a tree format, a gallery format, or a hierarchical folder format.

30. The non-transitory computer-readable storage medium of claim 28, wherein the control specification is incorporated in a HTML file.

31. The non-transitory computer-readable storage medium of claim 28, wherein each of the plurality of information items is associated with one of a single media content item provided by the one of the plurality of content sources and a folder containing multiple media content items provided by the one of the plurality of content sources.

32. The non-transitory computer-readable storage medium of claim 28, wherein the one of the plurality of content sources includes an Internet-based audio channel, and the media content associated with each of the plurality of information items includes an audio track.

33. The non-transitory computer-readable storage medium of claim 28, wherein each of the one or more playback controls is one or more of:

rewinding, stopping, playing, pausing, and fast-forwarding.