Pushing a user interface to a remote device that controls multiple displays

• US 9,471,207 B2
• Filed: 03/26/2014
• Issued: 10/18/2016
• Est. Priority Date: 05/13/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Schubert, Emily Clark, Langenfeld, Peter T.
• Primary Examiner(s): Ngo, Tony N
• Application Number:
  US14/226,705
• Publication Number:
  US 20140208224A1
• Time in Patent Office: 937 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications

Abstract

An electronic device can be coupled to an accessory that controls multiple displays. The device can receive accessory configuration information describing a configuration of the multiple displays. The device can generate one or more user interfaces for display by the multiple displays and transmit the one or more user interfaces to the accessory for display by the multiple displays. The device can receive information indicating user interaction with a user interface. In response to receiving the information, the device can generate an updated user interface and transmit the updated user interface to the accessory.

Claims

1. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while the electronic device is communicatively coupled to an accessory that controls a plurality of displays, receiving accessory configuration information that describes a configuration of the plurality of displays and identifies one or more input devices that are configured for use in controlling a user interface displayed on the display in one or more of the plurality of displays, wherein at least one of the input devices is a touchscreen input device; and

after receiving the accessory configuration information, generating one or more user interfaces for controlling the electronic device, in accordance with the accessory configuration information, for display by the accessory using one or more displays of the plurality of displays;

transmitting, to the accessory, the one or more user interfaces for controlling the electronic device, for display by the accessory using the one or more displays in the plurality of displays;

receiving, from the accessory, user-interaction information indicative of user interaction with a user interface of the one or more user interfaces, the user interface corresponding to a display in the plurality of displays, the user interaction information including an indicator of the one or more input devices that was operated and pixel coordinates of a location on the touchscreen input device that was touched by a user; and

in response to receiving the user-interaction information;

generating an updated user interface for controlling the electronic device in accordance with the user-interaction information; and

transmitting, to the accessory, the updated user interface for controlling the electronic device, for display by the accessory using the corresponding display in the plurality of displays.

2. The device of claim 1, wherein the accessory configuration information includes a description of a type of control associated with one or more displays in the plurality of displays.

3. The device of claim 1, wherein the accessory configuration information includes a description of a location associated with one or more displays in the plurality of displays.

4. The device of claim 1, wherein the accessory is configured to provide video data to displays in the plurality of displays.

5. The device of claim 1, wherein the accessory configuration information includes:

first configuration information for a first display; and

second configuration information for a second display, wherein the second configuration information is different from the first configuration information.

6. The device of claim 1, wherein the configuration of the plurality of displays includes a number of displays in the plurality of displays.

7. The device of claim 1, wherein transmitting the one or more user interfaces includes transmitting a parameter identifying the portion of the one or more displays in the plurality of displays to which pixel data applies.

8. A method comprising:

at an electronic device with one or more processors and memory;

while the electronic device is coupled to an accessory that controls a plurality of displays, receiving accessory configuration information that describes a configuration of the plurality of displays and identifies one or more input devices that are configured for use in controlling a user interface displayed on the display in one or more of the plurality of displays, wherein at least one of the input devices is a touchscreen input device; and

after receiving the accessory configuration information, generating one or more user interfaces for controlling the electronic device, in accordance with the accessory configuration information, for display by the accessory using one or more displays of the plurality of displays;

transmitting, to the accessory, the one or more user interfaces for controlling the electronic device, for display by the accessory using the one or more displays of the plurality of displays;

receiving, from the accessory, user-interaction information indicative of user interaction with a user interface in the one or more user interfaces, the user interface corresponding to a display in the plurality of displays, the user interaction information including an indicator of the one or more input devices that was operated and pixel coordinates of a location on the touchscreen input device that was touched by a user; and

in response to receiving the user-interaction information;

generating an updated user interface for controlling the electronic device in accordance with the user-interaction information; and

transmitting, to the accessory, the updated user interface for controlling the electronic device, for display by the accessory using the corresponding display in the plurality of displays.

9. The method of claim 8, wherein the accessory configuration information includes a description of a type of control associated with one or more displays in the plurality of displays.

10. The method of claim 8, wherein the accessory configuration information includes a description of a location associated with one or more displays in the plurality of displays.

11. The method of claim 8, wherein the accessory is configured to provide video data to displays in the plurality of displays.

12. The method of claim 8, wherein the accessory configuration information includes:

first configuration information for a first display; and

second configuration information for a second display, wherein the second configuration information is different from the first configuration information.

13. The method of claim 8, wherein the configuration of the plurality of displays includes a number of displays in the plurality of displays.

14. The method of claim 8, wherein transmitting the one or more user interfaces includes transmitting a parameter identifying the portion of the one or more displays in the plurality of displays to which pixel data applies.

15. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors and memory, cause the device to:

while the electronic device is coupled to an accessory that controls a plurality of displays, receive accessory configuration information that describes a configuration of the plurality of displays and identifies one or more input devices that are configured for use in controlling a user interface displayed on the display in one or more of the plurality of displays, wherein at least one of the input devices is a touchscreen input device; and

after receiving the accessory configuration information, generate one or more user interfaces for controlling the electronic device, in accordance with the accessory configuration information, for display by the accessory using one or more displays of the plurality of displays;

transmit, to the accessory, the one or more user interfaces for controlling the electronic device, for display by the accessory using the one or more displays of the plurality of displays;

receive, from the accessory, user-interaction information indicative of user interaction with a user interface in the one or more user interfaces, the user interface corresponding to a display in the plurality of displays, the user interaction information including an indicator of the one or more input devices that was operated and pixel coordinates of a location on the touchscreen input device that was touched by a user; and

in response to receiving the user-interaction information;

generate an updated user interface for controlling the electronic device in accordance with the user-interaction information; and

transmit, to the accessory, the updated user interface for controlling the electronic device, for display by the accessory using the corresponding display in the plurality of displays.

16. The non-transitory computer readable storage medium of claim 15, wherein the accessory configuration information includes a description of a type of control associated with one or more displays in the plurality of displays.

17. The non-transitory computer readable storage medium of claim 15, wherein the accessory configuration information includes a description of a location associated with one or more displays in the plurality of displays.

18. The non-transitory computer readable storage medium of claim 15, wherein the accessory is configured to provide video data to displays in the plurality of displays.

19. The non-transitory computer readable storage medium of claim 15, wherein the accessory configuration information includes:

first configuration information for a first display; and

second configuration information for a second display, wherein the second configuration information is different from the first configuration information.

20. The non-transitory computer readable storage medium of claim 15, wherein the configuration of the plurality of displays includes a number of displays in the plurality of displays.

21. The non-transitory computer readable storage medium of claim 15, wherein transmitting the one or more user interfaces includes transmitting a parameter identifying the portion of the one or more displays in the plurality of displays to which pixel data applies.