USER INTERFACE INCLUDING CONTENT FROM A REMOTE DEVICE

• US 20140208223A1
• Filed: 03/21/2014
• Published: 07/24/2014
• Est. Priority Date: 05/13/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Schubert, Emily Clark, Langenfeld, Peter T.
• Granted Patent: US 9,285,968 B2
• Time in Patent Office: Days
• US Class Current: 715/740
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications

Abstract

A user interface displayed by an electronic device can include a first user-interface portion generated by the electronic device and a second user interface portion generated by a remote device. The electronic device can detect received input and determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user interface portion. If the input corresponds to an interactive element in the first user-interface portion, the electronic device can generate a response to the input. If the input corresponds to an interactive element in the second user-interface portion, the electronic device can transmit information indicative of the input to the remote device.

Claims

1. An electronic device, comprising:

one or more processors;

a display;

one or more input devices;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and a second user-interface portion generated by a remote device;

detecting input from an input device of the one or more input devices; and

in response to detecting the input;

determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;

in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generating a response to the input; and

in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmitting information indicative of the input to the remote device.

2. The device of claim 1, wherein the one or more programs include instructions for:

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and

responding to the input in accordance with the remote response information.

3. The device of claim 2, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

4. The device of claim 1, wherein:

the one or more input devices include a touchscreen on which the user interface is displayed;

detecting the input includes detecting a contact on the touchscreen; and

determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

5. The device of claim 4, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

6. The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

7. The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

8. A method comprising:

at an electronic device with one or more processors, a display, one or more input devices, memory;

displaying, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and a second user-interface portion generated by a remote device;

detecting input from an input device of the one or more input devices; and

in response to detecting the input;

determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;

in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generating a response to the input; and

in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmitting information indicative of the input to the remote device.

9. The method of claim 8, further comprising:

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and

responding to the input in accordance with the remote response information.

10. The method of claim 9, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

11. The method of claim 8, wherein the one or more input devices include a touchscreen on which the user interface is displayed;

detecting the input includes detecting a contact on the touchscreen; and

determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

12. The method of claim 11, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

13. The method of claim 8, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

14. The method of claim 8, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

15. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors, a display, one or more input devices, and memory, cause the device to:

display, on the display, a user interface, wherein the user interface includes a first user-interface portion generated by the device and a second user-interface portion generated by a remote device;

detect input from an input device of the one or more input devices; and

in response to detecting the input;

determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion;

in accordance with a determination that the input corresponds to an interactive element in the first user-interface portion, generate a response to the input; and

in accordance with a determination that the input corresponds to an interactive element in the second user-interface portion, transmit information indicative of the input to the remote device.

16. The non-transitory computer readable storage medium of claim 15, wherein the one or more programs include instructions for:

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, remote response information for responding to the input; and

responding to the input in accordance with the remote response information.

17. The non-transitory computer readable storage medium of claim 16, wherein responding to the input in accordance with the remote response information includes updating the second user-interface portion in accordance with the remote response information.

18. The non-transitory computer readable storage medium of claim 15, wherein:

the one or more input devices include a touchscreen on which the user interface is displayed;

detecting the input includes detecting a contact on the touchscreen; and

determining whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user-interface portion includes determining whether the contact was detected on a portion of the touchscreen that displays the first user-interface portion or on a portion of the touchscreen that displays the second user-interface portion.

19. The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location on the touchscreen.

20. The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes Information identifying a control with which an interaction occurred.

21. The non-transitory computer readable storage medium of claim 15, wherein the information indicative of the input transmitted to the remote device includes Information indicating an operation associated with the input.

22. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while the device is coupled to an accessory that includes a display and an input device, receiving a request to provide a user interface for display at the accessory; and

in response to receiving the request;

generating a first user-interface portion for display in a first region of the display; and

initiating display of the first user-interface portion in the first region of the display of the accessory, wherein the first user-interface portion is displayed in the first region of the display while a second user-interface portion generated by the accessory is concurrently displayed in a second region of the display that is distinct from the first region of the display; and

after initiating display of the first user-interface portion in the first region of the display of the accessory, receiving input that is indicative of interaction with one or more controls in the first region of the display; and

in response to receiving the input;

generating an updated first user-interface portion in accordance with the input; and

initiating display of the updated first user-interface portion in the first region of the display.

23. The device of claim 22, wherein the one or more programs include instructions for transmitting information identifying a portion of the display corresponding to the first region of the display.

24. The device of claim 22, wherein generating a first user-interface portion includes generating placeholder data for the second user-interface portion.

25. The device of claim 22, wherein the request includes at least one of configuration, environment, and status information.

26. The device of claim 22, wherein the first user-interface portion is adapted to the configuration of the first region of the display.

27. The device of claim 22, wherein the one or more programs include instructions for:

after receiving the request, transmitting a list of remote user interface image types;

receiving a selection of a user interface image type from the list of remote user interface image types; and

generating the first user-interface portion using the selected user interface image type.

28. The device of claim 22, wherein the one or more programs include instructions for:

receiving a request to stop providing the user interface for display at the accessory; and

;

in response to receiving the request to stop providing the user interface for display at the accessory;

stop generating the updated first user-interface portion.