User interface including content from a remote device

• US 9,285,968 B2
• Filed: 03/21/2014
• Issued: 03/15/2016
• Est. Priority Date: 05/13/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Schubert, Emily Clark, Langenfeld, Peter T.
• Primary Examiner(s): Ng, Amy
• Assistant Examiner(s): CASILLAS, ROLAND J
• Application Number:
  US14/222,302
• Publication Number:
  US 20140208223A1
• Time in Patent Office: 725 Days
• Field of Search: 715/740, 715/746, 715748-749
• US Class Current: 1/1
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications

Abstract

A user interface displayed by an electronic device can include a first user-interface portion generated by the electronic device and a second user interface portion generated by a remote device. The electronic device can detect received input and determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user interface portion. If the input corresponds to an interactive element in the first user-interface portion, the electronic device can generate a response to the input. If the input corresponds to an interactive element in the second user-interface portion, the electronic device can transmit information indicative of the input to the remote device.

Claims

1. An electronic device, comprising:

one or more processors;

a touchscreen display;

one or more input devices;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, on the touchscreen display having a designated area for displaying a second user-interface portion received from a remote device, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and the second user-interface portion generated by the remote device, wherein the second user-interface portion is displayed in the designated area;

detecting input from an input device of the one or more input devices; and

in response to detecting the input;

determining whether the input is detected on the touchscreen display within the designated area or outside of the designated area

in accordance with a determination that the input is detected on the touchscreen outside of the designated area, conditionally generating a response to the input based on a location of the input; and

in accordance with a determination that the input is received on the touchscreen within the designated area;

transmitting information indicative of the input to the remote device;

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, one or more graphical elements; and

responding to the input by updating the second user-interface portion to display the one or more graphical elements received from the remote device,

wherein the information indicative of the input transmitted to the remote device includes location information indicating a location within the designated area of the touchscreen display at which a touch was detected.

2. The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location within the designated area of the touchscreen display.

3. The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

4. The device of claim 1, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

5. The device of claim 1, wherein the transmitting of information indicative of the input to the remote device occurs without interpretation by the remote device.

6. The device of claim 1, wherein, in response to receiving information indicative of the input transmitted to the remote device, the remote device is configured to perform at least one action.

7. The device of claim 6, wherein the at least one action performed by the remote device includes at least one of:

updating a user interface of the remote device, providing instructions to the electronic device to update a user interface of the electronic device, initiating playback of a media file;

pausing playback of a media file, modifying a setting of the remote device; and

sending instructions to the electronic device to modify a setting of the electronic device.

8. The device of claim 2, wherein the remote device is configured to interpret the pixel coordinates of the location within the designated area of the touchscreen display and perform at least one action based on the interpretation of the pixel coordinates.

9. The device of claim 2, wherein the one or more graphical elements are selected by the remote device in accordance with the pixel coordinates of the location within the designated area of the touchscreen display.

10. The device of claim 1, wherein the one or more graphical elements are selected by the remote device in accordance with the location information indicating the location within the designated area of the touchscreen display at which a touch was detected.

11. The device of claim 1, wherein the one or more graphical elements are generated by the remote device without any intervention by the electronic device.

12. The device of claim 1, wherein at least one of the one or more graphical elements indicate a functionality of the remote device that was invoked in response to interpretation of the input by the remote device.

13. The device of claim 1, wherein the electronic device is an in-vehicle media control unit and the remote device is a portable media device.

14. A method comprising:

at an electronic device with one or more processors, a touchscreen display, one or more input devices, memory;

displaying, on the touchscreen display having a designated area for displaying a second user-interface portion received from a remote device, a user interface, wherein the user interface includes a first user-interface portion generated by the electronic device and the second user-interface portion generated by the remote device, wherein the second user-interface portion is displayed in the designated area;

detecting input from an input device of the one or more input devices; and

in response to detecting the input;

determining whether the input is detected on the touchscreen display within the designated area or outside of the designated area;

in accordance with a determination that the input is detected on the touchscreen outside of the designated area, conditionally generating a response to the input based on a location of the input; and

in accordance with a determination that the input is received on the touchscreen within the designated area;

transmitting information indicative of the input to the remote device;

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, one or more graphical elements; and

responding to the input by updating the second user-interface portion to display the one or more graphical elements received from the remote device,

wherein the information indicative of the input transmitted to the remote device includes location information indicating a location within the designated area of the touchscreen display at which a touch was detected.

15. The method of claim 14, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location within the designated area of the touchscreen display.

16. The method of claim 14, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

17. The method of claim 14, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

18. The method of claim 14, wherein the transmitting of information indicative of the input to the remote device occurs without interpretation by the remote device.

19. The method of claim 14, wherein, in response to receiving information indicative of the input transmitted to the remote device, the remote device is configured to perform at least one action.

20. The method of claim 19, wherein the at least one action performed by the remote device includes at least one of:

updating a user interface of the remote device, providing instructions to the electronic device to update a user interface of the electronic device, initiating playback of a media file;

pausing playback of a media file, modifying a setting of the remote device; and

sending instructions to the electronic device to modify a setting of the electronic device.

21. The method of claim 15, wherein the remote device is configured to interpret the pixel coordinates of the location within the designated area of the touchscreen display and perform at least one action based on the interpretation of the pixel coordinates.

22. The method of claim 15, wherein the one or more graphical elements are selected by the remote device in accordance with the pixel coordinates of the location within the designated area of the touchscreen display.

23. The method of claim 14, wherein the one or more graphical elements are selected by the remote device in accordance with the location information indicating the location within the designated area of the touchscreen display at which a touch was detected.

24. The method of claim 14, wherein the one or more graphical elements are generated by the remote device without any intervention by the electronic device.

25. The method of claim 14, wherein at least one of the one or more graphical elements indicate a functionality of the remote device that was invoked in response to interpretation of the input by the remote device.

26. The method of claim 14, wherein the electronic device is an in-vehicle media control unit and the remote device is a portable media device.

27. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors, a touchscreen display, one or more input devices, and memory, cause the device to:

display, on the touchscreen display having a designated area for displaying a second user-interface portion received from a remote device, a user interface, wherein the user interface includes a first user-interface portion generated by the device and the second user-interface portion generated by the remote device;

detect input from an input device of the one or more input devices; and

in response to detecting the input;

determine whether the input is detected on the touchscreen display within the designated area or outside of the designated area;

in accordance with a determination that the input is detected on the touchscreen outside of the designated area, conditionally generate a response to the input based on a location of the input; and

in accordance with a determination that the input is received on the touchscreen within the designated area;

transmit information indicative of the input to the remote device;

after transmitting the information indicative of the input to the remote device, receiving, from the remote device, one or more graphical elements; and

responding to the input by updating the second user-interface portion to display the one or more graphical elements received from the remote device,

wherein the information indicative of the input transmitted to the remote device includes location information indicating a location within the designated area of the touchscreen display at which a touch was detected.

28. The non-transitory computer readable storage medium of claim 27, wherein the information indicative of the input transmitted to the remote device includes pixel coordinates of a location within the designated area of the touchscreen display.

29. The non-transitory computer readable storage medium of claim 27, wherein the information indicative of the input transmitted to the remote device includes information identifying a control with which an interaction occurred.

30. The non-transitory computer readable storage medium of claim 27, wherein the information indicative of the input transmitted to the remote device includes information indicating an operation associated with the input.

31. The non-transitory computer readable storage medium of claim 27, wherein the transmitting of information indicative of the input to the remote device occurs without interpretation by the remote device.

32. The non-transitory computer readable storage medium of claim 27, wherein, in response to receiving information indicative of the input transmitted to the remote device, the remote device is configured to perform at least one action.

33. The non-transitory computer readable storage medium of claim 32, wherein the at least one action performed by the remote device includes at least one of:

updating a user interface of the remote device, providing instructions to the electronic device to update a user interface of the electronic device, initiating playback of a media file;

pausing playback of a media file, modifying a setting of the remote device; and

sending instructions to the electronic device to modify a setting of the electronic device.

34. The non-transitory computer readable storage medium of claim 28, wherein the remote device is configured to interpret the pixel coordinates of the location within the designated area of the touchscreen display and perform at least one action based on the interpretation of the pixel coordinates.

35. The non-transitory computer readable storage medium of claim 28, wherein the one or more graphical elements are selected by the remote device in accordance with the pixel coordinates of the location within the designated area of the touchscreen display.

36. The non-transitory computer readable storage medium of claim 27, wherein the one or more graphical elements are selected by the remote device in accordance with the location information indicating the location within the designated area of the touchscreen display at which a touch was detected.

37. The non-transitory computer readable storage medium of claim 27, wherein the one or more graphical elements are generated by the remote device without any intervention by the electronic device.

38. The non-transitory computer readable storage medium of claim 27, wherein at least one of the one or more graphical elements indicate a functionality of the remote device that was invoked in response to interpretation of the input by the remote device.

39. The non-transitory computer readable storage medium of claim 27, wherein the electronic device is an in-vehicle media control unit and the remote device is a portable media device.

40. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

while the device is coupled to an accessory that includes a touchscreen display and an input device, receiving a request to provide a user interface for display at a designated area of the touchscreen display of the accessory; and

in response to receiving the request;

generating a first user-interface portion for display in the designated area of the touchscreen display; and

initiating display of the first user-interface portion in the designated area of the touchscreen display of the accessory, wherein the first user-interface portion is displayed in the designated area of the touchscreen display while a second user-interface portion generated by the accessory is concurrently displayed in a second region of the touchscreen display that is distinct from the designated area of the touchscreen display; and

after initiating display of the first user-interface portion in the designated area of the touchscreen display of the accessory, receiving input that is indicative of input received on the touchscreen within the designated area; and

in response to receiving the input;

generating an updated first user-interface portion including one or more graphical elements in accordance with the input; and

initiating display of the updated first user-interface portion in the designated area of the touchscreen display,

wherein the information indicative of the input received on the touchscreen includes location information indicating a location within the designated area of the touchscreen display at which a touch was detected.

41. The device of claim 40, wherein the one or more programs include instructions for transmitting information identifying a portion of the display corresponding to the designated area of the touchscreen display.

42. The device of claim 40, wherein generating a first user-interface portion includes generating placeholder data for the second user-interface portion.

43. The device of claim 40, wherein the request includes at least one of configuration, environment, and status information.

44. The device of claim 40, wherein the first user-interface portion is adapted to the configuration of the first region of the display.

45. The device of claim 40, wherein the one or more programs include instructions for:

after receiving the request, transmitting a list of remote user interface image types;

receiving a selection of a user interface image type from the list of remote user interface image types; and

generating the first user-interface portion using the selected user interface image type.

46. The device of claim 40, wherein the one or more programs include instructions for:

receiving a request to stop providing the user interface for display at the accessory; and

;

in response to receiving the request to stop providing the user interface for display at the accessory;

stop generating the updated first user-interface portion.