Remote user interface

• US 10,135,905 B2
• Filed: 07/21/2015
• Issued: 11/20/2018
• Est. Priority Date: 07/21/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chaudhri, Imran, Lemay, Stephen O., Dye, Alan C., Foss, Christopher Patrick, Yang, Lawrence Y., Graham, David Chance, Ive, Jonathan P.
• Primary Examiner(s): Lee, Ting Z
• Application Number:
  US14/805,403
• Publication Number:
  US 20160021168A1
• Time in Patent Office: 1,218 Days
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04M 1/72442 for playing music files
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041

Abstract

Target electronic devices can be remotely controlled using a personal electronic device are disclosed. The personal electronic device can identify target devices that are responsive to remote control. The personal electronic device displays graphical user interface objects that, when selected, cause the personal electronic device to instruct the target electronic device to perform an operation.

Claims

1. A non-transitory computer readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch-sensitive display, the one or more programs including instructions for:

displaying a plurality of icons representing applications, including an icon representing a remote control application;

receiving a first contact on the touch-sensitive display at a location of the icon;

identifying at least one target device out of a plurality of target devices, wherein identifying the at least one target device includes;

receiving data from the at least one target device; and

determining that the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control, selecting the identified single target device as the controllable target device; and

in response to receiving the first contact and identifying at least one of the target device out of the plurality of target devices being responsive to remote control;

displaying a first plurality of remote controls for the controllable target device;

while displaying the first plurality of remote controls, receiving a second contact on the touch-sensitive display, the second contact having maximum intensity;

in response to receiving the second contact and a determination that the maximum intensity is below a threshold intensity, instructing the controllable target device to perform an operation; and

in response to receiving the second contact and a determination that the maximum intensity is above the threshold intensity, display a second plurality of remote controls for the controllable target device.

2. A non-transitory computer readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch-sensitive display, the one or more programs including instructions for:

displaying a user interface object on the touch-sensitive display of the electronic device, wherein the displayed user interface object represents a remote control application;

receiving data representing a selection on the displayed user interface object;

identifying at least one target device, out of a plurality of target devices, wherein identifying the at least one target device includes;

receiving data from the at least one target device; and

determining that the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control, selecting the identified single target device as the controllable target device; and

in response to receiving the data representing the selection, displaying a plurality of graphical user interface objects for controlling operations of the controllable target device.

3. The non-transitory computer readable storage medium of claim 2, wherein the one or more programs further comprise instructions for:

in response to an identification that no target device is responsive to remote control, displaying a plurality of graphical user interface objects representing the plurality of target devices.

4. The non-transitory computer readable storage medium of claim 2, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and wherein the one or more programs further comprise instructions for:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity above a threshold intensity; and

in response to the second contact, invoking a user interface for registering a second target electronic device with the electronic device.

5. The non-transitory computer readable storage medium of claim 2, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and wherein the one or more programs further comprise instructions for:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity below a threshold intensity, the location associated with an operation of the first target device;

in response to the second contact, sending an instruction to the target device instructing the target device to perform the operation;

receiving third data representing a third contact on the display, the third contact having a maximum intensity above the threshold intensity; and

in response to the third contact, displaying a user interface for registering a second target device with the remote control application.

6. The non-transitory computer readable storage medium of claim 2, wherein the data representing the selection is a first data, wherein the one or more programs further comprise instructions for:

sending, from the electronic device to an intermediate electronic device, data representing an instruction instructing the target device to perform the operation,

wherein the instruction is communicated by the intermediate electronic device to the target device.

7. The non-transitory computer readable storage medium of claim 6, wherein the intermediate electronic device is a phone.

8. The non-transitory computer readable storage medium of claim 2, wherein identifying a target device comprises receiving network service information sent by the target device.

9. The non-transitory computer readable storage medium of claim 2, wherein identifying a target device comprises transmitting a request message and receiving an acknowledgement message sent by the target device, wherein the acknowledge message was sent by the target device in response to the request message.

10. The non-transitory computer readable storage medium of claim 2, wherein the target device is responsive to remote control when the target device is executing an application having an operation that can be requested using the remote control application.

11. The non-transitory computer readable storage medium of claim 10, wherein the application is executing in the background.

12. The non-transitory computer readable storage medium of claim 2, wherein the target device is a device other than the electronic device.

13. The non-transitory computer readable storage medium of claim 2, wherein the plurality of graphical user interface objects comprises transport controls.

14. The non-transitory computer readable storage medium of claim 2, wherein the plurality of graphical user interface objects are determined based on the controllable target device.

15. The non-transitory computer readable storage medium of claim 2, wherein the plurality of graphical user interface objects are shared by the controllable target device and at least one other device of the plurality of target devices.

16. A method, comprising:

at an electronic device with a touch-sensitive display, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display;

displaying a plurality of icons representing applications, including an icon representing a remote control application;

receiving a first contact on the touch-sensitive display at a location of the icon;

identifying at least one target device out of a plurality of target devices, wherein identifying the at least one target device includes;

receiving data from the at least one target device; and

determining that, the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control selecting the identified single target device as the controllable target device; and

in response to receiving the first contact and identifying at least one of the target device out of a plurality of target devices being responsive to remote control;

displaying a first plurality of remote controls for the controllable target device;

while displaying the first plurality of remote controls, receiving a second contact on the touch-sensitive display, the second contact having a maximum intensity;

in response to receiving the second contact and a determination that the maximum intensity is below a threshold intensity, instructing the controllable target device to perform an operation; and

in response to receiving the second contact and a determination that the maximum intensity is above the threshold intensity, displaying a second plurality of remote controls for the controllable target device.

17. A method, comprising:

at an electronic device with a touch-sensitive display;

displaying a user interface object on the touch-sensitive display of the electronic device, wherein the displayed user interface object represents a remote control application;

receiving data representing a selection on the displayed user interface object;

identifying at least one target device, out of a plurality of target devices, wherein identifying the at least one target device includes;

receiving data from the at least one target device; and

determining that the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control, selecting the identified single target device as the controllable target device; and

in response to receiving the data representing the selection, displaying a plurality of graphical user interface objects for controlling operations of the controllable target device.

19. The method of claim 17, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and further comprising:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity above a threshold intensity; and

in response to the second contact, invoking a user interface for registering a second target electronic device with the electronic device.

20. The method of claim 17. wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and further comprising:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity below a threshold intensity, the location associated with an operation of the first target device;

in response to the second contact, sending an instruction to the target device instructing the target device to perform the operation;

receiving third data representing a third contact on the display, the third contact having a maximum intensity above the threshold intensity; and

in response to the third contact, displaying a user interface for registering a second target device with the remote control application.

21. The method of claim 17, wherein the data representing the selection is a first data, further comprising:

sending, from the electronic device to an intermediate electronic device, data representing an instruction instructing the target device to perform the operation,

wherein the instruction is communicated by the intermediate electronic device to the target device.

22. The method of claim 21, wherein the intermediate electronic device is a phone.

23. The method of claim 17, wherein identifying a target device comprises receiving network service information sent by the target device.

24. The method of claim 17, wherein identifying a target device comprises transmitting a request message and receiving an acknowledgement message sent by the target device, wherein the acknowledge message was sent by the target device in response to the request message.

25. The method of claim 17, wherein the target device is responsive to remote control when the target device is executing an application having an operation that can be requested using the remote control application.

26. The method of claim 25, wherein the application is executing in the background.

27. The method of claim 17, wherein the target device is a device other than the electronic device.

28. The method of claim 17, wherein the plurality of graphical user interface objects comprises transport controls.

29. The method of claim 17, wherein the plurality of graphical user interface objects are determined based on the controllable target device.

30. The method of claim 17, wherein the plurality of graphical user interface objects are shared by the controllable target device and at least one other device of the plurality of target devices.

18. The method of clam further comprising:

in response to an identification that no target device is responsive to remote control, displaying a plurality of graphical user interface objects representing the plurality of target devices.

31. An electronic device, comprising:

a touch-sensitive display;

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a plurality of icons representing applications, including an icon representing a remote control application;

receiving a first contact on the touch-sensitive display at a location of the icon;

identifying at least one target device out of a plurality of target devices, wherein identifying the at least one target device includes;

receiving data from the at least one target device; and

determining that the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control selecting the identified single target device as the controllable target device; and

in response to receiving the first contact identifying at least one of the target device out of a plurality of target devices being responsive to remote control;

displaying a first plurality of remote controls for the controllable target device;

while displaying the first plurality of remote controls, receiving a second contact on the touch-sensitive display, the second contact having a maximum intensity;

in response to receiving the second contact and a determination that the maximum intensity is below a threshold intensity, instructing the controllable target device to perform an operation, and

in response to receiving the second contact and a determination that the maximum intensity is above the threshold intensity, displaying a second plurality of remote controls for the controllable target device.

32. An electronic device, comprising:

a touch-sensitive display;

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a user interface object on the touch-sensitive display of the electronic device, wherein the displayed user interface object represents a remote control application;

receiving data representing a selection on the displayed user interface object;

identifying at least one target device, out of a plurality of target devices, wherein identifying the at least one target, device includes;

receiving data from the at least one target device; and

determining that the at least one target device is responsive to remote control based on the data received from the at least one target device;

in response to identifying two or more target devices, out of the plurality of target devices, responsive to remote control, selecting one target device from the identified two or more target devices as a controllable target device based on a predetermined criteria;

in response to identifying a single target device, out of the plurality of target devices, responsive to remote control selecting the identified single target device as the controllable target device; and

in response to receiving the data representing the selection, displaying a plurality of graphical user interface objects for controlling operations of the controllable target device.

33. The electronic device of claim 32, wherein the one or more programs further comprise instructions for:

in response to an identification that no target device is responsive to remote control, displaying a plurality of graphical user interface objects representing the plurality of target devices.

34. The electronic device of claim 32, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and wherein the one or more programs further comprise instructions for:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity above a threshold intensity; and

in response to the second contact, invoking a user interface for registering a second target electronic device with the electronic device.

35. The electronic device of claim 32, wherein the touch-sensitive display comprises one or more sensors to detect the intensity of contacts with the touch-sensitive display, wherein the data representing the selection is a first data, wherein the target device is a first target device, and wherein the one or more programs further comprise instructions for:

receiving second data representing a second contact at a location on the touch-sensitive display, the second contact having a maximum intensity below a threshold intensity, the location associated with an operation of the first target device;

in response to the second contact, sending an instruction to the target device instructing the target device to perform the operation;

receiving third data representing a third contact on the display, the third contact having a maximum intensity above the threshold intensity; and

in response to the third contact, displaying a user interface for registering a second target device with the remote control application.

36. The electronic device of claim 32, wherein the data representing the selection is a first data, wherein the one or more programs further comprise instructions for:

sending, from the electronic device to an intermediate electronic device, data representing an instruction instructing the target device to perform the operation, wherein the instruction is communicated by the intermediate electronic device to the target device.

37. The electronic device of claim 36, wherein the intermediate electronic device is a phone.

38. The electronic device of claim 32, wherein identifying a target device comprises receiving network service information sent by the target device.

39. The electronic device of claim 32, wherein identifying a target device comprises transmitting a request message and receiving an acknowledgement message sent by the target device, wherein the acknowledge message was sent by the target device in response to the request message.

40. The electronic device of claim 32, wherein the target device is responsive to remote control when the target device is executing an application having an operation that can be requested using the remote control application.

41. The electronic device of claim 40, wherein the application is executing in the background.

42. The electronic device of claim 32, wherein the target device is a device other than the electronic device.

43. The electronic device of claim 32, wherein the plurality of graphical user interface objects comprises transport controls.

44. The electronic device of claim 32, wherein the plurality of graphical user interface objects are determined based on the controllable target device.

45. The electronic device of claim 32, wherein the plurality of graphical user interface objects are shared by the controllable target device and at least one other device of the plurality of target devices.