Dynamic user interface adaptable to multiple input tools

• US 9,665,206 B1
• Filed: 02/06/2015
• Issued: 05/30/2017
• Est. Priority Date: 09/18/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Khoe, May-Li, Missig, Julian, Costanzo, Bianca Cheng, Bernstein, Jeffrey Traer
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): Onyekaba, Amy C
• Application Number:
  US14/616,532
• Time in Patent Office: 844 Days
• Field of Search: 345156-179
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04162 for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
• CPC Class Codes: G06F 3/0441 using active external devices, e.g. active pens, for receiving changes in electrical potential transmitted by the digitiser, e.g. tablet driving signals
• CPC Class Codes: G06F 3/0442 using active external devices, e.g. active pens, for transmitting changes in electrical potential to be received by the digitiser
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A computing device can have a touchscreen user interface operable by different input tools, such as a finger or a stylus. The computing device can dynamically and automatically alter elements of a graphical user interface depending on the type of input tool in use. For example, a stylus can have sensors that detect touch and/or motion, and based on the sensor data, the stylus or the computing device can determine whether the stylus is currently being held in a hand. When the stylus is being held, the interface can be rendered with elements adapted for stylus input, and when the stylus is not being held, the interface can be rendered with elements adapted for finger input.

Claims

1. A method, performed by an electronic device with a touch-sensitive display, the method comprising:

displaying, on the touch-sensitive display, a user interface that includes a user interface object;

while displaying the user interface object on the touch-sensitive display, detecting, via the touch-sensitive display, an input that corresponds to a request to display a supplemental user interface for adjusting the user interface object;

determining, by the electronic device, whether a stylus associated with the device is in an active or idle state;

in response to detecting the input that corresponds to a request to display a supplemental user interface for adjusting the user interface object, selecting a mode for the supplemental user interface based on whether the stylus is in the active state or the idle state, the mode being selected from a stylus mode or a finger mode, including;

in accordance with a determination that the stylus is in the active state, displaying the supplemental user interface in the stylus mode, wherein the supplemental user interface in the stylus mode includes a first type of control for adjusting a first property of the user interface object; and

in accordance with a determination that the stylus is in the idle state, displaying the supplemental user interface in the finger mode, wherein the supplemental user interface in the finger mode includes a second type of control for adjusting the first property of the user interface object, and the second type of control is different from the first type of control.

2. The method of claim 1, wherein the supplemental user interface is displayed concurrently with at least a portion of the user interface object.

3. The method of claim 1, wherein the first type of control is adjusted by longitudinal rotation of the stylus, and the second type of control is adjusted by lateral movement of a finger along the touch-sensitive display.

4. The method of claim 1, wherein the supplemental user interface in the stylus mode includes an input element that is displayed at a smaller size in the stylus mode than a corresponding element of the supplemental user interface in the finger mode.

5. The method of claim 1, wherein the supplemental user interface in the stylus mode includes a larger number of input elements than the supplemental user interface in the finger mode.

6. The method of claim 1, wherein the supplemental user interface in the stylus mode includes a rotational input element operable to control a setting and the supplemental user interface in the finger mode includes a slider input element operable to control the setting.

7. The method of claim 1, further comprising, in the event that the selected mode for the supplemental user interface is the stylus mode and while displaying the supplemental user interface in the stylus mode:

detecting translational motion of a contact point of a tip of the stylus with a surface of the touch-sensitive display;

detecting rotational motion of the stylus about a longitudinal axis, the rotational motion being concurrent with the translational motion;

based on the translational motion, adjusting a second property of the displayed user interface object; and

based on the rotational motion, adjusting a second third property of the displayed user interface object.

8. A computing device comprising:

a touch-sensitive display; and

a processing subsystem coupled to the touch-sensitive display, the processing subsystem being configured to;

display, on the touch-sensitive display, a user interface that includes a user interface object;

detect, via the touch-sensitive display, while displaying the user interface object on the touch-sensitive display, an input that corresponds a request to display a supplemental user interface for adjusting the user interface object;

determine whether a stylus associated with the computing device is in an active or idle state;

select, in response to detecting the input that corresponds to a request to display a supplemental user interface for adjusting the user interface object, a mode for the supplemental user interface based on whether the stylus is in the active state or the idle state, the mode being selected from a stylus mode or a finger mode, including;

in accordance with a determination that the stylus is in the active state, displaying the supplemental user interface in the stylus mode, wherein the supplemental user interface in the stylus mode includes a first type of control for adjusting a first property of the user interface object; and

in accordance with a determination that the stylus is in the idle state, displaying the supplemental user interface in the finger mode, wherein the supplemental user interface in the finger mode includes a second type of control for adjusting the first property of the user interface object, and the second type of control is different from the first type of control.

9. The computing device of claim 8, wherein the supplemental user interface is displayed concurrently with at least a portion of the user interface object.

10. The computing device of claim 8, wherein the supplemental user interface in the stylus mode includes an input element that is displayed at a smaller size in the stylus mode than a corresponding element of the supplemental user interface in the finger mode.

11. The computing device of claim 8, wherein the supplemental user interface in the stylus mode includes a larger number of input elements than the supplemental user interface in the finger mode.

12. The computing device of claim 8, wherein the supplemental user interface in the stylus mode includes a rotational input element operable to control a setting and the supplemental user interface in the finger mode includes a slider input element operable to control the setting.

13. The computing device of claim 8, wherein the first type of control is adjusted by longitudinal rotation of the stylus, and the second type of control is adjusted by lateral movement of a finger along the touch-sensitive display.

14. The computing device of claim 8, wherein the processing subsystem is further configured to:

detect translational motion of a contact point of a tip of the stylus with a surface of the touch-sensitive display;

detect rotational motion of the stylus about a longitudinal axis, the rotational motion being concurrent with the translational motion;

based on the translational motion, adjust a second property of the displayed user interface object; and

based on the rotational motion, adjust a third property of the displayed user interface object.

15. A non-transitory computer readable storage medium storing one or more programs that when executed by a computing device having a touch-sensitive display and a processing subsystem coupled to the touch-sensitive display, causes the computing device to:

display, on the touch-sensitive display, a user interface that includes a user interface object;

detect, via the touch-sensitive display, while displaying the user interface object on the touch-sensitive display, an input that corresponds a request to display a supplemental user interface for adjusting the user interface object;

determine whether a stylus associated with the computing device is in an active or idle state;

select, in response to detecting the input that corresponds to a request to display a supplemental user interface for adjusting the user interface object, a mode for the supplemental user interface based on whether the stylus is in the active state or the idle state, the mode being selected from a stylus mode or a finger mode, including;

in accordance with a determination that the stylus is in the active state, displaying the supplemental user interface in the stylus mode, wherein the supplemental user interface in the stylus mode includes a first type of control for adjusting a first property of the user interface object; and

in accordance with a determination that the stylus is in the idle state, displaying the supplemental user interface in the finger mode, wherein the supplemental user interface in the finger mode includes a second type of control for adjusting the first property of the user interface object, and the second type of control is different from the first type of control.

16. The non-transitory computer readable storage medium of claim 15, wherein the supplemental user interface is displayed concurrently with at least a portion of the user interface object.

17. The non-transitory computer readable storage medium of claim 15, wherein the first type of control is adjusted by longitudinal rotation of the stylus, and the second type of control is adjusted by lateral movement of a finger along the touch-sensitive display.

18. The non-transitory computer readable storage medium of claim 15, wherein the supplemental user interface in the stylus mode includes an input element that is displayed at a smaller size in the stylus mode than a corresponding element of the supplemental user interface in the finger mode.

19. The non-transitory computer readable storage medium of claim 15, wherein the supplemental user interface in the stylus mode includes a larger number of input elements than the supplemental user interface in the finger mode.

20. The non-transitory computer readable storage medium of claim 15, wherein the supplemental user interface in the stylus mode includes a rotational input element operable to control a setting and the supplemental user interface in the finger mode includes a slider input element operable to control the setting.

21. The non-transitory computer readable storage medium of claim 15, wherein the one or more programs further including instructions that when executed by the computing device cause the computing device to:

detect translational motion of a contact point of a tip of the stylus with a surface of the touch-sensitive display;

detect rotational motion of the stylus about a longitudinal axis, the rotational motion being concurrent with the translational motion;

based on the translational motion, adjust a second property of the displayed user interface object; and

based on the rotational motion, adjust a third property of the displayed user interface object.