Interactions based on multiple stylus inputs

• US 9,459,794 B1
• Filed: 03/24/2014
• Issued: 10/04/2016
• Est. Priority Date: 03/24/2014
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Soegiono, Orry Wijanarko
• Primary Examiner(s): NGUYEN, JENNIFER T
• Application Number:
  US14/223,202
• Time in Patent Office: 925 Days
• Field of Search: 345/156, 345/157, 345/173, 345/179, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04807 Pen manipulated menu
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

In some examples, a stylus may be used to make a first input including a first shape at a first location on a display. The first location may correspond to a location of a graphic element presented on the display, and a mark consistent with the shape may be presented on the display, such as overlying the graphic element. Additionally, the stylus may be used to make a second input including a second shape. For example, the second shape may correspond to a command for performing an operation. In response, to the entry of the first shape and the second shape, the operation may be performed on or with respect to the graphic element. In some instances, various different shapes and combinations of shapes may be mapped to various different operations, designations, destinations, and so forth.

Claims

1. An electronic device comprising:

a display;

a touch surface overlying the display;

one or more processors;

one or more non-transitory computer-readable media; and

one or more modules maintained on the one or more non-transitory computer-readable media, which, when executed by the one or more processors, cause the one or more processors to;

receive first input information via a stylus on the touch surface;

determine that the first input information is received at a first location overlying, at least in part, a first graphic element presented on the display;

compare the first input information to one or more shape profiles to determine a first shape corresponding to the first input information;

receive second input information via the stylus on the touch surface;

determine that the second input information is received at a second location overlying, at least in part, a second graphic element presented on the display;

compare the second input information to the one or more shape profiles to determine a second shape corresponding to the second input information, wherein the second shape is different from the first shape;

receive third input information via the stylus on the touch surface;

compare the third input information to the one or more shape profiles to determine a third shape corresponding to the third input information, wherein the third shape is different from the first shape and the second shape;

determine a first operation to perform in relation to the first graphic element based at least in part on the first shape and the third shape;

determine a second operation to perform in relation to the second graphic element based at least in part on the second shape and the third shape; and

perform the first operation and the second operation, the first operation relating to the first graphic element and the second operation relating to the second graphic element, the first operation being a different operation from the second operation based at least in part on the first shape being different from the second shape.

2. The electronic device as recited in claim 1, wherein comparing the first input information to the one or more shape profiles further comprises:

determining, based on a sequence of scan data from the touch sensor, the first shape from the first input information;

comparing the first shape from the first input information with the one or more shape profiles; and

determining that the first shape from the first input information matches a particular one of the one or more shape profiles within at least a threshold amount.

3. The electronic device as recited in claim 1, wherein the one or more modules further cause the one or more processors to:

determine a first position of the stylus with respect to the touch surface;

present, on the display, at least one of the first graphic element or the second graphic element at a second location based at least in part on the first position of the stylus; and

in response to detecting movement of the stylus to a second position relative to the touch surface, present on the display the at least one of the first graphic element or the second graphic element at a third location based at least in part on the second position of the stylus.

4. The electronic device as recited in claim 3, wherein the one or more modules further cause the one or more processors to detect at least one of the first position of the stylus or the second position of the stylus while the stylus is within a threshold distance away from the touch surface.

5. The electronic device as recited in claim 1, wherein the one or more modules further cause the one or more processors to:

receive fourth input information via the stylus on the touch surface, the fourth input information being received at a third location overlying, at least in part, a third graphic element presented on the display;

compare the fourth input information to the one or more shape profiles to determine a fourth shape corresponding to the fourth input information, wherein the fourth shape is different from the first shape, the second shape, and the third shape;

determine a third operation to perform in relation to the third graphic element based at least in part on the fourth shape and the third shape; and

perform the third operation relating to the third graphic element, the third operation being a different operation from the first operation and the second operation based at least in part on the fourth shape being different from the first shape and the second shape.

6. A method comprising:

receiving, at a touch surface associated with a display of an electronic device, first input information;

determining, by the electronic device, that the first input information corresponds to a first shape;

determining, by the electronic device, a first graphic element presented on the display at a first location corresponding to the first input information;

receiving, at the touch surface, second input information corresponding to a second shape, wherein the second shape is different from the first shape;

determining, by the electronic device, a second graphic element presented on the display at a second location corresponding to the second input information;

receiving, at a time after the second input information is received, at the touch surface, third input information;

determining, by the electronic device, that the third input information corresponds to a third shape, wherein the third shape is different from the first shape and the second shape;

based at least in part on the first shape and the third shape, performing, by the electronic device, a first operation that is related to the first graphic element; and

based at least in part on the second shape and the third shape, performing, by the electronic device, a second operation that is different from the first operation and that is related to the second graphic element.

7. The method as recited in claim 6, wherein determining that the first input information corresponds to the first shape further comprises:

determining, based on scan data from a touch sensor, the first shape from the first input information; and

comparing the shape from the first input information to one or more shape profiles to determine that the first shape from the first input information matches the first shape.

8. The method as recited in claim 6, wherein:

performing the first operation comprises presenting the first graphic element at a different position on the display; and

performing the second operation comprises deleting the second graphic element from an image currently presented on the display.

9. The method as recited in claim 6, wherein:

performing the first operation comprises presenting the first graphic element at a different position on the display; and

performing the second operation comprises presenting a copy of the second graphic element on the display.

10. The method as recited in claim 6, wherein performing the first operation comprises:

determining a position of a tip of a stylus with respect to the touch surface of the electronic device; and

presenting the first graphic element at a location on the display that corresponds to the position of the tip of the stylus.

11. The method as recited in claim 10, wherein the location on the display that corresponds to the position of the tip of the stylus is different from the first location.

12. The method as recited in claim 6, further comprising determining the position of the tip of the stylus while the stylus is within a threshold distance away from the touch surface.

13. The method as recited in claim 6, wherein performing the first operation comprises performing an additional operation on an object represented by the first graphic element, wherein the object is at least one of:

a data object;

an application;

a control of the electronic device;

or

a feature of the electronic device.

14. The method as recited in claim 6, further comprising:

determining that the first shape indicates that the first graphic element is selected; and

determining that the second shape indicates a type of the first operation to be performed with respect to the first graphic element.

15. The method as recited in claim 6, further comprising:

determining that at least one of the first shape or the third shape indicates a number; and

wherein the first operation comprises changing an appearance of the first graphic element on the display based at least in part on the number.

16. A method comprising:

receiving, by an electronic device, a first input from a stylus;

determining, by the electronic device, that the first input includes a first shape;

receiving, by the electronic device, a second input from the stylus;

determining, by the electronic device, that the second input includes a second shape, wherein a first location of the second input corresponds to a first graphic element;

receiving, by the electronic device and at a time after the second input is received, a third input from the stylus;

determining, by the electronic device, that the third input includes a third shape;

determining, by the electronic device, that a second location of at least one of the first input or the third input corresponds to a third location at which a second graphic element is presented on a display associated with the electronic device;

performing, by the electronic device, based at least in part on a shape-to-operation mapping, a first operation corresponding to the first shape and the third shape; and

performing, by the electronic device, a second operation with respect to the first graphic element in response to the third input, based at least in part on the second shape being different from the first shape.

17. The method as recited in claim 16, further comprising:

determining that the first shape indicates selection of a control for the electronic device; and

determining that the third shape indicates an adjustment to the control, wherein performing the operation comprises adjusting the control based on the third shape.

18. The method as recited in claim 16, further comprising:

determining that at least one shape of the first shape or the third shape is a number; and

determining that a different shape of the first shape or the third shape indicates that the number is to be applied to at least one of;

adjusting a setting of a control based on the number;

or

modifying an appearance of the second graphic element based on the number.

19. The method as recited in claim 16, further comprising:

determining that the first shape indicates a selection of the second graphic element; and

determining, based at least in part on the shape-to-operation mapping, that receiving the first shape and the third shape indicates that the operation is at least one of;

changing an appearance of the second graphic element;

changing the third location of the second graphic element;

creating a copy of the second graphic element;

or

deleting the second graphic element.

20. The method as recited in claim 16, further comprising, prior to performing the first operation with respect to the second graphic element, comparing the first shape and the third shape with a plurality of combinations of shapes in the shape-to-operation mapping to determine that the second operation corresponds to receiving the first shape and the third shape.