Methods and Apparatus for Natural Media Painting Using a Realistic Brush and Tablet Stylus Gestures

• US 20130125068A1
• Filed: 07/08/2010
• Published: 05/16/2013
• Est. Priority Date: 07/10/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Harris, Jerry G., DiVerdi, Stephen J., Hadap, Sunil
• Granted Patent: US 9,483,138 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Systems and methods for providing a natural media painting application may receive user inputs through tablet stylus gestures. A user interface may detect stylus gestures that mimic real-world actions of artists based on information collected during user manipulation of the stylus, and may map the gestures to various digital painting and image editing tasks that may be invoked and/or controlled using the gesture-based inputs. The collected information may include spatial and/or directional information, acceleration data, an initial and/or ending position of the stylus, an initial and/or ending orientation of the stylus, and/or pressure data. The stylus gestures may include translations, rotations, twisting motions, mashing gestures, or jerking motions. The application may perform appropriate painting and image editing actions in response to detecting and recognizing the stylus gestures, and the actions taken may be dependent on the work mode and/or context of the graphics application in which stylus gesture was performed.

Claims

1. A method, comprising:

using a computer to perform;

collecting information about a user manipulation of a stylus in relation to a tablet device associated with the computer;

determining, from the collected information, a stylus gesture performed by the user via manipulation of the stylus, wherein the stylus gesture is one of a plurality of stylus gestures that each correspond to at least one of a plurality of actions in a graphics application and at least some of the stylus gestures are mapped to user manipulation of the stylus at a distance from the tablet device and at least some of the stylus gestures involve contact of the stylus with the tablet device;

determining a particular one of the actions that corresponds to the determined stylus gesture; and

performing a painting function for a digital image in the graphics application, wherein performing the painting function comprises performing the particular one of the actions.

2. The method of claim 1, wherein the tablet device is an integrated component of the computer.

3. The method of claim 1, wherein the collected information comprises information about the pressure applied to the tablet device by the stylus during the manipulation of the stylus.

4. The method of claim 1, wherein the graphics application comprises a natural media painting application, and wherein the painting function represents a real-world painting action using a brush tool.

5. The method of claim 1, wherein the plurality of stylus gestures includes a translation in one or more dimensions, a twisting motion, a barrel rotation, a fanning motion, a mashing down of the stylus into a tablet device, a jerking up motion, a jerking down motion, shaking the stylus away from the tablet device, and shaking the stylus toward the tablet device.

6. The method of claim 1, wherein the plurality of painting functions comprises one or more of:

sharpening a point of a brush tool, cleaning the brush tool, refilling the brush tool, changing a tool, drying the brush tool, splattering paint from the brush tool, adding paint to the brush tool saturating the brush tool, homogenizing the colors on the tip of the brush tool, splitting the tip of the brush tool or changing a work mode in the graphics application.

7. The method of claim 1, wherein the collected information comprises spatial or directional information collected during the manipulation of the stylus.

8. The method of claim 1, wherein the collected information comprises acceleration data collected during the manipulation of the stylus.

9. The method of claim 1, wherein the collected information comprises one or more of:

an initial position of the stylus, an ending position of the stylus, an initial orientation of the stylus, or an ending orientation of the stylus.

10. The method of claim 1, wherein the particular one of the actions is selected based, at least in part, on a context in which the user manipulation of the stylus was performed.

11. A system, comprising:

a stylus input module;

a computer device; and

a display device;

wherein during operation, the computer device executes the stylus input module to perform operations to;

collect information about a user manipulation of a stylus in relation to the computer device for both manipulation at a distance from the computer device and manipulation to provide touch input, the information including at least information regarding motion, orientation, and position of the stylus and an amount of pressure applied with the stylus;

determine, from the collected information, a stylus gesture performed by the user via manipulation of the stylus, wherein the stylus gesture is one of a plurality of stylus gestures that each correspond to at least one of a plurality of actions in a graphics application;

determine a particular one of the actions that corresponds to the determined stylus gesture; and

perform a painting function for a digital image displayed on the display device by the graphics application, wherein performing the painting function comprises performing the particular one of the actions.

12. The system of claim 11, further comprising:

a pressure sensitive surface;

wherein the collected information comprises information about the manipulation of the stylus in relation to the pressure sensitive surface.

13. The system of claim 11, wherein the graphics application comprises a natural media painting application, and wherein the painting function comprises a simulation of a real-world painting action using a brush tool.

14. The system of claim 11, wherein the collected information comprises one or more of:

spatial information collected during the manipulation of the stylus, directional information collected during the manipulation of the stylus, acceleration data collected during the manipulation of the stylus, an initial position of the stylus, an ending position of the stylus, an initial orientation of the stylus, or an ending orientation of the stylus.

15. The system of claim 11, wherein the plurality of stylus gestures includes a barrel rotation, a fanning motion, mashing down the stylus, a jerking up motion, a jerking down motion, shaking the stylus away from a tablet device, and shaking the stylus toward the tablet.

16. A computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to perform acts comprising:

collecting information about a user manipulation of a stylus indicative of six degrees of freedom movement of the stylus relative to an initial pose of the stylus in three-dimensional space relative to a touch input surface and information regarding touch or pressure input applied to the touch input surface;

determining, from the collected information, a stylus gesture performed by the user via manipulation of the stylus, wherein the stylus gesture is one of a plurality of stylus gestures that each correspond to at least one of a plurality of actions in a graphics application;

determining a particular one of the actions that corresponds to the determined stylus gesture; and

performing a painting function for a digital image in the graphics application, wherein performing the painting function comprises performing the particular one of the actions.

17. The storage medium of claim 16, wherein the collected information comprises information about the manipulation of the stylus in relation to a tablet device having the touch input surface.

18. The storage medium of claim 16, wherein the graphics application comprises a natural media painting application, and wherein the painting function comprises a simulation of a real-world painting action using a brush tool.

19. The storage medium of claim 16, wherein the collected information comprises one or more of:

spatial information collected during the manipulation of the stylus, directional information collected during the manipulation of the stylus, acceleration data collected during the manipulation of the stylus, an initial position of the stylus, an ending position of the stylus, an initial orientation of the stylus, or an ending orientation of the stylus.

20. The storage medium of claim 16, wherein the plurality of stylus gestures includes a barrel rotation, a fanning motion, mashing down the stylus, a jerking up motion, a jerking down motion, shaking the stylus away from the touch input surface, and shaking the stylus toward touch input surface.