Natural media painting using proximity-based tablet stylus gestures

• US 9,645,664 B2
• Filed: 10/16/2013
• Issued: 05/09/2017
• Est. Priority Date: 07/10/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Harris, Jerry G.
• Primary Examiner(s): McDowell, Jr., Maurice L
• Assistant Examiner(s): Ricks, Donna J
• Application Number:
  US14/055,327
• Publication Number:
  US 20140104211A1
• Time in Patent Office: 1,301 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Techniques for natural media painting using proximity-based tablet stylus gestures are described. A stylus is implemented for user manipulation to simulate a brush stroke of a paint brush, where the stylus includes an application tip formed from individual virtual bristles that simulate the paint brush. A graphics application is implemented to receive an input of the brush stroke, compute an effect of each individual virtual bristle of the brush stroke to determine simulated paint application on a virtual canvas, and initiate a display of the simulated paint application on the virtual canvas based on the computed effect of each individual virtual bristle of the brush stroke.

Claims

1. A system, comprising:

a stylus to track user manipulation to simulate paint deposition of a paint brush, the stylus comprising one or more accelerometers and one or more positional sensors;

a processor; and

a memory having computer readable media with instructions stored thereon that are executable on the processor to implement a graphics application to perform operations comprising;

receiving an input of paint deposition from the stylus, the input comprising accelerometer and positional data indicating that the stylus is located at an initial location above a tablet device not contacting the tablet device and followed by additional accelerometer and positional data indicating the stylus moved at a speed and a direction to a second location closer to the tablet device not contacting the tablet device;

employing a brush model to simulate the paint deposition of the paint brush, the brush model simulating individual virtual bristles of the paint brush being modeled to change shape during the paint deposition;

computing an effect of the individual virtual bristles during the paint deposition based on the input received to determine the simulated paint deposition on a virtual canvas; and

initiating a display of the simulated paint deposition on the virtual canvas based on the computed effect of the individual virtual bristles of the paint brush during the paint deposition.

2. The system as recited in claim 1, wherein the graphics application is configured to simulate the change of shape of the individual virtual bristles during the paint deposition.

3. The system as recited in claim 2, wherein the graphics application is further configured to said compute the effect of the individual virtual bristles during the paint deposition based on the change of shape of the individual virtual bristles during the paint deposition.

4. The system as recited in claim 1, wherein the stylus further comprises a tip fixed to a handle, the tip represented by a number of paint brush bristles.

5. The system as recited in claim 1, further comprising

a display device configured to display the virtual canvas and the simulated paint deposition, the display device configured as one of an integrated display device of the tablet device, or as a peripheral display device of the tablet device.

6. A system as recited in claim 1, wherein the operations further comprise determining if the stylus is within a defined proximity of the tablet device before receiving the input of the paint deposition from the stylus.

7. A system as recited in claim 1, wherein the stylus further comprises a wireless communication device, the wireless communication device transmitting the stylus input to a wireless receiving device in the tablet device.

8. A system as recited in claim 1, further comprising simulating splattering drops of paint on the virtual canvas based on recognition of a flicking motion of the stylus.

9. A method, comprising:

receiving an input of paint deposition from a stylus having one or more accelerometers and one or more positional sensors, the input comprising acceleration and positional data indicating that a position of the stylus moved relative to a tablet device from an initial location above and not contacting the tablet device to a second location closer to the tablet device and not contacting the tablet device;

employing a brush model to simulate the paint deposition for a virtual paint brush, the brush model simulating individual virtual bristles being modeled to change shape during the paint deposition;

computing an effect of the individual virtual bristles during the paint deposition based on the acceleration and positional data indicating that the position of the stylus moved from the initial location to the second location to determine the simulated paint deposition on the virtual canvas; and

displaying the simulated paint deposition on the virtual canvas based on the computed effect of the individual virtual bristles of the virtual paint brush during the paint deposition.

10. The method as recited in claim 9, further comprising simulating the change of shape of the individual virtual bristles during the paint deposition.

11. The method as recited in claim 10, wherein said computing the effect of the individual virtual bristles during the paint deposition is based on the change of shape of the individual virtual bristles during the paint deposition.

12. The method as recited in claim 9, wherein the stylus further comprises a tip fixed to a handle, the tip represented by a number of paint brush bristles.

13. The method as recited in claim 9, further comprising:

executing a graphics application with a processing system of the tablet device to implement the method, wherein;

the tablet device is configured as a representation of the virtual canvas and as a stylus input interface to the computing device; and

a display device associated with the tablet device displays the virtual canvas and the simulated paint deposition.

14. The method as recited in claim 13, further comprising:

receiving user contact input on a touch surface of the tablet device;

detecting stylus contact on the touch surface of the tablet device as an additional input to the graphics application;

determining a user and stylus combined input based on a combination of the user contact input and the stylus contact; and

initiating a user assist mode of the graphics application effective to assist completion of the paint deposition.

15. A method as recited in claim 9, further comprising determining if the stylus is within a defined proximity of the tablet device before receiving the input of paint deposition from the stylus.

16. A method as recited in claim 9, wherein the stylus comprises one or more accelerometers and one or more positional sensors, and the data being received from the one or more accelerometers and the one or more positional sensors.

17. A method as recited in claim 9, further comprising recognizing a flicking motion of the stylus, and based on the recognition simulating a flicking motion of the virtual brush, including splattering drops of paint on the virtual canvas.

18. A system, comprising:

a stylus to track user manipulation to simulate paint deposition of a paint brush on a virtual canvas;

a tablet device to represent the virtual canvas and a stylus input interface, the tablet device including a touch surface to detect stylus input and user contact on the touch surface; and

a processing system to implement a graphics application to;

receive the user contact on the touch surface of the tablet device;

receive the paint deposition as the stylus input, the input comprising data from at least one accelerometer and at least one positional sensor indicating that the stylus is located at an initial location above the tablet device not contacting the tablet device followed by data indicating the stylus moved at a speed and a direction to a second location closer to the tablet device not contacting the tablet device;

employ a brush model to simulate the paint deposition of the paint brush, the brush model simulating individual virtual bristles of the paint brush changing shape during the paint deposition;

compute an effect of the individual virtual bristles during the paint deposition to determine the simulated paint deposition on the virtual canvas;

determine a user and stylus combined input based on a combination of the user contact on the touch surface of the tablet device and the stylus input; and

initiate a user assist mode of the graphics application effective to assist completion of the paint deposition.

19. A system as recited in claim 18, wherein the user and stylus combined input comprises the user contact on the touch surface of the tablet device occurring at a same time as the stylus input.

20. A system as recited in claim 18, further comprising:

recognizing a flicking motion of the stylus by the user; and

simulating a flicking motion of the paint brush to splatter drops of paint on the virtual canvas.