3D digital painting

• US 8,817,017 B2
• Filed: 05/26/2011
• Issued: 08/26/2014
• Est. Priority Date: 06/01/2010
• Status: Active Grant

• Current Assignee: Vladimir Vaganov
• Original Assignee: Vladimir Vaganov
• Inventors: Vaganov, Vladimir
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Liu, Zhengxi
• Application Number:
  US13/116,015
• Publication Number:
  US 20110292042A1
• Time in Patent Office: 1,188 Days
• US Class Current: 345/419
• CPC Class Codes: B44D 2/002 Kits for drawing or painting
• CPC Class Codes: B44F 7/00 Designs imitating three-dimensional effects
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/31 using parallax barriers
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 using temporal multiplexing

Abstract

The present invention provides method and system for 3D hand-painting and 3D hand-drawing. The system corresponding to the invented method comprises: an electronic monitor and computer, means for 3D digital vision, multi-axis input control device allowing digital painting or drawing on the canvas, additional input control device for virtual changing the position of the canvas between the painter and the monitor, means for 3D image presentation. The system and method provide painting or drawing on the electronic canvas for each of the virtual positions of the canvas by changing this position with the input control device and verifying it with the means for 3D digital vision. The system provides complete digital 2D images of the painting for the right and the left eyes, which are used for presentation of 3D painting by available means for 3D vision.

Claims

1. A method of digital recording of 3D hand-painted and 3D hand-drawn images, said method comprising:

providing a digital electronic canvas for presenting two pictures for right and left eye for painting or drawing in 3D virtual space;

providing means for 3D digital vision (shutter glasses, polaroid glasses, splitting canvas on two canvases for the right and left eye and switching these two canvases between the left and the right eye, or projecting both pictures for the right and the left eye on the same digital electronic canvas) wherein instant positions of digital electronic canvas in 3D virtual space are displayed;

providing at least one hand-held three-axis input control device comprising a digital brush or digital pencil allowing digital painting or drawing on the digital electronic canvas;

painting or drawing on the digital electronic canvas and verifying a position of the digital brush or digital pencil within 3D virtual space with the means for 3D digital vision;

providing a description in digital format of images for the right and the left eyes for every and all instant virtual positions of the digital electronic canvas and of corresponding positions of the digital electronic canvas;

providing complete 2D images of a painting for the right and the left eyes at any stage of painting;

using left and right images of the painting for presentation of 3D painting by available means for 3D vision,

wherein two inputs of at least one hand-held three-axis input control device comprising a digital brush or pencil are used for painting of lines or strokes along two orthogonal axes X and Y in a plane of the digital electronic canvas and the third input is used for painting or drawing the lines or strokes along Z axis between a painter and the digital electronic canvas by changing a distance between the painter and a virtual position of the digital electronic canvas along the Z axis allowing making 3D paintings (drawings) equally continuous in (XY) plane and (XZ) and (YZ) virtual planes and

wherein a motion of the at least one hand-held three-axis input control device for making lines or strokes is provided simultaneously and continuously in all 3D (three dimensions) by free one hand and/or finger movements.

2. A method according to claim 1, further providing a description in digital format of images corresponding each instant virtual position of the canvas and of corresponding positions of the digital electronic canvas and providing complete 3D digital image of the painting for different applications associated with 3D image processing.

3. A method according to claim 1, wherein painting or drawing on the digital electronic canvas for each of the instant virtual positions of the digital electronic canvas in the third dimension will result in simultaneous painting on two canvases by switching canvases between right and left eye with frequency equals a half of a frame refreshing frequency of the digital electronic canvas such that each hand-painted stroke or line would appear on both canvases simultaneously with certain horizontal offsets with respect to a central vertical line on a physical canvas (computer monitor or TV screen) for each virtual canvas position on a distance corresponding to chosen depth of the virtual position of the digital electronic canvas.

4. A method according to claim 1, wherein complete 2D images of the painting for the right and the left eyes are provided by accumulation of all lines and strokes for the right and the left eye separately at any stage of painting.

5. A method according to claim 1, further providing an additional one or more at least one-axis input control device for controlling different functions necessary for a digital 3D painting process, said different functions being selected from a group of functions consisting of turning on and off, navigation of a cursor on a display of an electronic device, scrolling, zooming, shadowing, screening, selecting, deleting, restoring, saving, opening, closing, searching, setting up, previewing, undoing, clearing, repeating, pasting, finding, replacing, inserting, formatting, color selection, color mixing, selection of line or stroke width, selection of brush size, selection of swatch size, selection of sponge size, selection of eraser size, selection of a virtual position of the canvas, selection of a depth of focal plane around the canvas'"'"' virtual position, and selection of special effects.

6. A method according to claim 5, wherein different functions necessary for controlling digital 3D painting process are assigned to different inputs of different input control devices depending on the preferences of a painter and specifics of a required painting task.

7. A method according to claim 1, wherein a virtual instant current position of the canvas is presented by at least one virtual Z-axis having an origin located in a plane parallel to the canvas and having a scale describing a relative distance between the painter and the virtual position of the canvas and each virtual instant current position of the canvas is indicated on a scale of the at least one virtual Z-axis.

8. A method according to claim 7, wherein the virtual instant current position of the canvas is presented by more than one virtual Z-axis, which have different scale on different virtual Z-axes reflecting a different angular position of the virtual canvas with respect to different virtual Z-axes.

9. A method according to claim 8, wherein the virtual instant current position of the canvas is presented as a temporary plane within a 3D grid created by a system of lines including lines connecting points on different virtual Z-axes corresponding to the same position of the virtual canvas.

10. A system for digital recording of 3D hand-painted and 3D hand-drawn images, said system comprising:

a digital electronic canvas for presenting two pictures for right and left eye for painting or drawing in 3D virtual space;

means for 3D digital vision (shutter glasses, polaroid glasses, splitting canvas on two canvases for right and left eye and switching these two canvases between left and right eye, or projecting both pictures for right and left eye on the same digital electronic canvas) wherein instant positions of virtual canvas in 3D virtual space are displayed;

at least one hand-held three-axis input control device comprising a digital brush or digital pencil allowing digital painting or drawing on the digital electronic canvas;

means for 3D image presentation;

wherein two inputs of at least one hand-held three-axis input control device are used for painting of lines or strokes along two orthogonal axes X and Y in a plane of the digital electronic canvas and the third input is used for painting or drawing the lines or strokes along Z axis between a painter and the digital electronic canvas by changing a distance between the painter and a virtual position of the digital electronic canvas along the Z axis allowing making 3D paintings (drawings) equally continuous in (XY) plane and (XZ) and (YZ) virtual planes; and

wherein a motion of at least one hand-held three-axis input control device for making lines or strokes is provided simultaneously and continuously in all 3D (three dimensions) by free one hand and/or fingers movements; and

wherein the system also provides a description in digital format of images for the right and the left eyes for every and all instant virtual position of the digital electronic canvas and of corresponding positions of the digital electronic canvas; and

the system also provides complete 2D images of a painting for the right and the left eyes at any stage of painting as accumulation of all lines and strokes made up to that stage; and

the system also provides complete 3D images of the painting by superposition of all layers corresponding to all virtual positions of the digital electronic canvas and the system provides, using 2D left and right images, ability for presentation of 3D painting.

11. A system according to claim 10, wherein an additional one or more at least one-axis input control device for controlling different functions necessary for a digital 3D painting is selected from a group of control devices consisting of:

a scroll wheel, a sensitive pad, a mouse, a track ball, a joystick, a finger joystick, a key-button, a touch pad, a touch screen, and a motion sensor.

12. A system according to claim 10, wherein means for presentation of 3D images include:

optical projection of two images with polarized light onto one screen and using polarized glasses, optical projection of two images with colored light onto one screen and using corresponding colored glasses, stereoscopic glasses and two photographs or slides, 3D digital personal viewer, shutter glasses and synchronous switching display between left and right eye, polaroid glasses and switching display between left and right eye polarized images, 3D monitor with a system of vertical cylindrical lenses and corresponding striped overlapping images for the right and left eyes, and direct projection of left and right images onto retina of the eyes.

13. A system according to claim 12, wherein the 3D digital personal viewer comprises:

two micro-displays for left and right eye;

two lenses of glasses for projecting images from the micro-displays into eyes;

processor;

memory;

power supply;

transceiver; and

body of the 3D digital personal viewer integrating all elements of the viewer.

14. A system according to claim 10, wherein the at least one hand-held three-axis input control device allowing digital painting or drawing on the canvas comprises:

an end-point coupled to an integrated at least one three-axis force sensor within the at least one input control device;

integrated circuit (IC) for digitizing information from at least one three-axis force sensor and processing data related to a change of force vector components;

hardware and software for providing a description in digital format of how the input control device has been moved over a working surface based at least in part on the correlation between the force vector components and corresponding movement vector components; and

hardware and software for providing a description in digital format of how the input control device has been pressed to the working surface based at least in part on a force applied by the end-point to the working surface, wherein the input control device is moving with a hand and/or fingers such that the end-point is contacting the working surface suitable for such movement across the working surface in a process of painting, drawing, writing or cursor navigating while recording with the integrated at least one three-axis force sensor a change of X and Y force vector components representing a motion of the at least one input control device in XY plane and a change of Z force vector component applied to the at least one three-axis force sensor by the end-point representing a virtual motion of the input control device in Z direction.

15. A system according to claim 14, further comprising at least one sensor chosen from the group of sensors consisting of:

one-axis linear accelerometer, two-axis linear accelerometer, three-axis linear accelerometer, one-axis gyro, two-axis gyro, three-axis gyro, one-axis angular accelerometer, two-axis angular accelerometer, three-axis angular accelerometer, one-axis compass, two-axis compass, three-axis compass for determining motion parameters, at which the device has been moved between recording of two positions on the working surface.

16. A system according to claim 11, wherein the at least one hand-held three-axis input control device allowing digital painting or drawing on the canvas comprises:

a device having an end-point coupled to an integrated at least one three-axis force sensor within the device;

at least one at least two-axis motion sensor within the device;

any working surface suitable for moving the device having an end-point across the working surface in a process of painting or drawing;

IC (integrated circuit) for digitizing an information from at least one three-axis force sensor and from at least one at least two-axis motion sensor and processing data related to change of force vector components and motion vector components;

hardware and software for providing a description in digital format of how the device has been pressed to the working surface based at least in part on a vector force externally applied by the end-point to the working surface; and

hardware and software for providing a description in digital format of how the device has been moved over the working surface based at least in part on a correlation between force vector components and corresponding motion vector components, wherein the device is moving with a hand and/or fingers such that the end-point is contacting a surface suitable for such movement across the working surface in a process of painting, drawing, writing or cursor navigating while recording with the integrated at least one three-axis force sensor a change of a vector of force applied to the at least one three-axis force sensor by the end-point and recording with at least one at least two-axis motion sensor a change of a vector of motion representing together with three-axis force sensor an entire motion of the device in all three dimensions.

17. A system according to claim 16, wherein the at least one three-axis force sensor comprises of:

a semiconductor substrate sensor chip, said semiconductor substrate sensor chip comprising a frame element, a rigid island element and an elastic element mechanically coupling said frame element and said rigid island elements;

a set of three or more stress-sensitive IC components integrated into the elastic element;

at least one force-transferring element coupling the end-point to a rigid island element of a sensor die for transferring the applied external vector force from the end-point through at least one force-transferring element to the rigid island element of the sensor die, and thereby generating electrical outputs from application an external force vector via stress-sensitive IC components positioned in the elastic element, wherein each stress-sensitive IC component is carrying a unique orthogonal system components of signal from the external force vector, and the IC component outputs are functions of input component stresses developed from an orthogonal system of external force vector components.

18. A system according to claim 16, wherein the at least one at least two-axis motion sensor is chosen from a group:

linear acceleration sensor, angular acceleration sensor, angular rate sensor or combination of any above by measured parameter and/or number of axes.

19. A system according to claim 10, further comprising an additional one or more at least one-axis input control device for controlling different functions necessary for a digital 3D painting process, which include turning on and off, navigation of a cursor on a display of an electronic device, scrolling, zooming, rotation, shadowing, screening, selecting, deleting, restoring, saving, opening, closing, searching, setting up, previewing, undoing, clearing, repeating, pasting, finding, replacing, inserting, formatting, color selection, color mixing, selecting a line or stroke width, brush size, swatch size, sponge size, eraser size, a virtual position of the canvas, a depth of a focal plane around the canvas'"'"' virtual position, and special effects.