Zero parallax drawing within a three dimensional display

• US 10,019,130 B2
• Filed: 04/21/2014
• Issued: 07/10/2018
• Est. Priority Date: 04/21/2013
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Ullmann, Peter F., Champion, Clifford S.
• Primary Examiner(s): Vu, Kieu
• Assistant Examiner(s): Chung, Andrew
• Application Number:
  US14/257,623
• Publication Number:
  US 20140317575A1
• Time in Patent Office: 1,541 Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/334 using spectral multiplexing
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

Systems and methods for digitally drawing on virtual 3D object surfaces using a 3D display system. A 3D drawing mode may be enabled and a display screen of the system may correspond to a zero parallax plane of a 3D scene that may present a plurality of surfaces at non-zero parallax planes. User input may be received at a location on the display screen, and in response, a surface may be specified, rendered, and displayed at the zero parallax plane. Further, additional user input on the display screen may be received specifying drawing motion across the rendered and displayed surface. The drawing motion may start at the location and continue across a boundary between the surface and another contiguous surface. Accordingly, in response to the drawing motion crossing the boundary, the contiguous surface may be rendered and displayed at the zero parallax plane along with results of the drawing motion.

Claims

1. A method for digitally drawing on virtual three dimensional (3D) object surfaces using a 3D stereoscopic display system, comprising:

detecting enablement of a drawing mode of the 3D stereoscopic display system, wherein the 3D stereoscopic display system displays a first rendering of a 3D scene via a display screen, wherein the display screen corresponds to a zero parallax plane of the rendered 3D scene, and wherein the 3D scene comprises a plurality of surfaces presented in the 3D scene at non-zero parallax planes;

receiving first user input at a location on the display screen, wherein the first user input specifies a first surface of the plurality of surfaces, wherein the first surface is presented at a first non-zero parallax plane;

selecting, in response to said receiving first user input, the first surface;

displaying a second rendering, wherein the first surface is rendered at the zero parallax plane in the second rendering;

receiving second user input on the display screen specifying a painting motion across the second rendering of the first surface beginning at the location and continuing to a boundary between the first surface and a second surface, and wherein the second surface is not rendered at the zero parallax plane in the second rendering; and

displaying a third rendering in response to the painting motion crossing the boundary, wherein the second surface is rendered at the zero parallax plane and the first surface is rendered at a non-zero parallax plane in the third rendering, including rendering and displaying results of said painting motion on the first and second surfaces.

2. The method of claim 1, further comprising:

receiving one or more additional user inputs on the display screen specifying a corresponding one or more additional painting motions, wherein each of the one or more additional painting motions continues to a respective boundary between a respective pair of contiguous surfaces of the plurality of surfaces, and wherein a first additional painting motion of the one or more painting motions continues to a boundary between the second surface and a third surface, wherein the third surface is presented at a second non-zero parallax plane; and

displaying a fourth rendering in response to the first additional painting motion crossing the boundary between the second and third surfaces, wherein the third surface is rendered at the zero parallax plane and the first and second surfaces are rendered at a third non-zero parallax plane, including rendering and displaying results of said painting motion on the first and second surfaces sand rendering and displaying results of said first additional painting motion on the second and third surfaces.

3. The method of claim 1, wherein the first and second surfaces are contiguous.

4. The method of claim 1, wherein displaying the second rendering comprises transforming the first surface from the first non-zero parallax plane to the zero parallax plane, wherein said transforming comprises one or more of a translation or a rotation.

5. The method of claim 1, wherein the 3D scene comprises a plurality of surfaces presented in the 3D scene at non-zero parallax planes with respect to a camera POV, and wherein said displaying the second rendering comprises:

transforming the camera POV to render and display the first surface at the zero parallax plane.

6. The method of claim 1, wherein receiving first user input comprises receiving first user input via a user control element, wherein the user control element has a corresponding position, and wherein said selecting further comprises:

determining an intersection between a query ray and the first surface, wherein the query ray intersects the zero parallax plane at the corresponding position of the user control element, and wherein the query ray has a direction specified by one of;

a corresponding orientation of the user control element;

a corresponding orientation of the user'"'"'s point of view (POV) relative to the corresponding position of the user control element;

an orientation of a defined virtual POV relative to the corresponding position of the user control element;

or

a default orientation based on one or more of;

a type of the first surface;

or

a mode of the 3D stereoscopic display system.

7. The method of claim 6, wherein said displaying the second rendering comprises:

translating the first surface from the first non-zero parallax plane to the zero parallax plane along the query ray such that the intersection between the query ray and the first surface is coincident to the corresponding position of the user control element.

8. The method of claim 6, wherein said displaying a first rendering comprises:

rotating the first surface from the first non-zero parallax plane to the zero parallax plane such that a normal vector at the intersection between the query ray and the first surface is parallel to a normal vector of the zero parallax plane.

9. A non-transitory computer readable memory medium storing programming instructions for drawing on virtual three dimensional (3D) object surfaces using a 3D stereoscopic display system, wherein the programming instructions are executable by a processor to:

detect enablement of a drawing mode of the 3D stereoscopic display system, wherein the 3D stereoscopic display system displays a first rendering of a 3D scene via a display screen, wherein the display screen corresponds to a zero parallax plane of the 3D scene, and wherein the 3D scene comprises a plurality of surfaces presented in the 3D scene at non-zero parallax planes;

receive first user input at a location on the display screen, wherein the first user input specifies a first surface of the plurality of surfaces, wherein the first surface is presented at a first non-zero parallax plane;

select, in response to said receiving first user input, the first surface;

display a second rendering, wherein the first surface is rendered at the zero parallax plane in the second rendering;

receive second user input on the display screen specifying a painting motion across the second rendering of the first surface beginning at the location and continuing to a boundary between the first surface and a second surface, and wherein the second surface is not rendered at the zero parallax plane in the second rendering; and

display a third rendering in response to the painting motion crossing the boundary, wherein the second surface is rendered at the zero-parallax plane and the first surface is rendered at a non-zero parallax plane in the third rendering, including rendering and displaying results of said painting motion on the first and second surfaces.

10. The non-transitory computer readable memory medium of claim 9, wherein the programming instruction are further executable by a processor to:

receive one or more additional user inputs on the display screen specifying a corresponding one or more additional painting motions, wherein each of the one or more additional painting motions continues to a respective boundary between a respective pair of contiguous surfaces of the plurality of surfaces, and wherein a first additional painting motion of the one or more painting motions continues to a boundary between the second surface and a third surface, wherein the third surface is presented at a second non-zero parallax plane;

display a fourth rendering in response to the first additional painting motion crossing the boundary between the second and third surfaces, wherein the third surface is rendered at the zero parallax plane and the first and second surfaces are rendered at a third non-zero parallax plane, including rendering and displaying results of said painting motion on the first and second surfaces and rendering and displaying results of said first additional painting motion on the second and third surfaces.

11. The non-transitory computer readable memory medium of claim 9, wherein the first and second surfaces are contiguous.

12. The non-transitory computer readable memory medium of claim 9, wherein to display the second rendering, the programming instructions are further executable by a processor to transform the first surface from the first non-zero parallax plane to the zero parallax plane, wherein said transforming comprises on or more of a translation and a rotation.

13. The non-transitory computer readable memory medium of claim 9, wherein the 3D scene comprises a plurality of surfaces presented in the 3D scene at non-zero parallax planes with respect to a camera POV, and wherein to display the second rendering the programming instructions are further executable by a processor to:

transform the specified POV to render and display the first surface at the zero parallax plane.

14. The non-transitory-computer readable memory medium of claim 9, wherein to receive first user input, the programming instructions are further executable by a processor to receive first user input via a user control element, wherein the user control element has a corresponding position, and wherein to select the first surface, the programming instructions are further executable by a processor to:

determine an intersection between a query ray and the first surface, wherein the query ray intersects the zero parallax plane at the corresponding position of the user control element, and wherein the query ray has a direction specified by one of;

a corresponding orientation of the user control element;

a corresponding orientation of the user'"'"'s point of view (POV) relative to the corresponding position of the user control element;

an orientation of a defined virtual POV relative to the corresponding position of the user control element;

or

a default orientation based on one or more of;

a type of the first surface;

or

a mode of the 3D stereoscopic display system.

15. The non-transitory computer readable memory medium of claim 14, wherein to display the second rendering, the programming instructions are further executable by a processor to:

translate the first surface from the first non-zero parallax plane to the zero parallax plane along the query ray such that the intersection between the query ray and the first surface is coincident to the corresponding position of the user control element.

16. The non-transitory computer readable memory medium of claim 14, wherein to display the second rendering, the programming instructions are further executable by a processor to:

rotate the first surface from the first non-zero parallax plane to the zero parallax plane such that a normal vector at the intersection between the query ray and the first surface is parallel to a normal vector of the zero parallax plane.

17. A system for digitally drawing on virtual three dimensional (3D) object surfaces, comprising:

a 3D stereoscopic display system, comprising;

a display screen, and

a functional unit coupled to the display screen, wherein the functional unit is configured to;

detect enablement of a drawing mode of the 3D stereoscopic display system, wherein the 3D stereoscopic display system displays a first rendering of a 3D scene via a display screen, wherein the display screen corresponds to a zero parallax plane of the 3D scene, and wherein the 3D scene comprises a plurality of surfaces presented in the 3D scene at non-zero parallax planes;

receive first user input at a location on the display screen, wherein the first user input specifies a first surface of the plurality of surfaces, wherein the first surface is presented at a first non-zero parallax plane;

select, in response to said receiving first user input, the first surface;

display a second rendering, wherein the first surface is rendered at the zero parallax plane in the second rendering;

receive second user input on the display screen specifying a painting motion across the second rendering of the first surface beginning at the location and continuing to a boundary between the first surface and a second surface, and wherein the second surface is not rendered at the zero parallax plane in the second rendering; and

display a third rendering in response to the painting motion crossing the boundary, wherein the second surface is rendered at the zero parallax planet and the first surface is rendered at a non-zero parallax plane in the third rendering, including rendering and displaying results of said painting motion on the first and second surfaces.

18. The system of claim 17, wherein to receive first user input, the functional unit is further configured to:

receive first user input via a user control element, wherein the user control element has a corresponding position, and wherein to select the first surface, the functional unit is further configured to;

determine an intersection between a query ray and the first surface, wherein the query ray intersects the zero parallax plane at the corresponding position of the user control element, and wherein the query ray has a direction specified by one of;

a corresponding orientation of the user control element;

a corresponding orientation of the user'"'"'s point of view (POV) relative to the corresponding position of the user control element;

an orientation of a defined virtual POV relative to the corresponding position of the user control element;

or

a default orientation based on one or more of;

a type of the first surface;

or

a mode of the 3D stereoscopic display system.

19. The system of claim 18, wherein to display the second rendering, the functional unit is further configured to:

translate the first surface from the first non-zero parallax plane to the zero parallax plane along the query ray such that the intersection between the query ray and the first surface is coincident to the corresponding position of the user control element.

20. The system of claim 18, wherein to display the second rendering, the functional unit is further configured to:

rotate the first surface from the first non-zero parallax plane to the zero parallax plane such that a normal vector at the intersection between the query ray and the first surface is parallel to a normal vector of the zero parallax plane.