Virtual plane in a stylus based stereoscopic display system

• US 9,703,400 B2
• Filed: 10/09/2015
• Issued: 07/11/2017
• Est. Priority Date: 10/09/2015
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Hosenpud, Jonathan J., Berman, Arthur L., Champion, Clifford S., Chavez, David A., Lopez-Fresquet, Francisco, Kalnins, Robert D., Lelievre, Alexandre R., Venkat, Murugappan R., Sherman, Christopher W., Tu, Jerome C.
• Primary Examiner(s): HICKS, CHARLES V
• Application Number:
  US14/880,007
• Publication Number:
  US 20170102791A1
• Time in Patent Office: 641 Days
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

Virtual plane and use in a stylus based three dimensional (3D) stereoscopic display system. A virtual plane may be displayed in a virtual 3D space on a display of the 3D stereoscopic display system. The virtual plane may extend from a stylus of the 3D stereoscopic display system. Content may be generated in response to a geometric relationship of the virtual plane with at least one virtual object in the virtual 3D space. The generated content may indicate one or more attributes of the at least one virtual object. The content may be presented via the 3D stereoscopic display system.

Claims

1. A non-transitory computer readable memory medium that stores program instructions executable by a three dimensional (3D) stereoscopic display system to implement:

displaying a virtual plane in a virtual 3D space on a display of the 3D stereoscopic display system, wherein the virtual plane is specified by a long axis of a stylus, wherein the virtual plane is displayed as extending into the virtual 3D space from the long axis of the stylus, and wherein a movement of the stylus causes a corresponding movement of the virtual plane in the virtual 3D space;

generating content in response to a geometric relationship of the virtual plane with at least one virtual object in the virtual 3D space; and

presenting the content, wherein the content indicates one or more attributes of the at least one virtual object.

2. The non-transitory computer readable memory medium of claim 1, wherein said generating content comprises displaying a cross section of the virtual object defined by the virtual plane intersecting the virtual object.

3. The non-transitory computer readable memory medium of claim 1, wherein the virtual plane is displayed in accordance with a configuration, wherein the configuration specifies one or more of:

size of the virtual plane;

shape of the virtual plane;

position of the virtual plane relative to the display;

position of the virtual plane relative to the stylus;

or

appearance of the virtual plane.

4. The non-transitory computer readable memory medium of claim 3, wherein the configuration comprises one or more of:

a default configuration;

a configuration automatically determined based on a position of the stylus;

or

a user specified configuration.

5. The non-transitory computer readable memory medium of claim 1, wherein the content comprises imagery in the virtual plane, wherein the imagery visually indicates at least one of the one or more attributes.

6. The non-transitory computer readable memory medium of claim 5, wherein the imagery comprises at least one of:

a thermal map;

a virtual x-ray image;

a contour map;

a plot;

a cross section;

a video;

structural stresses;

fluid flow direction;

flux magnitude;

a cross sectional force field;

a vector field;

or

a density map.

7. The non-transitory computer readable memory medium of claim 1, wherein the geometric relationship of the virtual plane with the at least one virtual object comprises the virtual plane being proximate to the at least one virtual object.

8. The non-transitory computer readable memory medium of claim 1, wherein the content comprises text in the virtual plane, and wherein the text indicates at least one of the one or more attributes.

9. The non-transitory computer readable memory medium of claim 8, wherein the text comprises at least one value of the at least one of the one or more attributes.

10. The non-transitory computer readable memory medium of claim 1, wherein the stylus comprises one or more controls configured to receive user input, wherein the one or more controls comprises one or more of:

a button;

a scroll wheel;

a scroll ball;

a joystick;

a touch pad;

or

a slider.

11. The non-transitory computer readable memory medium of claim 10, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

updating the content in response to at least one of;

a user activating at least one of the one or more controls;

the virtual plane moving with respect to the at least one virtual object;

a change in at least one of the one or more attributes;

the user interacting, via the user'"'"'s hand, with the at least one virtual object;

or

the user interacting, via the stylus, with the at least one virtual object.

12. The non-transitory computer readable memory medium of claim 10, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

changing a position of the virtual plane in the 3D virtual space in response to at least one of;

a movement of the stylus according to one or more degrees of freedom of the stylus;

or

a user activating at least one of the one or more controls.

13. The non-transitory computer readable memory medium of claim 12, wherein the virtual plane changes position:

at a constant speed;

at a variable speed;

at a constant angle with respect to the plane of the display;

or

at a variable angle with respect to the plane of the display.

14. The non-transitory computer readable memory medium of claim 10, wherein the virtual plane extends from the stylus in accordance with:

a default extension;

or

an amount of time one of the one or more controls is activated by a user.

15. The non-transitory computer readable memory medium of claim 1, wherein an orientation of the virtual plane in the 3D virtual space is at least one of:

parallel to the stylus;

parallel to the display;

perpendicular to a viewing direction of a user;

or

automatically determined by the geometric relationship of the virtual plane with the at least one virtual object.

16. The non-transitory computer readable memory medium of claim 1, wherein the virtual plane extends from the stylus fully formed, or gradually.

17. The non-transitory computer readable memory medium of claim 1, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

fixing the virtual plane in a first position in the virtual 3D space.

18. The non-transitory computer readable memory medium of claim 17, wherein said fixing the virtual plane occurs in response to a change in an orientation of the stylus.

19. The non-transitory computer readable memory medium of claim 17, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

periodically updating the content.

20. The non-transitory computer readable memory medium of claim 1, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

displaying one or more additional virtual planes in the virtual 3D space on the display of the 3D stereoscopic display system, wherein each additional virtual plane specifies or presents content or at least partially defines a space.

21. The non-transitory computer readable memory medium of claim 1, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

displaying a plurality of selectable virtual tools in the 3D virtual space, the selectable virtual tools comprising at least one of;

at least one virtual button;

at least one virtual knob;

at least one virtual keyboard;

at least one virtual slider;

at least one picklist;

at least one color wheel;

at least one character map;

or

at least one virtual dial.

22. The non-transitory computer readable memory medium of claim 21, wherein display of the selectable virtual tools:

extends from the stylus;

is proximate to the at least one virtual object;

or

is in a specified region of the display.

23. The non-transitory computer readable memory medium of claim 21, wherein each tool of the array of selectable virtual tools is selectable via one or more of:

a keyboard input;

a user'"'"'s hand;

a control of the stylus;

a user'"'"'s gaze;

or

an indication by the stylus.

24. The non-transitory computer readable memory medium of claim 1, wherein the virtual plane is connected to the stylus via a flexible connection.

25. The non-transitory computer readable memory medium of claim 1, wherein the virtual plane is displayed in positive virtual 3D space.

26. The non-transitory computer readable memory medium of claim 1, wherein the program instructions are further executable by the 3D stereoscopic display system to implement:

defining a volume in the virtual 3D space by movement of the virtual plane in the virtual 3D space.

27. A three dimensional (3D) stereoscopic display system comprising:

at least one processor;

one or more displays, coupled to the at least one processor;

a stylus, communicatively coupled to the at least one processor; and

a memory coupled to the at least one processor, wherein the memory has stored thereon instructions executable by the at least one processor to;

display a virtual plane in a virtual 3D space on the one or more displays, wherein the virtual plane is specified by a long axis of a stylus, wherein the virtual plane is displayed as extending into the virtual 3D space from the long axis of the stylus, and wherein a movement of the stylus causes a corresponding movement of the virtual plane in the virtual 3D space;

generate content in response to a geometric relationship of the virtual plane with at least one virtual object in the virtual 3D space; and

present the content, wherein the content indicates one or more attributes of the at least one virtual object.

28. The 3D stereoscopic display system of claim 27, further comprising:

a tracking system coupled to the at least one processor, wherein the tracking system tracks one or more of a user'"'"'s head, a user'"'"'s hand, or the stylus, and wherein the tracking system is used to determine the geometric relationship of the virtual plane with the at least one virtual object.

30. The 3D stereoscopic display system of claim 27,

wherein to generate content in response to a geometric relationship of the virtual plane with at least one virtual object in the virtual 3D space, the instructions are further executable to:

display a cross section of the virtual object defined by the virtual plane intersecting the virtual object.

29. A computer implemented method comprising:

a three dimensional (3D) stereoscopic display system performing;

displaying a virtual plane in a virtual 3D space on a display of the 3D stereoscopic display system, wherein the virtual plane is specified by a long axis of a stylus, wherein the virtual plane is displayed as extending into the virtual 3D space from the long axis of the stylus, and wherein a movement of the stylus causes a corresponding movement of the virtual plane in the virtual 3D space;

generating content in response to a geometric relationship of the virtual plane with at least one virtual object in the virtual 3D space; and

presenting the content, wherein the content indicates one or more attributes of the at least one virtual object.