Presenting a view within a three dimensional scene

• US 8,717,360 B2
• Filed: 06/10/2010
• Issued: 05/06/2014
• Est. Priority Date: 01/29/2010
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Vesely, Michael A., Gray, Alan S.
• Primary Examiner(s): HAJNIK, DANIEL F
• Application Number:
  US12/797,958
• Publication Number:
  US 20110187706A1
• Time in Patent Office: 1,426 Days
• Field of Search: None
• US Class Current: 345/427
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

Presenting a view based on a virtual viewpoint in a three dimensional (3D) scene. The 3D scene may be presented by at least one display, which includes displaying at least one stereoscopic image of the 3D scene by the display(s). The 3D scene may be presented according to a first viewpoint. A virtual viewpoint may be determined within the 3D scene that is different than the first viewpoint. The view of the 3D scene may be presented on the display(s) according to the virtual viewpoint and/or the first view point. The presentation of the view of the 3D scene is performed concurrently with presenting the 3D scene.

Claims

1. A method for presenting a view based on a virtual viewpoint in a three dimensional (3D) scene, comprising:

tracking a position of a head of a user using a position input device, wherein said tracking comprises assessing X, Y, and Z coordinates, an angle, and an orientation;

assessing a first eyepoint of a user using a first head position;

presenting the 3D scene by at least one display according to a first viewpoint with respect to the at least one display, wherein said presenting the 3D scene comprises displaying at least one stereoscopic image of the 3D scene by the at least one display, wherein the first viewpoint corresponds to a first eyepoint of a user viewing the 3D scene;

assessing a second eyepoint of a user using a second head position;

presenting the 3D scene by at least one display according to a second viewpoint with respect to the at least one display, wherein the second viewpoint corresponds to a second eyepoint of a user;

determining a first virtual viewpoint within the 3D scene, wherein the first virtual viewpoint is different than the first viewpoint, wherein the first virtual view point corresponds to a first X, Y, and Z location and a first angle and a first orientation in physical space and maps to a first coordinate in the 3D scene, wherein the first X, Y, and Z location and the first angle and the first orientation is assessed using the position input device, wherein the first coordinate comprises a second X, Y, and Z location in the 3D scene, and wherein the first coordinate comprises a second angle and a second orientation in the 3D scene;

presenting the view of the 3D scene by the at least one display according to the first virtual viewpoint, wherein said presenting the view of the 3D scene according to the first virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first viewpoint;

determining a second virtual viewpoint within the 3D scene, wherein the second virtual viewpoint is different than the first virtual viewpoint, wherein the second virtual view point corresponds to a third X, Y, and Z location and a third angle and a third orientation in physical space and maps to a second coordinate in the 3D scene, wherein the third X, Y, and Z location and the third angle and the third orientation is assessed using the position input device, wherein the second coordinate comprises a fourth X, Y, and Z location in the 3D scene, and wherein the second coordinate comprises a fourth angle and a fourth orientation in the 3D scene; and

presenting the view of the 3D scene by the at least one display according to the second virtual viewpoint;

wherein said presenting the view of the 3D scene according to the second virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first or second viewpoint.

2. The method of claim 1,

wherein the first virtual viewpoint is selected by a user.

3. The method of claim 1,

wherein the first virtual viewpoint is selected by a user using a physical device positioned in an X, Y, and Z location and an angle and orientation in open space of the at least one display.

4. The method of claim 3,

wherein the first virtual viewpoint is selected via a virtual object presented in the 3D scene, wherein movement of the physical device by the user has a one to one correspondence with corresponding movement of the virtual object within the 3D scene.

5. The method of claim 1,

wherein said presenting the 3D scene comprises graphically indicating a frustum of the view of the 3D scene within the 3D scene.

6. The method of claim 1,

wherein said presenting the view of the 3D scene comprises displaying at least one stereoscopic image of the view of the 3D scene by the at least one display.

7. The method of claim 1,

wherein said presenting the view of the 3D scene is also based on the first or second viewpoint.

8. The method of claim 1, further comprising:

moving the first virtual viewpoint in response to user input;

wherein said determining the first virtual viewpoint and said presenting the view of the 3D scene are performed a plurality of times during said moving.

9. The method of claim 1,

wherein the first virtual viewpoint is from the position/orientation of a virtual object rendered and displayed within the 3D scene.

10. The method of claim 9,

moving the virtual object within the 3D scene in response to user input;

wherein said determining the first virtual viewpoint and said presenting the view of the 3D scene are performed a plurality of times during said moving the virtual object.

11. The method of claim 10,

wherein the 3D scene is presented at a first frame rate;

wherein said determining the first virtual viewpoint and said presenting the view of the 3D scene are performed during said moving the virtual object at substantially the first frame rate.

12. The method of claim 9,

wherein the virtual object is a virtual camera object, where the virtual camera object may be viewed as positioned and oriented commensurate with the second X, Y, and Z location and the second angle and second orientation.

13. The method of claim 9,

wherein the virtual object is a virtual magnifying glass object.

14. The method of claim 1, further comprising:

determining a field of view of the first virtual viewpoint;

wherein said presenting the view of the 3D scene is based on the field of view of the first virtual viewpoint.

15. The method of claim 1, further comprising:

determining a magnification of the 3D scene for the first virtual viewpoint;

wherein said presenting the view of the 3D scene is based on the magnification.

16. The method of claim 1,

determining a view volume for the first virtual viewpoint;

wherein said presenting the view of the 3D scene is based on the view volume.

17. The method of claim 1,

wherein the view of the 3D scene from the perspective of the first virtual viewpoint is magnified compared to a corresponding portion in the 3D scene.

18. The method of claim 1, wherein the at least one display comprises a vertical display and a horizontal display, wherein said presenting the 3D scene comprises 1) displaying, according to a horizontal perspective, a first stereoscopic image on the horizontal display, and 2) displaying, according to a vertical perspective, a second stereoscopic image on the vertical display.

19. The method of claim 1, wherein the at least one display comprises a plurality of displays, wherein said displaying the 3D scene is performed on a first display and said displaying the view of the 3D scene is performed on a second display.

20. The method of claim 1, wherein said presenting the view of the 3D scene is performed at a position and orientation that is different from a position and orientation of the at least one display.

21. A non-transitory computer accessible memory medium storing program instructions for presenting a view based on a virtual viewpoint in a three dimensional (3D) scene, wherein the program instructions are executable by a processor to:

track a position of a head of a user using a position input device, wherein said tracking comprises assessing X, Y, and Z coordinates, an angle, and an orientation;

assess a first eyepoint of a user using a first head position;

present the 3D scene via at least one display according to a first viewpoint with respect to the at least one display, wherein said presenting the 3D scene comprises the at least one display displaying at least one stereoscopic image of the 3D scene, and wherein the first viewpoint corresponds to a first eyepoint of a user viewing the 3D scene;

assess a second eyepoint of a user using a second head position;

present the 3D scene by at least one display according to a second viewpoint with respect to the at least one display, wherein the second viewpoint corresponds to a second eyepoint of a user;

determine a first virtual viewpoint within the 3D scene, wherein the virtual viewpoint is different than the first viewpoint, wherein the first virtual view point corresponds to a first X, Y, and Z location and a first angle and a first orientation in physical space and maps to a first coordinate in the 3D scene, wherein the first X, Y, and Z location and the first angle and the first orientation is assessed using the position input device, wherein the first coordinate comprises a second X, Y, and Z location in the 3D scene, and wherein the first coordinate comprises a second angle and a second orientation in the 3D scene;

present the view of the 3D scene via the at least one display according to the virtual viewpoint, wherein said presenting the view of the 3D scene according to the first virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first viewpoint;

determine a second virtual viewpoint within the 3D scene, wherein the second virtual viewpoint is different than the first virtual viewpoint, wherein the second virtual view point corresponds to a third X, Y, and Z location and a third angle and a third orientation in physical space and maps to a second coordinate in the 3D scene, wherein the third X, Y, and Z location and the third angle and the third orientation is assessed using the position input device, wherein the second coordinate comprises a fourth X, Y, and Z location in the 3D scene, and wherein the second coordinate comprises a fourth angle and a fourth orientation in the 3D scene; and

present the view of the 3D scene by the at least one display according to the second virtual viewpoint;

wherein said presenting the view of the 3D scene according to the second virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first or second viewpoint.

22. The memory medium of claim 21,

wherein the first virtual viewpoint is selected by a user.

23. The memory medium of claim 21,

wherein the first virtual viewpoint is selected by a user using a physical device in open space of at least one display positioned in an X, Y, and Z location and an angle and orientation.

24. The memory medium of claim 23,

wherein the first virtual viewpoint is selected via a virtual object presented in the 3D scene, wherein movement of the physical device by the user has a one to one correspondence with corresponding movement of the virtual object within the 3D scene.

25. The memory medium of claim 21,

wherein said presenting the view of the 3D scene comprises displaying at least one stereoscopic image of the view of the 3D scene by the at least one display.

26. The memory medium of claim 21,

wherein said presenting the view of the 3D scene is also based on the first viewpoint.

27. The memory medium of claim 21, wherein the program instructions are further executable to:

move the first virtual viewpoint in response to user input;

wherein said determining the first virtual viewpoint and said presenting the view of the 3D scene are performed a plurality of times during said moving.

28. The memory medium of claim 21, wherein the at least one display comprises a vertical display and a horizontal display, wherein said presenting the 3D scene comprises 1) displaying, according to a horizontal perspective, a first stereoscopic image on the horizontal display, and 2) displaying, according to a vertical perspective, a second stereoscopic image on the vertical display.

29. The memory medium of claim 21, wherein the view of the 3D scene is presented at a position and orientation that is different from a position and orientation of the at least one display.

30. A system for presenting a view based on a virtual viewpoint in a three dimensional (3D) scene, comprising:

a processor;

an input device configured to provide information to the processor indicating a current viewpoint of the user;

at least one display coupled to the processor;

a memory medium coupled to the processor which stores program instructions executable to;

track a position of a head of a user using a position input device, wherein said tracking comprises assessing X, Y, and Z coordinates, an angle, and an orientation;

assess a first eyepoint of a user using a first head position;

present the 3D scene via the at least one display according to a first viewpoint with respect to the at least one display, wherein said presenting the 3D scene comprises the at least one display displaying at least one stereoscopic image of the 3D scene, and wherein the first viewpoint corresponds to a first eyepoint of a user viewing the 3D scene;

assess a second eyepoint of a user using a second head position;

present the 3D scene by at least one display according to a second viewpoint with respect to the at least one display, wherein the second viewpoint corresponds to a second eyepoint of a user;

determine a first virtual viewpoint within the 3D scene, wherein the virtual viewpoint is different than the current viewpoint of the user, wherein the first virtual view point corresponds to a first X, Y, and Z location and a first angle and a first orientation in physical space and maps to a first coordinate in the 3D scene, wherein the first X, Y, and Z location and the first angle and the first orientation is assessed using the position input device, wherein the first coordinate comprises a second X, Y, and Z location in the 3D scene, and wherein the first coordinate comprises a second angle and a second orientation in the 3D scene;

present the view of the 3D scene via the at least one display according to the virtual viewpoint, wherein said presenting the view of the 3D scene according to the first virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first viewpoint;

determine a second virtual viewpoint within the 3D scene, wherein the second virtual viewpoint is different than the first virtual viewpoint, wherein the second virtual view point corresponds to a third X, Y, and Z location and a third angle and a third orientation in physical space and maps to a second coordinate in the 3D scene, wherein the third X, Y, and Z location and the third angle and the third orientation is assessed using the position input device, wherein the second coordinate comprises a fourth X, Y, and Z location in the 3D scene, and wherein the second coordinate comprises a fourth angle and a fourth orientation in the 3D scene; and

present the view of the 3D scene by the at least one display according to the second virtual viewpoint;

wherein said presenting the view of the 3D scene according to the second virtual viewpoint is performed concurrently with said presenting the 3D scene according to the first or second viewpoint.

31. The system of claim 30, further comprising:

an input device configured to communicate with the processor, wherein the input device is configured for use by a user to manipulate the 3D scene;

wherein the first virtual viewpoint is selected by a user using the input device.

32. The system of claim 30,

wherein said presenting the view of the 3D scene comprises displaying at least one stereoscopic image of the view of the 3D scene by the at least one display.

33. The system of claim 30,

wherein said presenting the view of the 3D scene is also based on the first viewpoint.

34. The system of claim 30, wherein the at least one display comprises a vertical display and a horizontal display, wherein said presenting the 3D scene comprises 1) displaying, according to a horizontal perspective, a first stereoscopic image on the horizontal display, and 2) displaying, according to a vertical perspective, a second stereoscopic image on the vertical display.