Modifying perspective of stereoscopic images based on changes in user viewpoint

• US 9,292,962 B2
• Filed: 05/02/2014
• Issued: 03/22/2016
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Vesely, Michael A., Clemens, Nancy L., Gray, Alan S.
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Hsu, Amy
• Application Number:
  US14/268,386
• Publication Number:
  US 20140313190A1
• Time in Patent Office: 690 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

Modifying perspective of stereoscopic images provided by one or more displays based on changes in user viewpoint. The one or more displays may include a first display that is provided substantially horizontal for displaying 3D horizontal perspective images and/or a second display that is provided substantially vertical for displaying text or conventional images such as 2D images, or 3D vertical perspective images. The horizontal display surface may be typically positioned directly in front of the user, and at a height of about a desktop surface so that the user can have about a 45° looking angle. The vertical display surface may be positioned in front of the user and preferably behind and above the horizontal display surface.

Claims

1. A method for presenting a three dimensional (3D) scene using a display device, comprising:

determining a first perspective of a user, wherein determining the first perspective comprises;

determining, via a tracking system coupled to the display, a first viewpoint of the user, including determining, by a processor of the display device and based on information from the tracking system, a position and orientation of the first viewpoint relative to a position and orientation of a surface of the display;

displaying a first 3D image on the display based on the first perspective of the user, wherein said displaying the first 3D image comprises displaying the 3D scene according to a first projection;

rendering, by the processor, the first projection based on a render surface, wherein the surface of the display has a first correlation to the first viewpoint in physical space, wherein the render surface has a second correlation to the first viewpoint in virtual space, and wherein the first correlation and second correlation are equivalent;

determining, via the tracking system, a second perspective of the user; and

displaying a second 3D image on the display based on the second perspective of the user, wherein said displaying the second 3D image comprises displaying an updated 3D scene according to a second projection;

wherein the second perspective is different from the first perspective, wherein the first perspective is a first oblique perspective and wherein the second perspective is a second oblique perspective.

2. The method of claim 1, wherein the render surface is at a non-normal position and orientation with respect to the first viewpoint.

3. The method of claim 1, wherein determining the second perspective comprises:

determining, via a tracking system coupled to the display, a second viewpoint of the user.

4. The method of claim 3, wherein said determining the second viewpoint is in response to a change of viewpoint of the user, wherein the second viewpoint is different than the first viewpoint, and wherein said determining the second viewpoint comprises:

determining, based on information from the tracking system, a position and orientation of the second viewpoint relative to a position and orientation of a surface of the display.

5. The method of claim 4, further comprising:

rendering the second projection based on a render surface, wherein the surface of the display has a first correlation to the second viewpoint in physical space, wherein the render surface has a second correlation to the second viewpoint in virtual space, and wherein the first correlation and second correlation are equivalent.

6. The method of claim 5, wherein the render surface is at a non-normal position and orientation to the second viewpoint.

7. The method of claim 1, wherein the display comprises a handheld display.

8. The method of claim 1, wherein the 3D scene comprises a stereoscopic 3D scene, where said displaying the first 3D image comprises displaying a first stereoscopic 3D image, and wherein said displaying the second 3D image comprises displaying a second stereoscopic 3D image.

9. A non-transitory computer readable memory medium storing program instructions executable by a processor to:

determine a first perspective of a user, wherein to determine the first perspective, the program instructions are further executable by the processor to;

determine, via a tracking system coupled to the display, a first viewpoint of the user, including determining, based on first information from the tracking system, a first position and first orientation of the first viewpoint relative to a position and orientation of a surface of the display;

display a first 3D image on a display based on the first perspective of the user, wherein to display the first 3D image, the program instructions are further executable by the processor to display a 3D scene according to a first projection;

render the first projection based on a first render surface, wherein the surface of the display has a first correlation to the first viewpoint in physical space, wherein the first render surface has a second correlation to the first viewpoint in virtual space, and wherein the first correlation and second correlation are equivalent;

determine a second perspective of the user; and

display a second 3D image on the display based on the second perspective of the user, wherein to display the second 3D image, the program instructions are further executable by the processor to display an updated 3D scene according to a second projection;

wherein the second perspective is different from the first perspective, wherein the first perspective is a first oblique perspective and wherein the second perspective is a second oblique perspective.

10. The non-transitory computer readable memory medium of claim 9, wherein the first render surface is at a first non-normal position and orientation with respect to the first viewpoint.

11. The non-transitory computer readable memory medium of claim 10,

wherein to determine the second perspective, the program instructions are further executable by the processor to determine, in response to a change in viewpoint of the use and via the tracking system coupled to the display, a second viewpoint of the user;

wherein to determine the second viewpoint, the program instructions are further executable by the processor to determine, based on second information from the tracking system, a second position and second orientation of the second viewpoint relative to the position and orientation of the surface of the display; and

wherein the program instructions are further executable by the processor to render the second projection based on a second render surface, wherein the surface of the display has a third correlation to the second viewpoint in physical space, wherein the second render surface has a fourth correlation to the second viewpoint in virtual space, wherein the third correlation and fourth correlation are equivalent, and wherein the second render surface is at a second non-normal position and orientation to the second viewpoint.

12. The non-transitory computer readable memory medium of claim 9, wherein the display comprises a handheld display.

13. The non-transitory computer readable memory medium of claim 9, wherein the 3D scene comprises a stereoscopic 3D scene, where to display the first 3D image the program instructions are further executable by the processor to displaying a first stereoscopic 3D image, and wherein to display the second 3D image the program instructions are further executable by the processor to displaying a second stereoscopic 3D image.

14. A system for presenting a three dimensional (3D) scene using a display, comprising:

a processor;

a display coupled to the processor; and

a memory medium coupled to the processor which stores program instructions executable to;

determine a first perspective of a user, wherein to determine the first perspective, the program instructions are further executable by the processor to;

determine, via a tracking system coupled to the display, a first viewpoint of the user, including determining, based on first information from the tracking system, a first position and first orientation of the first viewpoint relative to a position and orientation of a surface of the display;

display a first 3D image on the display based on the first perspective of the user, wherein to display the first 3D image, the program instructions are further executable by the processor to display the 3D scene according to a first projection;

render the first projection based on a first render surface, wherein the surface of the display has a first correlation to the first viewpoint in physical space, wherein the first render surface has a second correlation to the first viewpoint in virtual space, and wherein the first correlation and second correlation are equivalent;

determine a second perspective of the user; and

display a second 3D image on the display based on the second perspective of the user, wherein to display the second 3D image, the program instructions are further executable by the processor to display an updated 3D scene according to a second projection;

wherein the second perspective is different from the first perspective, wherein the first perspective is a first oblique perspective and wherein the second perspective is a second oblique perspective.

15. The system of claim 14, wherein the first render surface is at a non-normal position and orientation with respect to the first viewpoint.

16. The system of claim 14, wherein the display comprises a handheld display.

17. The system of claim 14, wherein the 3D scene comprises a stereoscopic 3D scene, wherein to display the first 3D image, the program instructions are executable by the processor to display a first stereoscopic 3D image, and wherein to display the second 3D image, the program instructions are executable by the processor to display a second stereoscopic 3D image.

19. The method of claim 15, wherein the render surface is at a non-normal position and orientation to the second viewpoint.

20. The method of claim 15, wherein the display comprises a handheld display.

21. The method of claim 15, wherein the 3D scene comprises a stereoscopic 3D scene, where said displaying the first 3D image comprises displaying a first stereoscopic 3D image, and wherein said displaying the second 3D image comprises displaying a second stereoscopic 3D image.

18. A method for presenting a three dimensional (3D) scene using a display device, comprising:

determining, by a processor of the display device, a first perspective of a user;

displaying a first 3D image on the display based on the first perspective of the user, wherein said displaying the first 3D image comprises displaying the 3D scene according to a first projection;

determining a second perspective of the user, wherein determining the second perspective comprises;

determining, via a tracking system coupled to the display, a second viewpoint of the user, wherein said determining the second viewpoint is in response to a change of viewpoint of the user, wherein the second viewpoint is different than the first viewpoint, and wherein said determining the second viewpoint comprises;

determining, by the processor and based on information from the tracking system, a position and orientation of the second viewpoint relative to a position and orientation of a surface of the display;

displaying a second 3D image on the display based on the second perspective of the user, wherein said displaying the second 3D image comprises displaying an updated 3D scene according to a second projection;

wherein the second perspective is different from the first perspective, wherein the first perspective is a first oblique perspective and wherein the second perspective is a second oblique perspective; and

rendering, by the processor, the second projection based on a render surface, wherein the surface of the display has a first correlation to the second viewpoint in physical space, wherein the render surface has a second correlation to the second viewpoint in virtual space, and wherein the first correlation and second correlation are equivalent.