SYSTEM AND METHOD FOR PRESENTING VIRTUAL AND AUGMENTED REALITY SCENES TO A USER

• US 20120218306A1
• Filed: 11/22/2011
• Published: 08/30/2012
• Est. Priority Date: 11/24/2010
• Status: Active Grant

• Current Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Original Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Inventors: McArdle, Terrence Edward, Newhouse, Benjamin Zeis
• Granted Patent: US 9,070,219 B2
• Time in Patent Office: Days
• US Class Current: 345/660
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A method of presenting a scene to a user according to a preferred embodiment includes determining a real orientation of a device relative to a projection matrix and determining a user orientation of the device relative to a nodal point. The method of the preferred embodiment can further include orienting a scene displayable on the device to the user in response to the real orientation and the user orientation; and displaying the scene on the device. The method of the preferred embodiment can be performed by an apparatus and/or embodied in computer program product including machine-readable code.

Claims

1. A method of presenting a scene to a user comprising:

determining a real orientation of a device relative to a projection matrix;

determining a user orientation of the device relative to a nodal point;

orienting a scene displayable on the device to the user in response to the real orientation and the user orientation; and

displaying the scene on the device.

2. The method of claim 1, further comprising creating a projection matrix representing an orientation of the device in a three-dimensional external frame of reference.

3. The method of claim 1, further comprising adapting the scene displayable on the device to the user in response to a change in one of the real orientation or the user orientation.

4. The method of claim 3, further comprising adjusting a virtual zoom of the scene in response to a change in a linear distance between the device and the nodal point.

5. The method of claim 3, further comprising adjusting a virtual parallax of the scene in response to a change in a translational distance between the device and the nodal point.

7. The method of claim 3, further comprising adjusting a virtual perspective of the scene in response to a change in a rotational orientation of the device and the nodal point.

8. The method of claim 3, further comprising adjusting a virtual origin of the scene in response to a change in a real position of the nodal point.

9. The method of claim 3, wherein the nodal point comprises a user'"'"'s head.

10. The method of claim 1, wherein the scene comprises a spherical image substantially symmetrically disposed about the nodal point.

11. The method of claim 10, wherein displaying the scene on the device comprises displaying a floating-point exposure of the spherical image in order to minimize lighting irregularities.

12. The method of claim 10, further comprising displaying a portion of the spherical image in response to the real orientation of the device.

13. The method of claim 1, wherein the device comprises a handheld device.

14. An apparatus comprising:

a user interface comprising a display and a camera, the display and the camera substantially oriented in a first direction;

a real orientation module configured to determine a three-dimensional spatial real orientation of the user interface relative to a projection matrix;

a user orientation module configured to determine a user orientation of the user interface relative to a nodal point; and

a processor connected to the user interface, the real orientation module, and the user orientation module, the processor configured to display a scene to the user on the display in response to the real orientation and the user orientation.

15. The apparatus of claim 14, wherein the real orientation module is further configured to create the projection matrix representing an orientation of the device in a three-dimensional external frame of reference.

16. The apparatus of claim 14, wherein the processor is further configured to adapt the scene displayable on the device to the user in response to a change in one of the real orientation or the user orientation.

17. The apparatus of claim 16, wherein the processor is further configured to adjust a virtual zoom of the scene in response to a change in a linear distance between the device and the nodal point.

18. The apparatus of claim 16, wherein the processor is further configured to adjust a virtual parallax of the scene in response to a change in a translational distance between the device and the nodal point.

19. The apparatus of claim 16, wherein the processor is further configured to adjust a virtual perspective of the scene in response to a change in a rotational orientation of the device and the nodal point.

20. The apparatus of claim 16, wherein the processor is further configured to adjust a virtual origin of the scene in response to a change in a real position of the nodal point.

21. The apparatus of claim 16, wherein the nodal point comprises a user'"'"'s head.

22. The apparatus of claim 14, wherein the scene comprises a spherical image substantially symmetrically disposed about the nodal point.

23. The apparatus of claim 22 wherein the processor is further configured to cause the display of the user interface to display a floating-point exposure of the spherical image in order to minimize lighting irregularities.

24. The apparatus of claim 22, wherein the processor is further configured to cause the display of the user interface to display a portion of the spherical image in response to the real orientation of the device.

25. The apparatus of claim 14, further comprising a MEMS gyroscope.

26. The apparatus of claim 25, wherein the MEMS gyroscope is integrated with the orientation module.

27. A computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code to determine a real orientation of a device relative to a projection matrix;

computer readable program code to determine a user orientation of the device relative to a nodal point;

computer readable program code to orient a scene displayable on the device to the user in response to the real orientation and the user orientation; and

computer readable program code to display the scene on the device.

28. The product of claim 27, further comprising computer readable program code to create a projection matrix representing an orientation of the device in a three-dimensional external frame of reference.

29. The product of claim 27, further comprising computer readable program code to adapt the scene displayable on the device to the user in response to a change in one of the real orientation or the user orientation.

30. The product of claim 29, further comprising computer readable program code to adjust a virtual zoom of the scene in response to a change in a linear distance between the device and the nodal point.

31. The product of claim 29, further comprising computer readable program code to adjust a virtual parallax of the scene in response to a change in a translational distance between the device and the nodal point.

32. The product of claim 29, further comprising computer readable program code to adjust a virtual perspective of the scene in response to a change in a rotational orientation of the device and the nodal point.

33. The product of claim 29, further comprising computer readable program code to adjust a virtual origin of the scene in response to a change in a real position of the nodal point.

34. The product of claim 29, wherein the nodal point comprises a user'"'"'s head.

35. The product of claim 27, wherein the scene comprises a spherical image substantially symmetrically disposed about the nodal point.

36. The product of claim 35, wherein the computer readable program code to display the scene on the device comprises computer readable program code to display a floating point exposure of the spherical image in order to minimize lighting irregularities.

37. The product of claim 35, further comprising computer readable program code to display a portion of the spherical image in response to the real orientation of the device.

38. The product of claim 27, wherein the computer readable program code to determine a real orientation comprises computer readable program code to generate a projection matrix in response to receiving a signal from a MEMS gyroscope.

39. The product of claim 27, wherein the device comprises a handheld device.