TRACKING LOCATION AND RESOLVING DRIFT IN AUGMENTED REALITY HEAD MOUNTED DISPLAYS WITH DOWNWARD PROJECTION

• US 20200134920A1
• Filed: 10/24/2018
• Published: 04/30/2020
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: DAVIS, Randall S., BAUMBACH, Elliott H., NOCON, Nathan D., GRAHAM, Todd Michael
• Granted Patent: US 10,621,789 B1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0161 characterised by the relative positioning of the constitutive elements
• CPC Class Codes: G02B 2027/0183 Adaptation to parameters characterising the motion of the vehicle
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

Embodiments provide for tracking location and resolving drift in Augmented Reality (AR) devices. The AR devices includes computing devices having screens on a first face and cameras on a second, opposite face to project an image onto optical arrangements for viewing by wearers of the AR devices. The AR devices map locations for real objects in the environment to a virtual environment; anchor virtual objects at anchor locations within the virtual environment; capture station keeping images of the environment from a first Field of View via the camera; determine a second, different Field of View in the environment for the wearer of the AR device based on the relative locations of real objects present in the station keeping images; and output images depicting the virtual objects at positions on the screen to depict the virtual objects in the physical environment at the anchor locations.

Claims

1. An Augmented Reality (AR) device, comprising:

a viewport;

a mount assembly configured to capture a mobile computing device, the mount assembly including;

a lower surface that is transmissive to light;

an upper surface, located above the lower surface and configured to capture the mobile computing device between the upper surface and the lower surface, the upper surface including a region transmissive to light;

an optical arrangement configured to receive light transmitted through the lower surface and reflect light onto the viewport for reflection to a wearer of the AR device; and

an additional optic, arranged above the region of the upper surface and configured to direct light from the environment outside of a field of view of the wearer.

2. The AR device of claim 1, wherein the additional optic is selectively positionable relative to the region of the upper surface via a slide-mount, comprising:

a first carrier captured via primary grooves defined in the mount assembly to travel in a first direction relative to the region; and

a second carrier captured via second grooved defined in the first carrier to travel in a second direction orthogonal to the first direction and onto which the additional optic is mounted.

3. The AR device of claim 1, wherein the additional optic is selectively positionable relative to the region of the upper surface via an arm-mount, comprising:

a pivot mounted to the mount assembly selectively that is rotatable in a first plane; and

an arm, including a telescopic segment to selectively adjust a length of the arm, connected to the pivot at a first end and to which the additional optic is mounted on a second end.

4. The AR device of claim 1, wherein the additional optic is selectively positionable relative to the region of the upper surface via an arm-mount, comprising:

a carrier beam, captured by the mount assembly in a slot defined in the mount assembly, to which the additional optic is mounted; and

wherein the slot is defined with a size relative to the carrier beam to allow the carrier beam to selectively rotate and translate relative to the region in a plane parallel to the upper surface.

5. The AR device of claim 1, additional optic is one of:

a wide angle lens;

a filter; and

a mirror configured to direct light transmitted in a plane parallel to the upper surface to a plane orthogonal to the upper surface.

6. The AR device of claim 1, further comprising one of:

a smart phone captured in the mount assembly as the mobile computing device; and

a tablet captured in the mount assembly as the mobile computing device with a screen oriented to output light toward the optical arrangement and a back camera oriented to capture light through the region of the upper surface.

7. The AR device of claim 1, wherein the optical assembly is located below the lower surface and below eyes of the wearer such that the optical assembly is outside of a field of view for the wearer.

8. An Augmented Reality (AR) system, comprising:

a viewport;

a computing device, including a screen on a first face and a camera on a second face opposite to the first face;

an optical arrangement configured to receive light generated by the screen and reflect the light onto the viewport for reflection to a wearer of the AR system; and

an additional optic, arranged above the camera to direct light from the environment outside of a field of view of the wearer to the camera.

9. The AR system of claim 8, wherein the additional optic is selectively positionable relative to the second face via a slide-mount, comprising:

a first carrier captured via primary grooves defined in a mount assembly in which the mobile computing device is captured to travel in a first direction relative to the camera; and

a second carrier captured via second grooved defined in the first carrier to travel in a second direction orthogonal to the first direction and onto which the additional optic is mounted.

10. The AR system of claim 8, wherein the additional optic is selectively positionable relative to the second face via an arm-mount, comprising:

a pivot mounted to a mount assembly in which the computing device is secured that is selectively rotatable in a first plane; and

an arm, including a telescopic segment to selectively adjust a length of the arm, connected to the pivot at a first end and to which the additional optic is mounted on a second end.

11. The AR system of claim 8, wherein the additional optic is selectively positionable relative to the second face via an arm-mount, comprising:

a carrier beam, captured by a mount assembly in which the computing device is secured in a slot defined in the mount assembly, to which the additional optic is mounted; and

wherein the slot is defined with a size relative to the carrier beam to allow the carrier beam to selectively rotate and translate relative to the camera in a parallel plane to the second face.

12. The AR system of claim 8, wherein the additional optic is one of:

a wide angle lens;

a filter; and

a mirror configured to direct light transmitted in a plane parallel to the second face surface to a plane orthogonal to the second face.

13. The AR system of claim 8, wherein the optical assembly is located below the lower surface and below eyes of the wearer such that the optical assembly is outside of a field of view for the wearer.

14. The AR system of claim 8, wherein the camera captures station keeping images of an upper portion of an environment in which the computing device provides an AR experience to the wearer that is outside of a field of view of the wearer to enable the computing device to determine an orientation of the system.

15. A method for tracking location and resolving drift in an Augmented Reality (AR) device, the AR device including a computing device having a screen on a first face and a camera on a second face opposite to the first face mounted in a mounting assembly to project an image from the screen onto an optical arrangement for viewing by a wearer of the AR device, the method comprising:

mapping locations for real objects in a physical environment in which the AR device resides to a virtual environment;

anchoring a virtual object at an anchor location within the virtual environment, based on the mapped locations of the real objects;

capturing a station keeping image of the environment from a first Field of View via the camera;

determining a second Field of View in the environment for the wearer of the AR device that is different from the first Field of View based on the relative locations of real objects present in the station keeping image; and

in response to determining that the second Field of View includes an area of the physical environment corresponding to the anchor location in the virtual environment, outputting an image depicting the virtual object at a position on the screen to display the image depicting the virtual object in the physical environment at the anchor location.

16. The method of claim 15, wherein determining the second Field of View further comprises:

determining a first orientation in the physical environment from the station keeping image;

determining a second orientation in the physical environment from a magnetometer included in the AR device; and

reconciling the first orientation and the second orientation to determine the second Field of View.

17. The method of claim 15, wherein determining the second Field of View further comprises:

determining a first orientation in the physical environment from the station keeping image;

determining a change in orientation from an initial orientation based on data from an accelerometer and a gyroscope included in the AR device; and

reconciling the first orientation and the change in orientation from the initial orientation to determine the second Field of View.

18. The method of claim 15, wherein the station keeping image is captured in response to one of a drift threshold and a saturation threshold being satisfied for one of a magnetometer, an accelerometer, and a gyroscope included in the AR device.

19. The method of claim 15, wherein the station keeping image is captured through a wide angle lens in a light path of the camera.

20. The method of claim 15, wherein the station keeping image tracks a position of an orientation image projected onto a ceiling of the physical environment.