OPTIMIZING HEAD MOUNTED DISPLAYS FOR AUGMENTED REALITY
First Claim
1. A computer system for causing a head mounted display (HMD) to render an augmented reality image, comprising:
- at least one processor; and
at least one memory comprising instructions configured to cause the at least one processor to perform a method comprising;
receiving visual data associated with a first orientation of the HMD;
receiving depth frame data associated with the first orientation of the HMD;
receiving IMU data associated with the first orientation of the HMD;
receiving localization results associated with the first orientation of the HMD;
determining a future rendering time when the HMD will present an image to a user;
determining a second, predicted orientation associated with the HMD at the future rendering time based upon the IMU data, the localization results, and the future rendering time;
generating a three-dimensional (3D) mesh from the depth frame data associated with the first orientation and specifying the texture for the 3D mesh from the visual data associated with the first orientation;
determining a transform between the first orientation and the second orientation;
applying the transform to the generated 3D mesh; and
causing at least a portion of the resulting textured 3D mesh to be rendered in the HMD to the user as a final image.
2 Assignments
0 Petitions
Accused Products
Abstract
While many augmented reality systems provide “see-through” transparent or translucent displays upon which to project virtual objects, many virtual reality systems instead employ opaque, enclosed screens. Indeed, eliminating the user'"'"'s perception of the real-world may be integral to some successful virtual reality experiences. Thus, head mounted displays designed exclusively for virtual reality experiences may not be easily repurposed to capture significant portions of the augmented reality market. Various of the disclosed embodiments facilitate the repurposing of a virtual reality device for augmented reality use. Particularly, by anticipating user head motion, embodiments may facilitate scene renderings better aligned with user expectations than naïve renderings generated within the enclosed field of view. In some embodiments, the system may use procedural mapping methods to generate a virtual model of the environment. The system may then use this model to supplement the anticipatory rendering.
41 Citations
20 Claims
-
1. A computer system for causing a head mounted display (HMD) to render an augmented reality image, comprising:
-
at least one processor; and at least one memory comprising instructions configured to cause the at least one processor to perform a method comprising; receiving visual data associated with a first orientation of the HMD; receiving depth frame data associated with the first orientation of the HMD; receiving IMU data associated with the first orientation of the HMD; receiving localization results associated with the first orientation of the HMD; determining a future rendering time when the HMD will present an image to a user; determining a second, predicted orientation associated with the HMD at the future rendering time based upon the IMU data, the localization results, and the future rendering time; generating a three-dimensional (3D) mesh from the depth frame data associated with the first orientation and specifying the texture for the 3D mesh from the visual data associated with the first orientation; determining a transform between the first orientation and the second orientation; applying the transform to the generated 3D mesh; and causing at least a portion of the resulting textured 3D mesh to be rendered in the HMD to the user as a final image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A computer-implemented method for rendering an augmented reality image in a head mounted display (HMD), comprising:
-
receiving visual data associated with a first orientation of the HMD; receiving depth frame data associated with the first orientation of the HMD; receiving IMU data associated with the first orientation of the HMD; receiving localization results associated with the first orientation of the HMD; determining a future rendering time when the HMD will present an image to a user; determining a second, predicted orientation associated with the HMD at the future rendering time based upon the IMU data, the localization results, and the future rendering time; generating a three-dimensional (3D) mesh from the depth frame data associated with the first orientation and specifying the texture for the 3D mesh from the visual data associated with the first orientation; determining a transform between the first orientation and the second orientation; applying the transform to the generated 3D mesh; and causing at least a portion of the resulting textured 3D mesh to be rendered in the HMD to the user as a final image. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable medium comprising instructions configured to cause a computer system to perform a method for rendering an augmented reality image in a head mounted display (HMD), comprising:
-
receiving visual data associated with a first orientation of the HMD; receiving depth frame data associated with the first orientation of the HMD; receiving IMU data associated with the first orientation of the HMD; receiving localization results associated with the first orientation of the HMD; determining a future rendering time when the HMD will present an image to a user; determining a second, predicted orientation associated with the HMD at the future rendering time based upon the IMU data, the localization results, and the future rendering time; generating a three-dimensional (3D) mesh from the depth frame data associated with the first orientation and specifying the texture for the 3D mesh from the visual data associated with the first orientation; determining a transform between the first orientation and the second orientation; applying the transform to the generated 3D mesh; and causing at least a portion of the resulting textured 3D mesh to be rendered in the HMD to the user as a final image. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification