AUGMENTED REALITY EXTRAPOLATION TECHNIQUES

• US 20120268490A1
• Filed: 04/20/2011
• Published: 10/25/2012
• Est. Priority Date: 04/20/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sugden, Benjamin J.
• Granted Patent: US 9,262,950 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2320/02 Improving the quality of display appearance
• CPC Class Codes: G09G 2320/0252 Improving the response speed
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2320/08 Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/12 Frame memory handling
• CPC Class Codes: G09G 2380/00 Specific applications
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

Augmented reality extrapolation techniques are described. In one or more implementations, an augmented-reality display is rendered based at least in part on a first basis that describes a likely orientation or position of at least a part of the computing device. The rendered augmented-reality display is updated based at least in part on data that describes a likely orientation or position of the part of the computing device that was assumed during the rendering of the augmented-reality display.

Claims

1. A method implemented by a computing device, the method comprising:

rendering an augmented-reality display based at least in part on a first basis that describes a likely orientation or position of at least a part of the computing device; and

updating the rendered augmented-reality display based at least in part on data that describes a likely orientation or position of the part of the computing device that was assumed during the rendering of the augmented-reality display.

2. A method as described in claim 1, wherein the rendering of the augmented-reality display causes a frame to be stored in a frame buffer of the computing device.

3. A method as described in claim 1, wherein the updating is performed before displaying the rendered augmented-reality display by a display device of the computing device.

4. A method as described in claim 1, wherein the updating is performed using a frame of the augmented-reality display that is stored in a frame buffer of the computing device.

5. A method as described in claim 1, wherein the updating is performed using data obtained from a gyroscope, an accelerometer, a magnetometer, or an inertial measurement unit (IMU) of the computing device.

6. A method as described in claim 1, wherein the updating uses the first basis and data obtained subsequent to calculation of the first basis.

7. A method as described in claim 1, wherein the first basis is a camera basis and the updating is performed by extrapolating a new camera basis by integrating velocities read from one or more sensors of the computing device that are not a camera and compositing the velocities with the first basis that was used in the rendering.

8. A method as described in claim 1, wherein the augmented-reality display includes a view of an image captured by a camera of the display device and an object generated by the computing device to be added to the image, the generation of the object based at least in part on the first basis.

9. A method implemented by a computing device, the method comprising:

calculating a first basis that describes a likely position or orientation of at least a part of the computing device;

generating an augmentation by the computing device for display as part of an augmented-reality display based at least in part on the first basis; and

calculating a modification to be applied to a rendering of the augmented-reality display that includes the augmentation, the modification based on a second basis that describes a likely orientation or position to which the part of the computing device was moved during the rendering of the augmented-reality display.

10. A method as described in claim 9, wherein the generated augmentation is configured such that an object appears to be added to a physical environment of the computing device.

11. A method as described in claim 9, wherein the second basis is computed by interpolating the first basis using data obtained from one or more sensors of the computing device.

12. A method as described in claim 11, wherein the one or more sensors are configured as a gyroscope, an accelerometer, a magnetometer, or an inertial measurement unit (IMU).

13. A method as described in claim 9, further comprising displaying the augmented-reality display by the computing device using one or more display devices.

14. A method as described in claim 9, wherein the first basis is formed by verifying an optical basis calculated using an image taken by a camera of the computing device with a sensor basis calculated using a sensor basis calculated using data obtained from one or more sensors that are not a camera.

15. An apparatus comprising:

a frame buffer;

one or more sensors; and

one or more modules that are configured to;

calculate a first basis that describes a likely orientation or position of at least a part of the apparatus in a physical environment;

render a frame of an augmented-reality display to the frame buffer that is based at least in part on the first basis;

obtain data from the one or more sensors that describes a likely position or orientation of the part of the apparatus in the physical environment during the rendering of the frame in the frame buffer;

calculate a second basis using the first basis and the obtained data; and

update the frame in the frame buffer based on the second basis.

16. An apparatus as described in claim 15, wherein the one or more sensors include an inertial measure unit (IMU), a gyroscope, an accelerometer, or a magnetometer.

17. An apparatus as described in claim 15, further comprising a display device to display the frame that is configured to be wearable by a user.

18. An apparatus as described in claim 15, wherein the updating is performed to the frame before the frame is displayed by the display device.

19. An apparatus as described in claim 15, wherein the one or more sensors are not a camera and the first basis is formed at least in part using data obtained via a camera.

20. An apparatus as described in claim 15, wherein the augmented-reality display includes an augmentation and an image captured using a camera of the apparatus.