VIRTUAL REALITY IMAGE COMPOSITING

• US 20180322669A1
• Filed: 05/03/2017
• Published: 11/08/2018
• Est. Priority Date: 05/03/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: ELLIOTT, Jack, YEUNG, Andrew
• Granted Patent: US 10,885,711 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

Methods and devices for performing one step compositing may include receiving at least a first surface from a first image source and a second surface from a second image source. The methods and devices may include generating a composite image with a combined first surface and second surface by contemporaneously applying position corrections to the first surface and the second surface and lens distortion corrections to the first surface and the second surface and blending the first surface and the second surface into respective positions in the composite image. The methods and devices may include transmitting the composite image representing a virtual environment for presentation to a downstream consumer.

Claims

1. A computer device, comprising:

a memory to store data and instructions;

a processor in communication with the memory; and

an operating system in communication with the memory and the processor, wherein the operating system is operable to;

receive at least a first surface from a first image source and a second surface from a second image source;

generate a composite image with a combined first surface and second surface by contemporaneously applying position corrections to the first surface and the second surface and lens distortion corrections to the first surface and the second surface and blending the first surface and the second surface into respective positions in the composite image; and

transmit the composite image representing a virtual environment for presentation to a downstream consumer.

2. The computer device of claim 1, wherein the operating system is further operable to:

receive a latest change in position for the first surface and the second surface; and

apply the latest change in position to the first surface and the second surface,

wherein the composite image is generated by contemporaneously applying the position corrections, the lens distortion corrections, and the latest change in position to the first surface and the second surface.

3. The computer device of claim 1, wherein to apply the position corrections to the first surface and the second surface the operating system is further operable to translate the first surface and the second surface into a common coordinate system.

4. The computer device of claim 1, wherein the operating system is further operable to:

receive a first position for the first surface corresponding to a first head position of a user; and

receive a second position for the second surface corresponding to a second head position of the user;

wherein to apply the position corrections to the first surface and the second surface the operating system is further operable to translate the first head position and the second head position to a same head position.

5. The computer device of claim 1, wherein the lens distortion corrections account for changes of the first surface and the second surface when projected through a lens of a display device.

6. The computer device of claim 1, wherein the first image source is from a virtual reality application, mixed reality application, or augmented reality application.

7. The computer device of claim 1, wherein the second image source is from a virtual reality application, mixed reality application, or augmented reality application.

8. The computer device of claim 1, wherein the composite image is presented on a two dimensional surface in the virtual environment.

9. The computer device of claim 1, wherein the operating system is operable to generate the composite image and contemporaneously apply the position corrections the lens distortion corrections to the first surface and the second surface in one step.

10. The computer device of claim 1, wherein the composite image is displayed to a user.

11. A method for performing one step compositing, comprising:

receiving, at an operating system executing on a computer device, at least a first surface from a first image source and a second surface from a second image source;

generating, at the operating system, a composite image with a combined first surface and second surface by contemporaneously applying position corrections to the first surface and the second surface and lens distortion corrections to the first surface and the second surface and blending the first surface and the second surface into respective positions in the composite image; and

transmitting the composite image representing a virtual environment for presentation to a downstream consumer.

12. The method of claim 11, further comprising:

receiving a latest change in position for the first surface and the second surface; and

applying the latest change in position to the first surface and the second surface,

wherein the composite image is generated by contemporaneously applying the position corrections, the lens distortion corrections, and the latest change in position to the first surface and the second surface.

13. The method of claim 11, wherein to apply the position corrections to the first surface and the second surface further includes translating the first surface and the second surface into a common coordinate system.

14. The method of claim 11, further comprising:

receiving a first position for the first surface corresponding to a first head position of a user; and

receiving a second position for the second surface corresponding to a second head position of the user,

wherein to apply the position corrections to the first surface and the second surface further includes translating the first head position and the second head position to a same head position.

15. The method of claim 11, wherein the lens distortion corrections account for changes of the first surface and the second surface when projected through a lens of a display device.

16. The method of claim 11, wherein the first image source is from a virtual reality application, mixed reality application, or augmented reality application.

17. The method of claim 11, wherein the second image source is from a virtual reality application, mixed reality application, or augmented reality application.

18. The method of claim 11, wherein the composite image is presented on a two dimensional surface in the virtual environment.

19. The method of claim 11, wherein generating the composite image further comprises contemporaneously applying the position corrections the lens distortion corrections to the first surface and the second surface in one step.

20. A computer-readable medium storing instructions executable by a computer device, comprising:

at least one instruction for causing the computer device to receive at least a first surface from a first image source and a second surface from a second image source;

at least one instruction for causing the computer device to generate a composite image with a combined first surface and second surface by contemporaneously applying position corrections to the first surface and the second surface and lens distortion corrections to the first surface and the second surface and blending the first surface and the second surface into respective positions in the composite image; and

at least one instruction for causing the computer device to transmit the composite image representing a virtual environment for presentation to a downstream consumer.