MULTI-PLANE DISPLAY IMAGE SYNTHESIS MECHANISM

• US 20200137380A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Supikov, Alexey, Huang, Qiong
• Application Number:
  US16/176,418
• Publication Number:
  US 20200137380A1
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: H04N 13/395 with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

An apparatus to facilitate generating a multi-focal/multi-plane (MF/MP) display is disclosed. The apparatus comprises one or more processors to generate a plurality full resolution views for each frame of a three-dimension (3D) scene, perform deep neural network (DNN) inferencing using the plurality of full resolution views to select two or more presentation planes from among a plurality of available planes for display.

Claims

1. An apparatus to facilitate generating a multi-focal/multi-plane (MF/MP) display, comprising:

one or more processors to generate a plurality of full resolution views for each frame of a three-dimension (3D) scene, perform deep neural network (DNN) inferencing using the plurality of full resolution views to select two or more presentation planes from among a plurality of available planes for display.

2. The apparatus of claim 1, wherein the full resolution views comprise a plurality of red, green, blue and depth (RGBD) images.

3. The apparatus of claim 2, wherein the one or more processors further to train the DNN.

4. The apparatus of claim 3, wherein training the DNN comprises performing a rendering pass on the 3D scene to generate Depth-of-Field (DoF) images, generating a focus stack of the DoF images and performing a decomposition to generate focused images of the 3D scene into presentation planes.

5. The apparatus of claim 4, wherein training the DNN further comprises performing a synthesis of all combinations of the presentation planes, selecting a combination of planes having a least percentage of error and generating a best combination of images of the presentation planes.

6. The apparatus of claim 5, wherein training the DNN further comprises applying training input data to a decomposition network to generate the best combination of images and plane labels.

7. The apparatus of claim 2, wherein the one or more processors further to perform optical eye box extension.

8. At least one computer readable medium having instructions stored thereon, which when executed by one or more processors, cause the processors to:

generate a plurality of full resolution views for each frame of a three-dimension (3D) scene;

perform deep neural network (DNN) inferencing using the plurality of full resolution views to select two or more presentation planes from among a plurality of available planes; and

display the two or more presentation planes.

9. The computer readable medium of claim 8, wherein the full resolution views comprise a plurality of red, green, blue and depth (RGBD) images.

10. The computer readable medium of claim 9, having instructions stored thereon, which when executed by one or more processors, further cause the processors to train the DNN.

11. The computer readable medium of claim 10, wherein training the DNN comprises performing a rendering pass on the 3D scene to generate Depth-of-Field (DoF) images, generating a focus stack of the DoF images and performing a decomposition to generate focused images of the 3D scene into presentation planes.

12. The computer readable medium of claim 11, wherein training the DNN further comprises performing a synthesis of all combinations of the presentation planes, selecting a combination of planes having a least percentage of error and generating a best combination of images of the presentation planes.

13. The computer readable medium of claim 12, wherein training the DNN further comprises applying training input data to a decomposition network to generate the best combination of images and plane labels.

14. The computer readable medium of claim 9, having instructions stored thereon, which when executed by one or more processors, further cause the processors to perform optical eye box extension.

15. A method to facilitate generating a multi-focal/multi-plane (MF/MP) display, comprising:

generating a plurality of full resolution views for each frame of a three-dimension (3D) scene;

performing deep neural network (DNN) inferencing using the plurality of full resolution views to select two or more presentation planes from among a plurality of available planes; and

displaying the two or more presentation planes.

16. The method of claim 15, wherein the full resolution views comprise a plurality red, green, blue and depth (RGBD) images.

17. The method of claim 16, further comprising training the DNN.

18. The method of claim 17, wherein training the DNN comprises performing a rendering pass on the 3D scene to generate Depth-of-Field (DoF) images, generating a focus stack of the DoF images and performing a decomposition to generate focused images of the 3D scene into presentation planes.

19. The method of claim 18, wherein training the DNN further comprises performing a synthesis of all combinations of the presentation planes, selecting a combination of planes having a least percentage of error and generating a best combination of images of the presentation planes.

20. The method of claim 19, wherein training the DNN further comprises applying training input data to a decomposition network to generate the best combination of images and plane labels.