SUPER-RESOLUTION BASED FOVEATED RENDERING

• US 20190026864A1
• Filed: 11/10/2017
• Published: 01/24/2019
• Est. Priority Date: 11/10/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Chen, Fan, Li, Zhengmin
• Granted Patent: US 10,643,307 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 3/4046 using neural networks
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

An embodiment of a semiconductor package apparatus may include technology to identify a region of interest portion of a first image, and render the region of interest portion with super-resolution. Other embodiments are disclosed and claimed.

Claims

1. An electronic processing system, comprising:

a graphics processor;

memory communicatively coupled to the graphics processor; and

logic communicatively coupled to the graphics processor to;

identify a region of interest portion of a first image, and

render the region of interest portion with super-resolution.

2. The system of claim 1, wherein the logic is further to:

provide the region of interest portion of the first image to a super-resolution network to generate a super-resolution enhanced image;

up-sample the first image to generate an up-sampled image; and

combine the super-resolution enhanced image with the up-sampled image to provide a foveated image.

3. The system of claim 1, wherein the logic is further to:

train a super-resolution network to provide a smooth transition between the region of interest portion of the first image and other portions of the first image.

4. The system of claim 3, wherein the logic is further to:

crop a training image based on the region of interest to generate a cropped image;

down-sample the cropped image to generate a down-sampled image;

up-sample the down-sampled image to generate an up-sampled image; and

blend the up-sampled image with the cropped image to generate a target image.

5. The system of claim 4, wherein the logic is further to:

train the super-resolution network with the down-sampled image as an input image for the super-resolution network and the target image as a target output image for the super-resolution network.

6. The system of claim 1, wherein a resolution of the first image is lower than a resolution of a target display.

7. A semiconductor package apparatus, comprising:

one or more substrates; and

logic coupled to the one or more substrates, wherein the logic is at least partly implemented in one or more of configurable logic and fixed-functionality hardware logic, the logic coupled to the one or more substrates to;

identify a region of interest portion of a first image, and

render the region of interest portion with super-resolution.

8. The apparatus of claim 7, wherein the logic is further to:

provide the region of interest portion of the first image to a super-resolution network to generate a super-resolution enhanced image;

up-sample the first image to generate an up-sampled image; and

combine the super-resolution enhanced image with the up-sampled image to provide a foveated image.

9. The apparatus of claim 7, wherein the logic is further to:

train a super-resolution network to provide a smooth transition between the region of interest portion of the first image and other portions of the first image.

10. The apparatus of claim 9, wherein the logic is further to:

crop a training image based on the region of interest to generate a cropped image;

down-sample the cropped image to generate a down-sampled image;

up-sample the down-sampled image to generate an up-sampled image; and

blend the up-sampled image with the cropped image to generate a target image.

11. The apparatus of claim 10, wherein the logic is further to:

train the super-resolution network with the down-sampled image as an input image for the super-resolution network and the target image as a target output image for the super-resolution network.

12. The apparatus of claim 7, wherein a resolution of the first image is lower than a resolution of a target display.

13. A method of rendering an image, comprising:

identifying a region of interest portion of a first image; and

rendering the region of interest portion with super-resolution.

14. The method of claim 13, further comprising:

providing the region of interest portion of the first image to a super-resolution network to generate a super-resolution enhanced image;

up-sampling the first image to generate an up-sampled image; and

combining the super-resolution enhanced image with the up-sampled image to provide a foveated image.

15. The method of claim 13, further comprising:

training a super-resolution network to provide a smooth transition between the region of interest portion of the first image and other portions of the first image.

16. The method of claim 15, further comprising:

cropping a training image based on the region of interest to generate a cropped image;

down-sampling the cropped image to generate a down-sampled image;

up-sampling the down-sampled image to generate an up-sampled image; and

blending the up-sampled image with the cropped image to generate a target image.

17. The method of claim 16, further comprising:

training the super-resolution network with the down-sampled image as an input image for the super-resolution network and the target image as a target output image for the super-resolution network.

18. The method of claim 13, wherein a resolution of the first image is lower than a resolution of a target display.

19. At least one non-transitory computer readable medium, comprising a set of instructions, which when executed by a computing device, cause the computing device to:

identify a region of interest portion of a first image; and

render the region of interest portion with super-resolution.

20. The at least one non-transitory computer readable medium of claim 19, comprising a further set of instructions, which when executed by the computing device, cause the computing device to:

provide the region of interest portion of the first image to a super-resolution network to generate a super-resolution enhanced image;

up-sample the first image to generate an up-sampled image; and

combine the super-resolution enhanced image with the up-sampled image to provide a foveated image.

21. The at least one non-transitory computer readable medium of claim 19, comprising a further set of instructions, which when executed by the computing device, cause the computing device to:

train a super-resolution network to provide a smooth transition between the region of interest portion of the first image and other portions of the first image.

22. The at least one non-transitory computer readable medium of claim 21, comprising a further set of instructions, which when executed by the computing device, cause the computing device to:

crop a training image based on the region of interest to generate a cropped image;

down-sample the cropped image to generate a down-sampled image;

up-sample the down-sampled image to generate an up-sampled image; and

blend the up-sampled image with the cropped image to generate a target image.

23. The at least one non-transitory computer readable medium of claim 22, comprising a further set of instructions, which when executed by the computing device, cause the computing device to:

train the super-resolution network with the down-sampled image as an input image for the super-resolution network and the target image as a target output image for the super-resolution network.

24. The at least one non-transitory computer readable medium of claim 19, wherein a resolution of the first image is lower than a resolution of a target display.