Dynamic foveation adjustment

• US 10,410,313 B2
• Filed: 08/05/2016
• Issued: 09/10/2019
• Est. Priority Date: 08/05/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Saleh, Skyler Jonathon, Ribble, Maurice Franklin, Garcia Garcia, David Rigel
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US15/230,073
• Publication Number:
  US 20180040097A1
• Time in Patent Office: 1,131 Days
• Field of Search: 345611
• CPC Class Codes: G06F 9/5094 where the allocation takes into account power or heat criteria power management in computers in general G06F1/3203; thermal management in computers in general G06F1/206
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

This disclosure describes examples for determining an amount of foveation that is to be applied for rendering an image. The example techniques may use information indicative of a performance of a graphics processing unit (GPU) to determine the amount of foveation that is to be applied. The GPU may render an image based on the determined amount of foveation.

Claims

1. A method of processing data, the method comprising:

receiving information indicative of a performance of a graphics processing unit (GPU), wherein the information includes a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage, and the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage;

determining a GPU performance based on at least one of the first value and the second value;

comparing the GPU performance to a GPU performance threshold;

determining an amount of foveation to apply based on the comparison; and

outputting information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

2. The method of claim 1, further comprising:

receiving information indicating an amount of foveation that is to be applied,

wherein determining the amount of foveation to apply comprises adjusting the amount of foveation that is to be applied.

3. The method of claim 1, wherein determining the amount of foveation to apply based on the comparison comprises repeatedly increasing an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

4. The method of claim 1,

further comprising:

weighting the first value by a first weighting factor to generate a first weighted factor; and

weighting the second value by a second, different weighting factor to generate a second weighted factor,

wherein determining the GPU performance comprises determining the GPU performance based at least in part on the first weighted factor and the second weighted factor.

5. The method of claim 1, wherein determining the amount of foveation to apply based on the received information comprises determining the amount of foveation to apply during the rendering of the image,

the method further comprising:

causing the GPU to render the image based on the determined amount of foveation to apply by updating the amount of foveation to apply during the rendering of the image.

6. The method of claim 1, wherein the image is for a virtual reality (VR) application.

19. The method of claim 5, wherein the image is rendered by the GPU with a first portion at a first resolution and a second portion at a second resolution, wherein the second resolution is lower than the first resolution, and the first portion and the second portion are determined by the amount of foveation to apply.

7. A device for processing data, the device comprising:

a graphics processing unit (GPU);

a performance estimation circuit configured to receive information indicative of a performance of the GPU, wherein the information includes a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage, and wherein the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage, and wherein the performance estimation circuit is configured to;

determine a GPU performance based on at least one of the first value and the second value; and

compare the GPU performance to a GPU performance threshold; and

a foveation gate circuit configured to;

determine an amount of foveation to apply based on the comparison; and

output information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

8. The device of claim 7, wherein the foveation gate circuit is configured to receive information indicating an amount of foveation that is to be applied, and wherein to determine the amount of foveation to apply, the foveation circuit is configured to adjust the amount of foveation that is to be applied.

9. The device of claim 7, wherein to determine the amount of foveation to apply based on the comparison, the foveation gate circuit is configured to repeatedly increase an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

10. The device of claim 7,

wherein the performance estimation circuit is configured to:

weight the first value by a first weighting factor to generate a first weighted factor, and

weight the second value by a second, different weighting factor to generate a second weighted factor,

wherein to determine the GPU performance, the foveation gate circuit is configured to determine the GPU performance based at least in part on the first weighted factor and the second weighted factor.

11. The device of claim 7, wherein to determine the amount of foveation to apply based on the received information, the foveation gate circuit is configured to determine the amount of foveation to apply during the rendering of the image.

12. The device of claim 7, wherein the image is for a virtual reality (VR) application.

13. A device for processing data, the device comprising:

means for receiving information indicative of a performance of a graphics processing unit (GPU), wherein the information includes a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage, and wherein the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage;

means for determining a GPU performance based on at least one of the first value and the second value;

means for comparing the GPU performance to a GPU performance threshold;

means for determining an amount of foveation to apply based on the comparison; and

means for outputting information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

14. The device of claim 13, further comprising:

means for receiving information indicating an amount of foveation that is to be applied,

wherein the means for determining the amount of foveation to apply comprises means for adjusting the amount of foveation that is to be applied.

15. The device of claim 13, wherein the means for determining the amount of foveation to apply based on the comparison comprises means for repeatedly increasing an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

16. A non-transitory computer-readable storage medium comprising instructions that when executed cause one or more processors of a device for processing data to:

receive information indicative of a performance of a graphics processing unit (GPU), wherein the information includes a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage, and the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, or the graphics API usage;

determine a GPU performance based on at least one of the first value and the second value;

compare the GPU performance to a GPU performance threshold;

determine an amount of foveation to apply based on the comparison; and

output information of the determined amount of foveation to apply to the GPU for the GPU to render an image based on the determined amount of foveation to apply.

17. The non-transitory computer-readable storage medium of claim 16, further comprising instructions that cause the one or more processors to:

receive information indicating an amount of foveation that is to be applied,

wherein the instructions that cause the one or more processors to determine the amount of foveation to apply comprise instructions that cause the one or more processors to adjust the amount of foveation that is to be applied.

18. The non-transitory computer-readable storage medium of claim 16, wherein instructions that cause the one or more processors to determine the amount of foveation to apply based on the comparison comprise instructions that cause the one or more processors to repeatedly increase an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.