DYNAMIC FOVEATION ADJUSTMENT

• US 20180040097A1
• Filed: 08/05/2016
• Published: 02/08/2018
• Est. Priority Date: 08/05/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Saleh, Skyler Jonathon, Ribble, Maurice Franklin, Garcia Garcia, David Rigel
• Granted Patent: US 10,410,313 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 9/5094 where the allocation takes into account power or heat criteria power management in computers in general G06F1/3203; thermal management in computers in general G06F1/206
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

This disclosure describes examples for determining an amount of foveation that is to be applied for rendering an image. The example techniques may use information indicative of a performance of a graphics processing unit (GPU) to determine the amount of foveation that is to be applied. The GPU may render an image based on the determined amount of foveation.

Claims

1. A method of processing data, the method comprising:

receiving information indicative of a performance of a graphics processing unit (GPU), wherein receiving information indicative of the performance of the GPU comprises receiving information indicative of one or more of power consumption, bandwidth usage, GPU core utilization, temperature, or graphics application programming interface (API) usage;

determining a GPU performance based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage;

comparing the GPU performance to a GPU performance threshold;

determining an amount of foveation to apply based on the comparison; and

outputting information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

2. The method of claim 1, further comprising:

receiving information indicating an amount of foveation that is to be applied,

wherein determining the amount of foveation to apply comprises adjusting the amount of foveation that is to be applied.

3. The method of claim 1,

wherein determining the GPU performance comprises determining a GPU performance value based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

wherein comparing the GPU performance comprises comparing the GPU performance value to a GPU performance threshold value, and

wherein determining the amount of foveation to apply comprises determining the amount of foveation to apply based on the comparison of the GPU performance value to the GPU performance threshold value.

4. The method of claim 3, wherein determining the amount of foveation to apply based on the comparison comprises repeatedly increasing an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

5. The method of claim 1,

wherein receiving information indicative of the performance of the GPU comprises receiving a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, temperature, or the graphics API usage, and wherein the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

the method further comprising:

weighting the first value by a first weighting factor to generate a first weighted factor;

weighting the second value by a second, different weighting factor to generate a second weighted factor; and

wherein determining the GPU performance comprises determining the GPU performance based at least in part on the first weighted factor and the second weighted factor.

7. The method of claim 1, wherein determining the amount of foveation to apply based on the received information comprises determining the amount of foveation to apply during the rendering of the image,

the method further comprising:

causing the GPU to render the image based on the determined amount of foveation to apply by updating the amount of foveation to apply during the rendering of the image.

8. The method of claim 1, wherein the image is for a virtual reality (VR) application.

6. (canceled)

9. A device for processing data, the device comprising:

a graphics processing unit (GPU);

a performance estimation circuit configured to receive information indicative of a performance of the GPU, wherein to receive information indicative of the performance of the GPU, the performance estimation circuit is configured to receive information indicative of one or more of power consumption, bandwidth usage, GPU core utilization, temperature, or graphics application programming interface (API) usage, and wherein the performance estimation circuit is configured to;

determine a GPU performance based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage; and

compare the GPU performance to a GPU performance threshold; and

a foveation gate circuit configured to;

determine an amount of foveation to apply based on the comparison; and

output information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

10. The device of claim 9, wherein the foveation gate circuit is configured to receive information indicating an amount of foveation that is to be applied, and wherein to determine the amount of foveation to apply, the foveation circuit is configured to adjust the amount of foveation that is to be applied.

11. The device of claim 9,

wherein to determine the GPU performance, the performance estimation circuit is configured to determine a GPU performance value based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

wherein to compare the GPU performance, the performance estimation circuit is configured to compare the GPU performance value to a GPU performance threshold value, and

wherein to determine the amount of foveation to apply, the foveation gate circuit is configured to determine the amount of foveation to apply based on the comparison of the GPU performance value to the GPU performance threshold value.

12. The device of claim 11, wherein to determine the amount of foveation to apply based on the comparison, the foveation gate circuit is configured to repeatedly increase an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

13. The device of claim 9,

wherein to receive information indicative of the performance of the GPU, the performance estimation circuit is configured to receive a first value indicative of the performance of the GPU and a second value indicative of the performance of the GPU, wherein the first value is indicative of one of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage, and wherein the second value is indicative of another one of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

wherein the performance estimation circuit is configured to:

weight the first value by a first weighting factor to generate a first weighted factor, and

weight the second value by a second, different weighting factor to generate a second weighted factor; and

wherein to determine the GPU performance, the foveation gate circuit is configured to determine the GPU performance based at least in part on the first weighted factor and the second weighted factor.

15. The device of claim 9, wherein to determine the amount of foveation to apply based on the received information, the foveation gate circuit is configured to determine the amount of foveation to apply during the rendering of the image.

16. The device of claim 9, wherein the image is for a virtual reality (VR) application.

14. (canceled)

17. A device for processing data, the device comprising:

means for receiving information indicative of a performance of a graphics processing unit (GPU), wherein the means for receiving information indicative of the performance of the GPU comprises means for receiving information indicative of one or more of power consumption, bandwidth usage, GPU core utilization, temperature, or graphics application programming interface (API) usage;

means for determining a GPU performance based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage;

means for comparing the GPU performance to a GPU performance threshold;

means for determining an amount of foveation to apply based on the comparison; and

means for outputting information of the determined amount of foveation to apply for the GPU to render an image based on the determined amount of foveation to apply.

18. The device of claim 17, further comprising:

means for receiving information indicating an amount of foveation that is to be applied,

wherein the means for determining the amount of foveation to apply comprises means for adjusting the amount of foveation that is to be applied.

19. The device of claim 17,

wherein the means for determining the GPU performance comprises means for determining a GPU performance value based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

wherein the means for comparing the GPU performance comprises means for comparing the GPU performance value to a GPU performance threshold value, and

wherein the means for determining the amount of foveation to apply comprises means for determining the amount of foveation to apply based on the comparison of the GPU performance value to the GPU performance threshold value.

20. The device of claim 19, wherein the means for determining the amount of foveation to apply based on the comparison comprises means for repeatedly increasing an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.

21. A non-transitory computer-readable storage medium comprising instructions that when executed cause one or more processors of a device for processing data to:

receive information indicative of a performance of a graphics processing unit (GPU), wherein the instruction that cause the one or more processors to receive information comprise instructions that cause the one or more processors to receive information indicative of one or more of power consumption, bandwidth usage, GPU core utilization, temperature, or graphics application programming interface (API) usage;

determine a GPU performance based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage;

compare the GPU performance to a GPU performance threshold;

determine an amount of foveation to apply based on the comparison; and

output information of the determined amount of foveation to apply to the GPU for the GPU to render an image based on the determined amount of foveation to apply.

22. The non-transitory computer-readable storage medium of claim 21, further comprising instructions that cause the one or more processors to:

receive information indicating an amount of foveation that is to be applied,

wherein the instructions that cause the one or more processors to determine the amount of foveation to apply comprise instructions that cause the one or more processors to adjust the amount of foveation that is to be applied.

23. The non-transitory computer-readable storage medium of claim 21,

wherein the instructions that cause the one or more processors to determine the GPU performance comprises instructions that cause the one or more processors to determine a GPU performance value based on one or more of the power consumption, the bandwidth usage, the GPU core utilization, the temperature, or the graphics API usage,

wherein the instructions that cause the one or more processors to compare the GPU performance comprise instructions that cause the one or more processors to compare the GPU performance value to a GPU performance threshold value, and

wherein the instructions that cause the one or more processors to determine the amount of foveation to apply comprise instructions that cause the one or more processors to determine the amount of foveation to apply based on the comparison of the GPU performance value to the GPU performance threshold value.

24. The non-transitory computer-readable storage medium of claim 23, wherein instructions that cause the one or more processors to determine the amount of foveation to apply based on the comparison comprise instructions that cause the one or more processors to repeatedly increase an amount of foveation being applied until the GPU performance value is less than or equal to the GPU performance threshold value.