Adaptive scheduling to maintain smooth frame rate

• US 20070057952A1
• Filed: 09/14/2005
• Published: 03/15/2007
• Est. Priority Date: 09/14/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schechter, Greg, Swedberg, Gregory, Evans, Glenn, Blanco, Leonardo, Ratanchandani, Prashant, Reneris, Kenneth, Nene, Sameer
• Granted Patent: US 7,450,130 B2
• Time in Patent Office: Days
• US Class Current: 345/474
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter

Abstract

Described is an adaptive scheduler associated with a desktop window manager that dynamically controls the rate at which graphics frames are composed. Values corresponding to performance when composing a frame are measured, and the frame composition rate is adjusted as necessary based on the values. The measured data is sampled to provide smooth adjustments. The sampled data is evaluated as to whether the current frame rate is too slow, too fast, or acceptable. If too slow, the frame rate may increased relative to the refresh rate, while if too fast, the frame rate is decreased relative to the refresh rate. In one implementation, the frame rate is too fast if a count of missed frames achieves a missed threshold value, or if a count of late frames achieves a late threshold value. The frame rate is too slow if a count of early frames exceeds an early threshold value.

Claims

1. At least one computer-readable medium having computer-executable instructions, which when executed perform steps, comprising:

measuring a set of values corresponding to performance with respect to composing a frame; and

adjusting a rate of composing frames based on the set of values.

2. The computer-readable medium of claim 1 wherein measuring a set of values comprises maintaining a count of missed frames, a count of late frames and a count of early frames, and comparing at least one of the counts to a threshold limit corresponding to that count.

3. The computer-readable medium of claim 2 further comprising obtaining the count of missed frames, the count of late frames and the count of early frames by evaluating actual frame composition times against time values corresponding to a hard deadline time, a budget deadline time and an early deadline time, respectively.

4. The computer-readable medium of claim 3 further comprising computing the hard deadline time, the budget deadline time and the early deadline time.

5. The computer-readable medium of claim 3 wherein maintaining the count of missed frames, the count of late frames and the count of early frames comprises maintaining a bit array that contains results of comparisons for each actual frame composition time against the hard deadline time, the budget deadline time and the early deadline time, respectively.

6. The computer-readable medium of claim 1 wherein adjusting a rate of composing frames based on the set of values comprises decreasing the rate upon detecting a number of missed frames or a number of late frames, or increasing the frame rate up to a graphics subsystem refresh rate upon detecting a number of early frames.

7. The computer-readable medium of claim 1 wherein adjusting a rate of composing frames based on the set of values comprises decreasing the rate when a count of missed frames achieves a missed threshold value, and if not achieved, decreasing the rate if a count of late frames achieves a late threshold value, and if not achieved, increasing the rate up to a graphics subsystem refresh rate if a count of early frames exceeds an early threshold value.

8. At least one computer-readable medium having computer-executable instructions, which when executed perform steps, comprising:

sampling data corresponding to a plurality of frame rate composition times;

determining whether the data indicates a pattern of a current frame rate being too slow, too fast, or acceptable relative to a graphics subsystem refresh rate; and

a) if the pattern indicates the frame rate is too slow, increasing the frame rate relative to the refresh rate; and

b) if the pattern indicates the frame rate is too fast, decreasing the frame rate relative to the refresh rate.

9. The computer-readable medium of claim 8 wherein determining whether the data indicates a pattern comprises maintaining values corresponding to missed frames, values corresponding to late frames and values corresponding to early frames, and evaluating the values.

10. The computer-readable medium of claim 9 wherein the values corresponding to missed frames correspond to a count of frame composition times that exceeded a hard deadline time, wherein the values corresponding to late frames correspond to a count of frame composition times that exceeded a budget deadline time, and wherein the values corresponding to early frames correspond to a count of frame composition times that preceded an early deadline time.

11. The computer-readable medium of claim 10 further comprising computing the hard deadline time, the budget deadline time and the early deadline time.

12. The computer-readable medium of claim 10 wherein maintaining the count of missed frames, the count of late frames and the count of early frames comprises maintaining a bit array that contains results of comparisons for each actual frame composition time against the hard deadline time, the budget deadline time and the early deadline time, respectively.

13. In a computing environment having at least one computing device, a system comprising:

a window manager that receives graphics instructions from application programs and composes frames of video data based on those instructions for sending to a graphics subsystem; and

an adaptive scheduler coupled to the window manager that controls a rate of frame composition of the window manager based on measured performance data with respect to composing frames.

14. The system of claim 13 wherein the adaptive scheduler maintains a buffer means for maintaining the measured performance data including at least one of missed frames, late frames and early frames.

15. The system of claim 14 wherein the measured performance data is maintained as data that can be counted for each of the missed frames, late frames and early frames.

16. The system of claim 15 wherein the data that can be counted comprises a bit array in which bits contain values indicative of whether each frame composition time corresponded to a missed frame, to a late frame and/or to an early frame.

17. The system of claim 13 wherein the adaptive scheduler includes comparison means for evaluating an actual frame composition time against a hard deadline time, a budget deadline time and an early deadline time.

18. The system of claim 17 wherein the adaptive scheduler includes means for computing the hard deadline time, the budget deadline time and the early deadline time for at least one frame.

19. The system of claim 13 wherein the adaptive scheduler adjusts the rate of composing frames by decreasing the rate upon detecting a number of missed frames or a number of late frames, or increasing the frame rate up to the graphics subsystem refresh rate upon detecting a number of early frames.

20. The system of claim 13 wherein the adaptive scheduler adjusts the rate of composing frames by:

a) decreasing the rate when a count of missed frames achieves a missed threshold value, and if not achieved, by

b) decreasing the rate if a count of late frames achieves a late threshold value, and if not achieved, by

c) increasing the rate if a count of early frames exceeds an early threshold value.

21. The system of claim 13 wherein the adaptive scheduler notifies at least one other component of a change in the rate of composing frames.

22. The system of claim 13 further comprising means for providing feedback to an application program such that the application program can adjust data submitted for compositing.