Method and system of display object pre-generation on windows of idle time available after each frame buffer fill tasks

• US 10,896,484 B2
• Filed: 10/25/2017
• Issued: 01/19/2021
• Est. Priority Date: 04/11/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Kaldor, Jonathan M.
• Primary Examiner(s): Chu Joy-Davila, Jorge A
• Application Number:
  US15/793,309
• Publication Number:
  US 20180047136A1
• Time in Patent Office: 1,182 Days
• CPC Class Codes: G06F 2209/5018 Thread allocation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/485 Task life-cycle, e.g. stopping, restarting, resuming execution G06F9/4881 takes precedence
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures

Abstract

In one embodiment, a computing device determines a window of time that is available after each operation to fill a frame buffer. The device determines a first number indicating a number of elements of a requested content that fit into a display region, an increment based on an application-specific rule, and a second number indicating a number of elements to pre-generate based on the first number and the increment. The device selects one or more pre-generation tasks for pre-generating content for the second number of select elements of the requested content. During the window of time, the device executes at least a subset of the pre-generation tasks to pre-generate content for a corresponding subset of the select elements. The pre-generated subset of the select elements is stored in an application-tailored recycler.

Claims

1. A method executed by a computing device comprising:

identifying a start point and a yield point of a window of idle time that is available after each frame buffer fill task for a displayable region;

determining that the window of idle time that is available after each frame buffer fill task is more than a predetermined minimum time period required for executing a pre-generation process comprising pre-generating one or more display elements for a requested content that is currently not visible and is outside of the displayable region;

determining a first number indicating a number of display elements of the requested content that fit into the displayable region;

determining an increment based on an application-specific rule;

determining a second number indicating a number of display elements to pre-generate based on the first number and the increment;

selecting one or more pre-generation tasks for pre-generating content for the second number of display elements of the requested content;

in response to the determination that the window of idle time that is available after each frame buffer fill task is more than the predetermined minimum time period required for pre-generating the one or more display elements of the requested content, starting, at the start point of the window of idle time, the pre-generation process for executing the one or more pre-generation tasks, wherein a pre-generation task for pre-generating content during the window of idle time is different from the frame buffer fill task;

executing, during the pre-generation process, a subset of the one or more pre-generation tasks to pre-generate content for a corresponding subset of the display elements;

stopping, at the yield point, the pre-generation process before completing the one or more pre-generation tasks in response to the pre-generation process taking longer than the window of idle time, wherein one or more stopped pre-generation tasks are added to a queue to continue execution in the window of idle time after a next frame buffer fill task; and

storing the pre-generated subset of the display elements in an application-tailored recycler.

2. The method of claim 1, wherein the application-specific rule comprises that the increment is associated with a multiple number of the first number, wherein the increment is determined based on a scroll speed of a user.

3. The method of claim 1, wherein the application-specific rule comprises that the increment is determined based on attributes of the requested content.

4. The method of claim 3, wherein the attributes are associated with display types of the one or more display elements of the requested content.

5. The method of claim 1, further comprising:

prioritizing one of the display elements for pre-generation based on a computational cost of generating the display element.

6. The method of claim 1, further comprising:

prioritizing one of the display elements for pre-generation based on a number of times content referenced by the display element is also referenced by other display elements in the one or more display elements.

7. The method of claim 1, wherein the one or more display elements are selected for pre-generation based on a determination that a total execution time for the one or more pre-generation tasks is less than the window of idle time.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

identify a start point and a yield point of a window of idle time that is available after each frame buffer fill task for a displayable region;

determine that the window of idle time that is available after each frame buffer fill task is more than a predetermined minimum time period required for executing a pre-generation process comprising pre-generating one or more display elements for a requested content that is currently not visible and is outside of the displayable region;

determine a first number indicating a number of display elements of the requested content that fit into the displayable region;

determine an increment based on an application-specific rule;

determine a second number indicating a number of display elements to pre-generate based on the first number and the increment;

select one or more pre-generation tasks for pre-generating content for the second number of display elements of the requested content;

in response to the determination that the window of idle time that is available after each frame buffer fill task is more than the predetermined minimum time period required for pre-generating the one or more display elements of the requested content, start, at the start point of the window of idle time, the pregeneration process for executing the one or more pre-generation tasks, wherein a pre-generation task for pre-generating content during the window of idle time is different from the frame buffer fill task;

execute, during the pre-generation process, a subset of the one or more pre-generation tasks to pre-generate content for a corresponding subset of the display elements;

stop, at the yield point, the pre-generation process before completing the one or more pre-generation tasks in response to the pre-generation process taking longer than the window of idle time, wherein one or more stopped pre-generation tasks are added to a queue to continue execution in the window of idle time after a next frame buffer fill task; and

store the pre-generated subset of the display elements in an application-tailored recycler.

9. The media of claim 8, wherein the application-specific rule comprises that the increment is associated with a multiple number of the first number, wherein the increment is determined based on a scroll speed of a user.

10. The media of claim 8, wherein the application-specific rule comprises that the increment is determined based on attributes of the requested content.

11. The media of claim 10, wherein the attributes are associated with display types of the one or more display elements of the requested content.

12. The media of claim 8, wherein the software is further operable when executed to:

prioritize one of the display elements for pre-generation based on a computational cost of generating the display element.

13. The media of claim 8, wherein the software is further operable when executed to:

prioritize one of the display elements for pre-generation based on a number of times content referenced by the display element is also referenced by other display elements in the one or more display elements.

14. The media of claim 8, wherein the one or more display elements are selected for pre-generation based on a determination that a total execution time for the one or more pre-generation tasks is less than the window of idle time.

15. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to;

identify a start point and a yield point of a window of idle time that is available after each frame buffer fill task for a displayable region;

determine that the window of idle time that is available after each frame buffer fill task is more than a predetermined minimum time period required for executing a pre-generation process comprising pre-generating one or more display elements for a requested content that is currently not visible and is outside of the displayable region;

determine a first number indicating a number of display elements of the requested content that fit into the displayable region;

determine an increment based on an application-specific rule;

determine a second number indicating a number of display elements to pre-generate based on the first number and the increment;

select one or more pre-generation tasks for pre-generating content for the second number of display elements of the requested content;

in response to the determination that the window of idle time that is available after each frame buffer fill task is more than the predetermined minimum time period required for pre-generating the one or more display elements of the requested content, start, at the start point of the window of idle time, the pregeneration process for executing the one or more pre-generation tasks, wherein a pre-generation task for pre-generating content during the window of idle time is different from the frame buffer fill task;

execute, during the pre-generation process, a subset of the one or more pre-generation tasks to pre-generate content for a corresponding subset of the display elements;

stop, at the yield point, the pre-generation process before completing the one or more pre-generation tasks in response to the pre-generation process taking longer than the window of idle time, wherein one or more stopped pre-generation tasks are added to a queue to continue execution in the window of idle time after a next frame buffer fill task; and

store the pre-generated subset of the display elements in an application-tailored recycler.

16. The system of claim 15, wherein the application-specific rule comprises that the increment is associated with a multiple number of the first number, wherein the increment is determined based on a scroll speed of a user.

17. The system of claim 15, wherein the application-specific rule comprises that the increment is determined based on attributes of the requested content.

18. The system of claim 15, wherein the processors are further operable when executing the instructions to:

prioritize one of the display elements for pre-generation based on a computational cost of generating the display element.

19. The system of claim 15, wherein the processors are further operable when executing the instructions to:

prioritize one of the display elements for pre-generation based on a number of times content referenced by the display element is also referenced by other display elements in the one or more display elements.

20. The system of claim 15, wherein the one or more display elements are selected for pre-generation based on a determination that a total execution time for the one or more pre-generation tasks is less than the window of idle time.