Predictive tiling

• US 9,383,917 B2
• Filed: 03/28/2011
• Issued: 07/05/2016
• Est. Priority Date: 03/28/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mouton, Laurent, Waggoner, Nicholas R., Brun, Nicolas J., Nelte, Michael A., Shaath, Hilal
• Primary Examiner(s): PITARO, RYAN F
• Application Number:
  US13/073,300
• Publication Number:
  US 20120254780A1
• Time in Patent Office: 1,926 Days
• Field of Search: 715800-801, 715/815
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

This document describes techniques and apparatuses for predictive tiling. These techniques predict tiles of content to pre-render so that an application will be ready to render content quickly in response to a user manipulation. By so doing, these techniques enable fast manipulation of content without unduly impacting drawing resources.

Claims

1. A computer-implemented method comprising:

receiving information concerning a current user manipulation made to an application interface implemented by a computing device;

predicting one or more tiles of content to be pre-rendered, the predicting comprising an amount and location of the one or more tiles to be pre-rendered based on;

the information to determine the location;

an availability of drawing resources on the computing device to use for the pre-rendering of the one or more tiles of content to determine the amount of tiles to be pre-rendered in the location determined from the information such that the amount is larger when the computing device has more resources available and smaller when the computing device has less resources available; and

a responsiveness of an application responsible for rendering content in the application interface to further determine the amount of tiles to be pre-rendered in the location determined from the information such that the amount is larger when the application is more responsive and smaller when the computing device is less responsive; and

indicating, to the application, the one or more tiles of content based on the prediction effective to enable the application to pre-render the tiles of content in preparation for responding to the current user manipulation.

2. A computer-implemented method as described in claim 1, further comprising repeating the method one or more times until completion of said current user manipulation.

3. A computer-implemented method as described in claim 2, wherein a prior repetition of the method resulted in pre-rendering of one or more prior, pre-rendered tiles, the one or more prior, pre-rendered tiles stored in memory, and further comprising deleting from the memory the one or more prior, pre-rendered tiles responsive to the one or more prior, pre-rendered tiles no longer including currently rendered content within the application interface or a threshold that includes the currently rendered content.

4. A computer-implemented method as described in claim 1, further comprising, prior to predicting the one or more tiles of content to be pre-rendered, determining a size of the tiles of content based on a total content size and a viewport size.

5. A computer-implemented method as described in claim 1, wherein the information includes a convergence/pinch rate or divergence/spread rate of the current user manipulation.

6. A computer-implemented method as described in claim 1, wherein predicting the one or more tiles of content includes determining a location in a total content based on a velocity or an inertia of the user manipulation, the velocity or inertia included within the information, the one or more tiles of content including the location or adjacent to the tile of content that includes the location.

7. A computer-implemented method as described in claim 6, wherein the location is an intermediate location intermediate to a final location at which the current user manipulation completes.

8. A computer-implemented method as described in claim 1, wherein the responsiveness of the application includes a time to pre-render content and predicting an amount of the one or more tiles of content to be pre-rendered is predicted proportional to the time to pre-render content.

9. A computer-implemented method as described in claim 8, further comprising determining the time to pre-render content based on a timing of calls received from the application.

10. A computer-implemented method as described in claim 1, wherein the availability of drawing resources is based on a number of other applications demanding drawing resources concurrent with the application or a quantity of drawing resources being demanded by the other applications.

11. A computer-implemented method as described in claim 1, wherein the availability of drawing resources is based on a size, an allocation, or an availability of video memory.

12. A computer-implemented method comprising:

receiving, at an application responsible for rendering content for an application interface implemented by a computing device, an indication of one or more tiles of content to be pre-rendered in preparation for responding to a current user manipulation made to the application interface, the one or more tiles predicted to be useful for responding to the current user manipulation comprising an amount and location of the one or more tiles to be pre-rendered based on information about the current user manipulation to determine the location, an availability of drawing resources on the computing device able to be utilized for the pre-rendering to determine the amount of tiles to be pre-rendered in the location determined from the information such that the amount is larger when the computing device has more resources available and smaller when the computing device has less resources available, and a responsiveness of the application to further determine the amount of tiles to be pre-rendered in the location determined from the information such that the amount is larger when the application is more responsive and smaller when the computing device is less responsive;

pre-rendering the one or more tiles based upon the prediction; and

rendering, in response to the current user manipulation, at least a portion of the pre-rendered content of the one or more tiles in a viewport of the application interface.

13. A computer-implemented method as described in claim 12, further comprising, prior to receiving the indication, providing the information to a predictor from which the indication of the one or more tiles of content to be pre-rendered is received, the information having a velocity or inertia of the current user manipulation.

14. A computer-implemented method as described in claim 12, wherein the information includes a velocity or inertia of the user manipulation and the responsiveness of the application is based at least in part on a time to pre-render content by the application.

15. A computer-implemented method as described in claim 12, further comprising, prior to receiving the indication of one or more tiles of content to be pre-rendered, providing a size of the viewport and a size of total content to a predictor and receiving an indication of initial tiles for pre-rendering, the initial tiles including content adjacent to content currently presented in the viewport.

16. A computer-implemented method as described in claim 12, further comprising:

receiving an indication of previously pre-rendered tiles to delete from memory, the previously pre-rendered tiles not including content currently rendered in the viewport; and

responsive to receiving the indication, deleting the previously pre-rendered tiles from the memory.

17. A computer-implemented method as described in claim 12, wherein the availability of drawing resources is determined based on a number of other applications demanding drawing resources concurrent with the application or a quantity of drawing resources being demanded by the other applications.

18. A computer-implemented method as described in claim 12, wherein the availability of drawing resources is determined based on a size, an allocation, or an availability of video memory on a graphics card usable by the application for rendering content in the application interface.

19. A computer-implemented method as described in claim 12, wherein the responsiveness of the application is based on an historical time needed by the application to pre-render content for display in the application interface or how often the application interface requests content for display from the application.

20. A computing device comprising:

one or more computer processors; and

one or more computer-readable storage media having instructions stored thereon that, responsive to execution by the one or more computer processors, perform operations comprising;

receiving, from an application associated with an application interface implemented on the computing device, a size of a viewport in the application interface through which a portion of total content can be displayed and a size of the total content;

determining, based on the viewport size and the size of the total content, a tile size and a threshold, the threshold including the viewport;

determining one or more initial content tiles that include at least a portion of the threshold;

indicating, to the application, the one or more initial content tiles effective to enable the application to pre-render the one or more initial content tiles to prepare for an initial portion of a user manipulation;

receiving information concerning the user manipulation made to the application interface;

predicting a location of one or more tiles of content to be pre-rendered based on the information and the threshold, the one or more tiles of content not including the initial content tiles but including at least a portion of a predicted threshold location;

predicting an amount of the one or more tiles of content to be pre-rendered in the location based on an availability of drawing resources on the computing device for pre-rendering content including an amount of video memory available for the application and a responsiveness of the application, the responsiveness based on an historical amount of time that the application has taken to pre-render content or an expected current amount of time that the application takes to pre-render content; and

indicating, to the application, the one or more tiles of content effective to enable the application to pre-render content of the one or more tiles of content in preparation for responding to a non-initial portion of the user manipulation.