Speculative render ahead and caching in multiple passes

• US 9,384,711 B2
• Filed: 02/15/2012
• Issued: 07/05/2016
• Est. Priority Date: 02/15/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ergan, Cenk, Huang, Li-Hsin
• Primary Examiner(s): NGUYEN, ANH TUAN V
• Application Number:
  US13/397,299
• Publication Number:
  US 20130208012A1
• Time in Patent Office: 1,602 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/08 Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
• CPC Class Codes: G09G 2360/121 using a cache memory
• CPC Class Codes: G09G 2370/027 Arrangements and methods specific for the display of internet documents
• CPC Class Codes: G09G 2370/10 Use of a protocol of communication by packets in interfaces along the display data pipeline
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/32 with means for controlling the display position see provisionally G09G5/42
• CPC Class Codes: G09G 5/346 for systems having a bit-mapped display memory

Abstract

Various embodiments are directed to intelligently rendering or pre-rendering content that may likely be brought into an associated viewport during a user'"'"'s interaction with the content. In at least some embodiments, the process of caching pre-rendered content is separated from a determination of which regions outside of the viewport are to be kept up to date over use and time. Further, at least some embodiments perform rendering activities in multiple passes in an effort to reduce potential delays in the user visible content reaching completion, without interfering with other work that an associated application or system performs during rendering.

Claims

1. A method comprising:

rendering content within a current viewport;

receiving a directional manipulation of the viewport;

responsive to receiving the directional manipulation of the viewport, starting a time-constrained rendering loop;

pre-rendering and caching content for the manipulation that lies in a virtual surface outside of the viewport, in the direction of the manipulation, wherein the content that lies in the virtual surface outside of the viewport is pre-rendered as long as time remains in the rendering loop and responsive to ascertaining that no invalidated content remains in the viewport;

maintaining cached;

at least some previously-rendered pre-rendered content, the at least some previously-rendered pre-rendered content having been identified based on one or more rendering policies that control which of the previously-rendered pre-rendered content is to remain cached; and

the content that is pre-rendered for the manipulation; and

rendering content, that includes at least some of the cached content, within a new current viewport.

2. The method of claim 1 further comprising discarding one or more portions of content that are determined invalidated in the virtual surface to enable rendering of placeholder content in place of the one or more portions.

3. The method of claim 1, wherein pre-rendering content comprises utilizing a threshold number of screenfuls of content within the virtual surface to determine an amount of content that is pre-rendered.

4. The method of claim 1 further comprising responsive to detecting a zoom out manipulation, beyond a threshold, to the previously-rendered pre-rendered content, locking the previously-rendered pre-rendered content and applying a transform to the previously-rendered pre-rendered content rather than re-rasterizing content that is zoomed out.

5. The method of claim 1 further comprising tracking invalidations over time to discard one or more portions of content.

6. The method of claim 1 further comprising tracking invalidations over time to discard one or more portions of content that are invalidated multiple times.

7. The method of claim 1, wherein said receiving a directional manipulation comprises receiving a touch manipulation.

8. The method of claim 1 further comprising prior to receiving the directional manipulation, pre-rendering and caching at least some other content that lies in the virtual surface.

9. The method of claim 1, wherein said pre-rendering is performed by prioritizing sub-portions within the virtual surface.

10. The method of claim 1, wherein said pre-rendering is performed by prioritizing sub-portions within the virtual surface based on an associated distance to the viewport.

11. The method of claim 1, further comprising ascertaining whether invalidated content remains in the viewport and if so, rendering content within the viewport that has become invalidated.

12. A device comprising:

one or more modules implemented at least partially in hardware, the one or more modules configured to perform operations comprising;

receiving a user manipulation of content that has been rendered in a viewport, at least some of the rendered content corresponding to one or more non-rendering actions that are performed by an associated application;

responsive to receiving the user manipulation of content, starting a time-constrained rendering loop;

rendering content within the viewport that has become invalidated during the rendering loop, including discarding at least some invalidated content to render placeholder content in place of the invalidated content;

suspending, during the rendering loop, performance of the one or more non-rendering actions that are performed by the associated application;

ascertaining whether time remains in the rendering loop;

yielding to the associated application if no time remains in the rendering loop, the yielding enabling the associated application to perform the one or more non-rendering actions; and

performing rendering if time remains in the rendering loop, including ascertaining whether invalidated content remains in the viewport and, if not, pre-rendering content within a virtual surface that lies outside of the viewport.

13. The device of claim 12, wherein the time-constrained rendering loop is constrained between 200-300 ms.

14. The device of claim 12, wherein performing rendering comprises ascertaining whether invalidated content remains in the viewport and if so, continuing to render content within the viewport that has become invalidated.

15. The device of claim 12, wherein the content within the virtual surface that lies outside of the viewport is pre-rendered in a direction of the user manipulation.

16. The device of claim 12, wherein pre-rendering the content within the virtual surface that lies outside of the viewport includes pre-rendering sub-portions of the virtual surface, at least some of which being in a direction of the user manipulation, effective to allow return of content rendering in the viewport during the rendering loop.

17. The device of claim 12, wherein pre-rendering the content within the virtual surface that lies outside of the viewport includes prioritizing sub-portions to render within the virtual surface.

18. The device of claim 12, wherein pre-rendering the content within the virtual surface that lies outside of the viewport includes prioritizing sub-portions to render within the virtual surface based on their associated distance to the viewport.

19. The device of claim 12, wherein performing rendering comprises:

ascertaining whether invalidated content remains in the viewport and if so, continuing to render content within the viewport that has become invalidated; and

wherein said rendering is performed by rendering in a direction of the user manipulation.

20. The device of claim 12, wherein performing rendering comprises performing rendering in a direction of the user manipulation or expanding in all directions in the event of a zoom out manipulation.