Cross-platform rendering engine

• US 10,157,593 B2
• Filed: 02/24/2014
• Issued: 12/18/2018
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Witman, Ben, McDonald, Dave, Joe, Michael Y., Ul Haq, Faaez
• Primary Examiner(s): PATEL, SHIVANG I
• Application Number:
  US14/187,897
• Publication Number:
  US 20150243257A1
• Time in Patent Office: 1,758 Days
• Field of Search: 345522
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

A cross-platform rendering engine. The cross-platform rendering engine serves as an intermediary between the application and the operating system for displaying application content on the screen allowing software developers to write platform-agnostic application code. The application sends content to the cross-platform rendering engine in the form of resource-efficient content descriptions describing the content to be displayed. In turn, cross-platform rendering engine stores the content descriptions and provides rasterized images generated from the content descriptions to the compositor as needed. In the event that a content description corresponding to the texture needed by the compositor is not available, the cross-platform rendering engine pulls the content description from the application. By producing rasterized images from a user interface thread on behalf of the application rather than waiting for the application thread, the cross-platform rendering engine improves the overall responsiveness of the application and contributes to a better user experience.

Claims

1. A method for rendering computer-generated display components associated with an application for display on a computer-enabled display surface, the method comprising the acts of:

receiving a plurality of content descriptions from an application, the plurality of content descriptions including commands on how to draw a display component;

storing the plurality of content descriptions in memory;

receiving a drawing request from an application to graphically display application content;

determining if the application content to be graphically displayed is stored as one or more of the plurality of content descriptions;

when the drawing request is stored as one or more of the plurality of content descriptions, rendering, from the stored content description, display components that are representative of the application content to be graphically displayed and displaying the display components; and

when the content description is not stored in memory, calling the application for a rasterized image representative of the application content to be graphically displayed and displaying the rasterized image.

2. The method of claim 1 further comprising the acts of:

requesting the application to provide an additional content description for which is there is no corresponding stored content description;

immediately rendering one or more display components from the additional content description; and

storing the additional content description for subsequent use.

3. The method of claim 1 wherein the plurality of stored content descriptions are platform-agnostic and wherein, rendering, from the one or more stored content descriptions, display components that are representative of the application content to be graphically displayed includes rendering, from the one or more stored content descriptions, display components that are platform-specific.

4. The method of claim 1 further comprising adding one or more of the plurality of content descriptions associated with a same virtual texture representing application content to a collection of content descriptions.

5. The method of claim 4 further comprising the acts of:

determining that at least three content descriptions in the collection of content descriptions have rendered the same at least one pixel;

requesting a new content description for the portion of the virtual texture enclosed by a bounding box encompassing the portions of the virtual texture drawn by the at least three content descriptions from the application; and

replacing the at least three content descriptions that rendered the same at least one pixel with the new content description in the collection of content descriptions.

6. The method of claim 1, further comprising:

determining that the display components need to be updated;

determining if a content description representative of the display components to be updated is stored as one or more of the plurality of content descriptions; and

when the content description representative of the display components to be updated is not stored as one or more of the plurality of content description,

pulling the content description representative of the display component to be updated from the application;

rendering the pulled content description to updated display components; and

displaying the updated display components.

7. The method of claim 6, wherein when the content description representative of the display components to be updated is stored as one or more of the plurality of content description, the method further comprising:

determining that the stored content description representative of the display component components to be updated needs to be manipulated;

manipulating the stored content description representative of the display component components to be updated;

notifying the application of the manipulation; and

rendering the manipulated content description to updated display components; and

displaying the updated display components.

8. The method of claim 1, wherein application is operating on a first processing thread and wherein the rendering of the display components from the stored content description is performed by a rendering engine operating on a second processing thread, the second processing thread, wherein a nominal response time of the second processing thread is at least twice as fast as a nominal response time of the first processing thread.

9. The method of claim 8, wherein the nominal response time of the second processing thread is at least three-times as fast as the nominal response time of the first processing thread.

10. The method of claim 9, wherein the nominal response time of the second processing thread is at least four-times as fast as the nominal response time of the first processing thread.

11. A system rendering application content for display on a computer-enabled display surface comprising:

a first content description memory for storing a content description containing platform-agnostic commands received from an application generating application content, the commands describing how to draw the application content; and

a processor executing the application and a rendering engine,

wherein the rendering engine receives a drawing request from the application to display specific application content on a computer-enabled display surface and determines if the specific application content to be displayed is found within the content description;

when the specific application content to be displayed is found within the content description, rendering the content description into platform-specific commands that render display components representative of the specific application content for display on the computer-enabled display surface; and

when the specific application content to be displayed is not found within the content description, calling the application for a rasterized image representative of the application content to be displayed for display on the computer-enabled display surface.

12. The system of claim 11 wherein the rendering engine further comprises a frontend in communication with a backend, the frontend in communication with and receiving commands from the application, the backend rendering the display components from the content description.

13. The system of claim 12 wherein the first content description memory is associated with the frontend, the system further comprising a second content description memory associated with the backend.

14. The system of claim 13 wherein the rendering engine includes a direct communication channel between the frontend and the backend, the frontend passing content descriptions from the first content description memory to the second content description memory via the communication channel.

15. The system of claim 12 further comprising:

an application thread associated with the frontend; and

a user interface thread associated with the backend.

16. The system of claim 15 further comprising a rendering thread for rasterizing the content descriptions into display component raster images, the rendering thread responsive to requests to render content sent by the user interface thread.

17. The system of claim 11 wherein the rendering engine is in communication with a compositor responsible for compositing a final image for display on the computer-enabled display surface.

18. A computer readable storage device containing computer executable instructions which, when executed by a computer, perform a method for displaying application content on a computer-enabled display surface, the method comprising the acts of:

receiving a plurality of content descriptions from an application, the plurality of content descriptions including commands on how to draw a display component;

storing the plurality of content descriptions in memory;

receiving a drawing request from an application to graphically display application content;

determining if the application content to be graphically displayed is stored as one or more of the plurality of content descriptions;

when the drawing request is stored as one or more of the plurality of content descriptions, rendering, from the stored content description, display components that are representative of the application content to be graphically displayed and displaying the display components; and

when the content description is not stored in memory, calling the application for a rasterized image representative of the application content to be graphically displayed and displaying the rasterized image.

19. The computer readable storage device of claim 18 wherein the method further comprises:

manipulating the stored content description in response to responding to a request to manipulate a selected area of application content.

20. The computer readable storage device of claim 19 wherein the request to manipulate a selected area of application content is a request to zoom or a request to pan.

21. The computer readable storage device of claim 18 wherein the method further comprises the act of grouping the stored content description with other content descriptions associated with a selected area of application content generated by the application into a content description composite.