Composition system thread

• US 9,563,971 B2
• Filed: 09/09/2011
• Issued: 02/07/2017
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Blanco, Leonardo E., Moncayo, Silvana Patricia, Liperi, Jonathan P., Fink, Reiner, Majid, Imran, Priestley, Joshua Warren
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US13/229,627
• Publication Number:
  US 20130063445A1
• Time in Patent Office: 1,978 Days
• Field of Search: 345/473, 345/619, 719/328
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites

Abstract

Composition system thread techniques are described. In one or more implementations, a composition system may be configured to compose visual elements received from applications on a thread that is executed separately than a user interface thread of the applications. As such, the composition system may execute asynchronously from a user interface thread of the application. Additionally, the composition system may be configured to expose one or more application programming interfaces (APIs) that are accessible to the applications. The APIs may be used for constructing a tree of objects representing the operations that are to be performed to compose one or more bitmaps. Further, these operations may be controlled by several API visual properties to allow applications to animate content within their windows and use disparate technologies to rasterize such content.

Claims

1. A method, implemented by a computer system that includes one or more processors and a display device, the method comprising:

receiving data at one or more application programming interfaces (APIs) from a plurality of applications including at least a first application executing in a first thread and a second application executing in a second thread, the data including one or more first visual elements received from the first application and to be rendered for the first application and one or more second visual elements received from the second application and to be rendered for the second application, wherein the one or more first visual elements correspond to a digitally-protected video to be rendered for the first application;

configuring the data for display on the display device through execution of a dedicated third thread that executes independent of and asynchronously with the first thread and the second thread, and that composes the data for the first application and the second application into displayable pixels using a single composition engine, the single composition engine composing the one or more first visual elements into protected memory that is inaccessible to at least the first thread and the second thread, to prevent the first and second applications from accessing any displayable pixels corresponding to the one or more first visual elements that are composed by the single composition engine; and

returning timing information usable by the plurality of applications to synchronize rasterization of bitmaps with one or more animations configured by the dedicated third thread.

2. A method as described in claim 1, wherein the execution of the dedicated third thread is performed as part of an operating system of the computing device.

3. A method as described in claim 1, wherein the first thread is a user interface thread responsible for generating a user interface of the first application for display by the display device.

4. A method as described in claim 1, wherein rasterizing is performed by the first and second threads, rather than the dedicated third thread.

5. A method as described in claim 1, wherein the dedicated third thread is implemented such that unbounded locks are not permitted.

6. A method as described in claim 1, wherein the data of the receiving includes a bitmap.

7. A method as described in claim 1, wherein the one or more application programming interfaces are configured to expose functionality to the plurality of applications to create one or more visuals, bind the visuals to a bitmap, and manipulate one or more visual properties as part of composition or animation of the data.

8. A method as described in claim 7, wherein at least one said visual property includes a two dimensional transformation usable to modify a coordinate system.

9. A method as described in claim 8, wherein the at least one said visual property includes a translation, scale, rotation, or skew.

10. A method as described in claim 7, wherein at least one said visual property includes a clip usable to restrict a set of pixels that are to be affected.

11. A method as described in claim 7, wherein at least one said visual property includes a bitmap effect.

12. A method as described in claim 11, wherein the at least one said visual property includes a three dimensional perspective transformation, constant-alpha opacity, or Porter-Duff compositional blend mode.

13. A method as described in claim 7, wherein at least one said visual property is configured to permit the plurality of applications to specify how an alpha channel is to be consumed by the dedicated third thread.

14. A method as described in claim 1, wherein configuring the data for display on the display device through execution of the dedicated third thread comprises the single composition engine managing priority of how the one or more first elements to be rendered for the first application and the one or more second elements to be rendered for the second application are rendered by the single composition engine, by at least performing an analysis of what is to be rendered by the plurality of applications and skipping rendering of a particular one of the one or more first visual elements received from the first application based at least on the analysis determining that the particular visual element is occluded by at least one of the one or more second visual elements received from the second application.

15. A method as described in claim 1, wherein configuring the data for display on the display device through execution of the dedicated third thread comprises the single composition engine updating the one or more first elements at a first frequency, and updating the one or more second elements at a second, different, frequency.

16. A method as described in claim 1, wherein configuring the data for display on the display device through execution of the dedicated third thread comprises the single composition engine using a common memory resource for at least one of the one or more first elements and at least one of the one or more second elements.

17. A method as described in claim 1, wherein configuring the data for display on the display device through execution of the dedicated third thread comprises the single composition engine refraining from composing at least one of the one or more first elements that would not be visible to a user, if rendered, due to a refresh of a display.

18. A method, implemented by a computer system that includes one or more processors and a display device, the method comprising:

exposing one or more application programming interfaces (APIs) to a plurality of applications executing in different processes to construct a single tree of objects, the plurality of applications including at least a first application executing in a first process and a second application executing in a second process, the single tree of objects describing a batch of different elements that are to be rendered together for the plurality of applications and representing operations to be performed by a single composition engine to compose one or more bitmaps, the batch of different elements including one or more first visual elements received from the first application and to be rendered for the first application and one or more second visual elements received from the second application and to be rendered for the second application, wherein the one or more first visual elements correspond to a digitally-protected video to be rendered for the first application;

composing the one or more bitmaps by the single composition engine using the operations represented in the single tree of objects, the composing being executed in a dedicated third thread of a process that is separate from and that executes asynchronously with the process of each of the plurality of applications executing a respective thread of a user interface, the composing including rendering the one or more first visual elements into protected memory that is inaccessible to at least the first process and the second process, to prevent the first and second applications from accessing any displayable pixels corresponding to the one or more first visual elements that are rendered by the single composition engine; and

outputting timing information usable by the plurality of applications to synchronize rasterization of bitmaps with one or more animations configured by the dedicated third thread.

19. A computing device comprising:

one or more processors; and

one or more computer-readable storage media storing computer-executable instructions that are executable by the one or more processors to cause the computing device to implement an operating system configured to;

expose one or more application programming interfaces (APIs) to a plurality of applications executing in different processes to construct a single tree of objects, the plurality of applications including at least a first application executing in a first process and a second application executing in a second process, the single tree of objects describing a batch of different elements that are to be rendered together for the plurality of applications and representing operations to be performed by a single composition engine to compose one or more bitmaps, the batch of different elements including one or more first visual elements received from the first application and to be rendered for the first application and one or more second visual elements received from the second application and to be rendered for the second application, wherein the one or more first visual elements correspond to a digitally-protected video to be rendered for the first application;

implement a dedicated third thread of a process that is separate from and that executes asynchronously with the process of each of the plurality of applications, to compose the one or more bitmaps by a single composition engine using the operations represented in the single tree of objects, the composing including rendering the one or more first visual elements into protected memory that is inaccessible to at least the first process and the second process, to prevent the first and second applications from accessing any displayable pixels corresponding to the one or more first visual elements that are rendered by the single composition engine; and

return timing information usable by the plurality of applications to synchronize rasterization of bitmaps with one or more animations configured by the dedicated third thread.