Rendering and compositing multiple applications in an interactive media environment

• US 20070002045A1
• Filed: 02/13/2006
• Published: 01/04/2007
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Colle, Olivier, Mazhar, Khurshed, Yovin, John, Finger, James, Freeman, Arthur
• Granted Patent: US 8,305,398 B2
• Time in Patent Office: Days
• US Class Current: 345/422
• CPC Class Codes: G06T 15/405 using Z-buffer

Abstract

In an interactive media environment, a model is provided where applications, each comprising zero or more script components and zero or more markup components, are themselves handled as visual elements having a Z order. A resource-efficient rendering order is provided where the application that has focus in an interactive media environment (by virtue of its receipt of user events) is rendered first. The remaining applications are rendered in top down order, i.e., in reverse Z order. Rendering is performed by drawing into one or more graphics buffers associated with respective applications. Frames for applications are composited from the graphics buffers bottoms up, in Z order, to provide the display with the correct appearance.

Claims

1. A method for compositing a display, the method comprising the steps of:

maintaining a Z order for each of a plurality of applications running in an interactive media environment where the Z order describes a relative order for visual elements on the display;

rendering applications into respective graphics buffers associated with the applications where the rendering is performed by application starting with an application having focus and continuing with remaining applications in the plurality of applications from highest to lowest Z order; and

compositing the display from visual elements in the graphics buffers where the compositing is performed by application from lowest to highest Z order.

2. The method of claim 1 where an initial Z order is set in a playlist.

3. The method of claim 1 where the rendering includes using a double buffering arrangement so that rendered elements are doubled buffered prior to being composited for display.

4. The method of claim 1 where the interactive media environment includes a plurality of display planes and the applications are assigned to render into a graphics plane in the plurality of display planes.

5. The method of claim 1 where the interactive media environment includes a plurality of display planes and the applications are assigned to render into a subpicture plane in the plurality of display planes.

6. The method of claim 1 where the respective graphics buffers are maintained as a single graphics buffer in which visual elements from all applications are rendered.

7. A playback system for connection with a display, the playback system comprising:

a navigation manager for maintaining a Z order for applications running in an interactive media environment where the Z order describes the relative order for visual elements on the display;

one or more graphics buffers for buffering visual elements generated by the applications;

a presentation engine, operatively coupled to the navigation manager, for rendering visual elements by application into the graphics buffers in reverse Z order; and

a graphics rendering engine for compositing frame images for the display from the one or more graphics buffers where the compositing is performed by application in Z order.

8. The playback system of claim 7 where the compositing includes clearing an opening within graphics buffers of all applications with a lower Z order than an application with a clear opening element in its markup so that a video plane may be seen through the opening.

9. The playback system of claim 8 in which the opening has a shape that is selected from one of a plurality of shapes, the shapes including rectangular, circular, oval, and arbitrary shapes.

10. The playback system of claim 7 where composited frame images include an application frame for every video frame.

11. The playback system of claim 10 where the application frame is a most recently available application frame if a new application frame is unavailable.

12. A computer-readable medium, which when executed by one or more processors in an electronic device, performs a method for setting drawing order for a plurality of N applications, each of the applications in the plurality being renderable as a visual element on a display, the method comprising the steps of:

enabling an application to invoke one or more methods for manipulating a Z order of the application on the display where the Z order describes the relative order for the N applications on the display, and values of Z order are selected from a set bounded by 0 and N-1; and

rendering the applications into respective graphics buffers associated with the applications where the rendering is performed by application in reverse Z order starting at N-1 and ending at 0.

13. The computer-readable medium of claim 12 where the rendering is performed using one or more workitems that are inserted into an event queue.

14. The computer-readable medium of claim 12 further including the step of supplying a notification to an application when its Z order changes.

15. The computer-readable medium of claim 12 where the rendering comprises drawing into a canvas object in an application'"'"'s markup.

16. The computer-readable medium of claim 15 where the canvas object is rendered into the application'"'"'s respective graphics buffer.

17. The computer-readable medium of claim 12 further including the step of compositing the display from the most recent buffer using a painter'"'"'s algorithm.

18. The computer-readable medium of claim 12 where the method for invoking includes moving an application to a highest Z order.

19. The computer-readable medium of claim 12 where the method for invoking includes moving an application to a lowest Z order.

20. The computer-readable medium of claim 12 where the method for invoking includes moving an application to a position immediately above an application having one higher Z order than the moving application.