TIME-SYNCHRONIZED GRAPHICS COMPOSITION IN A 2.5-DIMENSIONAL USER INTERFACE ENVIRONMENT

• US 20090322786A1
• Filed: 06/30/2008
• Published: 12/31/2009
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Deaquero, Joel B., Regen, Martin, Finger, James C., Reddish, Andrew Douglas, Russell, Richard W.
• Granted Patent: US 8,884,983 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally

Abstract

Software interfaces are configured to enable functionality that is commonly implemented in special-purpose hardware for mixing AV content into a set of 2.5-D graphics planes to be exposed to high level processes executing in a computing environment in a fully portable manner. Illustratively, the interfaces include a planar mixer (named “IPlanarMixer”) that abstracts the mixing hardware, and a graphics plane interface (named “IPlane”) that abstracts individual instances of planes that are retrieved from, and handed off to the planar mixer as the 2.5-D graphics planes are built up and rendered in a computing environment to support interactive graphic experiences for a user.

Claims

1. A computer-readable medium containing instructions which, when executed by one or more processors disposed in an electronic device, implement an interface for exposing a method for supporting interactive graphics composition in a rendering environment, the method comprising the steps of:

allocating memory in which a graphic plane may be rendered, the graphic plane being rendered as a component of a 2.5-D stack;

receiving the graphics plane as a finished graphics frame into which interactive content is rendered; and

applying business rules to compose the finished graphics frame into the 2.5-D stack in a time-synchronous manner.

2. The computer-readable medium of claim 1 in which the 2.5-D stack is an HDi stack.

3. The computer-readable medium of claim 1 in which the interface exposes functionality as substantially provided by SoC hardware.

4. The computer-readable medium of claim 1 in which the interactive content includes at least one of interactive navigation, interactive menu system, or interactive user experience.

5. The computer-readable medium of claim 1 in which the interactive content comprise Advanced Content as described by the DVD Forum.

6. The computer-readable medium of claim 1 in which the business rules pertains to at least one of Z order of graphics planes within the 2.5-D stack, position of clearrect elements used to implement openings through one or more graphic planes in the 2.5-D stack, coalescence of color spaces used in the graphic planes, size of a window in which a graphic plane is rendered, and position of the window in which the graphics plane is rendered.

7. The computer-readable medium of claim 1 in which the method includes a further step of queuing a graphics plane for presentation in the rendering environment in accordance with a current presentation time.

8. The computer-readable medium of claim 7 in which the method includes a further step of aborting composition of a graphics plane when time budgeted for the composition will result in presentation of the composed graphics plane beyond the current presentation time.

9. A computer-implemented method for exposing functionality for enabling time synchronous compositing of a graphics plane in a 2.5-D stack to a graphics composition process, the method comprising the steps of:

providing an abstraction of the graphics plane as an interface to the graphics composition process;

holding a rectangular array of pixels in the abstraction, the array of pixels being subject to manipulation through one or more functions included in a library of graphic primitives; and

holding data for constructing openings through the graphics plane, the openings constructed when the graphics plane is composited among a plurality of graphic planes used to implement the 2.5-D stack.

10. The computer-implemented method of claim 9 in which the graphics composition process is implemented using a presentation engine arranged for presenting the plurality of graphic planes utilized environment to mixing hardware that performs a mixing process for time-synchronously compositing and rendering a display from the plurality of graphic planes.

11. The computer-implemented method of claim 9 in which the plurality of graphics planes comprise one or more of OSD plane, cursor plane, interactive graphics plane, subpicture plane, subvideo plane, or main video plane.

12. The computer-implemented method of claim 11 including the further steps of tracking a number of openings used in a graphics plane, and tracking a destination plane, openings being generated in ones of the plurality of graphics planes that are positioned between the destination plane and the main video plane in the 2.5-D stack.

13. The computer-implemented method of claim 9 including a further step of representing a graphics plane as series of back buffers, each back buffer having an associated time stamp that is usable by the graphics composition process when compositing the graphics plane in the 2.5-D stack.

14. A software architecture arranged for implementing a 2.5-D user interface in a graphic display that is rendered by a device, comprising:

a presentation engine configured for presenting a plurality of graphic planes utilized with the 2.5-D user interface to mixing hardware that performs a mixing process for time-synchronously compositing and rendering the display from the plurality of graphic planes; and

interfaces accessible by the presentation engine including a mixing interface for abstracting the mixing hardware to the presentation engine and an a graphics plane interface for abstracting individual instances of the plurality of graphics plane.

15. The software architecture of claim 14 in which the mixing interface is further configured for applying business rules to compose finished graphics frame into a Z ordered stack in a time-synchronous manner for rendering the 2.5-D user interface.

16. The software architecture of claim 14 in which the interfaces are configured as COM interfaces.

17. The software architecture of claim 14 further including Win32 -specific APIs for exposing functionality associated with HDi graphics composition to one or more Win32 applications that operate on the device.

18. The software architecture of claim 14 in which the interfaces are configured to be portable across device types.

19. The software architecture of claim 14 in which the device is a thin client device selected from one of mobile phone, smart phone, pocket PC, PDA, GPS navigation unit, personal media player, set top box, handheld game device, or device combining one of more functionalities provided therein.

20. The software architecture of claim 14 in which the device is a thin client device that runs an embedded operating system.