Methods and systems for merging graphics for display on a computing device

• US 7,239,324 B2
• Filed: 02/15/2002
• Issued: 07/03/2007
• Est. Priority Date: 03/23/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Estrop, Stephen J., Wilt, Nicholas P., McCartney, Colin D.
• Primary Examiner(s): Zimmerman; Mark
• Assistant Examiner(s): Pappas; Peter-Anthony
• Application Number:
  US10/077,568
• Publication Number:
  US 20030067467A1
• Time in Patent Office: 1,964 Days
• Field of Search: 345/473
• US Class Current: 345/535
• CPC Class Codes: G09G 2320/103 Detection of image changes, e.g. determination of an index representative of the image change
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/393 Arrangements for updating the contents of the bit-mapped memory
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: G09G 5/399 using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers

Abstract

Disclosed are methods and systems that allow video applications to merge their outputs for display and to transform the outputs of other applications before display. A graphics arbiter tells applications the estimated time when the next frame will be displayed on a display screen. Applications tailor their output to the estimated display time. When output from a first application is incorporated into a scene produced by a second application, the graphics arbiter “offsets” the estimated display time it gives to the first application in order to compensate for the latency caused by the second application'"'"'s processing of the first application'"'"'s output. A set of overlay buffers parallels the traditional buffers used to prepare frames for the display screen. In composing a frame, the screen merges video information from a traditional buffer with that from an overlay buffer, conserving display resources at the final point in the display composition process.

Claims

1. An augmented primary surface system for displaying information on a display device, the system comprising:

a presentation surface set associated with the display device, the presentation surface set comprising a presentation flipping chain and an overlay flipping chain, the presentation flipping chain comprising a primary presentation surface and a presentation back buffer, the overlay flipping chain comprising an overlay primary surface and an overlay back buffer; and

a display interface driver for receiving display information from the primary presentation and overlay primary surfaces, merging the received display information, and transferring the merged information to the display device, wherein transferring the merged information comprises,

sending to the display device a pixel in a set that corresponds to the primary overlay surface if the pixel in the set that corresponds to the primary overlay surface matches a color key, and

sending to the display device a pixel in a set that corresponds to the primary presentation surface if the pixel in the set that corresponds to the primary overlay surface does not match the color key.

2. The system of claim 1 wherein the display interface driver comprises one of the following:

software executable, hardware, and firmware executable.

3. The system of claim 1 further comprising:

a graphics arbiter for transferring display information to the presentation and overlay back buffers.

4. The system of claim 1 wherein the display interface driver for merging the received display information comprises the display interface driver for merging wherein merging associates a Z order with each pixel from the received display information, the Z order defining a precedence of the display information.

5. The system of claim 1 wherein the display interface driver for merging the received display information comprises the display interface driver for merging wherein merging comprises matching pixels from the primary presentation surface to corresponding pixels from the primary overlay surface to corresponding pixel sets.

6. The system of claim 1 wherein the display interface driver for merging the received display information comprises the display interface driver for merging the display information wherein merging comprises each pixel in the primary presentation surface comprises an alpha value specifying an opacity of the corresponding pixel in the primary presentation surface.

7. A computer-readable medium which stores a set of instructions which when executed performs a method for displaying information on a display device, the method executed by the set of instructions comprising:

using a presentation surface set associated with the display device, the presentation surface set comprising a presentation flipping chain and an overlay flipping chain, the presentation flipping chain comprising a primary presentation surface and a presentation back buffer, the overlay flipping chain comprising an overlay primary surface and an overlay back buffer; and

using a display interface driver for receiving display information from the primary presentation and overlay primary surfaces, merging the received display information, and transferring the merged information to the display device, wherein transferring the merged information comprises,

sending to the display device a pixel in a set that corresponds to the primary presentation surface if the pixel in the set that corresponds to the primary presentation surface has an aloha value of 0;

sending to the display device a pixel in a set that corresponds to the primary overlay surface if the pixel in the set that corresponds to the primary presentation surface has the aloha value of 255; and

sending to the display device a pixel interpolated from the pixel in the set that corresponds to the primary presentation surface and the pixel in the set that corresponds to the primary overlay surface if the pixel that corresponds to the primary presentation surface has the alpha value between 0 and 255.

8. The computer-readable medium of claim 7 wherein merging the received display information comprises merging associates a Z order with each pixel from the received display information, the Z order defining a precedence of the display information.

9. The computer-readable medium of claim 7 wherein merging the received display information comprises merging the display information comprising matching pixels from the primary presentation surface to corresponding pixels from the primary overlay surface to corresponding pixel sets.

10. The computer-readable medium of claim 7 wherein merging the received display information comprises merging the display information wherein each pixel in the primary presentation surface comprises an alpha value specifying an opacity of the corresponding pixel in the primary presentation surface.

11. A method for displaying information on a display device, the method comprising:

receiving display information from a primary presentation surface of a presentation flipping chain of a presentation surface set associated with the display device;

receiving display information from a primary overlay surface of an overlay flipping chain of the presentation surface set, wherein the presentation flipping chain and the overlay flipping chain exist at the same time for at least a portion of their respective existences;

merging the display information received from the primary presentation and primary overlay surfaces wherein merging the display information comprises matching pixels from the primary presentation surface to corresponding pixels from the primary overlay surface to corresponding pixel sets;

transferring the merged information to the display device wherein transferring the merged information comprises,

sending to the display device, for each of the pixel sets, the pixel in the set that corresponds to the primary overlay surface if the pixel in the set that corresponds to the primary overlay surface matches a color key, and

sending to the display device, for each of the pixel sets, the pixel in the set that corresponds to the primary presentation surface if the pixel in the set that corresponds to the primary overlay surface does not match the color key; and

displaying the merged information on the display device.

12. A computer-readable medium which stores a set of instructions which when executed performs a method for displaying information on a display device, the method executed by the set of instructions comprising:

receiving display information from a primary presentation surface of a presentation flipping chain of a presentation surface set associated with the display device;

receiving display information from a primary overlay surface of an overlay flipping chain of the presentation surface set, wherein the presentation flipping chain and the overlay flipping chain exist at the same time for at least a portion of their respective existences;

merging the display information received from the primary presentation and primary overlay surfaces wherein merging the display information comprises matching pixels from the primary presentation surface to corresponding pixels from the primary overlay surface to corresponding pixel sets, each pixel in the primary presentation surface having an alpha value specifying an opacity of the corresponding pixel in the primary presentation surface; and

transferring the merged information to the display device wherein transferring the merged information comprises,

sending to the display device, for each of the pixel sets, the pixel in the set that corresponds to the primary presentation surface if the pixel in the set that corresponds to the primary presentation surface has the alpha value of 0;

sending to the display device, for each of the pixel sets, the pixel in the set that corresponds to the primary overlay surface if the pixel in the set that corresponds to the primary presentation surface has the alpha value of 255; and

sending to the display device, for each of the pixel sets, a pixel interpolated from the pixel in the set that corresponds to the primary presentation surface and the pixel in the set that corresponds to the primary overlay surface if the pixel that corresponds to the primary presentation surface has the alpha value between 0 and 255.