Rate allocation for mixed content video

• US 7,200,276 B2
• Filed: 10/26/2005
• Issued: 04/03/2007
• Est. Priority Date: 06/28/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): Bayat; Ali
• Application Number:
  US11/260,650
• Publication Number:
  US 20060045368A1
• Time in Patent Office: 524 Days
• Field of Search: 382/244, 382/243, 382/236, 382232-234, 382/238, 382245-246, 382/239, 382/176, 382/190, 382/240, 382/251, 375/240.1, 375/240.03, 375/240.16, 345/426, 341/55, 709/247, 708/203
• US Class Current: 382/239
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The present invention relates to regulating the quality and/or bitrate of content within mixed content video when the video is compressed subject to a bitrate constraint. For example, a screen capture encoder encodes palletized content within a frame of screen capture video. Subject to an overall bitrate constraint, the encoder then allocates bits for continuous tone content within the frame. By controlling the allocation of bits used to encode the continuous tone content, the encoder regulates bitrate for the continuous tone content. This in turn can allow the encoder to regulate spatial quality and/or overall temporal quality for the video. In one scenario, for screen capture video encoded to a (relatively) constant overall bitrate, the screen capture encoder reduces the bitrate (and quality) of the continuous tone content, instead spending bits to increase the overall frame rate of the video.

Claims

1. In a screen capture encoder, a method for encoding a screen capture video frame sequence, the method comprising:

encoding one or more components of plural screen capture video frames in the screen capture video frame sequence using lossless compression, each of the plural screen capture video frames showing at least part of a graphical user interface of a computer system recorded in operation as an image series over time in the screen capture video frame sequence;

allocating bits for continuous tone content of the plural screen capture video frames;

encoding the continuous tone content of the plural screen capture video frames, wherein the allocating regulates quality and bitrate for the continuous tone content; and

outputting a result of the encoding.

2. The method of claim 1 wherein the one or more components of the plural screen capture video frames encoded using lossless compression include palettized content in one of the plural screen capture video frames.

3. The method of claim 2 wherein the one or more components of the plural screen capture video frames encoded using lossless compression further include segmentation information.

4. The method of claim 2 wherein the one or more components of the plural screen capture video frames encoded using lossless compression further include motion information and frame header information.

5. The method of claim 1 further comprising:

repeating the encoding the one or more components, the allocating, and the encoding the continuous tone content on a frame-by-frame basis for each screen capture video frame in the sequence.

6. The method of claim 1 wherein the encoding the continuous tone content comprises using a combination of lossy compression and lossless compression.

7. A computer-readable medium having stored therein computer-executable instructions for causing a computer programmed thereby to perform the method of claim 1.

8. The method of claim 1 wherein the graphical user interface is a computer desktop environment of the computer system.

9. The method of claim 1 wherein each of the plural screen capture video frames depicts an entire screen area, only a single window, or only some other part of the entire screen area of the graphical user interface.

10. A computer-readable medium having stored therein computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a screen capture video frame sequence, the method comprising:

processing plural screen capture video frames in the screen capture video frame sequence, each of the plural screen capture video frames showing at least part of a graphical user interface of a computer system recorded in operation as an image series over time in the screen capture video frame sequence, wherein the processing comprises for each of the plural screen capture video frames;

encoding palettized content in the screen capture video frame;

checking for a number of bits available after the encoding the palettized content;

based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content; and

encoding continuous tone content in the screen capture video frame with the second number of bits; and

outputting a result of the processing.

11. The computer-readable medium of claim 10 wherein the encoding the palettized content uses lossless compression, and wherein the encoding the continuous tone content uses a combination of lossy compression and lossless compression.

12. A screen capture video encoder comprising:

a lossless content coder for encoding one or mare components of plural screen capture video frames in a series of screen capture video frames using lossless compression, each of the plural screen capture video frames showing at least part of a graphical user interface of a computer system recorded in operation as an image series over time in the series of screen capture video frames;

a continuous tone content coder for encoding continuous tone content of the plural screen capture video frames; and

a rate controller for controlling bitrate of the plural screen capture video frames by allocating resources for encoding the continuous tone content based at least in part on resources used for the lossless compression.

13. The screen capture video encoder of claim 12 further comprising a motion estimator for performing motion estimation on the plural screen capture video frames.

14. The screen capture video encoder of claim 12 wherein the one or more components encoded using lossless compression include palettized content.

15. The screen capture video encoder of claim 14 further comprising a segmenter for segmenting the continuous tone content and the palettized content.

16. The screen capture video encoder of claim 12 wherein the one or more components encoded using lossless compression include a segmentation map.

17. The screen capture video encoder of claim 16 wherein the segmentation map indicates locations of pixels corresponding to palettized content and pixels corresponding to continuous tone content among intra pixels of a P frame.

18. The screen capture video encoder of claim 12 wherein the one or more components encoded using lossless compression include motion information and frame header information.

19. The screen capture video encoder of claim 12 further comprising a screen capture module for recording the graphical user interface of the computer system in operation.