Rate allocation for mixed content video

• US 20040001638A1
• Filed: 06/28/2002
• Published: 01/01/2004
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev
• Granted Patent: US 6,980,695 B2
• Time in Patent Office: Days
• US Class Current: 382/244
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The present invention relates to regulating the quality and/or bitrate of content within mixed content video when the video is compressed subject to a bitrate constraint. For example, a screen capture encoder encodes palletized content within a frame of screen capture video. Subject to an overall bitrate constraint, the encoder then allocates bits for continuous tone content within the frame. By controlling the allocation of bits used to encode the continuous tone content, the encoder regulates bitrate for the continuous tone content. This in turn can allow the encoder to regulate spatial quality and/or overall temporal quality for the video. In one scenario, for screen capture video encoded to a (relatively) constant overall bitrate, the screen capture encoder reduces the bitrate (and quality) of the continuous tone content, instead spending bits to increase the overall frame rate of the video.

Claims

1. In a screen capture encoder, a method for encoding captured screen content, the method comprising:

encoding one or more components of the captured screen content using lossless compression;

allocating bits for continuous tone pixels of the captured screen content; and

encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.

2. The method of claim 1 wherein the one or more components of the captured screen content encoded using lossless compression include palettized pixels.

3. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include a segmentation map.

4. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include motion information and frame header information.

5. The method of claim 1 further comprising:

repeating the encoding the one or more components, the allocating, and the encoding the continuous tone pixels on a frame-by-frame basis for a sequence of plural frames.

6. The method of claim 1 wherein the allocating comprises:

computing a preliminary bit allocation;

comparing the preliminary bit allocation with a number of available bits in view of the encoding using lossless compression; and

allocating the bits based at least in part on results of the comparing.

7. The method of claim 1 wherein the allocating comprises:

computing a preliminary bit allocation;

computing a minimum quality bit allocation;

comparing the preliminary bit allocation and the minimum quality bit allocation; and

allocating the bits based at least in part on results of the comparing.

8. The method of claim 1 wherein the allocating comprises:

estimating a number of bits needed to encode the captured screen content;

estimating a number of bits needed to encode the continuous tone pixels; and

computing the minimum of;

the difference between the estimated number of bits needed to encode the captured screen content and a number of bits spent in the encoding using lossless compression;

a number of available bits in view of the encoding using lossless compression; and

the estimated number of bits needed to encode the continuous tone pixels.

9. The method of claim 8 wherein the number of available bits is further reduced according to a buffer filling factor.

10. The method of claim 1 wherein the encoding the continuous tone pixels comprises using a combination of lossy compression and lossless compression.

11. A computer-readable medium storing computer-executable instructions for causing a computer programmed thereby to perform a method of claim 1.

12. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform method of encoding a mixture of palettized screen capture content and continuous tone screen capture content, the method comprising:

encoding the palettized screen capture content;

allocating bits for encoding the continuous tone screen capture content; and

encoding the continuous tone screen capture content, wherein the allocating controls bitrate for the continuous tone screen capture content.

13. The computer-readable medium of claim 12 wherein the encoding the palettized screen capture content uses lossless compression and wherein the encoding the continuous tone screen capture content uses lossy and lossless compression.

14. The computer-readable medium of claim 12 wherein the method further comprises before the allocating, encoding a frame header and segmentation map for a frame that includes the palettized screen capture content and the continuous tone screen capture content.

15. The computer-readable medium of claim 12 wherein the allocating depends at least in part upon preliminary and minimum quality bit allocations for the continuous tone screen capture content and a number of available bits.

16. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a frame comprising palettized content and continuous tone content, the method comprising:

encoding the palettized content in the frame;

checking for a number of bits available after the encoding the palettized content;

based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content; and

encoding the continuous tone content in the frame with the second number of bits.

17. The computer-readable medium of claim 16 wherein the method further comprises:

repeating the encoding the palettized content, the checking, the allocating, and the encoding the continuous tone content on a frame-by-frame basis for a sequence of plural frames.

18. The computer-readable medium of claim 16 wherein the allocating comprises:

computing a preliminary bit allocation;

comparing the preliminary bit allocation with the number of bits available after the encoding the palettized content; and

allocating the bits based at least in part on results of the comparing.

19. The computer-readable medium of claim 16 wherein the allocating comprises:

computing a preliminary bit allocation;

computing a minimum quality bit allocation;

comparing the preliminary bit allocation and the minimum quality bit allocation; and

allocating the bits based at least in part on results of the comparing.

20. The computer-readable medium of claim 16 wherein the encoding the palettized content uses lossless compression, and wherein the encoding the continuous tone content uses a combination of lossy compression and lossless compression.

21. A screen capture encoder comprising:

means for encoding one or more components of captured screen content using lossless compression;

means for encoding continuous tone pixels of the captured screen content; and

means for controlling bitrate of the captured screen content by allocating resources for encoding the continuous tone pixels based at least in part on resources used for the lossless compression.

22. The screen capture encoder of claim 21 further comprising means for performing motion estimation on the captured screen content.

23. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include palettized pixels.

24. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include a segmentation map.

25. The screen capture encoder of claim 24 wherein the segmentation map indicates locations of palettized pixels and continuous tone pixels among intra pixels of a P frame.

26. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include motion information and frame header information.

27. An encoder comprising:

a palettized content coder for losslessly encoding palettized content;

a continuous tone content coder for encoding continuous tone content; and

a rate controller to regulate quality and bitrate for the continuous tone content following encoding of the palettized content.

28. The encoder of claim 27 further comprising a motion estimator.

29. The encoder of claim 27 further comprising a segmenter for segmenting the continuous tone content and palettized content.