Rate allocation for mixed content video

• US 20060045368A1
• Filed: 10/26/2005
• Published: 03/02/2006
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev
• Granted Patent: US 7,200,276 B2
• Time in Patent Office: Days
• US Class Current: 382/239
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The present invention relates to regulating the quality and/or bitrate of content within mixed content video when the video is compressed subject to a bitrate constraint. For example, a screen capture encoder encodes palletized content within a frame of screen capture video. Subject to an overall bitrate constraint, the encoder then allocates bits for continuous tone content within the frame. By controlling the allocation of bits used to encode the continuous tone content, the encoder regulates bitrate for the continuous tone content. This in turn can allow the encoder to regulate spatial quality and/or overall temporal quality for the video. In one scenario, for screen capture video encoded to a (relatively) constant overall bitrate, the screen capture encoder reduces the bitrate (and quality) of the continuous tone content, instead spending bits to increase the overall frame rate of the video.

Claims

1. In a screen capture encoder, a method for encoding captured screen content, the method comprising:

encoding one or more components of the captured screen content using lossless compression;

allocating bits for continuous tone pixels of the captured screen content; and

encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.

2. The method of claim 1 wherein the one or more components of the captured screen content encoded using lossless compression include palettized pixels.

3. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include a segmentation map.

4. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include motion information and frame header information.

5. The method of claim 1 further comprising:

repeating the encoding the one or more components, the allocating, and the encoding the continuous tone pixels on a frame-by-frame basis for a sequence of plural frames.

10. The method of claim 1 wherein the encoding the continuous tone pixels comprises using a combination of lossy compression and lossless compression.

11. A computer-readable medium storing computer-executable instructions for causing a computer programmed thereby to perform a method of claim 1.

31. The method of claim 1 wherein the captured screen content is an image of at least part of a computer desktop environment in operation.

6-9. -9. (canceled)

12-15. -15. (canceled)

16. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a screen capture frame comprising palettized content and continuous tone content, the method comprising:

encoding the palettized content in the screen capture frame;

checking for a number of bits available after the encoding the palettized content;

based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content; and

encoding the continuous tone content in the screen capture frame with the second number of bits.

17. The computer-readable medium of claim 16 wherein the method further comprises:

repeating the encoding the palettized content, the checking, the allocating, and the encoding the continuous tone content on a frame-by-frame basis for a sequence of plural screen capture frames.

20. The computer-readable medium of claim 16 wherein the encoding the palettized content uses lossless compression, and wherein the encoding the continuous tone content uses a combination of lossy compression and lossless compression.

32. The method of claim 16 wherein the screen capture frame comprises at least part of a computer desktop environment in operation.

18-19. -19. (canceled)

21. A screen capture encoder comprising:

a lossless content coder for encoding one or more components of captured screen content using lossless compression;

a continuous tone content coder for encoding continuous tone content of the captured screen content; and

a rate controller for controlling bitrate of the captured screen content by allocating resources for encoding the continuous tone content based at least in part on resources used for the lossless compression.

22. The screen capture encoder of claim 21 further comprising a motion estimator for performing motion estimation on the captured screen content.

23. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include palettized content.

24. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include a segmentation map.

25. The screen capture encoder of claim 24 wherein the segmentation map indicates locations of pixels corresponding to palettized content and continuous tone content among intra pixels of a P frame.

26. The screen capture encoder of claim 21 wherein the one or more components encoded using lossless compression include motion information and frame header information.

29. The screen capture encoder of claim 23 further comprising a segmenter for segmenting the continuous tone content and the palettized content.

30. The screen capture encoder of claim 21 further comprising a screen capture module for recording plural images of a computer desktop environment in operation as plural screen capture frames, the plural screen capture frames including the captured screen content.

27-28. -28. (canceled)