Motion estimation/compensation for screen capture video

• US 20040001544A1
• Filed: 06/28/2002
• Published: 01/01/2004
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev
• Granted Patent: US 7,224,731 B2
• Time in Patent Office: Days
• US Class Current: 375/240.1
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/557 Motion estimation characterised by stopping computation or iteration based on certain criteria, e.g. error magnitude being too large or early exit
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

The present invention relates to motion estimation and compensation. For example, a screen capture encoder performs motion estimation that is adapted to screen capture video in various respects. For example, the motion estimation uses a distortion measure based upon the count of equal/unequal pixels in two regions, sub-samples the distortion measure to speed up motion estimation, and/or uses a search pattern that prioritizes types of motion common in screen capture video. Or, a screen capture decoder performs motion compensation that is adapted to screen capture video in various respects. For example, the decoder performs the motion compensation for pixels with different values at corresponding locations in a current frame and a reference frame, but not for all pixels of the current frame. Alternatively, an encoder/decoder performs the motion estimation/compensation to compress/decompress other kinds of content.

Claims

1. A screen capture encoder comprising:

a frame buffer for storing a reference screen capture frame; and

a motion estimation module adapted to screen capture video, the motion estimation module for estimating motion from the reference screen capture frame for one or more regions of pixels in a current screen capture frame.

2. The screen capture encoder of claim 1 wherein the one or more regions include pixels with different values at corresponding locations in the reference screen capture frame and the current screen capture frame.

3. The screen capture encoder of claim 1 further comprising:

a segmenter for separating palettized content and continuous tone content in intra pixels in the current screen capture frame following motion estimation, the intra pixels having different values at corresponding locations in the reference screen capture frame and the current screen capture frame.

4. The screen capture encoder of claim 3 further comprising:

a first encoding module for encoding the continuous tone content; and

a second encoding module for encoding the palettized content.

5. The screen capture encoder of claim 1 wherein the motion estimation module checks for pure horizontal or vertical motion before checking for other kinds of motion.

6. The screen capture encoder of claim 1 wherein the motion estimation module uses Hamming distance as a distortion measure.

7. The screen capture encoder of claim 1 wherein the motion estimation module uses a distortion measure based on matching pixel values.

8. The screen capture encoder of claim 1 wherein the motion estimation module sub-samples a distortion measure.

9. The screen capture encoder of claim 1 wherein, for each of the one or more regions in the current screen capture frame, the motion estimation module searches for a match in a search area comprising the entire reference screen capture frame.

10. The screen capture encoder of claim 1 wherein the frame buffer stores an original version of the reference screen capture frame.

11. The screen capture encoder of claim 1 wherein the motion estimation module compares the current screen capture frame and the reference screen capture frame to detect value changes in pixels, and wherein the motion estimation module sets boundaries for the one or more regions around pixels with detected value changes.

12. The screen capture encoder of claim 1 wherein the reference screen capture frame immediately precedes the current screen capture frame.

13. The screen capture encoder of claim 1 wherein one or more two-dimensional motion vectors parameterize the motion.

14. A screen capture decoder comprising:

a buffer for storing a reference screen capture frame; and

a motion compensation module adapted to screen capture video, the motion compensation module for compensating for motion from the reference screen capture frame for one or more regions of pixels in a current screen capture frame.

15. The screen capture decoder of claim 14 wherein the decoder uses the motion compensation module for pixels with different values at corresponding locations in the reference screen capture frame and the current screen capture frame, and wherein the decoder uses the motion compensation module for less than all pixels of the current screen capture frame.

16. The screen capture decoder of claim 14 further comprising:

a frame assembler for assembling the current screen capture frame based at least in part upon the one or more regions, palettized content in intra pixels following motion estimation, and continuous tone content in the intra pixels following motion estimation.

17. The screen capture decoder of claim 16 further comprising:

a first decoding module for decoding the continuous tone content; and

a second decoding module for decoding the palettized content.

18. The screen capture decoder of claim 14 wherein, for each of the one or more regions in the current screen capture frame, the motion compensation module uses pixels from a match in an area comprising the entire reference screen capture frame.

19. The screen capture decoder of claim 14 wherein one or more two-dimensional motion vectors parameterize the motion.

20. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method comprising:

selecting a candidate motion vector for a current frame region of one or more pixels, wherein the candidate motion vector indicates a location for a reference frame region of one or more pixels;

computing a distortion measure based at least in part upon identical pixel value matching criteria for the candidate motion vector;

selecting a final motion vector for the current frame region; and

encoding the current frame region with the final motion vector.

21. The computer-readable medium of claim 20 wherein the computing comprises checking the identical pixel value matching criteria for less than all locations in the current frame region.

22. The computer-readable medium of claim 20 wherein the identical pixel value matching criteria comprises Hamming distance.

23. The computer-readable medium of claim 20 wherein screen capture video includes the reference frame and the current frame.

24. In a computer system, a method of motion estimation comprising:

for each of one or more motion vectors for a current frame region, computing a pixel count, each of the one or more motion vectors specifying a different reference frame region, wherein the computing includes,

comparing pixel values at each of one or more corresponding locations in the current frame region and the different reference frame region specified by the motion vector;

for each of the one or more corresponding locations, incrementing the pixel count based upon whether the compared pixel values are equal or not equal;

selecting a final motion vector for the current frame region; and

encoding the current frame region with the final motion vector.

25. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform the method of claim 24.

26. The method of claim 24 wherein the one or more corresponding locations are less than all locations in the current frame region and the different reference frame region specified by the motion vector.

27. The method of claim 24 wherein screen capture video includes the reference frame and the current frame.

28. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of motion estimation from a reference frame to a current frame, the method comprising:

for each of one or more candidate motion vectors for a current frame region,

selecting the candidate motion vector according to a search pattern that prioritizes horizontal motion vectors and vertical motion vectors above other motion vectors;

computing a distortion measure for the candidate motion vector;

selecting a final motion vector for the current frame region; and

encoding the current frame region with the final motion vector.

29. The computer-readable medium of claim 28 wherein screen capture video includes the reference frame and the current frame.

30. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method comprising:

evaluating a candidate motion vector for a current frame region in a current screen capture frame, wherein the candidate motion vector indicates a location for a reference frame region in a reference screen capture frame, and wherein the evaluating includes computing a sub-sampled distortion measure for the candidate motion vector;

selecting a final motion vector for the current frame region; and

encoding the current frame region with the final motion vector.

31. The computer-readable medium of claim 30 wherein the method further comprises:

for each of one or more additional candidate motion vectors, repeating the evaluating, wherein a set of n motion vectors includes n evaluated motion vectors, each of the n evaluated motion vectors having a sub-sampled distortion measure that satisfies a threshold, and wherein the selecting the final motion vector is based at least in part upon computation of a non-sub-sampled distortion measure for at least one evaluated motion vector in the set of n motion vectors.

32. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method comprising:

using segmentation data to identify one or more unchanged pixels between a current screen capture frame and a reference screen capture frame;

performing motion compensation for one or more regions of the current screen capture frame; and

assembling the current screen capture frame based at least in part upon the one or more identified unchanged pixels and the one or more motion compensated regions.

33. The computer-readable medium of claim 32 wherein the method further comprises decoding intra pixel data of the current screen capture frame, wherein the assembling is further based at least in part upon the decoded intra pixel data.

34. A method comprising:

comparing pixel values at corresponding locations in a current frame and a previous frame of a series to identify pixels in the current frame with changed values;

creating a bounding box around a group of the pixels with changed values;

performing motion estimation for the bounding box, thereby determining motion data for the bounding box relative to the previous frame;

comparing the bounding box with a predicted bounding box to identify intra pixels within the bounding box not predicted by the motion estimation; and

encoding the intra pixels.

35. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform the method of claim 34.