Texture sensitive temporal filter based on motion estimation

• US 8,619,861 B2
• Filed: 02/26/2008
• Issued: 12/31/2013
• Est. Priority Date: 02/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zuo, Peter X., Lin, Bruce, Siravara, Bharath, Ali, Walid
• Primary Examiner(s): Elliott, IV, Benjamin H
• Application Number:
  US12/037,925
• Publication Number:
  US 20090213933A1
• Time in Patent Office: 2,135 Days
• Field of Search: 375/240.01, 375/240.08, 375/240.016, 375/240.29, 382232-254
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Architecture that employs texture sensitive temporal filtering to reuse motion estimation information in a realtime encoder. The temporal filter is applied for classified static areas. The architecture reuses the motion estimation results on motion vectors, cost estimates (e.g., sum of absolute difference (SAD)), and edge awareness texture information to apply the temporal filter on the current picture. Filtering can be applied at the pixel level, block level or macroblock level.

Claims

1. An encoder system, comprising:

a memory;

a processing unit operatively coupled to the memory, the processing unit executing;

a motion estimation component for generating frame characteristics and a motion vector from an input frame, the frame characteristics comprising a cost function and texture data, the motion estimation component to determine whether to code the frame characteristics of an area of the input frame as a block that is inter-block or intra-block;

a detection component for pre-generating a texture threshold for different cost functions, taking the input of the motion vector, the cost function, and the texture information for each area in the input frame and updating a block static counter according to the texture information, the cost function and the motion vector, and determining that a current area of the input flame is static when the block static counter for the area exceeds the predetermined texture threshold value defining whether the area is static or dynamic;

a mapping component for generating a static map from the static area; and

a filter component for temporally filtering the input frame when the area is static, using the static map of the area and outputting a filtered frame to an encoding process.

2. The system of claim 1, wherein the motion estimation component, the mapping component and the filter component are part of an encoder.

3. The system of claim 1, the motion estimation component to generate variance information from the input frame.

4. The system of claim 1, wherein the cost function is a sum of absolute differences (SAD) cost function.

5. The system of claim 1, wherein the cost function and texture data are defined at the pixel level.

6. A computer-implemented encoding system, comprising:

a memory;

a processing unit operatively coupled to the memory, the processing unit executing;

a motion estimation component for generating frame characteristics comprising a motion vector, cost function, and texture data from an input frame, the motion estimation component to determine whether to code the frame characteristics of the input frame as a block that is inter-block or intra-block;

a detection component for pre-generating a texture threshold for different cost functions, taking the input of the motion vector, the cost function, and the texture information for each block in the input frame and updating a block static counter according to the texture information, the cost function and the motion vector, and creating a frame characteristics map based on the motion vector, cost function, and texture data, the detection component comprising the predetermined texture threshold value that defines if the frame characteristics map is static or dynamic and determining that a current block of the input frame is static when the block static counter for the block exceeds the predetermined texture threshold value;

a filter component for temporally filtering static blocks in the frame characteristics map and outputting a filtered frame; and

an encoding component for encoding the motion vector and the filtered input frame to output a compressed frame.

7. The system of claim 6, wherein the motion estimator component includes a cost component that employs a SAD cost function.

8. A computer-implemented method of encoding a picture, comprising:

generating motion estimation results comprising a motion vector, a cost associated with the motion vector, and texture information from a motion estimation process of input video information;

determining whether to code the motion estimation results of the input video information as a block that is inter-block or intra-block;

pre-generating a texture threshold for different cost functions;

taking the input of the motion vector, the cost function, and the texture information for each area in the input frame and updating a block static counter according to the texture information, the cost function and the motion vector;

determining that a current block of the input frame is static when the block static counter for the block exceeds the predetermined texture threshold value defining whether the area is static or dynamic;

creating a frame characteristics map based on detection of one or more of the motion estimation results, the cost, and the texture information;

filtering static blocks in the frame characteristics map temporally, to output a filtered input frame; and

encoding the filtered input frame and the motion vector to output a compressed representation of the input video information.

9. The method of claim 8, wherein the texture information includes edge awareness information.

10. The method of claim 8, wherein the frame characteristics map is created for each block of a current frame based on the motion vector, the cost, and the texture information.

11. The method of claim 8, further comprising applying weights across multiple frames as part of filtering the frame characteristics map.

12. The method of claim 8, further comprising employing block variance as the textual information for intra-block.