Motion estimation based on block level bilateral filter and local motion model

• US 10,448,008 B1
• Filed: 06/14/2013
• Issued: 10/15/2019
• Est. Priority Date: 04/30/2013
• Status: Active Grant

• Current Assignee: Pixelworks Incorporated
• Original Assignee: Pixelworks Incorporated
• Inventors: Sheng, Xiaojie, Woodall, Neil, Ma, Yue, Cheng, Guohua
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Noh, Jae N
• Application Number:
  US13/918,457
• Time in Patent Office: 2,314 Days
• Field of Search: 37524016
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method of estimating motion in video display data includes dividing a frame of the video display data into blocks, generating at least two motion models for a current block from motion models of neighboring blocks to the current block, and selecting one of the motion models based upon a similarity of the motion model that minimizes differences between corresponding blocks in reference frames to the current block.

Claims

1. A method of estimating motion in video display data, comprising:

dividing a frame of the video display data into blocks;

selecting at least two motion model candidate for a current block from motion models used in neighboring blocks;

modifying one or more of the at least two selected motion model candidates to generate at least two motion model candidates for the current block from motion models of blocks in a predetermined supporting region;

determining a cost for each of the motion model candidates for each block in the predetermined supporting region based on a difference between pixels in a first frame and pixels in a second frame with a relative position determined by each of the motion model candidates;

finding a content based weight for each block in the predetermined supporting region based on a similarity of content with the current block;

finding a distance based weight for each block in the predetermined supporting region based on a spatial distance of the block in the predetermined supporting region relative to the current block;

combining the content based weight and the distance based weight for each block in the predetermined supporting region together;

applying the combined weights to the cost for each motion model candidate for each block in the predetermined supporting region to produce weighted costs for each block in the predetermined supporting region;

summing the weighted cost for each motion model candidate for each block in the predetermined supporting region to create a total cost for each motion model candidate;

selecting a motion model from the motion model candidates based upon the total cost; and

calculating a motion vector for the current block based on the selected motion model.

2. The method of claim 1, wherein selecting the motion model is further based upon a similarity of content of the video display data in the current block to content of the video display data of a neighboring block from which the motion model is selected.

3. The method of claim 2, wherein selecting the motion model is further based upon the current block to the neighboring block of the selected motion model.

4. The method of claim 3, further comprising weighting the motion model candidates prior to selecting the motion model.

5. The method of claim 4, wherein weighting the motion model candidates comprises bilateral weighting.

6. The method of claim 1, wherein generating at least two motion model candidates comprises making random changes to parameters of motion model candidates for neighboring blocks.

7. The method of claim 1, wherein determining the cost comprises finding a difference in a motion vector calculated for a block and a motion vector calculated for the block using a motion model candidate and using the difference to set the cost.

8. The method of claim 1, wherein determining the cost comprises finding a sum of absolute differences for the block based upon the motion model candidate.

9. The method of claim 1, wherein weighting each of the blocks in the predetermined supporting region comprises using a sum of absolute differences between pixel values in a block in the predetermined supporting region and the current block.

10. The method of claim 1, wherein weighting each of the blocks in the predetermined supporting region comprises using a histogram of gradients.

11. The method of claim 1, wherein weighting each of blocks in the predetermined supporting region comprises using a distance between the current block and each of the blocks in the predetermined supporting region.

12. The method of claim 1, wherein combining the content based weight and the distance based weight for each block in the predetermined supporting region together includes multiplying the content based weight and the distance based weight for each block in the predetermined supporting region.

13. The method of claim 1, wherein the predetermined supporting region is rectangular.

14. The method of claim 1, wherein applying the combined weights to the cost for each motion model candidate for each block in the predetermined supporting region includes multiplying the combined weights to the cost for each motion model candidate for each block in the predetermined supporting region.

15. A method of estimating motion in video display data, comprising:

dividing a frame of the video display data into blocks;

selecting at least two motion model candidate for a current block from motion models used in neighboring blocks;

modifying one or more of the at least two selected motion model candidates to generate at least two motion model candidates for the current block from motion models of blocks in a predetermined supporting region;

determining a cost for each of the motion model candidates for each block in the predetermined supporting region based on a difference between pixels in a first frame and pixels in a second frame with a relative position determined by each of the motion model candidates;

determining a weight for each block in the predetermined supporting region based on a likelihood of the block in the predetermined supporting region belonging to a same object in the video display data as the current block and a spatial distance of the block in the predetermined supporting region relative to the current block;

applying the weight to the cost for each motion model candidate for each block in the predetermined supporting region to produce a weighted cost for each block in the predetermined supporting region;

summing the weighted cost for each motion model candidate for each block in the predetermined supporting region to create a total cost for each motion model candidate;

selecting a motion model from the motion model candidates based upon the total cost; and

calculating a motion vector for the current block based on the selected motion model.

16. The method of claim 15, wherein selecting the motion model is further based upon a similarity of content of the video display data in the current block to content of the video display data of a neighboring block from which the motion model is selected.

17. The method of claim 16, wherein selecting the motion model is further based upon the current block to the neighboring block of the selected motion model.