Method and system for optical flow based motion vector estimation for picture rate up-conversion

• US 8,218,638 B2
• Filed: 10/31/2007
• Issued: 07/10/2012
• Est. Priority Date: 10/31/2007
• Status: Expired due to Fees

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Chen, Xuemin
• Primary Examiner(s): Shah, Chirag
• Assistant Examiner(s): Rivas, Salvador E
• Application Number:
  US11/932,298
• Publication Number:
  US 20090110076A1
• Time in Patent Office: 1,714 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

Certain aspects of a method and system for optical flow based motion vector estimation for picture rate up-conversion (PRUC) may include generating one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function. The cost function may be constrained by any combination of a block matching constraint, a smoothness constraint and a bias constraint. The PRUC data may be extracted from a compressed video data stream while the compressed video data stream is being decompressed by a video decompression engine. The PRUC data may comprise local block motion vectors, block coding modes, quantization levels, quantized residual data and decoded pictures. A plurality of interpolated pictures may be generated based on extracting the PRUC data.

Claims

1. A method for processing video data, the method comprising:

generating one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function, wherein said cost function is constrained by one or more of;

a block matching constraint, a smoothness constraint and/or a bias constraint, wherein said block matching constraint is represented as

2. The method according to claim 1, wherein said smoothness constraint is represented as

3. The method according to claim 2, wherein said bias constraint is represented as where B( ν

, ν

_ref)=(v_x−

v_xref)²+(v_y−

v_yref)², where {right arrow over (v)}_ref, v_xref, and v_yref are reference motion vectors.

4. The method according to claim 3, wherein said cost function is represented as

5. The method according to claim 4, comprising dynamically updating said adaptive smoothness parameter w_S and said adaptive bias parameter w_B.

6. The method according to claim 3, wherein said cost function is represented as

7. The method according to claim 6, wherein a sufficient condition for a unique solution for said cost function is ν

=H<sup>−

1</sup>·

Q , where

8. The method according to claim 7, comprising solving said cost function using

9. The method according to claim 7, comprising solving said cost function using

10. A system for processing video data, the system comprising:

one or more circuits that enables generation of one of more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function, wherein said cost function is constrained by one or more of;

a block matching constraint, a smoothness constraint and/or bias constraint, wherein said block matching constraint is represented as

11. The system according to claim 10, wherein said smoothness constraint is represented as

12. The system according to claim 11, wherein said bias constraint is represented as B( ν

, ν

_ref)=(v_x−

v_xref)²+(v_y−

v_yref)², where {right arrow over (v)}_ref, v_xref and v_yref are reference motion vectors.

13. The system according to claim 12, wherein said cost function is represented as

14. The system according to claim 13, wherein said one or more circuits enables dynamic updating of said adaptive smoothness parameter w_S and said adaptive bias parameter w_B.

15. The system according to claim 12, wherein said cost function is represented as

16. The system according to claim 15, wherein a sufficient condition for a unique solution for said cost function is ν

=H<sup>−

1</sup>·

Q , where

17. The system according to claim 16, wherein said one or more circuits enables solving of said cost function using

18. The system according to claim 16, wherein said one or more circuits enables solving of said cost function using