Method and System for Optical Flow Based Motion Vector Estimation for Picture Rate Up-Conversion

• US 20090110076A1
• Filed: 10/31/2007
• Published: 04/30/2009
• Est. Priority Date: 10/31/2007
• Status: Active Grant

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Chen, Xuemin
• Granted Patent: US 8,218,638 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

Certain aspects of a method and system for optical flow based motion vector estimation for picture rate up-conversion (PRUC) may include generating one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function. The cost function may be constrained by any combination of a block matching constraint, a smoothness constraint and a bias constraint. The PRUC data may be extracted from a compressed video data stream while the compressed video data stream is being decompressed by a video decompression engine. The PRUC data may comprise local block motion vectors, block coding modes, quantization levels, quantized residual data and decoded pictures. A plurality of interpolated pictures may be generated based on extracting the PRUC data.

Claims

1. A method for processing video data, the method comprising:

generating one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function, wherein said cost function is constrained by one or more of;

a block matching constraint, a smoothness constraint and/or a bias constraint.

2. The method according to claim 1, comprising extracting said PRUC data from received compressed video data while said received compressed video data is being decompressed.

3. The method according to claim 1, comprising generating a plurality of interpolated pictures based on said extracted PRUC data.

4. The method according to claim 1, wherein said block matching constraint is represented as

5. The method according to claim 4, wherein said smoothness constraint is represented as

6. The method according to claim 5, wherein said bias constraint is represented as B( v, v_ref)=(v_x−

v_xref)²+(v_y−

v_yref)², where v_ref v_xref and v_yref reference motion vectors.

7. The method according to claim 6, wherein said cost function is represented as

8. The method according to claim 7, comprising dynamically updating said adaptive smoothness parameter w_S and said adaptive bias parameter w_B.

9. The method according to claim 6, wherein said cost function is represented as

10. The method according to claim 9, wherein a sufficient condition for a unique solution for said cost function is v=H⁻

1·

Q, where

11. The method according to claim 10, comprising solving said cost function

12. The method according to claim 10, comprising solving said cost function using

13. A system for processing video data, the system comprising:

one or more circuits that enables generation of one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function, wherein said cost function is constrained by one or more of;

a block matching constraint, a smoothness constraint and/or a bias constraint.

14. The system according to claim 13, wherein said one or more circuits enables extraction of said PRUC data from received compressed video data while said received compressed video data is being decompressed.

15. The system according to claim 13, wherein said one or more circuits enables generation of a plurality of interpolated pictures based on said extracted PRUC data.

16. The system according to claim 13, wherein said block matching constraint is represented as

17. The system according to claim 16, wherein said smoothness constraint is represented as

18. The system according to claim 17, wherein said bias constraint is represented as B( v, v_ref)=(v_x−

v_xref)²+(v_y−

v_yref), where v_ref, v_xref and v_yref are reference motion vectors.

19. The system according to claim 18, wherein said cost function is represented as

20. The system according to claim 19, wherein said one or more circuits enables dynamic updating of said adaptive smoothness parameter w_S and said adaptive bias parameter w_B.

21. The system according to claim 18, wherein said cost function is represented as

22. The system according to claim 21, wherein a sufficient condition for a unique solution for said cost function is v=H⁻

1·

Q, where

23. The system according to claim 22, wherein said one or more circuits enables solving of said cost function using

24. The system according to claim 22, wherein said one or more circuits enables solving of said cost function using

25. A machine-readable storage having stored thereon, a computer program having at least one code section for processing video data, the at least one code section being executable by a machine for causing the machine to perform steps comprising:

generating one or more motion vectors based on extracted picture rate up-conversion (PRUC) data by minimizing a cost function, wherein said cost function is constrained by one or more of;

a block matching constraint, a smoothness constraint and/or a bias constraint.