Image Processing

• US 20100182511A1
• Filed: 01/14/2010
• Published: 07/22/2010
• Est. Priority Date: 01/15/2009
• Status: Abandoned Application

• Current Assignee: Sanbao Xu
• Original Assignee: Sanbao Xu
• Inventors: Xu, Sanbao
• Application Number:
  US12/687,792
• Publication Number:
  US 20100182511A1
• Time in Patent Office: Days
• US Class Current: 348/699
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard

Abstract

Increasing spatial resolution of image frames in a sequence of image frames is described. Processing of a reference image frame and a current image frame is performed that involves updating a resulting image frame with data resulting from the processing. Calculation (203) of an upsampled current frame is performed by upsampling the current frame with an upsampling factor and interpolating the pixel data of the current frame. Calculation (205) of a plurality of motion vectors is performed via two-dimensional block motion estimation between the upsampled current frame and the reference frame resulting in a respective motion vector for each block. A motion mode in terms of whether the motion of the upsampled current frame is any of a global translation, a rotation and a complex motion is decided, involving analyzing the calculated motion vectors. Warping (207) of the upsampled current frame is performed, using at least one of the calculated motion vectors, the number of motion vectors used being dependent on the decided motion mode. The resulting image frame is updated (209) with the warped upsampled current frame, by weighted averaging using a weighting factor that is a confidence measure for the contribution of the upsampled current frame to the updating, the confidence measure being obtained from the motion vector calculation.

Claims

1. A method of increasing spatial resolution of image frames in a sequence of image frames by processing of a reference image frame and a current image frame, involving updating a resulting image frame with data resulting from said processing, the method comprising:

calculating an upsampled current frame by upsampling the current frame with an upsampling factor and interpolating the pixel data of the current frame,

calculating a plurality of motion vectors by performing two-dimensional block motion estimation between the upsampled current frame and the reference frame resulting in a respective motion vector for each block,

deciding a motion mode in terms of whether the motion of the upsampled current frame is any of a global translation, a rotation and a complex motion, the deciding involving analyzing the calculated motion vectors,

performing warping of the upsampled current frame, using at least one of the calculated motion vectors, the number of motion vectors used being dependent on the decided motion mode, and

updating the resulting image frame with the warped upsampled current frame, by weighted averaging using a weighting factor that is a confidence measure for the contribution of the upsampled current frame to the updating, the confidence measure being obtained from the motion vector calculation.

2. The method claim 1, where the confidence measure is inversely proportional to a prediction error obtained from the calculation of the at least one motion vector.

3. The method of claim 1, where the motion vector calculation is performed with an adaptive search block-matching algorithm using sub-pixel precision.

4. The method of claim 1, wherein:

in case the deciding involves deciding that the motion mode is a global translation of the upsampled current frame, the warping is performed using one motion vector that represents the motion of the upsampled current frame, and

in case the deciding involves deciding that the motion mode is any of a rotation and a complex motion of the upsampled current frame, the warping is performed block-wise using a plurality of motion vectors that represent the motion of respective blocks of the upsampled current frame.

5. The method of claim 1, where the performance of the warping and the updating is conditionally performed such that the warping and updating is omitted in case the analysis of the motion vectors indicate a motion complexity that is larger than a threshold complexity.

6. The method of claim 1, where the performance of the calculation of an upsampled current frame, the calculation of a plurality of motion vectors, the decision of a motion mode, the warping, and the updating is repeated a predetermined number of times using successive current image frames that are subsequent to the reference image frame.

7. The method of claim 1, where the performance of the calculation of an upsampled current frame, the calculation of a plurality of motion vectors, the decision of a motion mode, the warping, and the updating is repeated a predetermined number of times using successive current image frames that are prior to the reference image frame.

8. The method of claim 1, performed in a recursive manner involving the obtaining of a reference image frame and repeated obtaining of the current image frame, upsampling calculation, motion vector calculation, decision of a motion mode, and warping and the updating until processing of a predefined number of current image frames has been performed.

9. An apparatus comprising processing means and memory means that are configured to perform the method of claim 1.

10. A mobile communication device comprising the apparatus of claim 9.

11. A computer program comprising software instructions that, when executed in a processor, performs the method of claim 1.