One-pass super-resolution images

• US 20020172434A1
• Filed: 04/20/2001
• Published: 11/21/2002
• Est. Priority Date: 04/20/2001
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Information Technology Center America, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Freeman, William T., Jones, Thouis R.
• Granted Patent: US 6,766,067 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A super-resolution image is generated from a pixel image by first performing an initial image interpolation, creating an interpolated low resolution image. The interpolated low resolution image is then partitioned into overlapping low resolution patches. The low resolution patches are then processed in a raster scan order. For each low-resolution patch, a scaled mid band patch is generated. A search vector is constructed from pixels in the scaled mid band input patch, and pixels in an overlap region of adjacent previously predicted high band patches. A nearest index vector to the search vector is located in a training database, the nearest index vector has an associated high band output patch. The high band output patch is then combined with the interpolated low resolution patch to predict pixel values for the corresponding high resolution patch of the super-resolution image.

Claims

1. A method for generating a super-resolution image from a pixel image, comprising:

partitioning the pixel image into a plurality of overlapping low resolution patches;

partitioning the super-resolution image into a plurality of corresponding overlapping high resolution patches;

processing the low resolution patches in a sequential order, the processing for each low-resolution patch comprising;

generating a scaled mid band patch from the low resolution patch;

constructing a search vector from pixels in the scaled mid band input patch, and pixels in an overlap region of adjacent previously predicted high band patches;

locating a nearest index vector to the search vector in a training database, the nearest index vector having an associated high band output patch; and

combining the high band output patch with the interpolated low resolution patch to predict pixel values for the corresponding high resolution patch of the super-resolution image.

2. The method of claim 1 further comprising:

centering the overlapping patches on pixels spaced at coordinates (X, Y) that are P pixels apart.

3. The method of claim 1 wherein the high resolution patches overlap adjacent patches by one pixel and low resolution patches overlap adjacent low resolution patches by three pixels.

4. The method of claim 1 further comprising:

linearly arranging the pixels in the scaled mid band input patch and the pixels in the overlap region of the adjacent previously predicted high band patches to construct the search vector.

5. The method of claim 1 further comprising:

weighting the pixels of the mid band input patch by a weighting factor; and

normalizing contrast of the pixels of the search vector by a normalization factor.

6. The method of claim 5 wherein the weighting factor is

7. The method of claim 5 wherein the weighting factor is substantially a mean absolute value of the low resolution image across all of color values.

8. The method of claim 5 further comprising:

removing the normalizing on the high band output patch.

9. The method of claim 1 wherein the index vectors and the associated high band output patches are built from band-pass and high-pass pairs of training patches.

10. The method of claim 9 wherein the index vectors and the associated high band output patches are contrast normalized.

11. The method of claim 1 wherein the sequential order is a raster scan order.

12. The method of claim 2 wherein P is five.

13. The method of claim 1 wherein the nearest index vector is located using a L∞

-norm.

14. The method of claim 1 wherein similar index vectors in the training database are linked.