Apparatus and method for super-resolution enhancement processing

• US 7,715,658 B2
• Filed: 05/11/2006
• Issued: 05/11/2010
• Est. Priority Date: 08/03/2005
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Csaszar, Gergely, C zni, Laszlo, Kim, Hyun Mun, Cho, Daesung
• Primary Examiner(s): Tucker; Wesley
• Assistant Examiner(s): Yentrapati; Avinash
• Application Number:
  US11/431,514
• Publication Number:
  US 20070041663A1
• Time in Patent Office: 1,461 Days
• Field of Search: None
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

Provided is an apparatus and method for super-resolution enhancement processing. The apparatus includes a low-resolution image capturing unit, a patch pair generating unit, a patch pair database, and a high-resolution image reconstructing unit. The low-resolution image capturing unit captures a low-resolution image. The patch pair generating unit divides each of a plurality of high-resolution images into patches and generates a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches. The patch pair database stores the plurality of patch pairs generated by the patch pair generating unit. The high-resolution image reconstructing unit divides the low-resolution image into patches, searches in the patch pair database for a patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generates a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image.

Claims

1. An apparatus for super-resolution enhancement processing, the apparatus comprising:

a low-resolution image capturing unit capturing a low-resolution image;

a patch pair generating unit dividing each of a plurality of high-resolution images into patches and generating a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches;

a patch pair database storing the plurality of patch pairs generated by the patch pair generating unit; and

a high-resolution image reconstructing unit dividing the low-resolution image into patches, searching in the patch pair database for a patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generating a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image,

wherein the high-resolution image reconstructing unit performs simultaneous searching of the patch pair database for a plurality of geometrically converted versions of the medium-frequency patch using a shared candidate list.

2. The apparatus of claim 1, further comprising a patch merging unit which is positioned between the patch pair generating unit and the patch pair database and merges similar patches in leaves of a tree to reduce the redundancy of the patch pair database.

3. The apparatus of claim 2, wherein the patch merging unit obtains a difference between an average patch and an individual patch in each of the leaves of the tree and generates a new tree by replacing an individual patch that is similar to the average patch with the average patch.

4. The apparatus of claim 1, further comprising a patch classifying unit which is positioned between the patch pair generating unit and the patch pair database and classifies the medium-frequency patches to configure or use different databases according to the variance of the medium-frequency patches.

5. The apparatus of claim 1, wherein the patch pair generating unit comprises:

a high-frequency patch generating unit generating the high-frequency patches from the high-resolution image;

a medium-frequency patch generating unit generating the medium-frequency patches from an intermediate image; and

a patch pair configuring unit configuring the patch pairs with the high-frequency patches, the medium-frequency patches, and a plurality of overlapping edge pixels of the high-frequency patches.

6. The apparatus of claim 5, wherein the patch pair configuring unit performs smoothing on the high-frequency patches.

7. The apparatus of claim 1, wherein the high-resolution image reconstructing unit comprises:

a medium-frequency patch generating unit generating a medium-frequency patch from the low-resolution image;

a searching unit performing searching in the patch pair database using the medium-frequency patch and a plurality of edge pixels included in a high-frequency patch included in the reconstructed high-resolution image; and

an adding unit adding a high-frequency patch obtained by the searching of the searching unit to a corresponding patch of the low-resolution image.

8. The apparatus of claim 1, wherein the medium-frequency patch is configured not to overlap its adjacent patches.

9. The apparatus of claim 1, wherein the medium-frequency patch is configured to overlap its adjacent patches by one pixel in a gray channel.

10. A method for super-resolution enhancement processing, the method comprising:

dividing each of a plurality of high-resolution images into patches, generating a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches, and storing the generated patch pairs in a patch pair database; and

dividing a low-resolution image into patches, searching in the patch pair database for a patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generating a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image,

wherein the searching in the patch pair database performs simultaneous searching of the patch pair database for a plurality of geometrically converted versions of the medium-frequency patch using a shared candidate list.

11. The method of claim 10, further comprising merging similar patches in leaves of a tree to reduce the redundancy of the patch pair database.

12. The method of claim 11, wherein the merging of the similar patches comprises:

obtaining an average patch in each of the leaves of the tree;

obtaining a difference between the average patch and an individual patch in each of the leaves of the tree; and

generating a new tree by replacing an individual patch that is similar to the average patch with the average patch.

13. The method of claim 10, further comprising classifying the medium-frequency patches to configure or use different databases according to the variance of the medium-frequency patches.

14. The method of claim 13, wherein the classifying of the medium-frequency patches comprises:

calculating the variance of the medium-frequency patches of the high-resolution image during training;

determining a class according to the calculated variance;

storing the medium-frequency patches in the determined class in the patch pair database;

calculating the variance of the medium-frequency patches of the low-resolution image during reconstruction; and

searching in the patch pair database for the medium-frequency patches of the high-resolution image stored in the determined class according to the variance of the medium-frequency patches of the low-resolution image.

15. The method of claim 10, further comprising performing smoothing on the high-frequency patches of the high-resolution images.

16. At least one computer readable medium storing instructions that control at least one processor to perform a method for super-resolution enhancement processing claimed in claim 10.