Apparatus and method for super-resolution enhancement processing

• US 20070041663A1
• Filed: 05/11/2006
• Published: 02/22/2007
• Est. Priority Date: 08/03/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Hyun, Cho, Daesung, Csaszar, Gergely, Cuzni, Laszlo
• Granted Patent: US 7,715,658 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

Provided is an apparatus and method for super-resolution enhancement processing. The apparatus includes a low-resolution image capturing unit, a patch pair generating unit, a patch pair database, and a high-resolution image reconstructing unit. The low-resolution image capturing unit captures a low-resolution image. The patch pair generating unit divides each of a plurality of high-resolution images into patches and generates a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches. The patch pair database stores the plurality of patch pairs generated by the patch pair generating unit. The high-resolution image reconstructing unit divides the low-resolution image into patches, searches in the patch pair database for a patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generates a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image.

Claims

1. An apparatus for super-resolution enhancement processing, the apparatus comprising:

a low-resolution image capturing unit capturing a low-resolution image;

a patch pair generating unit dividing each of a plurality of high-resolution images into patches and generating a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches;

a patch pair database storing the plurality of patch pairs generated by the patch pair generating unit; and

a high-resolution image reconstructing unit dividing the low-resolution image into patches, searching in the patch pair database for a-patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generating a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image.

2. The apparatus of claim 1, further comprising a patch merging unit which is positioned between the patch pair generating unit and the patch pair database and merges similar patches in leaves of a tree to reduce the redundancy of the patch pair database.

3. The apparatus of claim 2, wherein the patch merging unit obtains a difference between an average patch and an individual patch in each of the leaves of the tree and generates a new tree by replacing an individual patch that is similar to the average patch with the average patch.

4. The apparatus of claim 1, further comprising a patch classifying unit which is positioned between the patch pair generating unit and the patch pair database and classifies the medium-frequency patches to configure or use different databases according to the variance of the medium-frequency patches.

5. The apparatus of claim 1, wherein the patch pair generating unit comprises:

a high-frequency patch generating unit generating the high-frequency patches from the high-resolution image;

a medium-frequency patch generating unit generating the medium-frequency patches from an intermediate image; and

a patch pair configuring unit configuring the patch pairs with the high-frequency patches, the medium-frequency patches, and a plurality of overlapping edge pixels of the high-frequency patches.

6. The apparatus of claim 5, wherein the patch pair configuring unit performs smoothing on the high-frequency patches.

7. The apparatus of claim 1, wherein the high-resolution image reconstructing unit comprises:

a medium-frequency patch generating unit generating a medium-frequency patch from the low-resolution image;

a searching unit performing searching in the patch pair database using the medium-frequency patch and a plurality of edge pixels included in a high-frequency patch included in the reconstructed high-resolution image; and

an adding unit adding a high-frequency patch obtained by the searching of the searching unit to a corresponding patch of the low-resolution image.

8. The apparatus of claim 7, wherein the searching unit performs searching in the patch pair database using a plurality of geometrically converted versions of the medium-frequency patch.

9. The apparatus of claim 8, wherein the searching unit performs simultaneous searching for the plurality of geometrically converted versions of the medium-frequency patch using a shared candidate list.

10. The apparatus of claim 1, wherein the medium-frequency patch is configured not to overlap its adjacent patches.

11. The apparatus of claim 1, wherein the medium-frequency patch is configured to overlap its adjacent patches by one pixel in a gray channel.

12. A method for super-resolution enhancement processing, the method comprising:

dividing each of a plurality of high-resolution images into patches, generating a plurality of patch pairs composed of medium-frequency patches and high-frequency patches for the patches, and storing the generated patch pairs in a patch pair database; and

dividing a low-resolution image into patches, searching in the patch pair database for a patch pair including a medium-frequency patch that is similar to that of the low-resolution image, and generating a reconstructed high-resolution image by inserting a high-frequency patch included in the found patch pair into a corresponding patch of the low-resolution image.

13. The method of claim 12, further comprising merging similar patches in leaves of a tree to reduce the redundancy of the patch pair database.

14. The method of claim 13, wherein the merging of the similar patches comprises:

obtaining an average patch in each of the leaves of the tree;

obtaining a difference between the average patch and an individual patch in each of the leaves of the tree; and

generating a new tree by replacing an individual patch that is similar to the average patch with the average patch.

15. The method of claim 12, further comprising classifying the medium-frequency patches to configure or use different databases according to the variance of the medium-frequency patches.

16. The method of claim 15, wherein the classifying of the medium-frequency patches comprises:

calculating the variance of the medium-frequency patches of the high-resolution image during training;

determining a class according to the calculated variance;

storing the medium-frequency patches in the determined class in the patch pair database;

calculating the variance of the medium-frequency patches of the low-resolution image during reconstruction; and

searching in the patch pair database for the medium-frequency patches of the high-resolution image stored in the determined class according to the variance of the medium-frequency patches of the low-resolution image.

17. The method of claim 12, wherein the generating of the reconstructed high-resolution image comprises performing searching in the patch pair database using a plurality of geometrically converted versions of the medium-frequency patch of the low-resolution image.

18. The method of claim 17, wherein the generating of the reconstructed high-resolution image comprises performing simultaneous searching for the plurality of geometrically converted versions of the medium-frequency patch using a shared candidate list.

19. The method of claim 12, further comprising performing smoothing on the high-frequency patches of the high-resolution images.

20. At least one computer readable medium storing instructions that control at least one processor to perform a method for super-resolution enhancement processing claimed in claim 12.