METHOD AND SYSTEM OF RECONSTRUCTING SUPER-RESOLUTION IMAGE

• US 20120051667A1
• Filed: 07/19/2011
• Published: 03/01/2012
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited, Korea University Research and Business Foundation
• Original Assignee: Samsung Techwin Corporation (Hanwha Vision Company Limited), Korea University Research and Business Foundation
• Inventors: KANG, Bong-hyup, KO, Han-seok, SONG, Tae-yup, KU, Bon-hwa, LEE, Young-hyun, KIM, Min-jae, CHUNG, Dae-sung, SHIN, Hyun-hak
• Granted Patent: US 8,682,109 B2
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A method and system of generating a super-resolution image is provided. fusing The method includes inputting an image; and generating an estimated high-resolution image of a current time by fusing an input image and an estimated high-resolution image of a previous time corresponding to the input image.

Claims

1. A method of generating a super-resolution image, the method comprising:

inputting an image; and

generating an estimated high-resolution image of a current time by fusing an input image and an estimated high-resolution image of a previous time corresponding to the input image.

2. The method of claim 1, wherein the estimated high-resolution image of the current time is generated by upsampling the input image to high resolution and fusing the upsampled input image and the high-resolution image of the previous time which is aligned with the upsampled input image.

3. The method of claim 2, wherein the generating the estimated high-resolution image of the current time comprising:

calculating an image registration error between the upsampled input image and the estimated high-resolution image of the previous time;

comparing the image registration error to a threshold value; and

generating the estimated high-resolution image of the current time by reflecting the estimated high-resolution image of the previous time and the upsampled input image if the image registration error is less than the threshold value,

wherein the image registration error corresponds to a degree of misalignment between the upsampled input image and the estimated high-resolution image of the previous time.

4. The method of claim 3, wherein the estimated high-resolution image of the current time is generated by reflecting the estimated high-resolution image of the previous time without the upsampled input image if the image registration error is not less than the threshold value.

5. The method of claim 3,

wherein the image registration error is calculated for each pixel between the upsampled input image and the estimated high-resolution image of the previous time, and

the estimated high-resolution image of the current time is generated for each pixel according to the image registration error.

6. The method of claim 3, further comprising:

calculating a number of invalid pixels each of which has the image registration error not less than the threshold value;

comparing the number of the invalid pixels to a limit pixel number; and

determining the estimated high-resolution image of the current image as an image of the current time if the number of the invalid pixels is less than the limit pixel number.

7. The method of claim 6, wherein the estimated high-resolution image of the previous time is initialized and a high-resolution image of the current time is generated without considering the estimated high-resolution image of the previous time before the initialization if the number of the invalid pixels is not less than the limit pixel number.

8. The method of claim 1, further comprising:

calculating a number of invalid pixels each of which has an image registration error not less than a threshold value;

comparing the number of the invalid pixels to a limit pixel number; and

determining the estimated high-resolution image of the current image as an image of the current time if the number of the invalid pixels is less than the limit pixel number, and generating a high-resolution image of the current image without considering the estimated high-resolution image of the previous time before the initialization if the number of the invalid pixels is not less than the limit pixel number,

wherein the image registration error corresponds to a degree of misalignment between the input image and the estimated high-resolution image of the previous time.

9. A surveillance system comprising:

an image inputting unit through which an image is input; and

an image generation unit which generates a super-resolution image having a higher resolution than a resolution of the input image,

wherein the super-resolution image is generated by generating an estimated high-resolution image of a current time by fusing the input image and an estimated high-resolution image of a previous time corresponding to the input image.

10. The system of claim 9, further comprising:

a detecting unit which detects a selected region from the input image; and

a recognizing unit which recognizes content included in the super-resolution image,

wherein the image generation unit generates the super-resolution image with respect to the selected region among an entire region of the input image.

11. The system of claim 10, wherein, if the selected region is not detected from the input image, the image generation unit generates the super-resolution image with respect to the entire region of the input image, and

wherein the detecting unit detects the selected region from the super-resolution image generated with respect to the entire region of the input image.

12. The system of claim 11, wherein the selected region comprises a sub-region comprising the content.

13. The system of claim 9, wherein the estimated high-resolution image of the current time is generated by upsampling the input image to high resolution and fusing the upsampled input image and the high-resolution image of the previous time which is aligned with the upsampled input image.

14. The system of claim 9, wherein the image generation unit calculates an image registration error between the input image and the estimated high-resolution image of the previous time, and generates the estimated high-resolution image of the current time by reflecting the estimated high-resolution image of the previous time and the upsampled input image if the image registration error is less than the threshold value,

wherein the image registration error corresponds to a degree of misalignment between the input image and the estimated high-resolution image of the previous time.

15. The system of claim 14, wherein the image generation unit generates the estimated high-resolution image of the current time by reflecting the estimated high-resolution image of the previous time without the upsampled input image if the image registration error is not less than the threshold value.

16. The system of claim 13, wherein the image generation unit calculates

the image registration error for each pixel between the upsampled input image and the estimated high-resolution image of the previous time, and

the estimated high-resolution image of the current time is updated for each pixel of the input image according to the image registration error.

17. The system of claim 14, wherein the image generation unit calculates a number of invalid pixels each of which has the image registration error not less than the threshold value and determines the estimated high-resolution image of the current time as an image of the current time if the number of the invalid pixels is less than a limit pixel number.

18. The system of claim 17, wherein the estimated high-resolution image of the previous time is initialized and a high-resolution image of the current time is generated without considering the estimated high-resolution image of the previous time before the initialization if the number of the invalid pixels is not less than the limit pixel number.

19. The system of claim 10, wherein the super-resolution image is generated with respect to the selected region only if the selected region is located within a predetermined distance from a corresponding selected region detected at a previous time.

20. The system of claim 10, wherein the image inputting unit comprises a single camera installed at a given location.