METHOD FOR IMAGE ENHANCEMENT, IMAGE PROCESSING APPARATUS AND COMPUTER READABLE MEDIUM USING THE SAME

• US 20150348234A1
• Filed: 10/24/2014
• Published: 12/03/2015
• Est. Priority Date: 05/30/2014
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Shieh, Han-Ping D., Peng, Kuo-Shiuan, Lin, Fang-Cheng, Huang, Yi-Pai
• Granted Patent: US 9,552,625 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A method, image processing apparatus, and computer readable medium for image enhancement are provided. In the method, a low-resolution image is received and upscaled to generate an upscaled image having first pixels. Edge detection is performed on the upscaled image to obtain edge information of the first pixels. Each piece of the edge information includes an edge direction and an edge intensity, and each of the edge directions includes a horizontal and vertical component having a constant summation. A non-edge region of the upscaled image is enhanced to generate a high-resolution detail image; based on unsharp masking sharpening methods, an edge region of the low-passed upscaled image is enhanced according to the horizontal and vertical components of the edge directions to generate a high-resolution edge image. Image fusion is performed on the high-resolution edge image and the high-resolution detail image according to the edge intensities to generate a super-resolution image.

Claims

1. A method for image enhancement, adapted to an image processing apparatus, comprising:

receiving a low-resolution image;

upscaling the low-resolution image so as to generate an upscaled image, wherein the upscaled image comprises a plurality of first pixels;

performing edge detection on the upscaled image so as to obtain edge information of each of the first pixels, wherein the edge information of each of the first pixels comprises an edge direction and an edge intensity, each of the edge directions comprises a horizontal component and a vertical component, and a summation of the horizontal component and the vertical component of each of the edge directions is a constant;

performing low-pass filtering on the upscaled image by using a low-pass filter so as to generate a low-passed upscaled image, and enhancing an edge region of the low-passed upscaled image according to the horizontal component and the vertical component of each of the edge directions by leveraging a one-dimensional horizontal unsharp masking sharpening method and a one-dimensional vertical unsharp masking sharpening method so as to generate a high-resolution edge image;

enhancing a non-edge region of the upscaled image by leveraging a two-dimensional unsharp masking sharpening method so as to generate a high-resolution detail image; and

performing image fusion on the high-resolution edge image and the high-resolution detail image according to each of the edge intensities so as to generate a super-resolution image.

2. The method according to claim 1, wherein the step of upscaling the low-resolution image so as to generate the upscaled image comprises:

interpolating and upscaling the low-resolution image by leveraging a bicubic interpolation method so as to generate the upscaled image.

3. The method according to claim 1, wherein the step of performing edge detection on the upscaled image so as to obtain the edge information of each of the first pixels comprises:

calculating a horizontal gradient and a vertical gradient of each of the first pixels by using an edge-detection filter;

calculating and normalizing an absolute summation of the horizontal gradient and the vertical gradient of each of the first pixels respectively so as to obtain the edge intensity of each of the first pixels; and

calculating and normalizing an absolute ratio corresponding to the horizontal gradient and the vertical gradient of each of the first pixels so as to obtain the horizontal component and the vertical component of the edge direction of each of the first pixels.

4. The method according to claim 3, wherein after the step of calculating and normalizing the absolute ratio corresponding to the horizontal gradient and the vertical gradient of each of the first pixels so as to obtain the horizontal component and the vertical component of the edge direction of each of the first pixels, the method further comprises:

performing low-pass filtering on the horizontal component of the edge direction of each of the first pixels by using another low-pass filter so as to generate a low-pass horizontal component of the edge direction of each of the first pixels;

obtaining a low-pass vertical component of the edge direction of each of the first pixels according to the low-pass horizontal component of the edge direction of each of the first pixels; and

setting the low-pass horizontal component and the low-pass vertical component of the edge direction of each of the first pixels respectively as the horizontal component and the vertical component of the edge direction of each of the first pixels.

5. The method according to claim 1, wherein the step of enhancing the edge region of the low-passed upscaled image according to the horizontal component and the vertical component of each of the edge directions by leveraging the one-dimensional horizontal unsharp masking sharpening method and the one-dimensional vertical unsharp masking sharpening method so as to generate the high-resolution edge image comprises:

setting the horizontal component of each of the edge directions as a horizontal weight corresponding to a horizontal unsharp mask;

setting the vertical component of each of the edge directions as a vertical weight corresponding to a vertical unsharp mask;

enhancing the edge region of the low-passed upscaled image according to the horizontal unsharp mask, the vertical unsharp mask, each of the horizontal weights, and each of the vertical weights so as to generate an edge-enhanced image; and

performing image clipping on the edge-enhanced image according to each of the edge intensities so as to generate the high-resolution edge image.

6. The method of claim 1, wherein the step of performing image fusion on the high-resolution edge image and the high-resolution detail image according to each of the edge intensities so as to generate the super-resolution image comprises:

setting each of the edge intensities as an edge intensity weight corresponding to the high-resolution edge image;

obtaining a corresponding non-edge intensity weight according to each of the edge intensity weights; and

performing image fusion on the high-resolution edge image and the high-resolution detail image by a fusion ratio of each of the edge intensity weight to the corresponding non-edge intensity weight so as to generate the super-resolution image.

7. An image processing apparatus comprising:

a storage unit, storing a plurality of instructions; and

a processing unit, coupled to the storage unit, accessing and executing the instructions for;

receiving a low-resolution image;

upscaling the low-resolution image so as to generate an upscaled image, wherein the upscaled image comprises a plurality of first pixels;

performing edge detection on the upscaled image so as to obtain edge information of each of the first pixels, wherein the edge information of each of the first pixels comprises an edge direction and an edge intensity, each of the edge directions comprises a horizontal component and a vertical component, and a summation of the horizontal component and the vertical component of each of the edge directions is a constant;

performing low-pass filtering on the upscaled image by using a low-pass filter so as to generate a low-passed upscaled image, and enhancing an edge region of the low-passed upscaled image according to the horizontal component and the vertical component of each of the edge directions by leveraging a one-dimensional horizontal unsharp masking sharpening method and a one-dimensional vertical unsharp masking sharpening method so as to generate a high-resolution edge image;

enhancing a non-edge region of the upscaled image by leveraging a two-dimensional unsharp masking sharpening method so as to generate a high-resolution detail image; and

performing image fusion on the high-resolution edge image and the high-resolution detail image according to each of the edge intensities so as to generate a super-resolution image.

8. The image processing apparatus of claim 7, wherein the processing unit executes the instruction for upscaling the low-resolution image so as to generate the upscaled image comprising:

interpolating and upscaling the low-resolution image by leveraging a bicubic interpolation method so as to generate the upscaled image.

9. The image processing apparatus of claim 7, wherein the processing unit executes the instruction for performing edge detection on the upscaled image so as to obtain the edge information of each of the first pixels comprising:

calculating a horizontal gradient and a vertical gradient of each of the first pixels by using an edge-detection filter;

calculating and normalizing an absolute summation of the horizontal gradient and the vertical gradient of each of the first pixels respectively so as to obtain the edge intensity of each of the first pixels; and

calculating and normalizing an absolute ratio corresponding to the horizontal gradient and the vertical gradient of each of the first pixels so as to obtain the horizontal component and the vertical component of the edge direction of each of the first pixels.

10. The image processing apparatus of claim 9, wherein after the processing unit executes the instruction for calculating and normalizing the absolute ratio corresponding to the horizontal gradient and the vertical gradient of each of the first pixels so as to obtain the horizontal component and the vertical component of the edge direction of each of the first pixels, the processing unit further executes another instruction for:

performing low-pass filtering on the horizontal component of the edge direction of each of the first pixels by using another low-pass filter so as to generate a low-pass horizontal component of the edge direction of each of the first pixels;

obtaining a low-pass vertical component of the edge direction of each of the first pixels according to the low-pass horizontal component of the edge direction of each of the first pixels; and

setting the low-pass horizontal component and the low-pass vertical component of the edge direction of each of the first pixels respectively as the horizontal component and the vertical component of the edge direction of each of the first pixels.

11. The image processing apparatus of claim 7, wherein the processing unit executes the instruction for enhancing the edge region of the low-passed upscaled image according to the horizontal component and the vertical component of each of the edge directions by leveraging the one-dimensional horizontal unsharp masking sharpening method and the one-dimensional vertical unsharp masking sharpening method so as to generate the high-resolution edge image comprising:

setting the horizontal component of each of the edge directions as a horizontal weight corresponding to a horizontal unsharp mask;

setting the vertical component of each of the edge directions as a vertical weight corresponding to a vertical unsharp mask;

enhancing the edge region of the low-passed upscaled image according to the horizontal unsharp mask, the vertical unsharp mask, each of the horizontal weights, and each of the vertical weights so as to generate an edge-enhanced image; and

performing image clipping on the edge-enhanced image according to each of the edge intensities so as to generate the high-resolution edge image.

12. The image processing apparatus of claim 7, wherein the processing unit executes the instruction for performing image fusion on the high-resolution edge image and the high-resolution detail image according to each of the edge intensities so as to generate the super-resolution image comprising:

setting each of the edge intensities as an edge intensity weight corresponding to the high-resolution edge image;

obtaining a corresponding non-edge intensity weight according to each of the edge intensity weights; and

performing image fusion on the high-resolution edge image and the high-resolution detail image by a fusion ratio of each of the edge intensity weight to the corresponding non-edge intensity weight so as to generate the super-resolution image.

13. A non-transitory computer readable medium, recording programs to be loaded into an image processing apparatus to perform steps of:

receiving a low-resolution image;

upscaling the low-resolution image so as to generate an upscaled image, wherein the upscaled image comprises a plurality of first pixels;

performing edge detection on the upscaled image so as to obtain edge information of each of the first pixels, wherein the edge information of each of the first pixels comprises an edge direction and an edge intensity, each of the edge directions comprises a horizontal component and a vertical component, and a summation of the horizontal component and the vertical component of each of the edge directions is a constant;

performing low-pass filtering on the upscaled image by using a low-pass filter so as to generate a low-passed upscaled image, and enhancing an edge region of the low-passed upscaled image according to the horizontal component and the vertical component of each of the edge directions by leveraging a one-dimensional horizontal unsharp masking sharpening method and a one-dimensional vertical unsharp masking sharpening method so as to generate a high-resolution edge image;

enhancing a non-edge region of the upscaled image by leveraging a two-dimensional unsharp masking sharpening method so as to generate a high-resolution detail image; and

performing image fusion on the high-resolution edge image and the high-resolution detail image according to each of the edge intensities so as to generate a super-resolution image.