REAL-TIME DEPTH-AWARE IMAGE ENHANCEMENT SYSTEM

• US 20120293615A1
• Filed: 05/17/2011
• Published: 11/22/2012
• Est. Priority Date: 05/17/2011
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Original Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Inventors: Chen, Liang-Gee, Li, Chung-Te, Cheng, Chao-Chung, Lai, Yen-Chieh, Huang, Ling-Hsiu
• Granted Patent: US 9,007,435 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

In accordance with at least some embodiments of the present disclosure, a process for enhancing a two-dimensional (2D) image is described. The process may include generating a depth map for the 2D image based on edges detected from the 2D image, wherein the depth map can be used to convert the 2D image to a 3D image. The process may further include enhancing the 2D image using the depth map.

Claims

1. A method for enhancing a two-dimensional (2D) image, comprising:

performing spatial filtering on the 2D image to generate an edge map;

performing weighted filtering on the edge map to generate a depth map, wherein the depth map can be used to convert the 2D image to a three-dimensional (3D) image; and

enhancing the 2D image using the depth map.

2. The method as recited in claim 1, wherein the performing spatial filtering on the 2D image comprising:

detecting edges in the 2D image based on a defocus depth cue.

3. The method as recited in claim 1, wherein the performing weighted filtering on the edge map comprising:

generating a weighted depth map based on a probability distribution of edges in the edge map;

applying a weighted low-pass filter to the weighted depth map to generate an estimated depth map; and

filtering the estimated depth map using a cross-bilateral filter to generate the depth map.

4. The method as recited in claim 1, wherein the enhancing comprising enhancing contrast of the 2D image.

5. The method as recited in claim 4, wherein the enhancing contrast of the 2D image comprising:

applying a high-pass filter to a first contrast value of a pixel in the 2D image to generate a second contrast value; and

applying a depth value from the depth map and associated with the pixel to the second contrast value.

6. The method as recited in claim 1, wherein the enhancing comprising enhancing color of the 2D image.

7. The method as recited in claim 6, wherein the enhancing color of the 2D image comprising:

determining an ambient light amount for the 2D image;

for a pixel selected from the 2D image, enhancing object light amount of the pixel based on a depth value from the depth map and associated with the pixel; and

adjusting color of the pixel based on the enhanced object light amount and the ambient light amount.

8. The method as recited in claim 7, wherein the determining the ambient light amount comprising:

identifying a set of pixels in the 2D image having low light levels; and

determining the ambient light amount based on light levels associated with the set of pixels.

9. The method as recited in claim 7, wherein the enhancing the object light amount comprising adjusting the object light amount to enhance depth characteristic associated with the pixel.

10. The method as recited in claim 1, wherein the method is embodied in a machine-readable storage medium as a set of instructions, which when executed by a processing unit, cause the processing unit to perform the method.

11. A method for enhancing a two-dimensional (2D) video, comprising:

extracting a 2D image from the 2D video;

generating a depth map associated with the 2D image;

de-interlacing the 2D image to generate a de-interlaced 2D image;

performing noise reduction on the de-interlaced 2D image to generate a noise-reduced 2D image;

enhancing contrast of the noise-reduced 2D image to generate a contrast-enhanced 2D image using the depth map;

enhancing color of the contrast-enhanced 2D image to generate a color-enhanced 2D image using the depth map;

enhancing sharpness of the color-enhanced 2D image to generate a sharpness-enhanced 2D image; and

scaling the sharpness-enhanced 2D image to generate an enhanced 2D image.

12. The method as recited in claim 11, wherein the enhancing contrast comprising:

applying a high-pass filter to a first contrast value of a pixel in the noise-reduced 2D image to generate a second contrast value; and

applying a depth value from the depth map and associated with the pixel to the second contrast value.

13. The method as recited in claim 11, wherein the enhancing color comprising:

determining an ambient light amount for the contrast-enhanced 2D image;

for a pixel selected from the contrast-enhanced 2D image, enhancing object light amount of the pixel based on a depth value from the depth map and associated with the pixel; and

adjusting color of the pixel based on the enhanced object light amount and the ambient light amount.

14. The method as recited in claim 13, wherein the determining the ambient light amount comprising:

identifying a set of pixels in the contrast-enhanced 2D image having low light levels, and

determining the ambient light amount based on light levels associated with the set of pixels.

15. A system configured to enhance a two-dimensional (2D) video, comprising:

a video decoder configured to extract a 2D image from the 2D video;

a depth map generator, coupled with the video decoder, configured to generate a depth map based on the 2D image, wherein the depth map can be used to convert the 2D image to a three-dimensional (3D) image; and

a video post-processor, coupled with the video decoder and the depth map generator, configured to enhance the 2D image using the depth map.

16. The system as recited in claim 15, wherein the depth map generator comprising an edge detector configured to generate an edge map based on the 2D image.

17. The system as recited in claim 15, wherein the depth map generator comprising a post-processing filter, coupled with the edge detector, configured to generate the depth map based on the edge map.

18. The system as recited in claim 15, wherein the video post-processor comprising:

a de-interlacing module configured to de-interlace the 2D image;

a noise-reduction module, coupled with the de-interlacing module, configured to perform noise-reduction on the 2D image; and

a scaling module, coupled with the noise-reduction module, configured to scale the 2D image.

19. The system as recited in claim 18, wherein the video post-processor comprising:

a contrast enhancement module, coupled with the noise-reduction module, configured to enhance contrast of the 2D image using the depth map.

20. The system as recited in claim 19, wherein the video post-processor further comprising:

a color enhancement module, coupled with the contrast enhancement module, configured to enhance color of the 2D image using the depth map.