Depth-Based Image Enhancement

• US 20100080485A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Original Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Inventors: Cheng, Chao-Chung Cheng, Chen, Liang-Gee Chen, Huang, Ling-Hsiu Huang, Li, Chung-To Li
• Granted Patent: US 8,059,911 B2
• Time in Patent Office: Days
• US Class Current: 382/266
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/50 Depth or shape recovery

Abstract

A depth-based image enhancement system is disclosed. A depth estimation unit generates three-dimensional (3D) depth information from a two-dimensional (2D) image. Subsequently, an image enhancement unit enhances the 2D image according to the 3D depth information.

Claims

1. A device for performing one or more of depth-based image enhancement and depth-based video enhancement, the device comprising:

a depth estimation unit that generates three-dimensional (3D) depth information from a two-dimensional (2D) image; and

an image enhancement unit that enhances the 2D image according to the 3D depth information.

2. The device of claim 1, wherein the image enhancement unit enhances objects in the 2D image having smaller depth value with larger enhancing weighting, and enhances objects in the 2D image having larger depth value with smaller enhancing weighting.

3. The device of claim 1, wherein the image enhancement unit enhances one or more of edge, contrast and saturation characteristics.

4. The device of claim 1, wherein the image enhancement unit comprises one or more of the following elements:

a depth-based edge enhancement unit that emphasizes edge details of the 2D image;

a color space converter that converts the 2D image between color spaces to obtain a luma component and a chroma component of the 2D image;

a depth-based contrast adjustment unit that enhances contrast characteristics of the luma component;

a depth-based saturation adjustment unit that enhances saturation characteristics of the chroma component; and

a color space back-converter that converts the adjusted luma and chroma components back to a color space of the original 2D image.

5. The device of claim 4, wherein the depth-based edge enhancement unit comprises:

a spatial convolution mask that is applied on a group of pixels of the 2D image surrounding a center pixel.

6. The device of claim 4, wherein the depth-based contrast adjustment unit comprises:

histogram stretching means for increasing a value of pixels having a gray level greater than a threshold, while decreasing a value of pixels having a gray level smaller than the threshold.

7. The device of claim 4, wherein the color space converter converts the 2D image from red/green/blue (RGB) color space to hue/saturation/intensity (HSI) color space.

8. The device of claim 7, wherein the color space back-converter converts the enhanced 2D image from the HSI color space back to the RGB color space.

9. The device of claim 1, further comprising an input device that receives the 2D image.

10. The device of claim 9, wherein the input device further stores the 2D image.

11. The device of claim 1, further comprising an output device that receives the enhanced 2D image.

12. The device of claim 11, wherein the output device further stores or displays the enhanced 2D image.

13. A method of using a device to perform one or more of depth-based image enhancement and depth-based video enhancement, the method comprising:

generating three-dimensional (3D) depth information from a two-dimensional (2D) image; and

enhancing the 2D image according to the 3D depth information.

14. The method of claim 13, wherein the enhancing step is performed by enhancing objects in the 2D image having smaller depth value with larger enhancing weighting, and enhancing objects in the 2D image having larger depth value with smaller enhancing weighting.

15. The method of claim 13, wherein the enhancing step is performed to enhance one or more of edge, contrast and saturation characteristics.

16. The method of claim 13, wherein the enhancing step comprises one or more of the following sub-steps:

enhancing edge details of the 2D image based on the 3D depth information;

converting the 2D image between color spaces to obtain a luma component and a chroma component of the 2D image;

adjusting contrast characteristics of the luma component based on the 3D depth information;

adjusting saturation characteristics of the chroma component based on the 3D depth information; and

converting the adjusted luma and chroma components back to a color space of the original 2D image.

17. The method of claim 16, wherein the edge-details enhancing sub-step comprises:

applying a spatial convolution mask on a group of pixels of the 2D image surrounding a center pixel.

18. The method of claim 16, wherein the contrast-characteristics adjusting sub-step comprises:

increasing a value of pixels having a gray level greater than a threshold, while decreasing a value of pixels having a gray level smaller than the threshold.

19. The method of claim 16, wherein the 2D image converting sub-step converts the 2D image from red/green/blue (RGB) color space to hue/saturation/intensity (HSI) color space.

20. The method of claim 19, wherein the luma/chroma back-converting sub-step converts the enhanced 2D image from the HSI color space back to the RGB color space.

21. The method of claim 13, further comprising a step of receiving the 2D image.

22. The method of claim 21, further comprising a step of storing the 2D image.

23. The method of claim 13, further comprising a step of receiving the enhanced 2D image.

24. The method of claim 23, further comprising a step of storing or displaying the enhanced 2D image.