Depth-based image enhancement

• US 8,059,911 B2
• Filed: 09/30/2008
• Issued: 11/15/2011
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Original Assignee: National Taiwan University (Government of The Republic of China), Himax Technologies Ltd. (Himax Technologies Incorporated)
• Inventors: Huang, Ling-Hsiu, Li, Chung-Te, Chen, Liang-Gee, Cheng, Chao-Chung
• Primary Examiner(s): Rahmjoo; Mike
• Application Number:
  US12/242,672
• Publication Number:
  US 20100080485A1
• Time in Patent Office: 1,141 Days
• Field of Search: 382/266, 382/264, 382/254, 382/300, 382/154, 382/118, 342/25, 342/179, 342/180, 342/181, 342/191
• US Class Current: 382/266
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/50 Depth or shape recovery

Abstract

A depth-based image enhancement system is disclosed. A depth estimation unit generates three-dimensional (3D) depth information from a two-dimensional (2D) image. Subsequently, an image enhancement unit enhances the 2D image according to the 3D depth information.

Claims

1. A device for performing one or more of depth-based image enhancement and depth-based video enhancement, the device comprising:

a depth estimation unit that generates three-dimensional (3D) depth information from a two-dimensional (2D) image;

an image enhancement unit that enhances the 2D image according to the 3D depth information, wherein the image enhancement unit comprises one or more of the following elements;

a depth-based edge enhancement unit that emphasizes edge details of the 2D image;

a color space converter that converts the 2D image between color spaces to obtain a luma component and a chroma component of the 2D image;

a depth-based contrast adjustment unit that enhances contrast characteristics of the luma component;

a depth-based saturation adjustment unit that enhances saturation characteristics of the chroma component; and

a color space back-converter that converts the adjusted luma and chroma components back to a color space of the original 2D image, wherein

the color space converter converts the 2D image from red/green/blue (RGB) color space to hue/saturation/intensity (HSI) color space, and wherein

the color space back-converter converts the enhanced 2D image from the HSI color space back to the RGB color space.

2. The device of claim 1, wherein the image enhancement unit enhances objects in the 2D image having smaller depth value with larger enhancing weighting, and enhances objects in the 2D image having larger depth value with smaller enhancing weighting.

3. The device of claim 1, wherein the image enhancement unit enhances one or more of edge, contrast and saturation characteristics.

4. The device of claim 1, wherein the depth-based edge enhancement unit comprises:

a spatial convolution mask that is applied on a group of pixels of the 2D image surrounding a center pixel.

5. The device of claim 1, wherein the depth-based contrast adjustment unit comprises:

histogram stretching means for increasing a value of pixels having a gray level greater than a threshold, while decreasing a value of pixels having a gray level smaller than the threshold.

6. The device of claim 1, further comprising an input device that receives the 2D image.

7. The device of claim 6, wherein the input device further stores the 2D image.

8. The device of claim 1, further comprising an output device that receives the enhanced 2D image.

9. The device of claim 8, wherein the output device further stores or displays the enhanced 2D image.

10. A method of using a device to perform one or more of depth-based image enhancement and depth-based video enhancement, the method comprising:

using a processor to perform the steps of;

generating three-dimensional (3D) depth information from a two-dimensional (2D) image;

enhancing the 2D image according to the 3D depth information, wherein the enhancing step comprises one or more of the following sub-steps;

enhancing edge details of the 2D image based on the 3D depth information;

converting the 2D image between color spaces to obtain a luma component and a chroma component of the 2D image;

adjusting contrast characteristics of the luma component based on the 3D depth information;

adjusting saturation characteristics of the chroma component based on the 3D depth information; and

converting the adjusted luma and chroma components back to a color space of the original 2D image;

wherein

the 2D image converting sub-step converts the 2D image from red/green/blue (RGB) color space to hue/saturation/intensity (HSI) color space, and wherein

the luma/chroma back-converting sub-step converts the enhanced 2D image from the HSI color space back to the RGB color space.

11. The method of claim 10, wherein the enhancing step is performed by enhancing objects in the 2D image having smaller depth value with larger enhancing weighting, and enhancing objects in the 2D image having larger depth value with smaller enhancing weighting.

12. The method of claim 10, wherein the enhancing step is performed to enhance one or more of edge, contrast and saturation characteristics.

13. The method of claim 10, wherein the edge-details enhancing sub-step comprises:

applying a spatial convolution mask on a group of pixels of the 2D image surrounding a center pixel.

14. The method of claim 10, wherein the contrast-characteristics adjusting sub-step comprises:

increasing a value of pixels having a gray level greater than a threshold, while decreasing a value of pixels having a gray level smaller than the threshold.

15. The method of claim 10, further comprising a step of receiving the 2D image.

16. The method of claim 15, further comprising a step of storing the 2D image.

17. The method of claim 10, further comprising a step of receiving the enhanced 2D image.

18. The method of claim 17, further comprising a step of storing or displaying the enhanced 2D image.