APPARATUS AND METHOD FOR PROCESSING COLOR IMAGE USING DEPTH IMAGE

• US 20140079288A1
• Filed: 08/28/2013
• Published: 03/20/2014
• Est. Priority Date: 09/20/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KIM, Sun Kwon, LEE, Seung Kyu, LEE, Hee Sae
• Granted Patent: US 9,542,728 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

An image processing apparatus and method using a depth image are provided. The image processing apparatus may include a region determination unit to determine a foreground region and a background region in a color image using a depth image, and a color compensation unit to compensate a color with respect to the foreground region and the background region.

Claims

1. An image processing apparatus comprising:

a region determination unit to determine a foreground region and a background region in a color image using a depth image; and

a color compensation unit to compensate a color with respect to the foreground region and the background region.

2. The image processing apparatus of claim 1, wherein the region determination unit determines the foreground region and the background region using a depth histogram related to a depth value of the depth image.

3. The image processing apparatus of claim 1, wherein the region determination unit determines a region having an approximate depth value of a person in the depth image to be the foreground region when the person is included in the color image.

4. The image processing apparatus of claim 1, wherein the region determination unit recognizes a face of a person included in the color image and determines a region having an approximate depth value of the face of the depth image to be the foreground region.

5. The image processing apparatus of claim 1, wherein the color compensation unit applies histogram equalization and white balance adjustment to the foreground region and/or the background region.

6. The image processing apparatus of claim 5, wherein the color compensation unit does not apply histogram equalization to the foreground region but applies histogram equalization to the background region differentially according to depth values of the background region.

7. The image processing apparatus of claim 5, wherein the color compensation unit compensates for color distortion of the background region using white balance extracted from the foreground region to which histogram equalization is not applied.

8. The image processing apparatus of claim 1, wherein the color compensation unit compensates the color by applying at least one of contrast, saturation, and hue to the foreground region and/or the background region.

9. The image processing apparatus of claim 8, wherein the color compensation unit increases contrast of the foreground region while reducing contrast of the background region.

10. The image processing apparatus of claim 8, wherein the color compensation unit maintains saturation of the foreground region while changing saturation of the background region.

11. The image processing apparatus of claim 8, wherein the color compensation unit compensates a hue of the background region with reference to a hue of the foreground region of which saturation is not changed.

12. An image processing apparatus comprising:

an image obtaining unit to obtain a color image and a depth image, in which an object is disposed in a focus region;

a boundary extraction unit to extract a boundary of the object in the depth image; and

a region blur unit to blur a region excluding the boundary of the object in the color image.

13. The image processing apparatus of claim 12, wherein the boundary extraction unit extracts the boundary of the object from the depth image using a depth value of the object disposed in the focus region.

36. The image processing apparatus of claim 12, further comprising:

a region determination unit to determine a foreground region and a background region in the color image using a depth image; and

a color compensation unit to compensate a color with respect to the foreground region and the background region.

14. An image processing apparatus comprising:

a movement information determination unit to determine movement information between depth images successively obtained for a predetermined time;

a region set unit to set a region including the movement information in the color image to be a focus region; and

a region blur unit to blur a region excluding the focus region in the color image.

37. The image processing apparatus of claim 14, further comprising:

a region determination unit to determine a foreground region and a background region in the successively obtained color images using successively obtained depth images; and

a color compensation unit to compensate a color with respect to the foreground region and the background region.

15. An image processing apparatus comprising:

an object recognition unit to recognize an object included in a focus region;

a region adjustment unit to adjust an object region using depth information according to movement information of the object; and

a region blur unit to blur a region excluding the adjusted object region in the color image.

16. The image processing apparatus of claim 15, wherein the region adjustment unit reduces size of the object region when the depth information according to the movement information increases.

17. The image processing apparatus of claim 15, wherein the region adjustment unit increases size of the object region when the depth information according to the movement information decreases.

38. The image processing apparatus of claim 15, further comprising:

a region determination unit to determine a foreground region and a background region in the color image using a depth image; and

a color compensation unit to compensate a color with respect to the foreground region and the background region.

18. An image processing method comprising:

determining a foreground region and a background region in a color image using a depth image; and

compensating a color with respect to the foreground region and the background region.

19. The image processing method of claim 18, wherein the determining comprises determining the foreground region and the background region in the color image using a depth histogram related to a depth value of the depth image.

20. The image processing method of claim 18, wherein the determining comprises determining a region having an approximate depth value of a person in the depth image to be the foreground region when the person is included in the color image.

21. The image processing method of claim 18, wherein the determining comprises recognizing a face of a person included in the color image and determining a region having an approximate depth value of the face of the depth image to be the foreground region.

22. The image processing method of claim 18, wherein the compensating comprises applying histogram equalization and white balance adjustment to the foreground region and/or the background region.

23. The image processing method of claim 22, wherein the compensating comprises not applying histogram equalization to the foreground region but applying histogram equalization to the background region differentially according to depth values of the background region.

24. The image processing method of claim 22, wherein the compensating comprises compensating for color distortion of the background region using white balance extracted from the foreground region to which histogram equalization is not applied.

25. The image processing method of claim 18, wherein the compensating comprises compensating the color by applying at least one of contrast, saturation, and hue to the foreground region and/or the background region.

26. The image processing method of claim 25, wherein the compensating comprises increasing contrast of the foreground region while reducing contrast of the background region.

27. The image processing method of claim 25, wherein the compensating comprises maintaining saturation of the foreground region while changing saturation of the background region.

28. The image processing method of claim 25, wherein the compensating comprises compensating a hue of the background region with reference to a hue of the foreground region of which saturation is not changed.

35. A non-transitory computer readable recording medium storing a program to cause a computer to implement the method of claim 18.

29. An image processing method comprising:

obtaining a color image and a depth image, in which an object is disposed in a focus region;

extracting a boundary of the object in the depth image; and

blurring a region excluding the boundary of the object in the color image.

30. The image processing method of claim 29, wherein the extracting comprises extracting the boundary of the object from the depth image using a depth value of the object disposed in the focus region.

39. The image processing method of claim 29, further comprising:

determining a foreground region and a background region in the color image using a depth image; and

compensating a color with respect to the foreground region and the background region.

31. An image processing method comprising:

determining movement information between depth images successively obtained for a predetermined time;

setting a region including the movement information in the color image to be a focus region; and

blurring a region excluding the focus region in the color image.

40. The image processing method of claim 31, further comprising:

determining a foreground region and a background region in the successively obtained color images using successively obtained depth images; and

compensating a color with respect to the foreground region and the background region.

32. An image processing method comprising:

recognizing an object included in a focus region;

adjusting an object region using depth information according to movement information of the object; and

blurring a region excluding the adjusted object region in the color image.

33. The image processing method of claim 32, wherein the adjusting comprises reducing size of the object region when the depth information according to the movement information increases.

34. The image processing method of claim 32, wherein the adjusting comprises increasing size of the object region when the depth information according to the movement information decreases.

41. The image processing method of claim 32, further comprising:

determining a foreground region and a background region in the color image using a depth image; and

compensating a color with respect to the foreground region and the background region.