Color balance correction based on skin color and highlight color

• US 9,002,107 B2
• Filed: 12/08/2010
• Issued: 04/07/2015
• Est. Priority Date: 07/16/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Nakagawa, Junichi, Sakai, Hiroyuki, Umeda, Kiyoshi
• Primary Examiner(s): Liu, Li
• Application Number:
  US12/962,895
• Publication Number:
  US 20120014598A1
• Time in Patent Office: 1,581 Days
• Field of Search: None
• US Class Current: 382/167
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77

Abstract

An image processing apparatus for applying a color balance correction to input image data, comprises a detection unit which detects an object included in the image data; an object color calculation unit which calculates an object color as a color of the object; an acquisition unit which acquires a highlight color in the image data; a calculation unit which calculates a correction value for the object color and the highlight color based on a relative positional relationship between the object color and the highlight color on a color space; and a color balance correction unit which applies the color balance correction to the highlight color and the object color using the correction value.

Claims

1. An image processing apparatus comprising:

a first acquisition unit which acquires a color of an object in image data;

a second acquisition unit which acquires a highlight color in the image data based on an analysis result of luminance components in the image data; and

a correction unit which corrects, in a case where a distance between the highlight color and the color of the object is equal to or larger than a predetermined distance on a color plane, the highlight color in the image data so that a correction amount of the highlight color becomes larger as the distance becomes longer.

2. The apparatus according to claim 1 further comprising:

a color balance correction unit which applies a color balance correction to the image data using the highlight color corrected by the correction unit.

3. A non-transitory computer-readable medium storing a program for controlling a computer to function as an image processing apparatus according to claims 1.

4. The apparatus according to claim 1, further comprising:

an estimation unit which estimates a light environment at the time of shooting of the image data based on the highlight color corrected by the correction unit.

5. The apparatus according to claim 1, wherein

the correction unit corrects the highlight color so that a relative position between the highlight color and the color of the object comes close to a relative position between a reference highlight color and a reference object color on the color space.

6. The apparatus according to claim 5, wherein

the reference highlight color and the reference object color are a highlight color and a color of an object under a predetermined light source.

7. The apparatus according to claim 1, wherein the color of the object is a skin color of a face in the image data.

8. The apparatus according to claim 1, wherein the color plane is a Cb-Cr plane of a YCbCr space.

9. An image processing method comprising:

a first acquisition step of acquiring a color of an object in image data;

a second acquisition step of acquiring a highlight color in the image data based on an analysis result of luminance components in the image data; and

a correction step of correcting, in a case where a distance between the highlight color and the color of the object is equal to or larger than a predetermined distance on a color plane, the highlight color in the image data so that a correction amount of the highlight color becomes larger as the distance becomes longer.

10. An image processing apparatus comprising:

a calculation unit which calculates an object color of an object included in the image data;

a detection unit which detects a highlight color of the image data;

a reliability calculation unit which calculates a reliability for a value of the object color calculated by said calculation unit and a reliability for a value of the highlight color acquired by said acquisition unit based on a relative positional relationship between the object color and the highlight color on a color space, and differences between the object color and the highlight color, and reference values; and

a color balance correction unit which applies the color balance correction to the object color and the highlight color by controlling a correction amount for the object color and the highlight color using the reliability for the value of the object color and the reliability for the value of the highlight color.

11. The apparatus according to claim 10, wherein said color balance correction unit further comprises:

a light source estimate unit which estimates a color temperature of a light source at the time of shooting of the image data; and

a color conversion unit which executes a color conversion of the image data based on the color temperature estimated by said light source estimation unit.

12. The apparatus according to claim 11, wherein said light source estimation unit further comprises a modification unit which modifies a highlight color of the image data at the estimated color temperature based on the reliability for the value of the object color and the reliability for the value of the highlight color.

13. The apparatus according to claim 11, wherein said color conversion unit suppresses a color conversion ratio in the color conversion as the reliabilities are lower.

14. The apparatus according to claim 11, wherein said light source estimation unit further comprises:

a setting unit which sets a gray axis on the color space based on the highlight color acquired by said acquisition unit; and

an estimation unit which executes a light source estimation using pixels adjacent to the gray axis and pixels adjacent to the object color.

15. The apparatus according to claim 10, wherein said calculation unit further comprises a face detection unit which calculates, as the object color, a skin color of a human face from the image data.

16. The apparatus according to claim 15, wherein said reliability calculation unit calculates the reliability for the value of the object color and the reliability for the value of the highlight color using a face detection reliability obtained by said face detection unit.

17. An image processing method comprising:

a calculation step of calculating an object color of an object included in image data;

a detection step of detecting a highlight color of the image data;

a reliability calculation step of calculating a reliability for a value of the object color calculated in the calculation step and a reliability for a value of the highlight color acquired in the acquisition step based on a relative positional relationship between the object color and the highlight color on a color space, and differences between the object color and the highlight color, and reference values; and

a color balance correction step of applying the color balance correction to the object color and the highlight color by controlling a correction amount for the object color and the highlight color using the reliability for the value of the object color and the reliability for the value of the highlight color.