Face region estimating device, face region estimating method, and face region estimating program

• US 20060126941A1
• Filed: 12/13/2005
• Published: 06/15/2006
• Est. Priority Date: 12/14/2004
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Higaki, Nobuo
• Granted Patent: US 7,756,299 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

The invention provides a face region estimating device capable of extracting a person'"'"'s face region even when an illumination environment, for example, is changed and persons with different skin colors are contained in a picked-up image taken by a camera. The face region estimating device includes: a mobile body detecting unit for extracting the outline of a mobile body from the picked-up image and for generating an object distance image composed of pixels corresponding to a distance in which the mobile body exists; a head region setting unit for setting a head region based on the pixel distribution in the object distance image corresponding to a predetermined range on the basis of a pixel position at the upper part of the outline; and a face region extracting unit for extracting the person'"'"'s face region from the picked-up image based on the color distribution in the picked-up image corresponding to the head region.

Claims

1. A face region estimating device for estimating a person'"'"'s face region contained in a plurality of picked-up images from the picked-up images taken of a mobile body by a plurality of cameras, comprising;

a distance information generating unit for generating a distance to the mobile body as distance information, based on an azimuth difference of the plurality of picked-up images;

a motion information generating unit for generating motion of the mobile body as motion information, based on a difference of the picked-up images inputted in time series from at least one of the plurality of cameras;

an object distance setting unit for setting an object distance in which the mobile body exists, based on the distance information and the motion information;

an object distance image generating unit for generating an object distance image composed of pixels corresponding to the object distance, based on the distance information;

an object region setting unit for setting an object region containing the region of the mobile body, corresponding to at least the object distance set by the object distance setting unit in the object distance image;

an outline extracting unit for extracting an outline from the object region set by the object region setting unit;

a head region setting unit for setting a head region assuming the mobile body to be a person, based on a pixel distribution of the object distance image corresponding to a predetermined range on the basis of a pixel position at the upper part of the outline extracted by the outline extracting unit; and

a face region extracting unit for extracting the person'"'"'s face region from the picked-up image, based on the color distribution of the picked-up image corresponding to the head region set by the head region setting unit.

2. The face region estimating device as claimed in claim 1, further comprising an edge image generating unit for generating an edge image obtained by extracting an edge of the picked-up image based on color information or shading information of each pixel of the picked-up image, wherein the distance image generating unit extracts the pixel of the edge image corresponding to the object distance based on the distance information and generates the object distance image.

3. The face region estimating device as claimed in claim 1, wherein the head region setting unit totals the pixels in a horizontal direction for every coordinate in a perpendicular direction in the predetermined range based on the pixel position at the upper part of the outline in the object distance image, and sets a range of the perpendicular direction of the head region based on the number of pixels in the horizontal direction.

4. The face region estimating device as claimed in claim 2, wherein the head region setting unit totals the pixels in a horizontal direction for every coordinate in a perpendicular direction in the predetermined range based on the pixel position at the upper part of the outline in the object distance image, and sets a range of the perpendicular direction of the head region based on the number of pixels in the horizontal direction.

5. The face region estimating device as claimed in claim 1, wherein the face region extracting unit includes an HLS threshold value setting unit for setting upper and lower limit values of hue, saturation, and lightness ranges in which the total of the pixels corresponding to each of the values is the largest in the predetermined range for every value of hue, saturation and lightness value in the head region, as hue, saturation, and lightness threshold values, respectively, and wherein a region composed of the pixels contained in the ranges of the hue, saturation and lightness threshold values is provided as the face region.

6. The face region estimating device as claimed in claim 2, wherein the face region extracting unit includes an HLS threshold value setting unit for setting upper and lower limit values of hue, saturation, and lightness ranges in which the total of the pixels corresponding to each of the values is the largest in the predetermined range for every value of hue, saturation and lightness value in the head region, as hue, saturation, and lightness threshold values, respectively, and wherein a region composed of the pixels contained in the ranges of the hue, saturation and lightness threshold values is provided as the face region.

7. The face region estimating device as claimed in claim 3, wherein the face region extracting unit includes an HLS threshold value setting unit for setting upper and lower limit values of hue, saturation, and lightness ranges in which the total of the pixels corresponding to each of the values is the largest in the predetermined range for every value of hue, saturation and lightness value in the head region, as hue, saturation, and lightness threshold values, respectively, and wherein a region composed of the pixels contained in the ranges of the hue, saturation and lightness threshold values is provided as the face region.

8. The face region estimating device as claimed in claim 4, wherein the face region extracting unit includes an HLS threshold value setting unit for setting upper and lower limit values of hue, saturation, and lightness ranges in which the total of the pixels corresponding to each of the values is the largest in the predetermined range for every value of hue, saturation and lightness value in the head region, as hue, saturation, and lightness threshold values, respectively, and wherein a region composed of the pixels contained in the ranges of the hue, saturation and lightness threshold values is provided as the face region.

9. The face region estimating device as claimed in claim 5, wherein the face region extracting unit includes an RGB threshold value setting unit for setting the upper and lower limit values of red, green, and blue ranges in which the total of the pixels corresponding to the concentration value is the largest in the predetermined range for every concentration value of red, green and blue, as three primary colors of light in the head region as red, green, and blue threshold values, respectively, and wherein the region composed of the pixels contained in the ranges of the red, green, and blue threshold values is set to the face region when the saturation threshold value is or below a predetermined value.

10. The face region estimating device as claimed in claim 6, wherein the face region extracting unit includes an RGB threshold value setting unit for setting the upper and lower limit values of red, green, and blue ranges in which the total of the pixels corresponding to the concentration value is the largest in the predetermined range for every concentration value of red, green and blue, as three primary colors of light in the head region as red, green, and blue threshold values, respectively, and wherein the region composed of the pixels contained in the ranges of the red, green, and blue threshold values is set to the face region when the saturation threshold value is or below a predetermined value.

11. The face region estimating device as claimed in claim 7, wherein the face region extracting unit includes an RGB threshold value setting unit for setting the upper and lower limit values of red, green, and blue ranges in which the total of the pixels corresponding to the concentration value is the largest in the predetermined range for every concentration value of red, green and blue, as three primary colors of light in the head region as red, green, and blue threshold values, respectively, and wherein the region composed of the pixels contained in the ranges of the red, green, and blue threshold values is set to the face region when the saturation threshold value is or below a predetermined value.

12. The face region estimating device as claimed in claim 8, wherein the face region extracting unit includes an RGB threshold value setting unit for setting the upper and lower limit values of red, green, and blue ranges in which the total of the pixels corresponding to the concentration value is the largest in the predetermined range for every concentration value of red, green and blue, as three primary colors of light in the head region as red, green, and blue threshold values, respectively, and wherein the region composed of the pixels contained in the ranges of the red, green, and blue threshold values is set to the face region when the saturation threshold value is or below a predetermined value.

13. A face region estimating method for estimating a person'"'"'s face region contained in a plurality of picked-up images from the picked-up images taken of a mobile body by a plurality of cameras, comprising;

generating a distance to the mobile body as distance information based on an azimuth difference of the plurality of picked-up images;

generating motion of the mobile body as motion information based on the difference of the picked-up images inputted in time series from at least one of the plurality of cameras;

setting an object distance in which the mobile body exists based on the distance information and the motion information;

generating an object distance image composed of pixels corresponding to the object distance based on the distance information;

setting an object region containing the region of the mobile body, corresponding to at least the object distance set by the object distance setting unit in the object distance image;

extracting an outline from the object region set by the setting of the object region;

setting a head region assuming the mobile body to be a person, based on the pixel distribution of the object distance image corresponding to a predetermined range on the basis of a pixel position at the upper part of the outline extracted by extracting of the outline; and

extracting the person'"'"'s face region from the picked-up image, based on the color distribution of the picked-up image corresponding to the head region set by the setting of the head region.

14. A program for allowing a computer to perform a method for estimating a person'"'"'s face region contained in a plurality of picked-up images from the picked-up images taken of a mobile body by a plurality of cameras, the method comprising:

generating a distance to the mobile body as distance information based on an azimuth difference of the plurality of picked-up images;

generating motion of the mobile body as motion information based on the difference of a picked-up images inputted in time series from at least one of the plurality of cameras;

setting an object distance in which the mobile body exists based on the distance information and the motion information;

generating an object distance image composed of pixels corresponding to the object distance based on the distance information;

setting an object region containing the region of the mobile body, corresponding to at least the object distance set by the object distance setting unit in the object distance image;

extracting an outline from the object region set by the setting of the object region;

setting a head region assuming the mobile body to be a person, based on the pixel distribution of the object distance image corresponding to a predetermined range on the basis of a pixel position at the upper part of the outline extracted by extracting of the outline; and

extracting the person'"'"'s face region from the picked-up image, based on the color distribution of the picked-up image corresponding to the head region set by the setting of the head region.