Segmenting human hairs and faces

• US 8,638,993 B2
• Filed: 04/05/2011
• Issued: 01/28/2014
• Est. Priority Date: 04/05/2010
• Status: Expired due to Fees

• Current Assignee: Flashfoto, Inc.
• Original Assignee: Flashfoto, Inc.
• Inventors: Piramuthu, Robinson, Ip, Katharine, Prochazka, Daniel, Lee, Kuang-chih
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US13/066,112
• Publication Number:
  US 20110299776A1
• Time in Patent Office: 1,029 Days
• Field of Search: 382/118, 382115-117, 382162-167
• US Class Current: 382/118
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

Systems for segmenting human hairs and faces in color images are disclosed, with methods and processes for making and using the same. The image may be cropped around the face area and roughly centered. Optionally, the illumination environment of the input image may be determined. If the image is taken under dark environment or the contrast between the face and hair regions and background is low, an extra image enhancement may be applied. Sub-processes for identifying the pose angle and chin contours may be performed. A preliminary mask for the face by using multiple cues, such as skin color, pose angle, face shape and contour information can be represented. An initial hair mask by using the abovementioned multiple cues plus texture and hair shape information may be created. The preliminary face and hair masks are globally refined using multiple techniques.

Claims

1. A method of identifying a facial and hair portions of a desired image comprising:

receiving the desired image;

receiving a first plurality of control data points associated with a facial feature of a person from a memory component, wherein the first plurality of control data points is based on a first plurality of training data point sets, wherein each training data point set of the first plurality of training data point sets is associated with a facial feature of a training image of a first plurality of training images;

applying the first plurality of control data points to the desired image, wherein the facial portion of the desired image is substantially a match between the first plurality of control data points and the desired image;

receiving a second plurality of control data points associated with a hair feature of a person from the memory component, wherein the second plurality of control data points is based on a second plurality of training data point sets, wherein each training data point set of the second plurality of training data point sets is associated with a hair feature of a training image of a second plurality of training images, and wherein the second plurality of control data points is based on averages of training data point sets of the second plurality of training data points;

applying the second plurality of control data points to the desired image to determine whether the desired image includes a hair portion; and

identifying the hair portion associated with the desired image in response to determining that the desired image includes the hair portion, wherein the identifying is based on a location, width, or forehead associated with the hair portion of the desired image.

2. The method as described by claim 1, wherein the first plurality of control data points is an average between training data point sets of the first plurality of training data point sets.

3. The method as described by claim 1 further comprising:

enhancing image quality of the facial portion of the desired image.

4. The method as described by claim 1 further comprising:

iteratively applying the first plurality of control data points to the desired image based on intensity profile of the desired image to identify the facial portion of the desired image.

5. The method as described by claim 1 further comprising:

identifying a pose angle associated with the facial portion of the desired image based on eye location, center of a face, width of the face, and a pitch angle associated with the pose angle.

6. The method as described by claim 1, wherein the first plurality of control data points is associated with a color of the facial portion of the desired image.

7. A method of identifying a chin and hair portions of a desired image comprising:

receiving the desired image;

receiving a first plurality of control data points from a memory component, wherein the first plurality of control data points is associated with a chin feature of a person that is substantially invariant from one person to another person, and wherein the first plurality of control data points is based on a first plurality of training data point sets, and wherein each training data point set of the first plurality of training data point sets is associated with a chin feature of the training image of a first plurality of training images;

applying the first plurality of control data points to the desired image, wherein the chin portion of the desired image is substantially a match between the first plurality of control data points and the desired image;

receiving a second plurality of control data points associated with a hair feature of a person from the memory component, wherein the second plurality of control data points is based on a second plurality of training data point sets, wherein each training data point set of the second plurality of training data point sets is associated with a hair feature of a training image of a second plurality of training images, and wherein the second plurality of control data points is based on averages of training data point sets of the second plurality of training data points;

applying the second plurality of control data points to the desired image to determine whether the desired image includes a hair portion; and

identifying the hair portion associated with the desired image in response to determining that the desired image includes the hair portion, wherein the identifying is based on a location, width, or forehead associated with the hair portion of the desired image.

8. The method as described by claim 7 further comprising:

enhancing image quality of the chin portion of the desired image.

9. The method as described by claim 7 further comprising:

iteratively applying the first plurality of control data points to the desired image based on intensity profile of the desired image to identify the chin portion of the desired image.

10. The method as described by claim 7 further comprising:

identifying a pose angle associated with the chin portion of the desired image based on eye location, center of a face, width of the face, and a pitch angle associated with the pose angle.

11. The method as described by claim 7, wherein the first plurality of control data points of the chin portion of the person has a higher contrast profile with portions outside of the object chin portion in comparison to areas within the chin portion of the person.

12. A non-transitory computer readable storage medium having computer-readable program code stored thereon for causing a processor to implement a method of identifying a facial and hair portions of a desired image comprising:

receiving the desired image;

receiving a first plurality of control data points associated with a facial feature of a person from a memory component, wherein the first plurality of control data points is based on a first plurality of training data point sets, wherein each training data point set of the first plurality of training data point sets is associated with a facial feature of a training image of a first plurality of training images;

applying the first plurality of control data points to the desired image, wherein the facial portion of the desired image is substantially a match between the first plurality of control data points and the desired image;

receiving a second plurality of control points associated with a hair feature of a person from the memory component, wherein the second plurality of control data points is based on a second plurality of training data point sets, wherein each training data point set of the second plurality of training data point sets is associated with a hair feature of a training image of a second plurality of training images, and wherein the second plurality of control data points is based on averages of training data points of the second plurality of training data points;

applying the second plurality of control data points to the desired image to determine whether the desired image includes a hair portion; and

identifying the hair portion associated with the desired image in response to determining that the desired image includes the hair portion, wherein the identifying is based on a location, width, or forehead associated with the hair portion of the desired image.

13. The non-transitory computer readable storage medium as described by claim 12, wherein the first plurality of control data points is an average between training data point sets of the first plurality of training data point sets.

14. The non-transitory computer readable storage medium as described by claim 12 further comprising:

iteratively applying the first plurality of control data points to the desired image based on intensity profile of the desired image to identify the facial portion of the desired image.

15. The non-transitory computer readable storage medium as described by claim 12 further comprising:

identifying a pose angle associated with the facial portion of the desired image based on eye location, center of a face, width of the face, and a pitch angle associated with the pose angle.