Systems and Methods for Distinguishing Facial Features for Cosmetic Application

• US 20170169285A1
• Filed: 12/09/2016
• Published: 06/15/2017
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: Perfect Corp. (Cyberlink Corporation)
• Original Assignee: Perfect Corp. (Cyberlink Corporation)
• Inventors: Cheng, Chih-Yu, Chen, Wei-Ming
• Granted Patent: US 9,984,282 B2
• Time in Patent Office: Days
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A media editing device obtains a digital image, and a facial region of an individual in the digital image is identified by a facial region analyzer. A plurality of facial features of the facial region is identified by a facial feature identifier. A skin color of the facial region is determined, and eyebrow regions in the facial region are identified based on the facial features. Each of the eyebrow regions is partitioned into a plurality of segments, and a determination is made, for one or more of the segments, whether an area above or an area below the segment has a color matching the skin color. The segment is designated as a verified eyebrow region or as an overlapped region depending on whether the area above or the area below the segment has a color matching the skin color. A special effect is applied on the verified eyebrow region.

Claims

1. A method implemented in a media editing device, comprising:

obtaining a digital image depicting an individual with hair overlapping a portion of at least one eyebrow of the individual, the hair overlapping the portion of the at least one eyebrow at an angle;

identifying a facial region of an individual in the digital image by a facial region analyzer;

identifying a plurality of facial features of the facial region by a facial feature identifier;

determining a skin color of the facial region;

identifying eyebrow regions in the facial region based on the facial features;

partitioning each of the eyebrow regions into a plurality of segments;

determining, for one or more of the segments, whether an area above or an area below the segment has a color matching the skin color;

in response to determining that the area above or that the area below the segment has a color matching the skin color, designating the segment as a verified eyebrow region;

in response to determining that the area above or that the area below the segment does not have a color matching the skin color, designating the segment as an overlapped region; and

applying a special effect to the verified eyebrow region.

2. The method of claim 1, further comprising:

determining a boundary of a hair region based on at least two points where a color transitions from the skin color to a non-skin color above or below the verified eyebrow region; and

extending the boundary of the hair region based on an angle between the at least two points.

3. The method of claim 2, further comprising identifying pixels falling within the verified eyebrow regions based on the extended boundary of the hair region and based on the verified eyebrow regions, wherein the special effect is applied to the identified pixels.

4. The method of claim 1, wherein applying the special effect on the verified eyebrow region comprises applying the special effect only to the verified eyebrow regions and not to the overlapped region.

5. The method of claim 1, further comprising:

determining a boundary of a hair region based on at least two points where a color transitions from the skin color to a non-skin color above the verified eyebrow region; and

extending the boundary of the hair region based on an aggregation among the at least two points.

6. The method of claim 1, wherein a set of the facial features comprises a plurality of eyebrow feature points, wherein identifying eyebrow regions in the facial region comprises:

connecting each eyebrow feature point to determine the eyebrow region.

7. The method of claim 1, wherein partitioning each of the eyebrow regions into a plurality of segments comprises:

partitioning the eyebrow regions into at least one row of segments.

8. The method of claim 1, wherein partitioning each of the eyebrow regions into a plurality of segments comprises:

partitioning the eyebrow regions into multiple columns of segments.

9. A system, comprising:

a memory storing instructions; and

a processor comprising a facial region analyzer and a facial feature identifier, the processor coupled to the memory and configured by the instructions to at least;

obtain a digital image depicting an individual with hair overlapping a portion of at least one eyebrow of the individual, the hair overlapping the portion of the at least one eyebrow at an angle;

identify a facial region of an individual in the digital image by a facial region analyzer;

identify a plurality of facial features of the facial region by a facial feature identifier;

determine a skin color of the facial region;

identify eyebrow regions in the facial region based on the facial features;

partition each of the eyebrow regions into a plurality of segments;

determine, for one or more of the segments, whether an area above or an area below the segment has a color matching the skin color;

in response to determining that the area above or that the area below the segment has a color matching the skin color, designate the segment as a verified eyebrow region;

in response to determining that the area above or that the area below the segment does not have a color matching the skin color, designate the segment as an overlapped region; and

apply a special effect to the verified eyebrow region.

10. The system of claim 9, wherein the processor is further configured by the instructions to:

determine a boundary of a hair region based on at least two points where a color transitions from the skin color to a non-skin color above the verified eyebrow region; and

extend the boundary of the hair region based on the angle between the least two points.

11. The system of claim 10, wherein the processor is further configured by the instructions to identify pixels falling within the verified eyebrow regions based on the extended boundary of the hair region and based on the verified eyebrow regions, wherein the special effect is applied to the identified pixels.

12. The system of claim 9, wherein a set of the facial features comprises a plurality of eyebrow feature points, and wherein the processor is configured by the instructions to identify eyebrow regions in the facial region by:

connecting each facial feature to determine the eyebrow region.

13. The system of claim 9, wherein the processor is configured by the instructions to partition each of the eyebrow regions into a plurality of segments by partitioning the eyebrow regions into at least one row of segments.

14. The system of claim 9, wherein the processor is configured by the instructions to partition each of the eyebrow regions into a plurality of segments by partitioning the eyebrow regions into multiple columns of segments.

15. A non-transitory computer-readable medium embodying a program executable in a computing device, comprising:

code for obtaining a digital image depicting an individual with hair overlapping a portion of at least one eyebrow of the individual, the hair overlapping the portion of the at least one eyebrow at an angle;

code for identifying a facial region of an individual in the digital image by a facial region analyzer;

code for identifying a plurality of facial features of the facial region by a facial feature identifier;

code for determining a skin color of the facial region;

code for identifying eyebrow regions in the facial region based on the facial features;

code for partitioning each of the eyebrow regions into a plurality of segments;

code for determining, for one or more of the segments, whether an area above or an area below the segment has a color matching the skin color;

code for designating the segment as a verified eyebrow region in response to determining that the area above or that the area below the segment has a color matching the skin color;

code for designating the segment as an overlapped region in response to determining that the area above or that the area below the segment does not have a color matching the skin color; and

code for applying a special effect to the verified eyebrow region.

16. The non-transitory computer-readable medium of claim 15, further comprising:

code for determining a boundary of a hair region based on at least two points where a color transitions from the skin color to a non-skin color above the verified eyebrow region; and

code for extending the boundary of the hair region based on the angle between the least two points.

17. The non-transitory computer-readable medium of claim 16, further comprising code for identifying pixels falling within the verified eyebrow regions based on the extended boundary of the hair region and based on the verified eyebrow regions, wherein the special effect is applied to the identified pixels.

18. The non-transitory computer-readable medium of claim 16, wherein the code for applying the special effect based on boundaries of the segments applies the special effect only to the eyebrow regions and not to the hair region.

19. The non-transitory computer-readable medium of claim 15, wherein a set of the facial features comprises a plurality of eyebrow feature points, and wherein the code for identifying eyebrow regions in the facial region connects each facial feature to determine the eyebrow region.

20. The non-transitory computer-readable medium of claim 15, wherein the code for partitioning each of the eyebrow regions into a plurality of segments partitions the eyebrow regions into at least one row of segments.

21. The non-transitory computer-readable medium of claim 15, wherein the code for partitioning each of the eyebrow regions into a plurality of segments partitions the eyebrow regions into multiple columns of segments.

22. A method implemented in a media editing device, comprising:

obtaining a digital image depicting an individual with hair overlapping a portion of at least one eyebrow of the individual, the hair overlapping the portion of the at least one eyebrow at an angle;

identifying a facial region of an individual in the digital image by a facial region analyzer;

identifying a plurality of facial features of the facial region by a facial feature identifier;

determining a hair color of the facial region;

identifying eyebrow regions in the facial region based on the facial features;

partitioning each of the eyebrow regions into a plurality of segments;

determining, for one or more of the segments, whether an area above or an area below the segment has a color matching the hair color;

in response to determining that the area above or that the area below the segment has a color matching the hair color, designating the segment as an overlapped region;

in response to determining that the area above or that the area below the segment does not have a color matching the hair color, designating the segment as a verified eyebrow region; and

applying a special effect to the verified eyebrow region.

23. The method of claim 22, further comprising:

determining a boundary of a hair region based on at least two points where a color transitions from a non-hair color to the hair color above or below the verified eyebrow region; and

extending the boundary of the hair region based on an angle between the at least two points.