RULE-BASED SEGMENTATION FOR OBJECTS WITH FRONTAL VIEW IN COLOR IMAGES

• US 20140219560A1
• Filed: 02/18/2014
• Published: 08/07/2014
• Est. Priority Date: 12/14/2007
• Status: Active Grant

• Current Assignee: Flashfoto, Inc.
• Original Assignee: Flashfoto, Inc.
• Inventors: Robinson, Piramuthu
• Granted Patent: US 9,042,650 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A method of labeling pixels in an image in which pixels in the image that represent human skin of one or more people are detected and one or more regions in the image are identified, where each region in the one or more regions includes all or a portion of a human face of a person in the one or people in the image. Pixels that represent each face in the image are identified using the pixels that represent skin and the regions that include faces of the people, thereby identifying a position of each face in the image. From this, a face mask for each face and a rough body map corresponding to each face is determined using the positions of the identified faces. Further still, a torso map corresponding to each face is determined using determined face positions. Then, the extracted face masks and the torso maps are used to refine a skin map. A person or people map is determined using the skin map and the rough body map.

Claims

1. A method comprising:

retrieving an image of a person from a memory component;

forming a skin map associated with a skin region of the person in the image;

forming a face map of the person based on the skin map, wherein the face map is associated with a face region of the person in the image;

forming a body map of the person based on the face map, wherein the body map is associated with a body region of the person in the image; and

forming a person map based on the face map and the body map that s associated with person in the image.

2. The method of claim 1, wherein forming the skin map comprises:

determining whether a position of a subset region of the skin map is at an edge of the image; and

removing the subset region from the skin map in response to determining that the position of the subset region is positioned at the edge of the image.

3. The method of claim 1, wherein forming the face map comprises:

determining an initial face map for the face region of the person based on a position of a face of the person in the image; and

determining whether to modify the initial face map by expanding the initial face map to include additional regions from the image.

4. The method of claim 1, wherein forming the face map includes identifying a region in the image that overlaps with the skin map by a given amount, and wherein the identified region is the face region of the person in response to a determination that the overlap between the identified region and the skin map meets or exceeds the given amount.

5. The method of claim 1, wherein forming a face map comprises:

positioning the image in a first position to identify the face region in the image in absence of a face of the person being tilted in the image; and

in absence of identifying the face region in response to the positioning, rotating the image by a given increment to a second position to identify the face region in the image.

6. The method of claim 1, wherein forming the body map comprises:

determining an initial body map for the body region of the person based on a position of a body of the person relative to a position of a face of the person in the image; and

determining whether to modify the initial body map by expanding the initial body map to include additional regions from the image.

7. The method of claim 1, wherein forming the person map comprises:

combining the face map and the body map to form an initial person map;

determining regions in the image in close proximity to the body map or the face map in the image that have a color matching a color of the body map or a color of the face map; and

combining the determined regions in the image having the color matching the color of the body map or the color of the face map with the initial person map.

8. The method of claim 1 further comprising:

determining whether a subset region of the person map is isolated from other subset regions of the person map; and

removing the subset region in response to the determination that the subset region is isolated and further in response to determining that a size of the subset region is below a certain threshold in comparison to the other subset regions and further in response to determining that a distance between the subset region and the other subset regions exceeds a given value.

9. The method of claim 1 further comprising:

determining whether the person map includes a hole;

determining whether a size associated with the hole is less than or equal to a threshold size, wherein the threshold size is based in part on size and location of the face of the person in the image; and

filling in the hole with color in response to determining that size of the hole is less than or equal to the threshold size.

10. A non-transitory computer readable medium having stored thereon, computer executable instructions that, if executed by a device, causes the device to perform a method comprising:

forming a face map of a person associated with a face region of a person in an image;

forming a body map of the person based on the face map, wherein the body map is associated with a body region of the person in the image;

forming a person map based on the face map and the body map that are associated with the person in the image; and

refining the person map by removing one or more isolated subset regions from the person map.

11. The non-transitory computer-readable storage medium of claim 10, wherein forming the person map comprises:

combining the face map and the body map to form an initial person map;

determining regions in the image in close proximity to the body map or the face map in the image that have a color matching a color of the body map or a color of the face map; and

combining the determined regions in the image having the color matching the color of the body map or the color of the face map with the initial person map.

12. The non-transitory computer-readable storage medium of claim 10, wherein the one or more isolated subset regions of the person map is identified based on a determination that a size of the one or more isolated subset regions is below a certain threshold in comparison to other subset regions of the person map and further based on a determination that a distance between the one or more isolated subset regions and the other subset regions of the person map exceeds a given value.

13. The non-transitory computer-readable storage medium of claim 10, wherein refining the person map further comprises:

determining whether the person map includes a hole;

determining whether a size associated with the hole is less than or equal to a threshold size, wherein the threshold size is based in part on size and location of the face of the person in the image; and

filling in the hole with color in response to determining that the size of the hole is less than or equal to the threshold size.

14. The non-transitory computer-readable storage medium of claim 10, the method further comprising:

forming a skin map associated with a skin region of the person in the image;

determining whether a position of a subset region of the skin map is at an edge of the image; and

removing the subset region in response to determining that the position of the subset is positioned at the edge of the image,

wherein the person map is further based on the skin map.

15. A system comprising:

a memory; and

a processor configured to;

retrieve an image of a person from the memory;

form a face map associated with a face region of the person in the image;

form a body map of the person based on the face map, wherein the body map is associated with a body region of the person in the image;

form a person map based on the face map and the body map that are associated with the person in the image; and

refine the person map by filling in one or more holes in the person map with color.

16. The system of claim 15, wherein forming the person map further comprises:

combining the face map and the body map to form an initial person map;

determining regions in the image in close proximity to the body map or the face map in the image that have a color matching a color of the body map or a color of the face map; and

combining the determined regions in the image having the color matching the color of the body map or the color of the face map with the initial person map.

17. The system of claim 15, wherein the processor is further configured to:

determine whether a subset region of the person map is isolated from other subset regions of the person map; and

remove the subset region in response to the determination that the subset region is isolated and further in response to determining that a size of the subset region is below a certain threshold in comparison to the other subset regions and further in response to determining that a distance between the subset region and the other subset regions exceeds a given value.

18. The system of claim 15, the processor is further configured to:

prior to refining the person map, determine whether the person map includes one or more holes; and

determine whether a size associated with the one or more holes is less than or equal to a threshold size, wherein the threshold size is based in part on size and location of a face of the person in the image,

wherein filling in the one or more holes with color is in response to the determination that the size associated with the one or more holes is less than or equal to the threshold size.

19. The system of claim 15, the processor is further configured to:

form a skin map associated with a skin of the person in the image, and wherein the person map is further based on the skin map.

20. The system of claim 19, the processor is further configured to:

determine whether a position of a subset region of the skin map is at an edge of the image; and

remove the subset region in response to determining that the position of the subset is positioned at the edge of the image.