Techniques for image segmentation

• US 9,665,945 B2
• Filed: 03/26/2015
• Issued: 05/30/2017
• Est. Priority Date: 07/23/2014
• Status: Active Grant

• Current Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Original Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Inventors: Wang, Lin, Chen, Zhijun, Xu, Xiaozhou
• Primary Examiner(s): MOYER, ANDREW M
• Application Number:
  US14/669,888
• Publication Number:
  US 20160027187A1
• Time in Patent Office: 796 Days
• Field of Search: 382164, 382171, 382173
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Techniques for image segmentation can include receiving image data of an image including a background and a face of a person in a foreground, and determining a respective a priori probability of a head-shoulder foreground pixel appearing per pixel of the plurality of pixels, according to a positioning result of a plurality of exterior contour points of the face. The techniques can also include selecting foreground and background pixels of the plurality of pixels, according to at least the a priori probabilities, and determining respective color likelihood probabilities of the foreground and the background, according to color feature vectors of the selected pixels. The techniques can also include determining respective posteriori probabilities of at least part of the foreground and at least part of the background, according to the a priori probabilities and the respective color likelihood probabilities. The techniques can also include performing segmentation on the plurality of pixels, according to the respective posteriori probabilities.

Claims

1. A method for segmenting an image, comprising:

receiving image data of an image including a background and a face of a person in a foreground, the image data including data representative of a plurality of pixels of the image and a positioning result of a plurality of exterior contour points of the face;

determining a respective a priori probability of a foreground pixel appearing per pixel of the plurality of pixels, according to the positioning result of the plurality of exterior contour points of the face and a sex characteristic of the face;

selecting foreground pixels and background pixels of the plurality of pixels, according to the a priori probabilities, a foreground probability threshold, and a background probability threshold;

determining a first color likelihood probability of the foreground and a second color likelihood probability of the background, according to color feature vectors of the selected foreground pixels and the selected background pixels;

determining a first a posteriori probability of at least part of the foreground and a second a posteriori probability of at least part of the background, according to the a priori probabilities, the first color likelihood probability and the second color likelihood probability; and

performing segmentation on the plurality of pixels, according to the first a posteriori probability and the second a posteriori probability.

2. The method according to claim 1, wherein the determining of a respective a priori probability of a foreground pixel appearing per pixel of the plurality of pixels is also according to an aggregation of corresponding pixels from a group of images including faces of people, the aggregation including:

selecting a first group of images including faces of people;

generating a calibrated image for each image of the group of images, each calibrated image based on a foreground calibration result of a corresponding image of the group of images;

positioning exterior contour characteristic points of a face for each image of the group of images to obtain a positioning result per image of the group of images;

performing alignment and size normalization on each image of the calibrated images, according to the corresponding positioning results to generate a second group of images having a set alignment, size, or both; and

determining the a priori probability of a foreground pixel appearing for a pixel at a same position in each of the second group of images;

according to the foreground calibration result.

3. The method according to claim 2, wherein the group of images including faces of people, only includes images of female faces.

4. The method according to claim 2, wherein the group of images including faces of people, only includes images of male faces.

5. The method according to claim 1, further comprising:

identifying the sex characteristic of the face in the foreground.

6. The method according to claim 1, further comprising:

identifying the sex characteristic of the face in the foreground; and

further selecting one or more of the foreground pixels and the background pixels, according the sex characteristic of the face in the foreground.

7. The method according to claim 1, further comprising:

identifying the sex characteristic of the face in the foreground; and

setting one or more of the foreground probability threshold and the background probability threshold;

according to the sex characteristic of the face in the foreground.

8. The method according to claim 1, wherein the determining of a respective a priori probability of a foreground pixel appearing per pixel of the plurality of pixels is according to the following formula:

9. The method according to claim 8, wherein I_(xi,yi)^j=0 denotes that the pixel (x_i,y_i) in the j^th image is calibrated as a background.

10. The method according to claim 1, wherein the selecting foreground pixels and the background pixels of the plurality of pixels, according to the a priori probabilities, the foreground probability threshold, and the background probability threshold, includes:

identifying and selecting a foreground pixel where a respective a priori probability of a given pixel of the plurality of pixels is greater than the foreground probability threshold; and

identifying and selecting a background pixel where a respective a priori probability of the given pixel of the plurality of pixels is less than the background probability threshold, wherein the background probability threshold is less than the foreground probability threshold.

11. The method according to claim 1, wherein the determining a first color likelihood probability of the foreground and a second color likelihood probability of the background, according to color feature vectors of the selected foreground pixels and the selected background pixels, is according to the following formulas:

12. The method according to claim 11, wherein β

_F and β

_B are defined by the following formulas;

13. The method according to claim 1, wherein the determining of the first a posteriori probability and the second a posteriori probability, is further according to the following formulas:

P(I_(xi,yi)=1|(R,G,B))∝

P(I_(xi,yi)=1)*P((R,G,B)|I_(xi,yi)=1), and

P(I_(xi,yi)=0|(R,G,B))∝

(1−

P(I_(xi,yi)=1))*P((R,G,B)|I_(xi,yi)=0),

wherein P(I_(xi,yi)=1|(R,G,B)) denotes the a posteriori probability of the foreground at a pixel,

wherein P(I_(xi,yi)=0|(R,G,B)) denotes the a posteriori probability of the background at a pixel (x_i,y_i),

wherein P(I_(xi,yi)=1) denotes an a priori probability of a foreground pixel appearing at a pixel (x_i,y_i),

wherein P((R,G,B)|I_(xi,yi)=1) denotes the first color likelihood probability, and

wherein P((R,G,B)|I_(xi,yi)=0) denotes the second color likelihood probability.

14. A system for segmenting an image, comprising:

receiver circuitry, configured to receive image data of an image including a background and a face of a person in a foreground, the image data including data representative of a plurality of pixels of the image and a positioning result of a plurality of exterior contour points of the face;

image processing circuitry communicatively coupled to the receiver circuitry, including;

sub-circuitry configured to determine a respective a priori probability of a head-shoulder foreground pixel appearing per pixel of the plurality of pixels, according to the positioning result of the plurality of exterior contour points of the face and a sex characteristic of the face;

sub-circuitry configured to select foreground pixels and background pixels of the plurality of pixels, according to the a priori probabilities, a foreground probability threshold, and a background probability threshold;

sub-circuitry configured to determine a first color likelihood probability of the foreground and a second color likelihood probability of the background, according to color feature vectors of the selected foreground pixels and the selected background pixels;

sub-circuitry configured to determine a first a posteriori probability of at least part of the foreground and a second a posteriori probability of at least part of the background, according to the a priori probabilities, the first color likelihood probability and the second color likelihood probability; and

sub-circuitry configured to perform segmentation on the plurality of pixels, according to the first a posteriori probability and the second a posteriori probability; and

output circuitry configured to communicate the image with the segmented plurality of pixels to display circuitry.

15. The system according to claim 14, wherein the sub-circuitry configured to determine the respective a priori probability is further configured to determine the respective a priori probability, according an aggregation of corresponding pixels from a group of images including faces of people;

the aggregation performed by sub-circuitry of the image processing circuitry, configured to;

select a first group of images including faces of people;

generate a calibrated image for each image of the group of images, each calibrated image based on a foreground calibration result of a corresponding image of the group of images;

position exterior contour characteristic points of a face for each image of the group of images to obtain a positioning result per image of the group of images;

perform alignment and size normalization on each image of the calibrated images, according to the positioning result to generate a second group of images with each image having a set alignment, size, or both; and

determine the a priori probability of a foreground pixel appearing for a pixel at a same position in each of the second group of images;

according to the foreground calibration result.

16. The system according to claim 15, wherein the group of images including faces of people, only includes images of female faces.

17. The system according to claim 15, wherein the group of images including faces of people, only includes images of male faces.

18. A non-transitory computer readable medium, comprising:

instructions executable by a processor to receive image data of an image including a background and a face of a person in a foreground, the image data including data representative of a plurality of pixels of the image and a positioning result of a plurality of exterior contour points of the face;

instructions executable by a processor to identify a sex characteristic of the face in the foreground;

instructions executable by a processor to determine a respective a priori probability of a foreground pixel appearing per pixel of the plurality of pixels, according to the positioning result of the plurality of exterior contour points of the face and the sex characteristic of the face;

instructions executable by a processor to select foreground pixels and background pixels of the plurality of pixels, according to the a priori probabilities, a foreground probability threshold, and a background probability threshold;

instructions executable by a processor to determine a first color likelihood probability of the foreground and a second color likelihood probability of the background, according to color feature vectors of the selected foreground pixels and the selected background pixels;

instructions executable by a processor to determine a first a posteriori probability of at leak part of the foreground and a second a posteriori probability of at least part of the background, according to the a priori probabilities, the first color likelihood probability and the second color likelihood probability; and

instructions executable by a processor to perform segmentation on the plurality of pixels, according to the first a posteriori probability and the second a posteriori probability.

19. The non-transitory computer readable medium according to claim 18, further comprising instructions executable to determine the first color likelihood probability of the foreground and the second color likelihood probability of the background, according to the color feature vectors of the selected foreground pixels and the selected background pixels, according to the following formulas:

20. The non-transitory computer readable medium according to claim 18, further comprising instructions executable to determine the first a posteriori probability and the second a posteriori probability, according to the a priori probabilities, the first color likelihood probability, and the second color likelihood probability and according to the following formulas:

P(I_(xi,yi)=1|(R,G,B))∝

P(I_(xi,yi)=1)*P((R,G,B)|I_(xi,yi)=1), and

P(I_(xi,yi)=0|(R,G,B))∝

(1−

P(I_(xi,yi)=1))*P((R,G,B)|I_(xi,yi)=0),

wherein P(I_(xi,yi)=1|(R,G,B)) denotes the a posteriori probability of the foreground at a pixel,

wherein P(I_(xi,yi)=0|(R,G,B)) denotes the a posteriori probability of the background at a pixel (x_i,y_i),

wherein P(I_(xi,yi)=1) denotes an a priori probability of a foreground pixel appearing at a pixel (x_i,y_i),

wherein P((R,G,B)|I_(xi,yi)=1) denotes the first color likelihood probability, and

wherein P((R,G,B)|I_(xi,yi)=0) denotes the second color likelihood probability.