Methods and systems for generating a user-hair-color model

• US 9,607,397 B2
• Filed: 09/01/2015
• Issued: 03/28/2017
• Est. Priority Date: 09/01/2015
• Status: Active Grant

• Current Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Original Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Inventors: Lin, Dennis, Francisco, Glenn, Zhang, Yi, Dang, Gia, Venshtain, Simion, Nguyen, Quang, Dang, Long
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US14/842,525
• Publication Number:
  US 20170061640A1
• Time in Patent Office: 574 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

Disclosed herein are methods and systems for generating a user-hair-color model. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user. The process also includes determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user. The process also includes grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels. The three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set. The process also includes updating a user hair-color model based at least in part on the skin-pixel set.

Claims

1. A method comprising:

obtaining video data depicting a head of a user;

determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user;

grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels, wherein the three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set; and

updating a user hair-color model based at least in part on the hair-pixel set.

2. The method of claim 1, further comprising generating the upper contour of the depicted head of the user.

3. The method of claim 2, wherein generating the upper contour of the depicted head of the user comprises:

obtaining depth data that is associated with the video data;

identifying a region of the video data that corresponds with depth values that are within a threshold depth range; and

identifying an upper boundary of the identified region as being the upper contour of the depicted head of the user.

4. The method of claim 2, wherein generating the upper contour of the depicted head of the user comprises:

obtaining a head contour that estimates an outline of the depicted head of the user; and

identifying an upper boundary of the obtained head contour as being the upper contour of the depicted head of the user.

5. The method of claim 1, wherein determining the set of line segments of pixels of the video data comprises:

identifying a center of the depicted head of the user;

identifying at least one line segment of pixels of the video data that originates from the identified center of the depicted head of the user and intersects the upper contour; and

determining the identified at least one line segment of pixels of the video data to be in the set of line segments.

6. The method of claim 1, wherein the grouping assigns each of the pixels to one and only one of the three sets of pixels.

7. The method of claim 1, wherein the grouping assigns at least one of the pixels to more than one of the three sets of pixels.

8. The method of claim 1, wherein grouping the pixels comprises employing a k-means clustering algorithm to group the pixels.

9. The method of claim 1, wherein grouping the pixels comprises employing a Viterbi algorithm to group the pixels.

10. The method of claim 1, wherein grouping the pixels comprises employing a support vector machine algorithm to group the pixels.

11. The method of claim 1, wherein grouping the pixels comprises employing an expectation maximization algorithm to group the pixels.

12. The method of claim 1, wherein:

grouping the pixels comprises performing multiple independent groupings of at least some of the pixels of at least one of the line segments in the determined set of line segments;

each performed grouping respectively outputs three sets of pixels based at least in part on respective color data of the pixels;

each respective three sets of pixels includes a respective skin-pixel set, a respective hair-pixel set, and a respective background-pixel set;

the method further comprises selecting one of the respective hair-pixel sets based on one or more hair-pixel-set quality metrics; and

updating the user hair-color model based at least in part on the hair-pixel set comprises updating the user hair-color model based at least in part on the selected hair-pixel set.

13. The method of claim 12, wherein each independent grouping of at least some of the pixels of at least one of the line segments in the determined set of line segments employs a different type of grouping algorithm.

14. The method of claim 12, wherein each independent grouping of at least some of the pixels of at least one of the line segments in the determined set of line segments employs a same type of grouping algorithm and a different set of grouping algorithm parameters.

15. The method of claim 1, further comprising updating a background-color model based at least in part on the background-pixel set.

16. The method of claim 1, wherein the user hair-color model is at least one of a histogram, a machine learning classifier, a Gaussian mixture, and an array of color values with respective color counts.

17. The method of claim 1, wherein updating the user hair-color model based at least in part on the hair-pixel set comprises, for each pixel in the hair-pixel set:

adding a plurality of user-hair-color-model entries to the user hair-color model, the entries corresponding to a color of the pixel as well as colors that are within a threshold tolerance of the color of the pixel.

18. A system comprising:

a communication interface;

a processor; and

non-transitory computer-readable data storage containing instructions executable by the processor for causing the system to carry out a set of functions, the set of functions including;

obtaining video data depicting a head of a user;

determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user;

grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels, wherein the three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set; and

updating a user hair-color model based at least in part on the hair-pixel set.

19. A method comprising:

obtaining video data depicting a head of a user;

obtaining an upper contour of the head of the user;

determining a set of line segments of pixels, each line segment in the set of line segments crossing the upper contour;

selecting a first number of k-means initial configurations, each selected k-means initial configuration including, for each line segment in the set of line segments, two initial mean colors that are configuration-independent and one initial mean color that is configuration-dependent;

performing the first number of independent k-means analyses on the set of line segments, each independent k-means analysis (i) using a different one of the selected k-means initial configurations and (ii) identifying, for each line segment in the set of line segments, three final mean colors respectively derived from the three initial mean colors;

for each of the performed k-means analyses, using the final mean colors derived from the configuration-dependent initial mean colors to identify a respective set of hair-color pixels from the pixels in the line segments;

selecting one of the identified sets of hair-color pixels based on one or more hair-color-pixel-set criteria; and

using the selected set of hair-color pixels to generate a user-hair-color model of the head of the user.

20. The method of claim 19, wherein one of the two configuration-independent initial mean colors is representative of a color of a face of the user, the other of the two configuration-independent initial mean colors is representative of a color of background of the video data, and the configuration-dependent initial mean color is representative of a color of hair of the user.