Inferring the gender of a face in an image

• US 9,355,300 B1
• Filed: 10/17/2013
• Issued: 05/31/2016
• Est. Priority Date: 11/02/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Baluja, Shumeet, Jing, Yushi
• Primary Examiner(s): Chawan, Sheela C
• Application Number:
  US14/056,852
• Time in Patent Office: 957 Days
• Field of Search: 382/118, 382/276, 382/159, 382/190, 382/115, 382/100, 382/224, 707/E15.042, 707/E15.017, 707/803, 707/713, 707/714, 707/716, 707/731, 707/796, 707/798, 707/E17.009, 707/E17.999, 705/14.4, 705/319
• US Class Current: 1/1
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The subject matter of this specification can be embodied in, among other things, a computer-implemented method that includes receiving a plurality of images having human faces. The method further includes generating a data structure having representations of the faces and associations that link the representations based on similarities in appearance between the faces. The method further includes outputting a first gender value for a first representation of a first face that indicates a gender of the first face based on one or more other gender values of one or more other representations of one or more other faces that are linked to the first representation.

Claims

1. A computer-implemented method, comprising:

receiving, by a computing system, images that display a particular visible physical feature for multiple people displayed in the images;

determining, by the computing system, similarities in appearance among the particular visible physical features for at least some of the people displayed in the images;

for each person in at least a subset of the people displayed in the images;

identifying one or more people displayed in the images whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person,

identifying a respective classification for the particular visible physical feature of each of the one or more people, and

determining a classification for the particular visible physical feature of the person based on the respective classifications for the particular visible physical features of the one or more people whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person; and

outputting, by the computing system, information that characterizes the classification of the particular visible physical feature determined for at least one of the people displayed in the images.

2. The method of claim 1, further comprising generating a data structure that includes representations of the particular visible physical features of at least a second subset of the people displayed in the images and that specifies links among at least some of the representations of the particular visible physical features.

3. The method of claim 2, wherein the data structure comprises a plurality of nodes and edges, each node in the data structure corresponding to a particular representation of the particular visible physical feature of a person displayed in the images, and each edge in the data structure linking a pair of nodes among the plurality of nodes and indicating a respective similarity score for the representations of the particular visible physical feature corresponding to the pair of nodes.

4. The method of claim 1, wherein the particular visible physical feature comprises a face.

5. The method of claim 1, wherein, for each person in the at least subset of people displayed in the images, determining the classification for the particular visible physical feature of the person comprises determining a likelihood that the person is male or female based on the respective classifications for the particular physical features of the one or more people indicating likelihoods that each of the one or more people are male or female, respectively.

6. The method of claim 1, further comprising identifying the one or more people whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person by performing a random walk algorithm on a data structure that links representations of the particular visible physical feature of at least some of the people displayed in the images.

7. The method of claim 6, further comprising performing multiple iterations of the random walk algorithm on the data structure by performing the random walk algorithm a predetermined number of times or until respective classification values for at least some representations of the particular visible physical feature from among the multiple representations of the particular visible physical feature linked in the data structure converge such that the respective classification values change less than a threshold amount over corresponding classification values determined from a previous iteration of the random walk algorithm.

8. The method of claim 1, wherein the classification for the particular visible physical feature of at least one of the one or more people is seeded from user input.

9. The method of claim 1, wherein the classification for the particular visible physical feature of at least one person from among the at least subset of the people displayed in the images is determined further based on respective classifications for the particular visible physical feature of one or more additional people whose particular visible physical feature is determined to be similar to the particular visible physical feature of at least one of the one or more people.

10. The method of claim 1, wherein determining the classification for the particular visible physical feature of the person comprises performing an adsorption algorithm.

11. A computer program product tangibly embodied in one or more non-transitory computer-readable storage devices that, when executed by one or more processors, causes performance of operations comprising:

receiving, by a computing system, images that display a particular visible physical feature for multiple people displayed in the images;

determining, by the computing system, similarities in appearance among the particular visible physical features for at least some of the people displayed in the images;

for each person in at least a subset of the people displayed in the images;

identifying one or more people displayed in the images whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person,

identifying a respective classification for the particular visible physical feature of each of the one or more people, and

determining a classification for the particular visible physical feature of the person based on the respective classifications for the particular visible physical features of the one or more people whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person; and

outputting, by the computing system, information that characterizes the classification of the particular visible physical feature determined for at least one of the people displayed in the images.

12. The computer program product of claim 11, wherein the operations further comprise generating a data structure that includes representations of the particular visible physical features of at least a second subset of the people displayed in the images and that specifies links among at least some of the representations of the particular visible physical features.

13. The computer program product of claim 12, wherein the data structure comprises a plurality of nodes and edges, each node in the data structure corresponding to a particular representation of the particular visible physical feature of a person displayed in the images, and each edge in the data structure linking a pair of nodes among the plurality of nodes and indicating a respective similarity score for the representations of the particular visible physical feature for the pair of nodes.

14. The computer program product of claim 11, wherein the particular visible physical feature comprises a face.

15. The computer program product of claim 11, wherein, for each person in the at least subset of people displayed in the images, determining the classification for the particular visible physical feature of the person comprises determining a likelihood that the person is male or female based on the respective classifications for the particular physical features of the one or more people indicating likelihoods that each of the one or more people are male or female, respectively.

16. The computer program product of claim 11, wherein the operations further comprise identifying the one or more people whose particular visible physical feature is determined to be similar to the particular visible physical feature of the person by performing a random walk algorithm on a data structure that links multiple representations of the particular visible physical feature of at least some of the people displayed in the images.

17. The computer program product of claim 16, wherein the operations further comprise performing multiple iterations of the random walk algorithm on the data structure by performing the random walk algorithm a predetermined number of times or until respective classification values for at least some representations of the particular visible physical feature from among the multiple representations of the particular visible physical feature linked in the data structure converge such that the respective classification values change less than a threshold amount over corresponding classification values determined from a previous iteration of the random walk algorithm.

18. The computer program product of claim 11, wherein the classification for the particular visible physical feature of at least one of the one or more people is seeded from user input.

19. The computer program product of claim 11, wherein the classification for the particular visible physical feature of at least one person from among the at least subset of the people displayed in the images is determined further based on respective classifications for the particular visible physical feature of one or more additional people whose particular visible physical feature is determined to be similar to the particular visible physical feature of at least one of the one or more people.

20. A system comprising:

one or more computers configured to provide;

an image repository to store a plurality of images;

a visible physical feature detector to identify respective visible physical features among people displayed in the plurality of images;

a similarity calculator to determine similarities in appearance among representations of the visible physical features of the people displayed in the images;

a graph generator to link the representations of the visible physical features based on the determined similarities in appearance among the representations of the visible physical features; and

a classification generator to determine classification values for particular ones of the representations of the visible physical features based on the determined similarities in appearance among the particular representations of the visible physical features and other ones of the representations of the visible physical features that are linked to the particular representations, wherein the classification values indicate a likelihood that respective ones of the people displayed in the plurality of images corresponding to the representations of the visible physical features have a particular visually discernible characteristic.