USER CLASSIFICATION BASED UPON IMAGES

• US 20170098124A1
• Filed: 10/06/2015
• Published: 04/06/2017
• Est. Priority Date: 10/06/2015
• Status: Active Grant

• Current Assignee: Yahoo AD Tech LLC
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Jonsson, Kristjan, Rao, Sanjeev Jagannatha, Tasse, Daniel J.
• Granted Patent: US 9,704,045 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 18/2453 non-linear, e.g. polynomial classifier
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene

Abstract

One or more systems and/or methods for providing content to a user are provided. An image, associated with a user, may be evaluated utilizing an image classifier to identify an object within the image. The object may be utilized to identify a predicted class for the user. In an example, the predicted class may correspond to a life event (e.g., graduating college, having a baby, buying a house, etc.) and/or a life stage (e.g., adolescence, retirement, etc.). Locational information (e.g., a geotag) for the image may be evaluated to determine an image location (e.g., a location where the image was generated). Responsive to the image location corresponding to a home location of the user, the predicted class may be determined to be a class associated with the user. Content (e.g., promotional content) may be selected from a content repository based upon the class and subsequently provided to the user.

Claims

1. A method of providing content to a user, comprising:

evaluating an image, associated with a user, utilizing an image classifier to identify an object within the image;

identifying a predicted class based upon the object;

evaluating locational information for the image to determine an image location;

responsive to the image location corresponding to a home location of the user, determining that the predicted class is a class associated with the user;

selecting content from a content repository based upon the class; and

providing the content to the user.

2. The method of claim 1, comprising:

determining the home location for the user based upon one or more additional images associated with the user.

3. The method of claim 2, the determining the home location of the user comprising:

evaluating an image repository associated with the user to identify a set of images comprising locational information;

grouping one or more images of the set of images into a first cluster based upon the one or more images corresponding to a first location;

determining a total number of images in the first cluster;

determining a timespan for the images in the first cluster; and

responsive to the total number of images in the first cluster exceeding an image count threshold and the timespan for the first cluster exceeding a timespan threshold, determining that the first location corresponds to the home location of the user.

4. The method of claim 1, comprising:

determining a confidence score for the predicted class; and

the identifying the predicted class as a class associated with the user comprising identifying the predicted class as the class based upon the confidence score exceeding a confidence threshold.

5. The method of claim 4, the determining a confidence score comprising:

comparing the object utilized to identify the predicted class to a set of known objects to determine a match score for the object; and

responsive to the match score exceeding a threshold amount, increasing the confidence score for the predicted class.

6. The method of claim 4, the determining a confidence score comprising:

evaluating an individual within the image utilizing a facial recognition technique to determine an identity of the individual; and

responsive to the identity of the individual corresponding to the user, increasing the confidence score for the predicted class.

7. The method of claim 1, the class comprising at least one of:

an athletic class, a college class, a school class, a graduate class, an engagement class, a wedding class, a parent class, a new homeowner class, a career class, a hobby class, or a pet owner class.

8. The method of claim 1, the image classifier comprising a non-linear classifier utilizing a convolutional neural network configured to transform image elements.

9. The method of claim 8, comprising:

configuring the image classifier to transform image elements through a series of layers, wherein each layer extracts features from an output of a previous layer.

10. The method of claim 9, the series of layers comprising:

at least one of a pooling layer, a rectified linear unit layer, a convolutional layer, or fully connected layer.

11. The method of claim 1, comprising:

receiving feedback associated with the identification of at least one of the predicted class or the object from the user; and

training the image classifier based upon the user feedback.

12. The method of claim 1, comprising:

receiving feedback associated with the identification of at least one of the predicted class or the object from the user; and

responsive to the feedback indicating an improper identification for at least one of the predicted class or the image feature, prompting the user to provide a proper identification of at least one of the predicted class or the image feature.

13. The method of claim 1, the evaluating an image, associated with a user, utilizing an image classifier to identify an object comprising:

aggregating pixel attributes of the image into patch-level features;

evaluating the patch-level features to determine aggregated patch-level features;

evaluating the aggregated patch-level features to identify one or more image features for the image; and

evaluating the one or more image features to identify the object.

14. The method of claim 2, comprising:

evaluating locational information for the one or more additional images utilizing a leader-follower algorithm to determine the home location based upon a locational grouping of the one or more additional images over a set time span.

15. The method of claim 1, the evaluating an image comprising:

obtaining the image from at least one of a social media community, a microblogging community, a workgroup community, an online forum community, or an image sharing community.

16. A system for providing content to a user, comprising:

an image classifier component configured to;

evaluate an image, associated with a user, to identify an object within the image;

identify a predicted class for the user based upon the object;

evaluate locational information of the image to determine an image location for the image;

determine a home location of the user by utilizing a leader-follower algorithm to evaluate one or more additional images associated with the user; and

responsive to the image location corresponding to the home location, determine that the predicted class is a class associated with the user; and

a content provider component configured to;

select content from a content repository based upon the class associated with the user; and

provide the content to the user.

17. The system of claim 16, the image classifier component configured to:

evaluate an image repository associated with the user to identify a set of images comprising locational information;

group one or more images of the set of images into a first cluster based upon the one or more images corresponding to a first location;

determine a total number of images in the first cluster;

determine a timespan for the images in the first cluster; and

responsive to the total number of images in the first cluster exceeding an image count threshold and the timespan for the first cluster exceeding a timespan threshold, determine that the first location corresponds to the home location of the user.

18. The system of claim 16, the image classifier component configured to:

determine a confidence score for the predicted class; and

identify the predicted class as the class based upon the confidence score exceeding a confidence threshold.

19. The system of claim 16, comprising:

a training component configured to;

train an untrained image classifier using a training image database to create the image classifier.

20. A non-transitory computer readable medium comprising computer executable instructions that when executed by a processor perform a method for providing content to a user, comprising:

evaluating an image, associated with a user, utilizing an image classifier to identify an image feature within the image;

identifying a predicted class for the user based upon the image feature;

evaluating locational information of the image to determine an image location for the image;

determining a home location of the user based upon one or more additional images associated with the user, the determining the home location comprising;

evaluating locational information for the one or more additional images utilizing a leader-follower algorithm to determine the home location based upon a locational grouping of the one or more additional images over a timespan;

responsive to the image location corresponding to a home location of the user, determining that the predicted class is a class associated with the user;

selecting promotional content from a content repository based upon the class; and

providing the promotional content to the user.