SYSTEM AND METHODS FOR ANALYSIS OF USER-ASSOCIATED IMAGES TO GENERATE NON-USER GENERATED LABELS AND UTILIZATION OF THE GENERATED LABELS

• US 20170206420A1
• Filed: 07/22/2015
• Published: 07/20/2017
• Est. Priority Date: 07/22/2014
• Status: Active Grant

• Current Assignee: Hong Kong University of Science and Technology
• Original Assignee: Hong Kong University of Science and Technology
• Inventors: SHE, James Pei Man, CHEUNG, Pak Ming
• Granted Patent: US 10,460,174 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video

Abstract

A system for annotating user images with non-user generated labels corresponding to a plurality of users and generating user profiles includes: one or more processor-readable media, configured to store processor-executable instructions for image analysis and user profile generation; and one or more processors, configured to, based on execution of, the stored processor-executable instructions: annotate user images with non-user-generated labels, wherein the non-user-generated labels correspond to visual features of the user images and generate user profiles based on non-user-generated tags associated with images corresponding to each respective user.

Claims

1. A system for analyzing user images corresponding to a plurality of users and generating distributions corresponding to users, comprising:

one or more processor-readable media, configured to store processor-executable instructions for image analysis and user profile generation;

one or more processors, configured to, based on execution of the stored processor-executable instructions;

annotate user images with one or more non-user-generated labels, wherein the non-user-generated labels correspond to visual features of the user images; and

generate a distribution based on non-user-generated labels associated with images corresponding to each respective user.

2. The system according to claim 1, wherein the user images comprise at least one of the following:

images generated by users from user-related devices or smart devices via image taking, drawing, screen capturing, receiving, liking or sharing, and/or saving;

images generated by users from cameras of user-related devices or smart devices, wherein the user-related devices or smart devices include;

smartphones, wearable devices, and/or human control devices;

images taken by cameras and stored in one or more digital formats; and

/or

images captured from videos.

3. The system according to claim 1, wherein the one or more processors are further configured to:

extract features of the user images;

generate visual words representing extracted features;

encode each image of a plurality of user images with at least one feature vector based on the visual words and the extracted features corresponding to the visual words associated with the image;

cluster images of the plurality of user images into multiple groups based on feature vectors corresponding to the images;

determine at least one non-user-generated label corresponding to each group of the multiple groups; and

determine a distribution of non-user-generated labels associated with images corresponding to each respective user.

4. The system according to claim 3, wherein clustering the images into multiple groups is based on visual similarity of the images;

and

wherein each group of the multiple groups is represented by the at least one non-user-generated label corresponding to the group.

5. The system according to claim 3, wherein clustering the images into multiple groups is based on pre-definitions of the multiple groups;

and

wherein each group of the multiple groups is represented by the at least one non-user-generated label corresponding to the group.

6. The system according to claim 3, wherein the visual features comprise color, edges, textures, region, objects and faces, and/or other unique feature descriptors based on SIFT, GIST or other computer vision and image/signal processing techniques.

7. The system according to claim 3, wherein extracting the features utilizes at least one of a Harris Affine detector, a Maximally Stable Extremal Regions detector, a Kadir-Brady saliency detector, and a Scale-invariant feature transform (SIFT).

8. The system according to claim 7, wherein a unique visual word is generated from a set of features as a feature representation.

9. The system according to claim 3, wherein the visual word is represented by combining corresponding features of the plurality of images.

10. The system according to claim 3, wherein the feature vector is represented by a number of occurrences of each visual word in each image.

11. The system according to claim 3, wherein the clustering is based on similarities between feature vectors of the images being clustered.

12. The system according to claim 4, wherein the similarity calculation is based on cosine similarity.

13. The system according to claim 3, wherein the label distribution for each respective user is indicative of a number of occurrences of unique non-user-generated labels associated with images corresponding to the respective user.

14. The system according to claim 3, wherein the label distribution for each respective label is determined based on a transform function corresponding to multiple labels.

15. The system according to claim 1, wherein the one or more processors are configured to:

determine that two users are related based on similarities in their respective distributions of non-user generated labels.

16. The system according to claim 15, wherein the determination of whether two users are related is performed via direct comparison between the respective distributions of the two users.

17. The system according to claim 1, wherein the one or more processors are further configured to:

compare profiles of at least two of the plurality of users to calculate a similarity value corresponding to the two users.

18. The system according to claim 17, wherein the one or more processors are further configured to:

identify or recommend connections between users based on calculated similarity values from the distribution generated by non-user generated labels.

19. The system according to claim 18, wherein the identification or recommendation of connections is conducted without utilizing social graphs or user-annotated tags.

20. The system according to claim 18, wherein social graphs or user-annotated tags are used to enhance the results of the identifying or recommending.

21. The system according to claim 17, wherein the one or more processors are further configured to:

identify or recommend user communities for users based on calculated similarity values.

22. The system according to claim 17, wherein the one or more processors are further configured to:

identify user gender based on the calculated similarity values of the user with others, or the similarity graph with the similarity among all users.

23. One or more non-transitory processor-readable media having processor-executable instructions stored thereon for analyzing user images corresponding to a plurality of users and generating user profiles, wherein the processor-executable instructions are executable by one or more processors to facilitate the following steps:

annotating user images with non-user-generated labels, wherein the non-user-generated labels correspond to visual features of the user images; and

generating a distribution for scalable calculation based on non-user-generated tags associated with images corresponding to each respective user.

24. The one or more non-transitory processor-readable media according to claim 23, wherein the processor-executable instructions are further executable by the one or more processors to facilitate the following steps:

extracting features of the user images;

generating visual words representing extracted features;

encoding each image of a plurality of user images with at least one feature vector based on the visual words and the extracted features corresponding to the visual words associated with the image;

clustering images of the plurality of user images into multiple groups based on feature vectors corresponding to the images;

assigning each group of the multiple groups with at least one non-user-generated label; and

determining a distribution of non-user-generated labels associated with images corresponding to each respective user.

25. A method for analyzing user images corresponding to a plurality of users and generating user profiles, comprising:

annotating user images with non-user-generated labels, wherein the non-user-generated labels correspond to visual features of the user images; and

generating a distribution for scalable calculation based on non-user-generated tags associated with images corresponding to each respective user.