Photo management

• US 10,043,112 B2
• Filed: 10/30/2014
• Issued: 08/07/2018
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Talathi, Sachin Subhash, Julian, David Jonathan
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US14/528,935
• Publication Number:
  US 20150254532A1
• Time in Patent Office: 1,377 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/19133 Interactive pattern learning with a human teacher
• CPC Class Codes: G06V 30/19167 Active pattern learning
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A method for image processing includes determining features of multiple stored images from a pre-trained deep convolutional network. The method also includes clustering each image of the multiple stored images based on the determined features.

Claims

1. A method for image processing, comprising:

determining, with a first pre-trained classifier, features of a plurality of stored images, the plurality of stored images being different from a plurality of training images used for training the first pre-trained classifier;

assigning each image of the plurality of stored images to a cluster of a plurality of clusters based at least in part on the determined features;

assigning a label to an image of the plurality of stored images based on a distance of the image to a closest cluster center corresponding to the image being less than a first threshold, a pre-defined number of images associated with the closest cluster center being less than a second threshold, and/or a percentage of images associated with the closest cluster center being less than a third threshold;

modifying the assigned label;

re-training the first pre-trained classifier with the image when the assigned label is modified; and

re-sorting the image based at least in part on the re-trained classifier.

2. The method of claim 1, in which the first pre-trained classifier is a support vector machine (SVM), logistic regression, nearest neighbor, or multilayer perceptron (MLP).

3. The method of claim 1, further comprising classifying images of the plurality of stored images into groups based at least in part on corresponding labels.

4. The method of claim 3, further comprising placing each image of the plurality of stored images into a folder based at least in part on a corresponding group.

5. The method of claim 4, further comprising naming the folders with nonspecific names.

6. The method of claim 4, further comprising determining a specific folder name with a second pre-trained classifier.

7. The method of claim 6, further comprising determining, with the second pre-trained classifier, the specific folder name based at least in part on a classification of a cluster center associated with a label for an associated group.

8. The method of claim 7, in which the specific folder name is further determined based at least in part on a majority rule on labeled images of a given folder.

9. The method of claim 4, further comprising receiving user input to modify a specific folder name or assign a new specific folder name.

10. The method of claim 9, further comprising re-training a second pre-trained classifier to include the modified specific folder name or the new specific folder name based at least in part on a cluster center or at least one labeled image feature vector.

11. The method of claim 4, further comprising placing a new image in a folder based at least in part on determining a feature vector of the new image and applying the classifier to determine a group for the new image.

12. The method of claim 3, further comprising receiving user input to modify the assigned label.

13. The method of claim 12, in which the assigned label is modified based at least in part on user input to move the image from a first folder to a second folder.

14. The method of claim 1, further comprising sharing at least one classifier or a list of cluster centers with a second user.

15. The method of claim 14, in which the shared list of cluster centers is combined with a list of cluster centers for the second user to create labels for images of the second user and to update a classifier for the second user.

16. The method of claim 14, further comprising sharing user defined categories for classes or clusters with the second user.

17. An apparatus for image processing, the apparatus comprising:

a memory unit; and

at least one processor coupled to the memory unit, the at least one processor being configured;

to determine, with a first pre-trained classifier, features of a plurality of stored images, the plurality of stored images being different from a plurality of training images used for training the first pre-trained classifier;

to assign each image of the plurality of stored images to a cluster of a plurality of clusters based at least in part on the determined features;

to assign a label to an image of the plurality of stored images based on a distance of the image to a closest cluster center corresponding to the image being less than a first threshold, a pre-defined number of images associated with the closest cluster center being less than a second threshold, and/or a percentage of images associated with the closest cluster center being less than a third threshold;

to modify the assigned label;

to re-train the first pre-trained classifier with the image when the assigned label is modified; and

to re-sort the image based at least in part on the re-trained classifier.

18. The apparatus of claim 17, in which the first pre-trained classifier is a support vector machine (SVM), logistic regression, nearest neighbor, or multilayer perceptron (MLP).

19. The apparatus of claim 17, in which the at least one processor is configured to classify the plurality of stored images into groups based at least in part on corresponding labels.

20. The apparatus of claim 19, in which the at least one processor is configured to place each image of the plurality of stored images into a folder based at least in part on a corresponding group.

21. The apparatus of claim 20, in which the at least one processor is configured to name the folders with nonspecific names.

22. The apparatus of claim 20, in which the at least one processor is configured to determine a specific folder name with a second pre-trained classifier.

23. The apparatus of claim 22, in which the second pre-trained classifier is configured to determine the specific folder name based at least in part on a classification of a cluster center associated with a label for an associated group.

24. The apparatus of claim 23, in which the at least one processor is configured to determine the specific folder name based at least in part on a majority rule on labeled images of a given folder.

25. The apparatus of claim 20, in which the at least one processor is further configured to receive user input to modify a specific folder name or assign a new specific folder name.

26. The apparatus of claim 25, in which the at least one processor is configured to re-train a second pre-trained classifier to include the modified specific folder name or the new specific folder name based at least in part on a cluster center or at least one labeled image feature vector.

27. The apparatus of claim 20, in which the at least one processor is further configured to place a new image in a folder based at least in part on determining a feature vector of the new image and applying the classifier to determine a group for the new image.

28. The apparatus of claim 19, in which the at least one processor is further configured to receive user input to modify the assigned label.

29. The apparatus of claim 28, in which the assigned label is modified based at least in part on user input to move the image from a first folder to a second folder.

30. The apparatus of claim 17, in which the at least one processor is further configured to share at least one classifier or a list of cluster centers with a second user.

31. The apparatus of claim 30, in which the at least one processor is further configured to share user defined categories for classes or clusters with the second user.

32. A non-transitory computer-readable medium having program code recorded thereon for image processing, the program code being executed by a processor and comprising:

program code to determine, with a first pre-trained classifier, features of a plurality of stored images, the plurality of stored images being different from a plurality of training images used for training the first pre-trained classifier;

program code to assign each image of the plurality of stored images to a cluster of a plurality of clusters based at least in part on the determined features;

program code to assign a label to an image of the plurality of stored images based on a distance of the image to a closest cluster center corresponding to the image being less than a first threshold, a pre-defined number of images associated with the closest cluster center being less than a second threshold, and/or a percentage of images associated with the closest cluster center being less than a third threshold;

program code to modify the assigned label;

program code to re-train the first pre-trained classifier with the image when the assigned label is modified; and

program code to re-sort the image based at least in part on the re-trained classifier.

33. An apparatus for image processing, the apparatus comprising:

means for determining, with a first pre-trained classifier, features of a plurality of stored images, the plurality of stored images being different from a plurality of training images used for training the first pre-trained classifier;

means for assigning each image of the plurality of stored images to a cluster of a plurality of clusters based at least in part on the determined features;

means for assigning a label to an image of the plurality of stored images based on a distance of the image to a closest cluster center corresponding to the image being less than a first threshold, a pre-defined number of images associated with the closest cluster center being less than a second threshold, and/or a percentage of images associated with the closest cluster center being less than a third threshold;

means for modifying the assigned label;

means for re-training the first pre-trained classifier with the image when the assigned label is modified; and

means for re-sorting the image based at least in part on the re-trained classifier.