Filter and shutter based on image emotion content

• US 10,185,869 B2
• Filed: 08/11/2016
• Issued: 01/22/2019
• Est. Priority Date: 08/02/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Emotient, Inc. (Apple Inc.)
• Inventors: Movellan, Javier, Denman, Ken, Susskind, Joshua
• Primary Examiner(s): Tucker, Wesley J
• Application Number:
  US15/234,763
• Publication Number:
  US 20160350588A1
• Time in Patent Office: 894 Days
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A computer-implemented (including method implemented using laptop, desktop, mobile, and wearable devices) method for image filtering. The method includes analyzing each image to generate a content vector for the image; applying an interest operator to the content vector, the interest operator being based on a plurality of pictures with desirable characteristics, thereby obtaining an interest index for the image; comparing the interest index for the image to an interest threshold; and taking one or more actions or abstaining from one or more actions based on a result of the step of comparing. Also, related systems and articles of manufacture.

Claims

1. A method for image filtering, the method comprising:

analyzing, by a computer, a collection of images using a plurality of classifiers corresponding to one of a plurality of characteristics;

generating, by the computer, a content vector for each image in the collection of images, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

receiving a selection of two or more images in the collection of images;

identifying a preference point vector, wherein the preference point vector comprises a plurality of values, wherein each value in the preference point vector corresponds to a value in the content vectors of the images in the collection of images, and wherein the preference point vector is within a predetermined distance of the content vectors of a subset of the selected two or more images;

receiving a burst of two or more images from a camera;

selecting one or more additional images from the burst based, at least in part, on a comparison of a distance between a content vector of a respective additional image and the preference point vector to an interest threshold;

discarding one or more unselected images from the burst; and

taking one or more actions on the selected one or more additional images.

2. The method of claim 1, wherein selecting one or more additional images further comprises:

obtaining the one or more additional images;

generating a content vector for the one or more additional images; and

determining a level of interest of the one or more additional images using the content vectors for the one or more additional images.

3. The method of claim 1, wherein the one or more actions comprise one or more actions selected from a group consisting of:

amending metadata associated with one of the one or more additional images to indicate the determined level of interest,

store one of the one or more additional images in a directory,

display one of the one or more additional images, and

erase one of the one or more additional images.

4. The method of claim 1, wherein the selection of the two or more images is received from a first user, wherein the first user is associated with first user characteristics, and wherein the method further comprises:

comparing the first user characteristics to second user characteristics associated with a second user; and

determining a preference point vector for the second user based on the content vectors for each image of the selected two or more images and the comparison of the first user characteristics and the second user characteristics.

5. The method of claim 1, wherein at least one classifier of the plurality of classifiers is related to emotional content in facial expressions.

6. The method of claim 1, wherein the subset comprises a predetermined percentage of the selected two or more images.

19. The method of claim 1, wherein the content vector for at least one image in the collection of images comprises values corresponding to classifiers for low-level features and classifiers for facial expressions.

20. The method of claim 1, wherein the collection of images comprises images from the burst of two or more images from the camera.

7. A non-transitory computer readable medium comprising computer readable code which, when executed by one or more processors, causes the one or more processors to:

analyze a collection of images using a plurality of classifiers corresponding to one of a plurality of characteristics;

generate a content vector for each image in the collection of images, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

receive a selection of two or more images in the collection of images;

identify a preference point vector, wherein the preference point vector comprises a plurality of values, wherein each value in the preference point vector corresponds to a value in the content vectors of the images in the collection of images, and wherein the preference point vector is within a predetermined distance of the content vectors of a subset of the selected two or more images;

receive a burst of two or more images from a camera;

select one or more additional images from the burst based, at least in part, on a comparison of a distance between a content vector of a respective additional image and the preference point vector to an interest threshold;

discard one or more unselected images from the burst; and

take one or more actions on the selected one or more additional images.

8. The computer readable medium of claim 7, wherein the computer readable code to cause the one or more processors to select one or more additional images comprises computer readable code to further cause the one or more processors to:

obtain the one or more additional images;

generate a content vector for the one or more additional images; and

determine a level of interest of the one or more additional images using the content vectors for the one or more additional images.

9. The computer readable medium of claim 7, wherein the one or more actions comprise one or more actions selected from a group consisting of:

amending metadata associated with one of the one or more additional images to indicate the determined level of interest,

store one of the one or more additional images in a directory,

display one of the one or more additional images, and

erase one of the one or more additional images.

21. The computer readable medium of claim 7, wherein the content r for at least one image in the collection of images comprises values corresponding to classifiers for low-level features and classifiers for facial expressions.

22. The computer readable medium of claim 7, wherein the collection of images comprises images from the burst of two or more images from the camera.

10. The computer readable medium of 7, wherein the selection of the two or more images is received from a first user, wherein the first user is associated with first user characteristics, and wherein the computer readable code further causes the one or more processors to:

compare the first user characteristics to second user characteristics associated with a second user; and

determine a preference point vector for the second user based on the content vectors for each image of the selected two or more images and the comparison of the first user characteristics and the second user characteristics.

11. The computer readable medium of 7, wherein at least one classifier of the plurality of classifiers is related to emotional content in facial expressions.

12. The computer readable medium of 7, wherein the subset comprises a predetermined percentage of the selected two or more images.

13. A system for image filtering, comprising:

one or more processors; and

a memory coupled to the one or more processors and comprising computer readable code which, when executed by the one or more processors, causes the system to;

analyze a collection of images using a plurality of classifiers corresponding to one of a plurality of characteristics;

generate a content vector for each image in the collection of images, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

receive a selection of two or more images in the collection of images;

identify a preference point vector, wherein the preference point vector comprises a plurality of values, wherein each value in the preference point vector corresponds to a value in the content vectors of the images in the collection of images, and wherein the preference point vector is within a predetermined distance of the content vectors of a subset of the selected two or more images;

receive a burst of two or more images from a camera;

select one or more additional images from the burst based, at least in part, on a comparison of a distance between a content vector of a respective additional image and the preference point vector to an interest threshold;

discard one or more unselected images from the burst; and

take one or more actions on the selected one or more additional images.

14. The system of claim 13, wherein the computer readable code to cause the system to select one or more additional images comprises computer readable code to further cause the system to:

obtain the one or more additional images;

generate a content vector for the one or more additional images; and

determine a level of interest of the one or more additional images using the content vectors for the one or more additional images.

15. The system of claim 13, wherein the one or more actions comprise one or more actions selected from a group consisting of:

amending metadata associated with one of the one or more additional images to indicate the determined level of interest,

store the one of the one or more additional images in a directory,

display the one of the one or more additional images, and

erase the one of the one or more additional images.

16. The system of claim 13, wherein the selection of the one two or more images is received from a first user, wherein the first user is associated with first user characteristics, and wherein the computer readable code further causes the system to:

compare the first user characteristics to second user characteristics associated with a second user; and

determine a preference point vector for the second user based on the content vectors for each image of the selected two or more images and the comparison of the first user characteristics and the second user characteristics.

17. The system of claim 13, wherein at least one classifier of the plurality of classifiers is related to emotional content in facial expressions.

18. The system of claim 13, wherein the subset comprises a predetermined percentage of the selected two or more images.

23. The system of claim 13, wherein the content vector for at least one image in the collection of images comprises values corresponding to classifiers for low-level features and classifiers for facial expressions.

24. The system of claim 13, wherein the collection of images comprises images from the burst of two or more images from the camera.