Filter and shutter based on image emotion content

• US 9,443,167 B2
• Filed: 08/04/2014
• Issued: 09/13/2016
• Est. Priority Date: 08/02/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Emotient, Inc. (Apple Inc.)
• Inventors: Movellan, Javier, Denman, Ken, Susskind, Joshua
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US14/451,409
• Publication Number:
  US 20150036934A1
• Time in Patent Office: 771 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A computer-implemented (including method implemented using laptop, desktop, mobile, and wearable devices) method for image filtering. The method includes analyzing each image to generate a content vector for the image; applying an interest operator to the content vector, the interest operator being based on a plurality of pictures with desirable characteristics, thereby obtaining an interest index for the image; comparing the interest index for the image to an interest threshold; and taking one or more actions or abstaining from one or more actions based on a result of the step of comparing. Also, related systems and articles of manufacture.

Claims

1. A computer-implemented method for image filtering, the method comprising:

analyzing, by a computer, an image using a plurality of classifiers corresponding to one of a plurality of characteristics;

generating, by the computer, a content vector for the image, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

applying, by the computer, an interest operator to the content vector, wherein the interest operator is based on a plurality of pictures with desirable characteristics, thereby obtaining an interest index for the image represented by the content vector;

comparing, by the computer, the interest index for the image to an interest threshold to obtain a result; and

taking one or more actions by the computer based on the result.

2. The computer-implemented method according to claim 1, wherein the interest operator is predominantly based on content of one or more expressions of one or more faces in the plurality of pictures.

3. The computer-implemented method according to claim 2, wherein the desirable characteristics relate to emotional content or sentiment in the expressions.

4. The computer-implemented method according to claim 1, wherein the interest operator is predominantly based on content of one or more human appearance characteristics in the plurality of pictures.

5. The computer-implemented method according to claim 1, wherein the interest operator is predominantly based on content of one or more action units in the plurality of pictures.

6. The computer-implemented method according to claim 1, wherein the interest operator is predominantly based on content of one or more low level features in the plurality of pictures.

7. The computer-implemented method according to claim 1, wherein the image is captured by a camera viewing an audience of an event.

8. A system comprising:

one or more processors; and

a memory coupled to the one or more processors and comprising computer code which, when executed by the one or more processors, causes the one or more processors to;

analyze an image using a plurality of classifiers corresponding to one of a plurality of characteristics;

generate a content vector for the image, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

apply an interest operator to the content vector, wherein the interest operator is based on a plurality of pictures with desirable characteristics, thereby obtaining an interest index for the image represented by the content vector;

compare the interest index for the image to an interest threshold, to obtain a result; and

take one or more actions based on the result.

9. The system according to claim 8, wherein the interest operator is predominantly based on content of one or more expressions of one or more faces in the plurality of pictures.

10. The system according to claim 9, wherein the desirable characteristics relate to emotional content or sentiment in the expressions.

11. The system according to claim 8, wherein the interest operator is predominantly based on content of one or more human appearance characteristics in the plurality of images.

12. The system according to claim 8, wherein the interest operator is predominantly based on content of one or more action units in the plurality of images.

13. The system according to claim 8, wherein the interest operator is predominantly based on content of one or more low level features in the plurality of images.

14. The system according to claim 8, wherein the plurality of images are captured by a camera viewing an audience of an event.

15. A non-transitory machine-readable medium embedded with computer code for image filtering executable by one or more processors to:

analyze, by a computer, an image using a plurality of classifiers corresponding to one of a plurality of characteristics;

generate, by the computer, a content vector for the image, wherein the content vector comprises a plurality of values, wherein each value corresponds to a classifier of the plurality of classifiers;

apply an interest operator to the content vector, the interest operator being based on a plurality of pictures with desirable characteristics, thereby obtaining an interest index for the image;

comparing the interest index for the image to an interest threshold to obtain a result; and

taking one or more actions based on the result.

16. The computer readable medium according to claim 15, wherein the interest operator is predominantly based on content of one or more expressions of one or more faces in the plurality of pictures.

17. The computer readable medium according to claim 16, wherein the desirable characteristics relate to emotional content or sentiment in the expressions.

18. The computer readable medium according to claim 15, wherein the interest operator is predominantly based on content of one or more human appearance characteristics in the plurality of pictures.

19. The computer readable medium according to claim 15, wherein the interest operator is predominantly based on content of one or more action units in the plurality of pictures.

20. The computer readable medium according to claim 15, wherein the interest operator is predominantly based on content of one or more low level features in the plurality of pictures.