Method for using facial expression to determine affective information in an imaging system

• US 20030165270A1
• Filed: 02/19/2002
• Published: 09/04/2003
• Est. Priority Date: 02/19/2002
• Status: Active Grant

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Fedorovskaya, Elena A., Parulski, Kenneth A., Matraszek, Tomasz A., Endrikhovski, Serguei, Mir, Jose M.
• Granted Patent: US 7,003,139 B2
• Time in Patent Office: Days
• US Class Current: 382/189
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection

Abstract

A method for determining affective information for at least one image in an imaging system includes displaying a digital image for viewing by a user; monitoring the facial expression of the user as the user views the digital image; and using the facial expression of the user to determine affective information for the digital image.

Claims

1. A method for determining affective information for at least one image in an imaging system, comprising the steps of:

a) displaying a digital image for viewing by a user;

b) monitoring the facial expression of the user as the user views the digital image; and

c) using the facial expression of the user to determine affective information for the digital image.

2. The method of claim 1 further including the step of:

d) associating the affective information with the digital image.

3. The method of claim 1 wherein a video camera is used to monitor the facial expression of the user.

4. The method of claim 3 wherein at least one image from the video camera is analyzed to determine the smile size of the user.

5. The method of claim 1 wherein a plurality of digital images are displayed for viewing by the user.

6. The method of claim 5 wherein the smile size of the user is determined for each of the plurality of digital images.

7. The method of claim 6 wherein a degree of preference is determined for each of the plurality of digital images by relating the smile size corresponding to each digital image to an average smile size.

8. The method of claim 7 wherein the degree of preference is stored along with the corresponding digital image in separate digital image files.

9. A method for providing affective information for images in an imaging system, comprising the steps of:

a) sequentially displaying a plurality of digital images for viewing by a user;

b) monitoring the facial expression of the user as the user views each of the plurality of digital images; and

c) using the facial expression of the user to determine affective information.

10. The method of claim 9 wherein the 1 wherein a video camera is used to monitor the facial expression of the user, and images from the video camera are analyzed to determine the smile size of the user as the user view each of the plurality of digital images.

11. A system for providing affective information for images in an imaging system, comprising:

a) a digital memory which stores a set of digital images;

b) a display which sequentially displays the set of digital images for viewing by a user;

c) a camera for automatically measuring the user'"'"'s facial expression; and

d) a processor for processing the signal from the camera to provide affective information for the set of digital images.

12. The system of claim 11 wherein the camera is a video camera.

13. The system of claim 12 wherein the processor processes the signal from the video camera in order to determine the user'"'"'s smile size.

14. The system of claim 13 wherein the processor determines the normalized smile size for each digital image in the set.

15. The system of claim 13 wherein the smile size is determined using the maximum distance between mouth corners.

16. The system of claim 11 wherein the system further includes a sensor for measuring the user'"'"'s physiology.

17. The system of claim 16 wherein the sensor measures the user'"'"'s galvanic skin response.

18. The system of claim 11 wherein the affective information is stored in the digital memory.

19. The system of claim 11 wherein the affective information is stored with each digital image in a digital image file.

20. The system of claim 19 wherein the digital image file includes affective information and user identifiers for a plurality of users.