Tagging Images With Emotional State Information

• US 20150178915A1
• Filed: 12/19/2013
• Published: 06/25/2015
• Est. Priority Date: 12/19/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chatterjee, Monique, Collette, Benoit, VanEngelen, Fernd, Hughes, Stephanie
• Application Number:
  US14/134,863
• Publication Number:
  US 20150178915A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Images, such as photographs or videos, are tagged with emotional state and/or biometric information. Emotional state information (or mood) may be stored in metadata of an electronic image. A computing device, such as a cellular telephone, receives an image from a camera as well as biometric information from a sensor. Sensors may be located on the computing device, or alternatively on a user wearable device. Biometric information may be from a user taking a photograph or from a user viewing a photograph. Biometric information may include heart rate, galvanic skin response (GSR), facial expression and the like. The computing device may calculate an emotional state of a user, such as happiness, based on the biometric information. The tagged biometric and/or emotional state information allows for a way to retrieve, sort and organize images. Tagged images may be used in social media connections or broadcasting, such as blogging specific emotional images.

Claims

1. A method to operate a computing device, the method comprising:

receiving, from an image capture device, an image obtained from the image capture device;

receiving, from a sensor, sensor information that represents biometric information when the image was obtained from the image capture device;

determining emotional state information associated with the image based on the sensor information that represents biometric information when the image was obtained from the image capture device;

associating the emotional state information with the image; and

storing the image and emotional state information.

2. The method of claim 1, wherein the sensor and image capture device is included in a single computing device.

3. The method of claim 1, wherein the sensor is a wearable sensor and the image capture device is a camera, wherein the wearable sensor is included in a first computing device and the camera is included in a second separate computing device.

4. The method of claim 1, wherein the determining the emotional state is performed, at least in part, by a processor in the computing device executing processor readable instructions, stored in a processor readable memory, in response to the sensor information.

5. The method of claim 4, wherein storing the image and emotional state information includes storing emotional state information as a number in metadata of the image, wherein the image and emotional state information is stored in processor readable memory.

6. The method of claim 1, wherein the sensor is configured so as to obtain sensor information that includes at least one of include heart rate, galvanic skin response (GSR), facial expression, temperature, glucose level or hydration.

7. The method of claim 6, wherein the sensor information is obtained from a user that causes the image to be obtained from the image capture device.

8. The method of claim 7, wherein the associating includes storing the emotional state information in metadata of the image.

9. The method of claim 8, wherein the determining the emotional state information includes assigning a number in a range of numbers associated with a range of emotions of the user, wherein the assigning the number in the range of numbers is based on the sensor information.

10. The method of claim 9, wherein the range of emotions of the user includes at least happiness, sadness and anger, wherein a first set of numbers in the range of numbers is associated with happiness, a second set of numbers in the range of numbers is associated with sadness and a third set of numbers in the range of numbers is associated with anger.

11. An apparatus comprising;

at least one sensor to obtain biometric data;

at least one camera to obtain an image;

at least one processor; and

at least one processor readable memory to store processor readable instructions,

wherein the at least one processor executes the processor readable instructions to;

receive sensor information that represents biometric information from the sensor,

receive the image from the camera,

calculate emotional state information associated with the image based on the sensor information that represents biometric information, and

store the emotional state information with the image.

12. The apparatus of claim 11, wherein at least one sensor to obtain biometric information, at least one camera to obtain the image, at least one processor and at least one processor readable memory to store processor readable instructions are stored in a single computing device.

13. The apparatus of claim 11, wherein the at least one sensor to obtain biometric information is included in a wearable device and the at least one camera to obtain the image, at least one processor and at least one processor readable memory to store processor readable instructions are included in a separate computing device.

14. The apparatus of claim 11, wherein the at least one sensor is configured so as to obtain sensor information that includes at least one of heart rate, galvanic skin response (GSR), facial expression, temperature, glucose level or hydration, wherein the sensor information is obtained from a user that causes the image to be obtained from the camera, and wherein calculate the emotional state information includes assign a number in a range of numbers associated with a range of emotions of the user, wherein the assign the number in the range of numbers is based on the sensor information.

15. The apparatus of claim 11, wherein the apparatus is included in a game and media console.

16. One or more processor readable memories having instructions encoded thereon which when executed cause one or more processors to perform a method for processing an image, the method comprising:

receiving sensor information that represents biometric information from a sensor;

receiving an image from a camera;

calculating emotional state information associated with the image based on the sensor information that represents biometric information;

storing the emotional state information with the image;

receiving a request for the image having a requested emotional state; and

providing the image in response to the request for the image having the requested emotional state.

17. The one or more processor readable memories of claim 16, wherein sensor information includes at least one of heart rate, galvanic skin response (GSR), facial expression, temperature, glucose level or hydration.

18. The one or more processor readable memories of claim 17, wherein the sensor information represent biometric information of a user and the user causes the camera to obtain the image, wherein the calculating the emotional state information includes assigning a number in a range of numbers associated with a range of emotions of the user, wherein the assigning the number in the range of numbers is based on the sensor information.

19. The one or more processor readable memories of claim 18, wherein the range of emotions of the user includes at least happiness, sadness and anger, wherein a first set of numbers in the range of numbers is associated with happiness, a second set of numbers in the range of numbers is associated with sadness and a third set of numbers in the range of numbers is associated with anger.

20. The one or more processor readable memories of claim 16, wherein the storing the emotional state information with the image includes storing the emotional state information in metadata of the image that is retrievable during providing the image in response for the request for the image having the requested emotional state.