MULTIPLE SENSORY CHANNEL APPROACH FOR TRANSLATING HUMAN EMOTIONS IN A COMPUTING ENVIRONMENT

• US 20110040155A1
• Filed: 08/13/2009
• Published: 02/17/2011
• Est. Priority Date: 08/13/2009
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: GUZAK, BARBARA S., KWAN, HUNG-TACK, VORA, JANKI Y.
• Application Number:
  US12/540,735
• Publication Number:
  US 20110040155A1
• Time in Patent Office: Days
• US Class Current: 600/300
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/0531 Measuring skin impedance
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/245 specially adapted for magnetoencephalographic [MEG] signals
• CPC Class Codes: A61B 5/6804 Garments; Clothes
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

Sensory inputs of a user can be received by a computing device. At least one of the sensory inputs can include a physiological input providing a physiological measurement from a body of the user. Each sensory input can be processed in a unique one of a set of standards-defined sensory channels, each corresponding to a specific emotion dimension. Processing the sensory inputs can transform the physiological measurement into an emotion dimension value. The emotion dimension values from each of the sensory channels can be aggregated to generate at least one emotion datum value, which is a standards-defined value for an emotional characteristic of the user. Historical data for a user can be optionally collected and used by a learning and calibration component to improve the accuracy of the generated emotion datum values for a specific individual. A programmatic action driven by the emotion datum value can be performed.

Claims

1. A method for discerning human emotions in a computing environment comprising:

receiving a plurality of sensory inputs of a user, wherein at least one of the sensory inputs comprises a physiological input providing a physiological measurement from a body of the user, said physiological measurement being obtained using a physiological sensor;

processing each sensory input in a unique one of a plurality of standards-defined sensory channels, each standards-defined sensory channel corresponding to a specific emotion dimension, wherein said processing comprises transforming the physiological measurement into an emotion dimension value, said emotion dimension value abstracting the physiological measurement from specifics attributable to unique characteristics of the physiological sensor;

aggregating the emotion dimension values from each of the sensory channels to generate at least one emotion datum value, which is a standards-defined value for an emotional characteristic of said user, wherein said emotion datum value is a value independent of any of said sensory devices and is a value independent of any single one of said standards-defined sensory channels; and

performing a programmatic action driven by the emotion datum value.

2. The method of claim 1, wherein said user is interacting with a graphical user interface to participate within a virtual environment in which said user is represented in said virtual environment as an avatar, which is presented to a plurality of other users of said virtual environment, wherein a presentation characteristic of said avatar is adjusted for the emotion datum value so that an emotion corresponding to the emotion datum value is presented to the other users.

3. The method of claim 2, wherein the presentation characteristic alters at least one of a facial expression of the avatar and a speech characteristic of the avatar in accordance with the emotion corresponding to the emotion datum value.

4. The method of claim 1, wherein the emotion datum value is generated without the user ever explicitly manually inputting semantic content for an emotion.

5. The method of claim 1, wherein said emotion datum value is an application independent value conforming to a defined standard, wherein a plurality of independent applications are able to consume the emotion datum value to drive application specific changes based upon emotions of the user.

6. The method of claim 1, further comprising:

when aggregating the emotion dimension values, adjusting user specific factors specified within a stored user specific data set to normalize the generated emotion datum value to a baseline of an average person.

7. The method of claim 1, wherein said user participates in an application session of an application which performs said programmatic action driven by the emotion datum value, wherein during the application session, the plurality of sensory inputs from the user are continuously received, processed, and aggregated to continuously generate different emotion datum values, which drive programmatic actions of the application that are based upon emotional variances of the user during the application session, wherein the continuous receiving, processing, aggregating, and performing of the programmatic actions occur in at least one of real-time and near real time, wherein the application session is a communication session in which said user communicates in real-time or near real-time with at least one other user, wherein during said communication session said other user is continuously appraised of emotional changes of said user determined from the continuously generated emotion datum values.

8. The method of claim 1, wherein processing for different ones of the plurality of standards-defined is handled by different networked computing devices, each of the different networked computing devices being explicitly dedicated to handle at least one specific sensory channel specific processing task.

9. The method of claim 1, further comprising:

collecting historical data for said user comprising emotion data of the user;

analyzing said historical data on an iterative basis; and

adjusting parameters used to generate said emotion datum value in accordance with results of said analyzing to improve an accuracy of the generated emotion datum value over time, thereby instituting a learning process to continuously improve method produced results in a user specific manner.

10. The method of claim 1, further comprising:

collecting channel specific historical data for said user on a sensory channel specific basis for each of said plurality of standards-defined sensory channels;

analyzing said channel specific historical data for each sensory channel on an iterative basis;

adjusting parameters used to generate said emotion dimension values in accordance with results of said analyzing of the channel specific historical data to improve an accuracy of the generated emotion dimension values over time; and

collecting aggregation specific historical data for said user comprising emotion datum values and emotion datum values generated from said emotion datum values;

analyzing said aggregation specific historical data on an iterative basis; and

adjusting parameters used to generate said emotion datum values in accordance with results of said analyzing of the aggregation specific historical data to improve an accuracy of the generated emotion datum values over time.

11. The method of claim 1, wherein said aggregating of the dimension values comprises:

querying an emotional data set specific to said user, wherein said emotion data set weights the dimension values relative to each other to generate the emotion datum value in a user specific manner that accounts for a user specific point in a human emotional variance range.

12. The method of claim 1, wherein the processing of each sensory input is performed as a software service by a computing device remotely located from a computing device proximate to the user, and wherein the aggregating of the emotion dimension values is performed as a software service by a computing device remotely located from a computing device proximate to the user.

13. A computer program product for discerning human emotions in a computing environment comprising:

a computer usable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to receive a plurality of sensory inputs of a user, wherein at least one of the sensory inputs comprises a physiological input providing a physiological measurement from a body of the user, said physiological measurement being obtained using a physiological sensor;

computer readable program code configured to process each sensory input in a unique one of a plurality of standards-defined sensory channels, each standards-defined sensory channel corresponding to a specific emotion dimension, wherein said processing comprises transforming the physiological measurement into an emotion dimension value, said emotion dimension value abstracting the physiological measurement from specifics attributable to unique characteristics of the physiological sensor;

computer readable program code configured to aggregate the emotion dimension values from each of the sensory channels to generate at least one emotion datum value, which is a standards-defined value for an emotional characteristic of said user, wherein said emotion datum value is a value independent of any of said sensory devices, independent of any single one of said standards-defined sensory channels; and

computer readable program code configured to perform a programmatic action driven by the emotion datum value.

14. A system for incorporating human emotions in a computing environment comprising:

a plurality of discrete sensory channels for handling sensory input, wherein each of the discrete sensory channels is a standards-defined sensory channel corresponding to a specific emotion dimension, wherein sensory input handled within the sensory channels comprises physiological input providing a physiological measurement from a body of the user;

a plurality of in-channel processors that process sensory input specific to the channel and that generate emotion dimension values from the sensory input, wherein each emotion dimension value has been transformed to be independent of idiosyncrasies of a sensory capture device from which the sensory input was originally obtained; and

a sensory aggregator for aggregating emotion dimension values generated in a per-channel basis by the in-channel processors to generate at least one emotion datum value, which is a standards-defined value for an emotional characteristic of a user from whom the sensory input was gathered, wherein said emotion datum value is a value independent of any single one of said standards-defined sensory channels, and is an application independent value that is able to be utilized by a plurality of independent applications to discern emotions of said user and to cause application specific code of the independent applications to be reactive to changes in sensory aggregator generated emotion datum values.

15. The system of claim 14, further comprising:

a plurality of sensory capture devices, wherein said sensory capture devices comprise a physiological sensor, an audio or video capture device, and a user input peripheral for manual user input, wherein input obtained from the physiological sensor, from the audio or video capture device, and from the user input peripheral is handled by separately by different ones of the discrete sensory channels, and wherein said physiological sensor obtains a measure from a system of the human body selected from a group of systems consisting of a cardiovascular system, a muscular system, a circulatory system, a nervous system, a respiratory system, an excretory system, an endocrine system, and a digestive system.

16. The system of claim 14, further comprising:

a plurality of emotion data consumers each performing a programmatic actions driven by emotion datum values generated by the sensory aggregator.

17. The system of claim 16, wherein at least one of the emotion data consumers comprises a virtual world software application in which a plurality of humans interact though user specific avatars, wherein said user specific avatars express emotions corresponding to emotions of their users driven by the emotion data values.

18. The system of claim 14, wherein at least a portion of the in-channel processers are computer program products hosted by a physical computing device remotely located and connected by network to another physical computing device from which the sensory input handled by the portion of the in-channel processers originated.

19. The system of claim 14, wherein at least one of the in-channel processors analyzes speech input to determine emotions of the user who provided the speech input based upon vocal characteristics of the speech input, and wherein at least one of the in-channel processors analyzes images to determine facial expressions of the user.

20. The system of claim 14, wherein said in-channel processors and said sensory aggregator are components of a service oriented architecture (SOA), where software services are provided for processing said sensory input and for aggregating said emotion dimension values.