Method and system for measuring emotional state

• US 20180032126A1
• Filed: 08/01/2016
• Published: 02/01/2018
• Est. Priority Date: 08/01/2016
• Status: Abandoned Application

• Current Assignee: Yadong Liu
• Original Assignee: Yadong Liu
• Inventors: LIU, Yadong
• Application Number:
  US15/224,665
• Publication Number:
  US 20180032126A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 2562/0247 Pressure sensors
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Techniques for measuring the emotional state of minds in a person based upon a set of biological data captured from a person in a natural environment, where the person is not restricted at all in his movement. Various sensing data from sensors are captured with or without the intervention of the person. In addition, expression data is also captured before, during or after the sensing signals are captured. The data sets are processed to filter out those uncorrelated. A dedicated computing device is provided to collect the data along with other necessary data available on the Internet, where the measurement of the emotion is measured, derived or calculated based on the collected data, and/or any historical measurements of the person. The result may be shared with a list of contacts selected by the person who may also views theirs as well.

Claims

1. A method for measuring an emotional state of mind in a user using a smartphone along with a plurality of sensors, the method comprising:

receiving a set of biological data from the user in the smartphone, wherein the biological data includes sensing data generated from the sensors to be disposed on different body parts of the user, and expression data provided by the user before, during or after the sensing data is collected, the sensors are coupled to the smartphone wirelessly and activated by commands from the smartphone, wherein the smartphone is in communication with a remote server over a wireless network;

preprocessing, in the smartphone, at least one type the biological data to derive a representative of data captured over a period of time;

transferring from the smartphone the biological data along with geographic locations of the user to the remote server, wherein the geographic locations are automatically obtained within the smartphone over a period the biological data was captured, the remote server is configured to further preprocess the biological data to filter out those that do not make any sense when external, ambient or surrounding conditions at the time the biological data was captured respectively around the geographic locations, and obtain an emotion index based on the preprocessed biological and expression data; and

causing the smartphone to display the emotion index to the user.

2. The method as recited in claim 1, wherein the smartphone is coupled wirelessly with a wrist watch integrating a first set of the sensors, wherein the wrist watch captures some of the sensing data near a wrist of the user when the user wears the wrist watch.

3. The method as recited in claim 2, wherein the smartphone is coupled wirelessly with a pair of glasses integrating a second set of the sensors, wherein the glasses capture some of the sensing data near the head of the user when the user wears the glasses.

4. The method as recited in claim 3, wherein the expression data is related to one or both of a facial image of the user and a voice from the user, and captured by an camera and a microphone equipped in the smartphone, the expression data is preprocessed to eliminate some of the expression data from fake expression in conjunction with some of the sensing data.

5. The method as recited in claim 4, further comprising obtaining the other data by the server from a number of predefined network resources, wherein the other data pertains to the geographic locations of the user, a condition of the geographic locations or any event related to the user or the geographic locations.

6. The method as recited in claim 5, further comprising retrieving historical data about the user.

7. The method as recited in claim 6, further comprising modifying the emotion index in conjunction with the emotion from the expression data, the other data and the historical data.

8. The method as recited in claim 1, further comprising:

sharing the emotion index with a list of contacts determined by the user; and

displaying an emotion index of each of the contacts.

9. The method as recited in claim 8, further comprising:

retrieving a profile of the user;

allocating a commercial message for the user in accordance with the emotion index; and

monitoring how and when the user has interacted with the commercial message.

10. The method as recited in claim 1, further comprising:

instructing the user to try on a virtual reality device that is driven to provide corresponding content for the user to interact with to improve the emotion index.

11. A server for measuring an emotion, the server coupled to a smartphone being carried by a user, the server comprising:

a processor;

a wireless interface to communicate with the smartphone wirelessly over a data network;

a memory space, coupled to the processor, provided to store a server module, wherein the server module is executed by the processor to cause the server to perform operations of;

receiving a set of biological data from the smartphone, wherein the biological data includes sensing data generated from a plurality of sensors to be disposed on different body parts of the user, and expression data provided by the user before, during or after the sensing data is collected, the biological data further includes geographic locations of the user automatically obtained within the smartphone, the sensors are coupled to the smartphone wirelessly and activated by commands from the smartphone;

feeding the biological data to a data processing unit together with other data;

preprocessing the biological data to filter out those that do not make any sense when external, ambient or surrounding conditions at the time the biological data was captured respectively around the geographic locations;

providing processed data to an emotion measurement engine to obtain an emotion index from the processed data; and

causing the smartphone to display the emotion index to the user.

12. The server as recited in claim 11, wherein one part of the biological data is obtained from a wrist watch integrating a first set of the sensors, wherein the wrist watch captures some of the sensing data near a wrist of the user when the user wears the wrist watch.

13. The server as recited in claim 12, wherein another part of the biological data is obtained from a pair of glasses integrating a second set of the sensors, wherein the glasses capture some of the sensing data near the head of the user when the user wears the glasses.

14. The server as recited in claim 13, wherein the expression data is related to one or both of a facial image of the user and a voice from the user, and captured by an camera and a microphone equipped in the smartphone.

15. The server as recited in claim 14, wherein the other data is obtained by the server from a number of predefined network resources and pertains to a location of the user, a condition of the location or any event related to the user or the location.

16. The server as recited in claim 15, wherein the processor is caused to modify the emotion index in conjunction with the emotion from the expression data, the other data and historical data of the user.

17. The server as recited in claim 11, wherein the operation further comprises:

sharing the emotion index with a list of contacts determined by the user; and

displaying an emotion index of each of the contacts.

18. The server as recited in claim 11, further comprising:

an interface to be coupled to a virtual reality device that is controlled to display corresponding content for the user to interact with to improve the emotion index.