Apparatus and method for perceiving physical and emotional state

• US 20030078505A1
• Filed: 08/20/2001
• Published: 04/24/2003
• Est. Priority Date: 09/02/2000
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Bang, Seok-won, Kim, Jay-woo, Lee, Hyoung-ki, Kim, Jeong-hwan
• Granted Patent: US 6,656,116 B2
• Time in Patent Office: Days
• US Class Current: 600/485
• CPC Class Codes: A61B 5/0008 Temperature signals
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1455 using optical sensors, e.g. spectral photometrical oximeters
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

An apparatus and method for perceiving a physical and emotional state, which allows easy attachment to and detachment from a human body, and through which a bio-signal is simply detected, are provided. The apparatus includes a bio-signal detection part attached to a predetermined portion of a body for performing analog signal processing on at least one bio-signal detected from the body and outputting the processed bio-signal, and a bio-signal recognizing part for performing digital signal processing on the processed bio-signal received from the bio-signal detection part, perceiving the physical and emotional state from the result of the digital signal processing, and representing the physical and emotional state. Accordingly, the apparatus can be conveniently attached to a predetermined portion of a user'"'"'s body, a bio-signal transmitted wirelessly or through a wire can be easily detected, a physical and emotional state which is perceived based on the detected bio-signal can be reported to the user, and a rapidly changing emotional state or an emotional state which remains for a long time can be perceived in real time.

Claims

1. An apparatus for perceiving a physical and emotional state, comprising:

a bio-signal detection part attached to a predetermined portion of a body for performing analog signal processing on at least one bio-signal detected from the body and outputting the processed bio-signal; and

a bio-signal recognizing part for performing digital signal processing on the processed bio-signal received from the bio-signal detection part, perceiving the physical and emotional state from the result of the digital signal processing, and reporting the physical and emotional state.

2. The apparatus of claim 1, wherein the bio-signal detection part comprises:

a sensor for detecting the bio-signal from the body in analog form;

an analog signal processor for amplifying and filtering the analog bio-signal detected by the sensor; and

an analog-to-digital converter for converting the amplified and filtered analog bio-signal into a digital bio-signal, and

the bio-signal recognizing part comprises;

a digital signal processor for performing data processing on the digital bio-signal;

a controller for perceiving the physical and emotional state from data processed by the digital signal processor and outputting the perceived result as a control signal; and

a data output unit for outputting the data processed by the digital signal processor in predetermined form in response to the control signal.

3. The apparatus of claim 2, wherein the sensor comprises:

a blood pressure sensor for measuring blood pressure;

a skin temperature sensor for measuring skin temperature; and

a skin resistivity sensor for measuring skin resistivity.

4. The apparatus of claim 3, wherein the blood sensor is a pressure sensor such as a Piezo sensor or a strain gauge.

5. The apparatus of claim 3, wherein the skin temperature sensor measures skin temperature using an electrode and a thermocouple.

6. The apparatus of claim 3, wherein the skin resistivity sensor measures skin resistivity using an electrode and a comparator.

7. The apparatus of claim 2, wherein the bio-signal detection part further comprises a wireless signal transmitter for converting the digital bio-signal output from the analog-to-digital converter into a wireless signal and transmitting the wireless signal, and the bio-signal recognizing part further comprises a wireless signal receiver for receiving the wireless signal and outputting it to the digital signal processor as the digital bio-signal.

8. The apparatus of any one of claim 1, wherein the bio-signal detection part is realized as a strap-on type and is removably attached to a predetermined portion of the body.

9. The apparatus of claim 2, wherein the controller stores at least one reference value, compares the stored reference value with the data processed by the digital signal processor, and perceives the physical and emotional state from the result of the comparison.

10. The apparatus of claim 9, wherein the controller calculates an average value of at least one bio-signal detected by the sensor from data previously processed by the digital signal processor, selects one of the average value and the reference value in response to the difference between the average value and the stored reference value, and compares the selected average value or the selected reference value with the data currently processed by the digital signal processor, to perceive the physical and emotional state.

11. A method of perceiving a physical and emotional state, comprising the steps of:

(a) performing analog signal processing on at least one bio-signal detected from a body; and

(b) performing digital signal processing on the analog signal processed bio-signal, perceiving the physical and emotional state from the result of the digital signal processing, and reporting the physical and emotional state.

12. The method of claim 11, wherein step (a) comprises the steps of:

(a1) detecting the bio-signal from the body;

(a2) amplifying and filtering the detected analog bio-signal; and

(a3) converting the amplified and filtered analog bio-signal into a digital bio-signal, and

step (b) comprises the steps of;

(b1) performing data processing on the digital bio-signal;

(b2) perceiving the physical and emotional state from the resultant data of the data processing of step (b1); and

(b3) providing the data to a user in response to the perceived result.

13. The method of claim 12, wherein step (a) further comprises the step of converting the digital bio-signal into a wireless signal and transmitting the wireless signal after step (a3), and step (b) further comprises the step of receiving the wireless signal before step (b1).

14. The method of claim 12, wherein step (b2) comprises the steps of:

storing at least one reference value; and

comparing the stored reference value with the resultant data of step (b1) and perceiving the physical and emotional state based on the result of the comparison.

15. The method of claim 12, wherein step (b2) comprises the steps of:

storing at least one reference value;

calculating an average value of least one bio-signal detected in the step (a1) from data which is previously processed in the step (b1);

obtaining the difference between the average value and the reference value;

determining whether the difference is greater than a predetermined allowable error;

comparing the average value with the resultant data of step (b1) and perceiving the physical and emotional state from the result of the comparison, when it is determined that the difference is greater than the predetermined allowable error;

comparing the reference value with the resultant data of step (b1) and perceiving the physical and emotional state from the result of the comparison, when it is determined that the difference is not greater than the predetermined allowable error.

16. A method of perceiving an emotional state, comprising the steps of:

(c) extracting at least one bio-parameter representing a characteristic of a body from at least one bio-signal which is detected from the body for a predetermined time; and

(d) determining the amount of variation of the extracted bio-parameter and determining a current emotional state using the determined amount of variation,

wherein the value of the bio-parameter varies with a change in the emotional state.

17. The method of claim 16, wherein in step (c), bio-parameters representing the frequency of an electrocardiogram, number of heartbeats and an electrocardiogram attendant upon respiration are extracted from a bio-signal having information as to blood pressure, and bio-parameters representing skin temperature and skin resistivity are extracted from a bio-signal having information as to skin temperature and a bio-signal having information as to skin resistivity.

18. The method of claim 17, wherein step (d) comprises the steps of:

(d11) perceiving happiness using the bio-parameters representing the number of heartbeats, the skin temperature and the frequency of an electrocardiogram;

(d12) perceiving stress using the bio-parameters representing the number of heartbeats and the frequency of an electrocardiogram;

(d13) perceiving sadness using the bio-parameters representing the number of heartbeats and the skin temperature;

(d14) perceiving boredom using the bio-parameters representing the skin temperature and the skin resistivity; and

(d15) perceiving anger using the bio-parameters representing the skin resistivity and the frequency of an electrocardiogram, and

the predetermined time is a minimum time taken for a changed emotion to be represented by a bio-signal when there is a change in the emotion.

19. The method of claim 17, wherein the step (d) comprises the steps of:

(d21) perceiving boredom using the bio-parameters representing the skin temperature, the skin resistivity, the frequency of an electrocardiogram and the electrocardiogram attendant upon respiration;

(d22) perceiving anger using the bio-parameters representing the skin resistivity and the electrocardiogram attendant upon respiration;

(d2) perceiving happiness using the bio-parameters representing the number of heartbeats, the frequency of an electrocardiogram and a rate of the respiration;

(d24) perceiving sadness using the bio-parameters representing the number of heartbeats and the skin resistivity; and

(d25) perceiving stress using the bio-parameters representing the skin resistivity, and

the predetermined time is longer than a minimum time taken for a changed emotion to be represented by a bio-signal when there is a change in the emotion, and the rate of the respiration is extracted from the bio-signal having information as to the blood pressure in step (c).

20. The method of claim 17, wherein the bio-parameters representing the frequency of an electrocardiogram are a LF indicating a value obtained by integrating a power spectrum in a low frequency range over the Fast Fourier Transform (FFT) of the bio-signal having the information as to the blood pressure, a HF indicating a value obtained by integrating a power spectrum in a high frequency range over the FFT of the bio-signal having the information as to the blood pressure, and a LF/HF indicating a value obtained by dividing the LF by the HF;

the bio-parameters representing the number of heartbeats are a RR indicating an average value of the intervals between the positive peaks of the blood pressure and a HR indicating the number of heartbeats for a unit time period;

the bio-parameter representing the electrocardiogram attendant upon respiration is a RSA indicating a value obtained by dividing the result of subtracting a minimum interval from a maximum interval among the intervals between positive peaks of the blood pressure by the minimum interval;

the bio-parameter representing the skin temperature is a SKT indicating an average skin temperature; and

the bio-parameters representing the skin resistivity are a SCL indicating an average of reciprocals of the values of the skin resistivity, a N-SCR indicating the number of negative slope zero crossings measured from a graph expressing the reciprocals of the values of the skin resistivity, and SCRM equal to the sum of reciprocals of values of skin resistivity between a negative slope zero crossing and a positive slope zero crossing.

21. The method of claim 20, wherein when the predetermined time is the minimum time, step (d11) comprises the steps of:

(d111) determining whether a HR value decreases by at least a HR threshold value;

(d112) determining whether a SKT value increases by at least a SKT threshold value if it is determined that the HR value decreases by at least the HR threshold value;

(d113) determining whether a LF value decreases by at least a LF threshold value or whether a LF/HF value decreases by at least a LF/HF threshold value if it is determined that the SKT value increases by at least the SKT threshold value;

(d114) determining that probability that the emotional state is happiness is a first predetermined percentage if it is determined that the LF value decreases by at least the LF threshold value or that the LF/HF value decreases by at least the LF/HF threshold value;

(d115) determining whether a RR value increases by at least a RR threshold value and going back to step (d114) if it is determined that the RR value does not increase by at least the RR threshold value; and

(d116) determining that probability that the emotional state is happiness is a second predetermined percentage greater than the first predetermined percentage if it is determined that the RR value increases by at least the RR threshold value.

22. The method of claim 21, wherein step (d12) comprises the steps of:

(d121) determining whether the LF value decreases by at least the LF threshold value if it is determined that the SKT value does not increase by at least the SKT threshold value in step (d112); and

(d122) determining that probability that the emotional state is stress is the second predetermined percentage if it is determined that the LF value decreases by at least the LF threshold value.

23. The method of claim 22, wherein step (d13) comprises the steps of:

(d131) determining whether the RR value increases by at least the RR threshold value if it is determined that the LF value does not decrease by at least the LF threshold value and that the LF/HF value does not decrease by at least the LF/HF threshold value in step (d113); and

(d132) determining that probability that the emotional state is sadness is the second predetermined percentage if it is determined that the RR value increases by at least the RR threshold value.

24. The method of claim 23, wherein step (d14) comprises the steps of:

(d141) determining whether a N-SCR value increases by at least a N-SCR threshold value and whether a SCRM value increases by at least a SCRM threshold value if it is determined that the HR value does not decrease by at least the HR threshold value in step (d111);

(d142) determining whether the SKT value increases by at least the SKT threshold value if it is determined that the N-SCR value increases by at least the N-SCR threshold value and that the SCRM value increases by at least the SCRM threshold value; and

(d143) determining that probability that the emotional state is boredom is the second predetermined percentage if it is determined that the SKT value increases by at least the SKT threshold value.

25. The method of claim 24, wherein step (d15) comprises the steps of:

(d151) determining whether the LF/HF value decreases by at least the LF/HF threshold value if it is determined that the SKT value does not increase by at least the SKT threshold value in step (d142) and going back to step (d142) if it is determined that the LF/HF value does not decrease by at least the LF/HF threshold value;

(d152) determining that probability that the emotional state is anger is the first predetermined percentage if it is determined that the LF/HF value decreases by at least the LF/HF threshold value;

(d153) determining whether a SCL value increases by at least a SCL threshold value and going back to step (d152) if it is determined that the SCL value does not increase by at least the SCL threshold value; and

(d154) determining that probability that the emotional state is anger is the second predetermined percentage if it is determined that the SCL value increases by at least the SCL threshold value.

26. The method of claim 25, wherein step (d) further comprises the step of determining that no dominant emotional state exists if it is determined that the LF value does not decrease by at least the LF threshold value in step (d121), if it is determined that the N-SCR value does not increase by at least the N-SCR threshold value or that the SCRM value does not increase by at least the SCRM threshold value in step (d141), or if it is determined that the RR value does not increase by at least the RR threshold value in step (d131).

27. The method of claim 20, wherein when the predetermined time is longer than the minimum time, step (d21) comprises the steps of:

(d211) determining whether a RSA value decreases by at least a RSA threshold value, whether a SCL value increases by at least a SCL threshold value, and whether a SCRM value increases by at least a SCRM threshold value;

(d212) determining whether a N-SCR value increases by at least a N-SCR threshold value if it is determined that the RSA value decreases by at least the RSA threshold value, that the SCL value increases by at least the SCL threshold value, and that the SCRM value increases by at least the SCRM threshold value;

(d213) determining whether a SKT value increases by at least a SKT threshold value if it is determined that the N-SCR value increases by at least the N-SCR threshold value;

(d214) determining that probability that the emotional state is boredom is a third predetermined percentage if it is determined that the SKT value increases by at least the SKT threshold value;

(d215) determining whether a HF value decreases by at least a HF threshold value and going back to step (d214) if it is determined that the HF value does not decrease by at least the HF threshold value;

(d216) determining that probability that the emotional state is boredom is a fourth predetermined percentage greater than the third predetermined percentage if it is determined that the HF value decreases by at least the HF threshold value;

(d217) determining whether a LF value decreases by at least a LF threshold value and going back to step (d216) if it is determined that the LF does not decrease by at least the LF threshold value; and

(d218) determining that probability that the emotional state is boredom is a fifth predetermined percentage greater than the fourth predetermined percentage if it is determined that the LF decreases by at least the LF threshold value.

28. The method of claim 27, wherein step (d22) comprises the steps of.

(d221) determining that probability that the emotional state is anger is the fifth predetermined percentage if it is determined that the N-SCR value does not increase by at least the N-SCR threshold value in step (d212);

and

(d222) determining that probability that the emotional state is anger is the third predetermined percentage if it is determined that the SKT value does not increase by at least the SKT threshold value in step (d213).

29. The method of claim 28, wherein step (d23) comprises the steps of:

(d231) determining whether a HR value decreases by at least a HR threshold value if it is determined that the RSA value does not decrease by at least the RSA threshold value, that the SCL value does not increase by at least the SCL threshold value, or that the SCRM value does not increase by at least the SCRM threshold value in step (d211);

(d232) determining whether a RESP value increases by at least a RESP threshold value if it is determined that the HR value decreases by at least the HR value;

(d233) determining that probability that the emotional state is happiness is the third predetermined percentage if it is determined that the RESP value increases by at least the RESP threshold value;

(d234) determining whether a LF/HF value decreases by at least a LF/HF threshold value and going back to step (d233) if it is determined that the LF/HF value does not decrease by at least the LF/HF threshold value;

(d235) determining that probability that the emotional state is happiness is the fourth predetermined percentage if it is determined that the LF/HF value decreases by at least the LF/HF threshold value;

(d236) determining whether the LF value decreases by at least the LF threshold value and going back to step (d235) if it is determined that the LF value does not decrease by at least the LF threshold value; and

(d237) determining that probability that the emotional state is happiness is the fifth predetermined percentage if it is determined that the LF value decreases by at least the LF threshold value.

30. The method of claim 29, wherein step (d24) comprises the steps of:

(d241) determining whether the SCL value decreases by at least the SCL threshold value if it is determined that the RESP value does not increase by at least the RESP threshold value in step (d232); and

(d242) determining that probability that the emotional state is sadness is the fifth predetermined percentage if it is determined that the SCL value decreases by at least the SCL threshold value.

31. The method of claim 30, wherein step (d25) comprises the steps of:

(d251) determining whether the N-SCR value increases by at least the N-SCR threshold value if it is determined that the HR value does not decrease by at least the HR threshold value in step (d231); and

(d252) determining that probability that the emotional state is stress is the fourth predetermined percentage if it is determined that the N-SCR value increases by at least the N-SCR threshold value.

32. The method of claim 31, wherein step (d) further comprises the step of determining that no dominant emotional state exists if it is determined that the SCL value does not decrease by at least the SCL threshold value in step (d241) or if it is determined that the N-SCR value does not increase by at least the N-SCR threshold value in step (d251).