EMOTION-BASED VEHICLE SERVICE SYSTEM, EMOTION COGNITION PROCESSING APPARATUS, SAFE DRIVING SERVICE APPARATUS, AND EMOTION-BASED SAFE DRIVING SERVICE METHOD

• US 20130054090A1
• Filed: 08/24/2012
• Published: 02/28/2013
• Est. Priority Date: 08/29/2011
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: SHIN, Hyun Soon, LEE, Yong Kwi, JO, Jun
• Granted Patent: US 8,862,317 B2
• Time in Patent Office: Days
• US Class Current: 701/36
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: Y02T 10/84 Data processing systems or methods, management, administration

Abstract

An emotion-based vehicle service system for safe driving includes a multi-emotion sensor node configured to sense emotion information of a driver and passenger in a vehicle and condition information of the vehicle. The emotion-based vehicle service system further includes an emotion cognition processing apparatus configured to analyze sensing information provided from the multi-emotion sensor node to generate emotion cognition information, and a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information.

Claims

1. An emotion-based vehicle service system for safe driving, including:

a multi-emotion sensor node configured to sense emotion information of a user in a vehicle and condition information of the vehicle;

an emotion cognition processing apparatus configured to analyze sensing information provided from the multi-emotion sensor node to generate emotion cognition information for cognizing the emotion of the user; and

a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information.

2. The emotion-based vehicle service system of claim 1, wherein the multi-emotion sensor node is constituted with at least one of a handle sensor node, a driver seat sensor node, a passenger seat sensor node, a transmission sensor node, a ceiling sensor node, and a front glass sensor node.

3. The emotion-based vehicle service system of claim 2, wherein the multi-emotion sensor node includes at least one of a heart rate sensor, a skin reaction sensor, a body temperature sensor, a voice sensor, an image sensor, an acceleration sensor, and a slope sensor.

4. The emotion-based vehicle service system of claim 1, wherein the emotion cognition processing apparatus is further configured to combine the emotion information of the user and the condition information of the vehicle for analyzing the combined emotion information and the condition information to cognize the emotion of the user.

5. An emotion recognition processing apparatus for a safe driving service, including:

an emotion cognition unit configured to cognize emotion information of a driver and passenger in a vehicle; and

a vehicle condition cognizing unit configured to cognize condition information of the vehicle,

wherein the emotion cognition unit includes;

a multi-bio signal-based emotion signal analyzer configured to analyze information necessary for cognizing emotion on the basis of bio signals which have been sensed through multiple channels;

an image-based emotion signal analyzer configured to analyze information necessary for cognizing emotion on the basis of image signals which have been sensed through the multiple channels;

a voice-based emotion signal analyzer configured to analyze information necessary for cognizing emotion on the basis of voice signals which have been sensed through the multi-channels;

a motion-based emotion signal analyzer configured to analyze information necessary for recognizing emotion on the basis of motion signals which have been sensed through the multi-channels;

a smell recognition-based emotion signal analyzer configured to analyze information necessary for cognizing emotion on the basis of smell information which has been sensed through the multi-channels;

a temperature/humidity signal analyzer configured to analyze information necessary for recognizing emotion on the basis of a temperature and humidity which have been sensed through the multi-channels;

a position recognizer and analyzer configured to analyze information necessary for cognizing emotion on the basis of position information which has been sensed through the multi-channels;

a driver emotion condition cognizer configured to analyze information necessary for cognizing emotion on the basis of a driver'"'"'s emotion conditions which have been sensed through the multi-channels; and

a passenger emotion condition cognizer configured to analyze information necessary for recognizing emotion on the basis of a passenger'"'"'s emotion conditions which have been sensed through the multi-channels, and

wherein the vehicle condition cognizer includes;

a manifold absolute pressure sensor (MAPS) configured to sense a pressure change of a vehicle'"'"'s intake manifold;

a coolant sensor configured to sense an amount of a coolant in the vehicle;

an illumination sensor configured to sense ambient brightness of the vehicle;

an acceleration sensor configured to sense a speed of the vehicle;

a throttle position sensor mounted on a side of a throttle body to sense a degree of opening of a throttle valve; and

a knock sensor configured to sense an absolute combustion of the vehicle'"'"'s engine.

6. The emotion recognition processing apparatus of claim 5, wherein the emotion cognition processing apparatus receives the emotion information of the driver and passenger and the condition information of the vehicle over a local wireless network.

7. An emotion-based safe driving service apparatus, including:

a vehicle terminal user interface (UI) configured to provide a UI environment of the safe driving service apparatus;

an emotion information management module configured to manage emotion cognition information provided thereto;

an engine control unit (ECU)-linked safe driving control module configured to control ECU-linked safe driving of a vehicle in accordance with to the emotion cognition information managed by the emotion information management unit; and

an emotion care safe driving service module configured to provide an emotion care safe driving service to the vehicle when the vehicle is controlled by the ECU-linked safe driving control module.

8. The emotion-based safe driving service apparatus of claim 7, wherein the emotion information management module includes:

a user emotion information manager;

an EUC linker;

a vehicle condition information manager;

a personal terminal linkage manager; and

an external network connection interface.

9. The emotion-based safe driving service apparatus of claim 7, wherein the ECU-linked safe driving control module includes:

a handle controller;

an acceleration pedal controller;

a break controller;

a starting lock controller;

an automatic vehicle speed reduction controller; and

an automatic vehicle stop controller.

10. The emotion-based safe driving service apparatus of claim 7, wherein the emotion care safe driving service module includes:

a vehicle external UI platform;

a guide broadcaster configured to provide emotion care-related information around a vehicle;

an emotion care audio servicer configured to provide emotion care-related audio information;

an emotion care video servicer configured to providing emotion care-related video information;

a vehicle condition image transmitter configured to provide real-time traffic information around the vehicle as an image;

a vehicle condition information message processor configured to process an operation of sending an emergency report message, a message for reporting to an emergency service center, or a message for sending notice information to an acquaintance;

an illumination setter configured to set emotion care-related illumination; and

a ventilation adjuster configured to adjust air ventilation associated with emotion care around the vehicle.

11. A safe driving service method of a multi-emotion sensor node including a plurality of sensors, the safe driving service method including:

recognizing a driver and a passenger in a vehicle to objectify a sensing target;

sensing emotion cognition information using the plurality of sensors, wherein the emotion cognition information includes emotion information of the driver and passenger and condition information of the vehicle;

analyzing the emotion cognition information;

converting the analyzed emotion cognition information into message information; and

transmitting the message information corresponding to the emotion cognition information to an emotion recognition processing apparatus over a network.

12. The safe driving service method of claim 11, wherein the sensors include one or more a heart rate sensor, a skin reaction sensor, a body temperature sensor, a voice sensor, an image sensor, and an acceleration and slope sensor.

13. The safe driving service method of claim 12, wherein said analyzing the emotion cognition information includes:

analyzing a plurality of signals sensed by the heart rate sensor to detect an HRV/pulse wave, a hematocele oxidization saturation, and intensity of a hematocele;

analyzing a plurality of signals sensed by the skin reaction sensor to detect a skin conductance;

analyzing a signal sensed by the body temperature sensor to detect a skin temperature;

analyzing a signal sensed by the voice sensor to detect voice and a sound wave;

analyzing a signal sensed by the image sensor to detect conditions of a pupil and face; and

analyzing a signal sensed by the acceleration and slope sensor to detect a motion and an activity.

14. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result.

15. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a drowsy driving state when a current condition of the driver is in the drowsy driving state;

executing an ECU-linked process for controlling safe driving in the drowsy driving state to control a reduction in a vehicle speed, an intensity of a handle, or an intensity of a pedal in accordance with a drowsy intensity;

performing a drowsy emotion service to perform guide broadcasting to a drowsy state with a vehicle terminal and ventilate air in the vehicle through the control of an air conditioning system;

providing a music service for awaking drowsiness, and performing guide broadcasting for inducing parking of the vehicle onto a side road;

continuously monitoring a drowsy state to induce parking of the vehicle onto the side road in operational connection to a maneuver patrol when the drowsy state is continuously maintained;

controlling parking of the vehicle onto the side road to stop drowsy driving by linking up with the ECU; and

informing the driver being awaked from the drowsy state to a drivable state.

16. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a driver-fatigued state when a current condition of the driver is a fatigued or health-deteriorated state;

executing a safe driving service process in accordance with a health condition of the driver to perform guide broadcasting for a fatigue and health condition with a vehicle terminal and display the health condition on the vehicle terminal;

determining whether the fatigue and health condition reaches an undrivable threshold;

reporting to an emergency service when it is determined that the fatigue and health condition of the driver is an undrivable state;

collecting position information of the vehicle;

notifying an acquaintance of a health-deteriorated state; and

executing an ECU-linked process for controlling safe driving in a fatigued state to control a safe speed of the vehicle and control safe driving to a destination in operational connection with a vehicle terminal navigation when it is determined that the fatigue and health condition of the driver is not the undrivable state.

17. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a drunk driving state when a current condition of the driver is a drunk driving state;

executing a drunk driving prevention service process to display an alcohol concentration with a vehicle terminal;

providing guide broadcasting for warning against drunk driving with the vehicle terminal;

determining whether the vehicle is currently stopped;

executing an ECU-linked process to control parking of the vehicle onto a side road and control a starting lock such that the driver cannot drive the vehicle in the drunk state any longer, when it is determined that the vehicle is continuously driven;

collecting position information of the vehicle; and

notifying a maneuver patrol of the collected position information.

18. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a reckless driving state of the driver;

executing a reckless driving prevention service process to provide guide broadcasting for warning against reckless driving with the vehicle terminal, when a current condition of the driver is an angry state;

checking whether the driver is continuously performing reckless driving;

executing an ECU-linked process to control an intensity of a handle and control an intensity of an acceleration pedal, when it is checked that the driver is continuously performing the reckless driving;

reproducing an emotion care music;

notifying a maneuver patrol of information on a reckless-driving vehicle in automatic connection with the maneuver patrol; and

collecting position information of the vehicle to notify the maneuver patrol of the collected position information.

19. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a feared state in the vehicle;

executing a rescue service process for rescuing the driver from the feared state to provide a service which protects the driver from fear, when it is determined that the driver or the passenger has a sensation of fear;

collecting real-time image or voice data on situations inside the vehicle;

automatically reporting to a maneuver center, maneuver center, or maneuver patrol center a maneuver patrol;

sending the collected data to the maneuver center, maneuver center, or maneuver patrol center in real time;

collecting position information of the vehicle;

sending the collected position information to the maneuver center, maneuver center, or maneuver patrol center in real time;

analyzing, by 112 maneuver center, 119 maneuver center or maneuver patrol center, the situations and position information to determine whether it is required to control a track and stop of the vehicle;

reiteratively performing the operations until the feared state is removed, when it is determined that controlling the stop of the vehicle is not required; and

executing an ECU-linked process to control the stop of the vehicle and control a starting lock until the maneuver center, maneuver center, or the maneuver patrol arrives, when it is determined that controlling the stop of the vehicle is required.

20. The safe driving service method of claim 14, wherein said performing the selective safe driving service includes:

displaying a state of talking over a phone while driving;

executing a call prevention service process while driving to broadcast guidance for stopping to talk over the phone while driving with the vehicle terminal and induce a handsfree call, when the driver is talking over the phone while driving;

checking whether the driver voluntarily performs the handsfree call;

guiding provision of an automatic wireless handsfree calling with a vehicle terminal;

checking whether to allow provision of the automatic wireless handsfree calling, when it is determined that the driver does not voluntarily perform the handsfree call;

performing the automatic wireless handsfree calling to provide a handsfree call service, when it is determined that the provision is allowed;

executing an ECU-linked process to control an intensity of a handle, control a reduction in an acceleration pedal, and control stop of the vehicle onto a side road for building a traffic flow and a safe driving environment, when it is determined that the provision is not allowed;

executing a call abandonment induction service process to perform a call quality down function to induce stop of a call;

checking whether the driver abandons talking over a phone while driving; and

continuously performing the call quality down function until a call is ended when it is checked that the driver does not abandon talking over the phone while driving.