Emotion-based vehicle service system, emotion cognition processing apparatus, safe driving apparatus, and emotion-based safe driving service method

• US 8,862,317 B2
• Filed: 08/24/2012
• Issued: 10/14/2014
• Est. Priority Date: 08/29/2011
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Shin, Hyun Soon, Lee, Yong Kwi, Jo, Jun
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US13/594,003
• Publication Number:
  US 20130054090A1
• Time in Patent Office: 781 Days
• Field of Search: 701/36
• US Class Current: 701/36
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: Y02T 10/84 Data processing systems or methods, management, administration

Abstract

An emotion-based vehicle service system for safe driving includes a multi-emotion sensor node configured to sense emotion information of a driver and passenger in a vehicle and condition information of the vehicle. The emotion-based vehicle service system further includes an emotion cognition processing apparatus configured to analyze sensing information provided from the multi-emotion sensor node to generate emotion cognition information, and a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information.

Claims

1. An emotion-based vehicle service system for safe driving, including:

a multi-emotion sensor node configured to sense emotion information of a user in a vehicle and condition information of the vehicle;

an emotion cognition processing apparatus configured to analyze sensing information provided from the multi-emotion sensor node to generate emotion cognition information for cognizing the emotion of the user; and

a safe driving service apparatus configured to provide an emotion-based safe driving service depending on the emotion cognition information,

wherein the safe driving service apparatus is configured to;

display a drowsy driving state when a current condition of the driver is in the drowsy driving state;

execute an ECU-linked process for controlling safe driving in the drowsy driving state to control a reduction in a vehicle speed, an intensity of a handle, or an intensity of a pedal in accordance with a drowsy intensity;

perform a drowsy emotion service to perform guide broadcasting to a drowsy state with a vehicle terminal and ventilate air in the vehicle through the control of an air conditioning system;

provide a music service for awaking drowsiness, and performing guide broadcasting for inducing parking of the vehicle onto a side road;

continuously monitor a drowsy state to induce parking of the vehicle onto the side road in operational connection to a maneuver patrol when the drowsy state is continuously maintained;

control parking of the vehicle onto the side road to stop drowsy driving by linking up with the ECU; and

inform the driver being awaked from the drowsy state to a drivable state.

2. The emotion-based vehicle service system of claim 1, wherein the multi-emotion sensor node is constituted with at least one of a handle sensor node, a driver seat sensor node, a passenger seat sensor node, a transmission sensor node, a ceiling sensor node, and a front glass sensor node.

3. The emotion-based vehicle service system of claim 2, wherein the multi-emotion sensor node includes at least one of a heart rate sensor, a skin reaction sensor, a body temperature sensor, a voice sensor, an image sensor, an acceleration sensor, and a slope sensor.

4. The emotion-based vehicle service system of claim 1, wherein the emotion cognition processing apparatus is further configured to combine the emotion information of the user and the condition information of the vehicle for analyzing the combined emotion information and the condition information to cognize the emotion of the user.

5. A safe driving service method of a multi-emotion sensor node including a plurality of sensors, the safe driving service method including:

recognizing a driver and a passenger in a vehicle to objectify a sensing target;

sensing emotion cognition information using the plurality of sensors, wherein the emotion cognition information includes emotion information of the driver and passenger and condition information of the vehicle;

analyzing the emotion cognition information;

performing a safe driving service based on the analyzed emotion cognition information;

converting the analyzed emotion cognition information into message information; and

transmitting the message information corresponding to the emotion cognition information to an emotion recognition processing apparatus over a network,

wherein said performing the safe driving service includes;

displaying a drowsy driving state when a current condition of the driver is in the drowsy driving state;

executing an ECU-linked process for controlling safe driving in the drowsy driving state to control a reduction in a vehicle speed, an intensity of a handle, or an intensity of a pedal in accordance with a drowsy intensity;

performing a drowsy emotion service to perform guide broadcasting to a drowsy state with a vehicle terminal and ventilate air in the vehicle through the control of an air conditioning system;

providing a music service for awaking drowsiness, and performing guide broadcasting for inducing parking of the vehicle onto a side road;

continuously monitoring a drowsy state to induce parking of the vehicle onto the side road in operational connection to a maneuver patrol when the drowsy state is continuously maintained;

controlling parking of the vehicle onto the side road to stop drowsy driving by linking up with the ECU; and

informing the driver being awaked from the drowsy state to a drivable state.

6. The safe driving service method of claim 5, wherein the sensors include one or more a heart rate sensor, a skin reaction sensor, a body temperature sensor, a voice sensor, an image sensor, and an acceleration and slope sensor.

7. The safe driving service method of claim 6, wherein said analyzing the emotion cognition information includes:

analyzing a plurality of signals sensed by the heart rate sensor to detect an HRV/pulse wave, a hematocele oxidization saturation, and intensity of a hematocele;

analyzing a plurality of signals sensed by the skin reaction sensor to detect a skin conductance;

analyzing a signal sensed by the body temperature sensor to detect a skin temperature;

analyzing a signal sensed by the voice sensor to detect voice and a sound wave;

analyzing a signal sensed by the image sensor to detect conditions of a pupil and face; and

analyzing a signal sensed by the acceleration and slope sensor to detect a motion and an activity.

8. An emotion-based safe driving service apparatus, including:

a vehicle terminal user interface (UI) configured to provide a UI environment of the safe driving service apparatus;

an emotion information management module configured to manage emotion cognition information provided thereto;

an engine control unit (ECU)-linked safe driving control module configured to control ECU-linked safe driving of a vehicle in accordance with to the emotion cognition information managed by the emotion information management unit; and

an emotion care safe driving service module configured to provide an emotion care safe driving service to the vehicle when the vehicle based on the emotion cognition information is controlled by the ECU-linked safe driving control module, wherein the emotion care safe driving service module includes;

a vehicle external UI platform;

a guide broadcaster configured to provide emotion care-related information around a vehicle;

an emotion care audio servicer configured to provide emotion care-related audio information;

an emotion care video servicer configured to providing emotion care-related video information;

a vehicle condition image transmitter configured to provide real-time traffic information around the vehicle as an image;

a vehicle condition information message processor configured to process an operation of sending an emergency report message, a message for reporting to an emergency service center, or a message for sending notice information to an acquaintance;

an illumination setter configured to set emotion care-related illumination; and

a ventilation adjuster configured to adjust air ventilation associated with emotion care around the vehicle.

9. The emotion-based safe driving service apparatus of claim 8, wherein the emotion information management module includes:

a user emotion information manager;

an engine control unit (ECU) linker;

a vehicle condition information manager;

a personal terminal linkage manager; and

an external network connection interface.

10. The emotion-based safe driving service apparatus of claim 8, wherein the ECU-linked safe driving control module includes:

a handle controller,

an acceleration pedal controller;

a break controller;

a starting lock controller;

an automatic vehicle speed reduction controller; and

an automatic vehicle stop controller.

11. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a drowsy driving state when a current condition of the driver is in the drowsy driving state;

executing an ECU-linked process for controlling safe driving in the drowsy driving state to control a reduction in a vehicle speed, an intensity of a handle, or an intensity of a pedal in accordance with a drowsy intensity;

performing a drowsy emotion service to perform guide broadcasting to a drowsy state with a vehicle terminal and ventilate air in the vehicle through the control of an air conditioning system;

providing a music service for awaking drowsiness, and performing guide broadcasting for inducing parking of the vehicle onto a side road;

continuously monitoring a drowsy state to induce parking of the vehicle onto the side road in operational connection to a maneuver patrol when the drowsy state is continuously maintained;

controlling parking of the vehicle onto the side road to stop drowsy driving by linking up with the ECU; and

informing the driver being awaked from the drowsy state to a drivable state.

12. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a driver-fatigued state when a current condition of the driver is a fatigued or health-deteriorated state;

executing a safe driving service process in accordance with a health condition of the driver to perform guide broadcasting for a fatigue and health condition with a vehicle terminal and display the health condition on the vehicle terminal;

determining whether the fatigue and health condition reaches an undrivable threshold;

reporting to an emergency service when it is determined that the fatigue and health condition of the driver is an undrivable state;

collecting position information of the vehicle;

notifying an acquaintance of a health-deteriorated state; and

executing an ECU-linked process for controlling safe driving in a fatigued state to control a safe speed of the vehicle and control safe driving to a destination in operational connection with a vehicle terminal navigation when it is determined that the fatigue and health condition of the driver is not the undrivable state.

13. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a drunk driving state when a current condition of the driver is a drunk driving state;

executing a drunk driving prevention service process to display an alcohol concentration with a vehicle terminal;

providing guide broadcasting for warning against drunk driving with the vehicle terminal;

determining whether the vehicle is currently stopped;

executing an ECU-linked process to control parking of the vehicle onto a side road and control a starting lock such that the driver cannot drive the vehicle in the drunk state any longer, when it is determined that the vehicle is continuously driven;

collecting position information of the vehicle; and

notifying a maneuver patrol of the collected position information.

14. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a reckless driving state of the driver;

executing a reckless driving prevention service process to provide guide broadcasting for warning against reckless driving with the vehicle terminal, when a current condition of the driver is an angry state;

checking whether the driver is continuously performing reckless driving;

executing an ECU-linked process to control an intensity of a handle and control an intensity of an acceleration pedal, when it is checked that the driver is continuously performing the reckless driving;

reproducing an emotion care music;

notifying a maneuver patrol of information on a reckless-driving vehicle in automatic connection with the maneuver patrol; and

collecting position information of the vehicle to notify the maneuver patrol of the collected position information.

15. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a feared state in the vehicle;

executing a rescue service process for rescuing the driver from the feared state to provide a service which protects the driver from fear, when it is determined that the driver or the passenger has a sensation of fear;

collecting real-time image or voice data on situations inside the vehicle;

automatically reporting to a maneuver center, maneuver center, or maneuver patrol center a maneuver patrol;

sending the collected data to the maneuver center, maneuver center, or maneuver patrol center in real time;

collecting position information of the vehicle;

sending the collected position information to the maneuver center, maneuver center, or maneuver patrol center in real time;

analyzing, by maneuver center, maneuver center or maneuver patrol center, the situations and position information to determine whether it is required to control a track and stop of the vehicle;

reiteratively performing the operations until the feared state is removed, when it is determined that controlling the stop of the vehicle is not required; and

executing an ECU-linked process to control the stop of the vehicle and control a starting lock until the maneuver center, maneuver center, or the maneuver patrol arrives, when it is determined that controlling the stop of the vehicle is required.

16. A safe driving service method of an emotion-based safe driving service apparatus, the safe driving service method including:

classifying a sensing signal received from a multi-emotion sensor node;

analyzing the classified sensing signal depending on characteristic;

applying an emotion inference algorithm to a driver and a passenger in a vehicle in accordance with the analyzed result to extract emotion information of the driver and passenger;

analyzing the classified sensing signal depending on characteristic of the sensor node to extract condition information of the vehicle;

analyzing the emotion information of the driver and passenger and the condition information of the vehicle to provide the analyzed result for a safe driving service; and

performing a selective safe driving service depending on the analyzed result, wherein said performing the selective safe driving service includes;

displaying a state of talking over a phone while driving;

executing a call prevention service process while driving to broadcast guidance for stopping to talk over the phone while driving with the vehicle terminal and induce a handsfree call, when the driver is talking over the phone while driving;

checking whether the driver voluntarily performs the handsfree call;

guiding provision of an automatic wireless handsfree calling with a vehicle terminal;

checking whether to allow provision of the automatic wireless handsfree calling, when it is determined that the driver does not voluntarily perform the handsfree call;

performing the automatic wireless handsfree calling to provide a handsfree call service, when it is determined that the provision is allowed;

executing an ECU-linked process to control an intensity of a handle, control a reduction in an acceleration pedal, and control stop of the vehicle onto a side road for building a traffic flow and a safe driving environment, when it is determined that the provision is not allowed;

executing a call abandonment induction service process to perform a call quality down function to induce stop of a call;

checking whether the driver abandons talking over a phone while driving; and

continuously performing the call quality down function until a call is ended when it is checked that the driver does not abandon talking over the phone while driving.