Driver monitoring system

• US 9,682,622 B2
• Filed: 03/30/2015
• Issued: 06/20/2017
• Est. Priority Date: 05/15/2014
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Hong, Sungwook, Kim, Joomin, Kim, Sunghyun
• Primary Examiner(s): Fleming, Faye M
• Application Number:
  US14/672,520
• Publication Number:
  US 20150328985A1
• Time in Patent Office: 813 Days
• CPC Class Codes: A61B 5/0261 using optical means, e.g. infrared light
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/389 Electromyography [EMG]
• CPC Class Codes: A61B 5/398 Electrooculography [EOG], e.g. detecting nystagmus; Electroretinography [ERG]
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/681 Wristwatch-type devices
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 : Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G08B 21/06 : indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A driver state monitoring (DSM) system includes an image obtainment apparatus configured to obtain image information of a user driving a vehicle; a biological information obtainment unit configured to be worn on a specific portion of the user'"'"'s body and to obtain biological information from the specific portion of the user'"'"'s body; and a controller configured to detect a dangerous driving-state in which the user drives the vehicle, based on the image information and the biological information of the user.

Claims

1. A driver state monitoring (DSM) system comprising:

an image obtainment apparatus configured to obtain image information of a user driving a vehicle;

a biological information obtainment unit configured to be worn on a specific portion of the user'"'"'s body and to obtain biological information from the specific portion of the user'"'"'s body; and

a controller configured to detect a dangerous driving-state in which the user drives the vehicle based on a weighted combination of the image information and the biological information of the user.

2. The system of claim 1, wherein the controller is configured to, based on the image information and the biological information, determine whether the dangerous driving-state corresponds to a drowsy driving state, a distracted driving-state, or a stressful driving-state.

3. The system of claim 2, wherein the controller is configured to, based on the image information and the biological information, generate information on a danger level indicating an extent of danger corresponding to the determined dangerous driving-state.

4. The system of claim 3, wherein the controller is configured to generate the information on the determined danger level by applying a weighting factor to at least one of the image information or the biological information.

5. The system of claim 4, wherein the applied weighting factor differs depending on whether the dangerous driving-state corresponds to the drowsy driving state, the distracted driving-state, or the stressful driving-state.

6. The system of claim 3, wherein the controller is configured to output the generated information on the determined danger level through an output unit.

7. The system of claim 6, wherein the controller is configured to adjust an output strength of the information on the danger level as the information on the danger level changes, and wherein the information on the danger level is output according to the adjusted output strength.

8. The system of claim 7, wherein the output unit is provided in a mobile terminal and is configured to output the information on the danger level in the form of a vibration output or an audio output.

9. The system of claim 8, wherein the controller is configured to adjust the output strength by changing at least one of a frequency of the vibration output, an amplitude of the vibration output, an output period of the vibration output, an amplitude of the audio output, contents of the audio output, or an output period of the audio output.

10. The system of claim 6, wherein the output unit is provided in the vehicle, and a vehicle control apparatus provided in the vehicle is configured to control the output unit to output the information on the danger level in the form of an image output or an audio output, and

wherein the output unit is at least one of an emergency light, a light emitting diode (LED) installed in a dashboard of the vehicle, a vibration-enabled seat of the vehicle, or a vibration-enabled wheel steering apparatus of the vehicle.

11. The system of claim 10, wherein the controller is configured to adjust the output strength by changing at least one of an output period of the emergency light, a light color of the LED, an output period of the LED, a vibration frequency of the vibration-enabled seat or of the vibration-enabled wheel steering apparatus, an amplitude of the vibration, or an output period of the vibration.

12. The system of claim 2, wherein the biological information includes at least one of sleep-state information or biorhythm information of the user that is measured before the user drives the vehicle.

13. The system of claim 12, wherein the dangerous driving-state corresponds to a drowsy driving-state, and

wherein the controller is configured to generate a drowsiness trend line over time for the user, based on at least one of the sleep-state information or the biorhythm information, and to detect the dangerous driving-state, based on the generated drowsiness trend line and the image information.

14. The system of claim 2, wherein the controller is configured to generate a driving-state learning model for determining the user'"'"'s dangerous driving-state based on past image information and past biological information of the user, and to detect the dangerous driving-state based on the image information, the biological information, and the generated driving-state learning model.

15. The system of claim 1, further comprising a mobile terminal configured to be worn on the specific portion of the user'"'"'s body, wherein the mobile terminal comprises:

the biological information obtainment unit, and

a communication unit that receives the image information from an image obtainment apparatus provided in the vehicle.

16. The system of claim 15, wherein the mobile terminal further comprises the controller.