Method and system for detecting a driving state of a driver in a vehicle

• US 8,552,873 B2
• Filed: 12/28/2010
• Issued: 10/08/2013
• Est. Priority Date: 12/28/2010
• Status: Active Grant

• Current Assignee: Automotive Research & Testing Center
• Original Assignee: Automotive Research & Testing Center
• Inventors: Chen, Chia-Tseng, Liao, Yu-Sheng, Chen, Yu-Sung
• Primary Examiner(s): Ghayour, Mohammad
• Assistant Examiner(s): WILSON, BRIAN P
• Application Number:
  US12/979,939
• Publication Number:
  US 20120161954A1
• Time in Patent Office: 1,015 Days
• Field of Search: None
• US Class Current: 340/576
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

In a method and system for detecting a driving state of a driver in a vehicle, a face recognition standard corresponding to an environmental condition within the vehicle is established based on a series of image frames generated by capturing consecutively images of the driver upon movement of the vehicle. Upon detection of a variance in the environmental condition during movement of vehicle, the face recognition standard is updated with a corrected face recognition standard corresponding to the environmental condition with the variance. It is determined based on the corrected face recognition standard and the image frames whether the driving state of the driver is a dangerous state, where the driver'"'"'s face does not face a front windshield of the vehicle or where the driver'"'"'s face faces the front windshield of the vehicle but one of the driver'"'"'s eyes is closed.

Claims

1. A method of detecting a driving state of a driver in a vehicle, comprising the steps of:

a) capturing consecutively images of the driver in the vehicle to generate a series of image frames;

b) establishing a face recognition standard corresponding to an environmental condition within the vehicle based on the image frames generated in step a) upon movement of the vehicle;

c) during movement of the vehicle, determining whether there is a variance in the environmental condition;

d) upon determining that there is a variance in the environmental condition, updating the face recognition standard with a corrected face recognition standard that corresponds to the environmental condition with the variance and that is generated based on the image frames generated in step a) under the environmental condition with the variance; and

e) determining, based on the corrected face recognition standard, and the image frames generated after the corrected face recognition standard is generated, whether the driving state of the driver is a dangerous state, where the driver'"'"'s face does not face a front windshield of the vehicle or where the driver'"'"'s face faces the front windshield of the vehicle but at least one of the driver'"'"'s eyes is closed;

wherein step e) further includes the sub-steps of;

e-1) generating a face detection result of a number of the image frames generated in step a) upon generation the corrected face recognition standard; and

e-2) determining whether the face detection result generated in sub-step e-1) matches the corrected face recognition standard, the driving state of the driver being in the dangerous state upon determining that the face detection result does not match the corrected face recognition standard;

wherein each of the face recognition standard and the corrected face recognition standard is associated with a facial feature of the driver that corresponds a portion of the driver'"'"'s face with at least one of eyes, nose and mouth; and

wherein;

each of the face recognition standard and the corrected face recognition standard includes a first threshold value that is obtained from face detection scores of sub-image frames of the image frames corresponding to the portion of the driver'"'"'s face;

the face detection result includes a first detection value that is an average value of face detection scores of sub-image frames of the number of the image frames corresponding to the portion of the driver'"'"'s face; and

when the first detection value is less than the first threshold value of the corrected face recognition standard, the face detection result does not match the corrected face recognition threshold standard, and the dangerous state indicates that the driver'"'"'s face does not face the front windshield of the vehicle.

2. The method as claimed in claim 1, further comprising the step of:

f) upon determining that the driving state of the drive is the dangerous state in step e), generating a warning output.

3. The method as claimed in claim 1, wherein the variance of the environmental condition includes at least one of variation in illumination within the vehicle, and variation in an orientation of an external light source relative to the driver'"'"'s face.

4. The method as claimed in claim 1, wherein each of the face recognition standard and the corrected face recognition standard is further associated with a predetermined open-eye image sample.

5. The method as claimed in claim 4, wherein:

each of the face recognition standard and the corrected face recognition standard further includes a second threshold value that is obtained from correlation coefficients between sub-image frames of the image frames corresponding to one driver'"'"'s eye, and the predetermined open-eye image sample;

the face detection result further includes a second detection value that is an average value of correlation coefficients between sub-image frames of the number of the image frames corresponding to the driver'"'"'s eye, and the predetermined open-eye image sample; and

when the first detection value is greater than the first threshold value of the corrected face recognition standard but the second detection value is less than the second threshold value of the corrected face recognition standard, the face detection result does not match the face recognition standard, and the dangerous state indicates that the driver'"'"'s face faces the front windshield of the vehicle but the driver'"'"'s eye is closed.

6. A system for detecting a driving state of a driver of a vehicle, comprising:

an image capturing unit for capturing consecutively images of the driver in the vehicle to generate a series of image frames;

a processing unit coupled to said image capturing unit, receiving the image frames from said image capturing unit, and operable to process the image frames to establish a face recognition standard corresponding to an environmental condition within the vehicle upon movement of the vehicle; and

a detecting unit coupled to said processing unit for detecting the environmental condition during movement of the vehicle, said detecting unit outputting a trigger signal to said processing unit upon detecting a variance in the environmental condition;

wherein said processing unit

generates a corrected face recognition standard corresponding to the environmental condition with the variance based on the image frames from said image capturing unit upon receipt of the trigger signal from said detecting unit,

updates the face recognition standard with the corrected face recognition standard, and

determines, based on the corrected face recognition standard and the image frames from said image capturing unit, whether the driving state of the driver is a dangerous state, where the driver'"'"'s face does not face a front windshield of the vehicle or where the driver'"'"'s face faces the front windshield of the vehicle but at least one of the driver'"'"'s eyes is closed;

wherein;

said processing unit is operable to generate a face detection result of a number of the image frames from said image capturing unit upon generation the corrected face recognition standard and to determine whether the face detection result matches the corrected face recognition standard; and

when said processing unit determines that the face detection result does not match the corrected face recognition standard, the driving state of the driver is the dangerous state;

wherein each of the face recognition standard and the corrected face recognition standard is associated with a facial feature of the driver that includes a portion of the driver'"'"'s face with at least one of eyes, nose and mouth; and

wherein said processing unit is operable so that;

each of the face recognition standard and the corrected face recognition standard includes a first threshold value that is associated with face detection scores of sub-image frames of the image frames corresponding to the portion of the driver'"'"'s face;

the face detection result includes a first detection value that is an average value of face detection scores of sub-image frames of the number of the image frames corresponding to the portion of the driver'"'"'s face; and

when the first detection value is less than the first threshold value of the corrected face recognition standard, the face detection result does not match the corrected face recognition standard, and the dangerous state indicates that the driver'"'"'s face does not face the front windshield of the vehicle.

7. The system as claimed in claim 6, wherein said processing unit is operable to generate a warning output upon determining that the driving state of the driver is the dangerous state, said system further comprising a warning unit coupled to said processing unit for receiving and reproducing the warning output from said processing unit.

8. The system as claimed in claim 6, wherein the variance of the environmental condition includes at least one of variation in illumination within the vehicle, and variation in an orientation of an external light source relative to the driver'"'"'s face.

9. The system as claimed in claim 6, wherein each of the face recognition standard and the corrected face recognition standard is further associated with a predetermined open-eye image sample.

10. The system as claimed in claim 9, wherein said processing unit is operable so that:

each of the face recognition standard and the corrected face recognition standard further includes a second threshold value that is obtained from correlation coefficients between sub-image frames of the image frames corresponding to one driver'"'"'s eye, and the predetermined open-eye image sample;

the face detection result further includes a second detection value that is an average value of correlation coefficients between sub-image frames of the number of the image frames corresponding to the driver'"'"'s eye, and the predetermined open-eye image sample; and

when the first detection value is greater than the first threshold value of the corrected face recognition standard but the second detection value is less than the second threshold value of the corrected face recognition standard, the face detection result does not match the face recognition standard, and the dangerous state indicates that the driver'"'"'s face faces the front windshield of the vehicle but the driver'"'"'s eye is closed.