Method and a system for detecting drowsiness state of a vehicle user

• US 10,121,084 B2
• Filed: 03/20/2017
• Issued: 11/06/2018
• Est. Priority Date: 03/07/2017
• Status: Active Grant

• Current Assignee: Knapp Investment Company Limited
• Original Assignee: Wipro Limited
• Inventors: Yadhunandan, Ullam Subbaraya, Kalra, Ankita, Jain, Rahul
• Primary Examiner(s): Gilliard, Delomia L
• Application Number:
  US15/463,455
• Publication Number:
  US 20180260641A1
• Time in Patent Office: 596 Days
• Field of Search: None
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

Disclosed subject matter relates generally to image processing that includes a method for detecting drowsiness state of a vehicle user independent of factors such ethnicities, gender and other differences of an individual. A drowsiness detection system receives current images of the vehicle user from an image capturing device in a current time frame. Further, an eye closure ratio of the vehicle user is determined in the current time frame using eye closure parameters extracted from the current images in real-time and a profile of the vehicle user. Further, the eye closure ratio is normalized using a scaling factor computed in real-time using normalizing parameters extracted from the current images in real-time and the profile. Finally, a Percentage Eye Closure (PEC) value of the vehicle user is determined in the current time frame using the normalized eye closure ratio of the vehicle to detect drowsiness state of the vehicle user.

Claims

1. A method for detecting drowsiness state of a vehicle user, the method comprising:

receiving, by a drowsiness detection system, one or more current images of the vehicle user from an image capturing device associated with the drowsiness detection system in a current time frame;

determining, by the drowsiness detection system, an eye closure ratio of the vehicle user in the current time frame using one or more eye closure parameters extracted from the one or more current images in real-time, and a profile of the vehicle user received from a user profile database associated with the drowsiness detection system;

normalizing, by the drowsiness detection system, the eye closure ratio using a scaling factor computed in real-time, wherein the scaling factor is computed using one or more normalizing parameters extracted from the one or more current images in real-time and the profile of the vehicle user;

determining, by the drowsiness detection system, a Percentage Eye Closure (PEC) value of the vehicle user in the current time frame using the normalized eye closure ratio of the vehicle user; and

comparing, by the drowsiness detection system, the PEC value of the current time frame and PEC values of plurality of previous time frames with a predefined threshold to detect drowsiness state of the vehicle user.

2. The method as claimed in claim 1, wherein the PEC value of each of the previous time frames is determined by reiterating the steps of determining the eye closure ratio, normalizing the eye closure ratio and determining the PEC value for each of the plurality of previous time frames.

3. The method as claimed in claim 1 further comprising notifying, by the drowsiness detection system, the drowsiness state of the vehicle user to one or more end users.

4. The method as claimed in claim 1, wherein the profile of the vehicle user is created by:

capturing, by the drowsiness detection system, one or more images of eyes and face of the vehicle user using the image capturing device, wherein the vehicle user is in a stationary position;

extracting, by the drowsiness detection system, plurality of eye parameters and plurality of face parameters from the one or more images;

creating, by the drowsiness detection system, the profile of the vehicle user comprising the plurality of eye parameters and the plurality of face parameters; and

storing, by the drowsiness detection system, the profile of the vehicle user in the user profile database.

5. The method as claimed in claim 4, wherein the plurality of eye parameters comprises average height of an eye, average width of the eye, maximum distance between upper eye lash and eyebrow of the vehicle user, and minimum distance between upper eye lash and eyebrow of the vehicle user.

6. The method as claimed in claim 4, wherein the plurality of face parameters comprises average height of the face, average width of the face, location of the eye on the face, and distance of the face from the image capturing device.

7. The method as claimed in claim 1, wherein the one or more eye closure parameters comprises distance between upper eye lash and eyebrow of the vehicle user in the current time frame and distance between lower eye lash and the eyebrow of the vehicle user in the current time frame.

8. The method as claimed in claim 1, wherein the one or more normalization parameters comprises width of face of the vehicle user in the current time frame, height of the face in the current time frame, and distance between the face and the image capturing device in the current time frame.

9. The method as claimed in claim 1, wherein determining the PEC value comprises:

determining, by the drowsiness detection system, an intermediate-PEC value using the normalized eye closure ratio in the current time frame;

comparing, by the drowsiness detection system, the intermediate-PEC value with a dynamically predicted PEC value based on historical data; and

allocating, by the drowsiness detection system, the intermediate-PEC value as the PEC value if deviation between the intermediate-PEC value and the dynamically predicted PEC value is within a predefined range.

10. The method as claimed in claim 9, further comprising allocating, by the drowsiness detection system, the dynamically predicted PEC value as the PEC value if the deviation between the intermediate-PEC value and the dynamically predicted PEC value exceeds the predefined range.

11. A drowsiness detection system for detecting drowsiness state of a vehicle user, the drowsiness detection system comprising:

a processor; and

a memory communicatively coupled to the processor, wherein the memory stores the processor-executable instructions, which, on execution, causes the processor to;

receive one or more current images of the vehicle user from an image capturing device associated with the drowsiness detection system in a current time frame;

determine an eye closure ratio of the vehicle user in the current time frame using one or more eye closure parameters extracted from the one or more current images in real-time and a profile of the vehicle user received from a user profile database associated with the drowsiness detection system;

normalize the eye closure ratio using a scaling factor computed in real-time, wherein the scaling factor is computed using one or more normalizing parameters extracted from the one or more current images in real-time and the profile of the vehicle user;

determine a Percentage Eye Closure (PEC) value of the vehicle user in the current time frame using the normalized eye closure ratio of the vehicle user; and

compare the PEC value of the current time frame and PEC values of plurality of previous time frames with a predefined threshold to detect drowsiness state of the vehicle user.

12. The drowsiness detection system as claimed in claim 11, wherein the processor determines the PEC values of each of the plurality of previous time frames by reiterating the steps of determining the eye closure ratio, normalizing the eye closure ratio, and determining the PEC value for each of the plurality of previous time frames.

13. The drowsiness detection system as claimed in claim 11, wherein the processor further notifies to one or more end users, the drowsiness state of the vehicle user.

14. The drowsiness detection system as claimed in claim 11, wherein to create the profile of the vehicle user, the instructions cause the processor to:

capture one or more images of eyes and face of the vehicle user using the image capturing device, wherein the vehicle user is in a stationary position;

extract plurality of eye parameters and plurality of face parameters from the one or more images;

create the profile of the vehicle user comprising the plurality of eye parameters and the plurality of face parameters; and

store the profile of the vehicle user in the user profile database.

15. The drowsiness detection system as claimed in claim 14, wherein the plurality of eye parameters comprises average height of an eye, average width of the eye, maximum distance between upper eye lash and eyebrow of the vehicle user, and minimum distance between upper eye lash and eyebrow of the vehicle user.

16. The drowsiness detection system as claimed in claim 14, wherein the plurality of face parameters comprises average height of the face, average width of the face, location of the eye on the face, and distance of the face from the image capturing device.

17. The drowsiness detection system as claimed in claim 11, wherein the one or more eye closure parameters comprises distance between upper eye lash and eyebrow of the vehicle user in the current time frame, and distance between lower eye lash and the eyebrow of the vehicle user in the current time frame.

18. The drowsiness detection system as claimed in claim 11, wherein the one or more normalization parameters comprises width of face of the vehicle user in the current time frame, height of the face in the current time frame, and distance between the face and the image capturing device in the current time frame.

19. The drowsiness detection system as claimed in claim 11, wherein to determine the PEC value, the instructions cause the processor to:

determine an intermediate-PEC value using the normalized eye closure ratio in the current time frame;

compare the intermediate-PEC value with a dynamically predicted PEC value based on historical data; and

allocate the intermediate-PEC value as the PEC value if deviation between the intermediate-PEC value and the dynamically predicted PEC value is within a predefined range.

20. A non-transitory computer readable medium including instructions stored thereon that when processed by at least one processor causes a drowsiness detection system to perform operations comprising:

receiving one or more current images of a vehicle user from an image capturing device associated with the drowsiness detection system in a current time frame;

determining an eye closure ratio of the vehicle user in the current time frame using one or more eye closure parameters extracted from the one or more current images in real-time and a profile of the vehicle user received from a user profile database associated with the drowsiness detection system;

normalizing the eye closure ratio using a scaling factor computed in real-time, wherein the scaling factor is computed using one or more normalizing parameters extracted from the one or more current images in real-time and the profile of the vehicle user;

determining a Percentage Eye Closure (PEC) value of the vehicle user in the current time frame using the normalized eye closure ratio of the vehicle user; and

comparing the PEC value of the current time frame and PEC values of plurality of previous time frames with a predefined threshold to detect drowsiness state of the vehicle user.