Cost effective and robust system and method for eye tracking and driver drowsiness identification

• US 9,483,695 B2
• Filed: 12/02/2010
• Issued: 11/01/2016
• Est. Priority Date: 12/02/2009
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: S., Chidanand K., Bhowmick, Brojeshwar
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Braniff, Christopher T
• Application Number:
  US13/513,495
• Publication Number:
  US 20130010096A1
• Time in Patent Office: 2,161 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A cost-effective and robust method for localizing and tracking drowsiness state of the eyes of driver by using images captured by near infrared (IR) camera disposed on the vehicle, the said method comprising the processor implemented steps of: Real-time tracking of the face and localizing eye bounding box within the face bounding box in the captured image by comparing the gray values with threshold using the segmentation process; tracking the eyes by computing the centroid of the eye, target model histogram and target candidate model histogram for one location to another by comparing them to identify distance and calculating the displacement of the target center by the weighted means, wherein the target model histogram and target candidate model histogram are computed based on the feature space; and detecting the drowsiness state of the eyes using histogram equalization, Morphological operations and texture based parameters using histogram and grey level co-occurrence matrices.

Claims

1. A computer implemented method for determining in real time a drowsiness state of a driver while driving by using images captured by a near infrared (IR) camera disposed on a vehicle, the method comprising:

determining a face bounding box by determining face coordinates using a segmentation process by collecting one or more features of a face and by determining a face height based on a difference between at least one of a nose tip position, an eye brow position, and co-ordinates of the eye brow position;

real time tracking of the face by collecting grey values of features of the face greater than threshold values of said features, obtained from the segmentation process;

tracking the eyes by computing a centroid of the eye, and calculating a target model histogram and a target candidate model histogram based on a range of intensity of a histogram equalized image and a morphology transformed image;

real time tracking of the eyes within a face bounding box and collecting a histogram equalization and a morphology transformation in the face bounding box;

calculating the distance between the target model histogram and the target candidate model histogram and calculating a displacement of the target centre; and

detecting a drowsiness state of a driver from the eyes by using at least one of histogram equalization, morphological operations and texture based parameters by using histogram and grey level co-occurrence matrices.

2. The method of claim 1, wherein performing the histogram equalization and the morphological operation results in a contrast enhancement of the face bounding box.

3. The method of claim 1, further comprising generating an alert for warning the driver based on the drowsiness state of the driver.

4. The method of claim 1, wherein the near IR camera is disposed outside or inside the vehicle, and wherein the near IR camera faces towards the driver.

5. The method of claim 1, wherein the tracking of the eyes in real time further comprise detecting a change in head position in a current frame with respect to a previous frame, wherein the change in the head position is detected using the Kernel tracking algorithm.

6. The method of claim 1, wherein detecting the drowsiness state of the driver further comprises:

extending a bounding box of the eyes upwards to a centroid of the eyebrows;

applying the histogram equalization on the bounding box of the eyes;

eliminating a brighter pupil effect of a histogram equalized image by computing a line erosion of the histogram equalized image with a structuring element, wherein a width of the structuring element is equal to ⅓

of eyebrow width and a height of the structuring element is equal to one; and

using a histogram based approach to texture analysis which is based on the intensity value concentrations on all or part of an image represented as a histogram to identify the state of the eye, wherein the value of uniformity or Angular secondary moment (ASM) texture parameter occurs high for closed eye and low for open eye.

7. A system for determining drowsiness state of a driver for avoiding accidents by using images captured by a near infrared (IR) camera disposed on a vehicle, the system comprises:

a processor; and

a memory coupled to the processor, wherein the processor is capable of executing programmed instructions stored in the memory to;

determine a face bounding box by determining face coordinates using a segmentation process by collecting one or more features of a face and determine a face height based on a difference between at least one of a nose tip position, an eye brow position, and co-ordinates of the eye brow position;

track the face real time by collecting grey values of features of the face greater than threshold values of said features, obtained from the segmentation process;

track the eyes by computing a centroid of the eye, and calculating a target model histogram and a target candidate model histogram based on a range of intensity of a histogram equalized image and a morphology transformed image, wherein the eyes are tracked real time within a face bounding box and a histogram equalization and a morphology transformation in the face bounding box are collected;

calculate the distance between the target model histogram and the target candidate model histogram and calculating a displacement of the target centre; and

detect a drowsiness state of a driver from the eyes by using at least one of histogram equalization, morphological operations and texture based parameters by using histogram and grey level co-occurrence matrices.

8. The system of claim 7, wherein the processor is further configured to generate an alert for warning the driver based on the drowsiness state of the driver.