Object tracking and eye state identification method

• US 20050232461A1
• Filed: 04/20/2004
• Published: 10/20/2005
• Est. Priority Date: 04/20/2004
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Hammoud, Riad I.
• Granted Patent: US 7,362,885 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

An object tracking method tracks a target object between successively generated infrared video images using a grey-scale hat filter to extract the target object from the background. The filtered image is binarized, and candidate binary blobs are extracted. The binary blob that minimizes the Euclidian spatial distance to the previous position of the object and satisfies a specified appearance model is selected, and its center of mass is taken as the current position of the object. Where the object is a person'"'"'s eye, the eye state and decision confidence are determined by analyzing the shape and appearance of the binary blob along with changes in its size and the previous eye state, and applying corresponding parameters to an eye state decision matrix.

Claims

1. A method of tracking movement of an object between first and second successively generated video images after a position of the object in said first video image has been identified, comprising the steps of:

defining a first state vector for the first video image corresponding to the identified position of said object;

defining a search window in said second video image based on said first state vector;

bottom hat filtering said search window with a non-flat structuring element to form a filtered image;

binarizing said filtered image and identifying candidate binary blobs that possibly correspond to said object;

computing a spatial Euclidian distance between each candidate binary blob and said first state vector, and selecting a candidate binary blob for which the computed spatial Euclidian distance is smallest;

determining a center of mass of the selected binary blob; and

defining a second state vector based on said center of mass for identifying the location of said object in said second video image.

2. The method of claim 1, wherein the step of bottom hat filtering said search window includes the steps of:

scanning said non-flat structuring element over said search window using a first convolution function to form a dilation image;

rotating said structuring element;

scanning the rotated structuring element over said dilation image using a second convolution function to form an erosion image; and

subtracting said erosion image from said search window to form said filtered image.

3. The method of claim 1, wherein said non-flat structuring element is ellipsoidal.

4. The method of claim 1, where the object is a person'"'"'s eye, the method including the steps of:

establishing an eye model defining image characteristics of human eyes and a non-eye model defining image characteristics of facial features other than human eyes;

extracting a patch of said search window based on said center of mass;

computing deviations of said patch from said eye model and said non-eye model; and

rejecting the selected candidate binary blob when the deviation of said patch from said eye model is greater than the deviation of said patch from said non-eye model.

5. The method of claim 4, wherein said eye model includes an open-eye mode defining image characteristics of open eyes and a closed-eye model defining image characteristics of closed eyes, and the method includes the steps of:

computing deviations of said patch from said open-eye model and said closed-eye model; and

rejecting the selected candidate binary blob when the deviation of said patch from said non-eye model is less than both the deviation of said patch from said open-eye model and the deviation of said patch from said closed-eye model.

6. The method of claim 1, where the object is a subject'"'"'s eye, the method including the steps of:

selecting candidate binary blobs in successively generated video images based on minimum spatial Euclidian distance;

determining a size of the selected candidate binary blobs, and computing a change in size of successively selected candidate binary blobs; and

determining an eye state designating either that said eye is open or that said eye is closed based on a sign of said change in size when said change in size has a magnitude that exceeds a threshold magnitude.

7. The method of claim 6, including the steps of:

establishing an open-eye model defining image characteristics of open eyes and a closed-eye model defining image characteristics of closed eyes;

computing deviations of said patch from said open-eye model and said closed-eye model;

forming a model-based determination that said eye is open when the deviation of said patch from said open-eye model is less than the deviation of said patch from said closed-eye model, and closed when the deviation of said patch from said closed-eye model is less than the deviation of said patch from said open-eye model;

determining said eye state based on said model-based determination and said sign of said change in size.

8. The method of claim 6, including the step of:

determining a shape parameter of said selected candidate binary blob according to a difference in standard deviation calculations along orthogonal axes;

forming an appearance-based determination that said eye is open if said shape parameter is less than a predefined threshold, and closed if said shape parameter is greater than said predefined threshold; and

determining said eye state based on said appearance-based determination and said sign of said change in size.

9. The method of claim 6, including the step of:

determining said eye state based on said sign of said change in size and a previously determined eye state.

10. The method of claim 9, including the steps of:

applying at least said sign of said change in size and said previously determined eye state to a decision matrix to determine said eye state and a confidence that the determined eye state is accurate.