DETERMINATION APPARATUS AND DETERMINATION METHOD

• US 20160171321A1
• Filed: 12/14/2015
• Published: 06/16/2016
• Est. Priority Date: 12/15/2014
• Status: Abandoned Application

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: OHSUGA, Shin, KOJIMA, Shinichi
• Application Number:
  US14/968,001
• Publication Number:
  US 20160171321A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/168 Evaluating attention deficit, hyperactivity
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A determination apparatus includes: a memory memorizing a three-dimensional face model in which a three-dimensional face shape and eye positions of a test subject, and change information mapping each state of change in eye shape to each shape parameter are registered; a matching unit matching a picked-up image of a driver picked up by an image pickup device with the three-dimensional face shape to specify the eye position; a shape parameter determination unit changing the shape parameter to specify a shape parameter corresponding to the eye state in the picked-up image; an eye-opening degree calculating unit calculating an eye-opening degree indicating a state of opening of the eye in the picked-up image; and a determination unit determining a state of a face and an eye of the driver on the basis of the eye-opening degree and the specified shape parameter.

Claims

1. A determination apparatus comprising:

a memory configured to memorize a three-dimensional face model in which a three-dimensional face shape of a test subject, eye positions of the test subject, and change information that maps each state of change in eye shape to each shape parameter are registered;

a matching unit configured to match a picked-up image of a driver picked up by an image pickup device with the three-dimensional face shape to specify the eye position;

a shape parameter determination unit configured to change the shape parameter to specify a shape parameter corresponding to the eye state in the picked-up image;

an eye-opening degree calculating unit configured to calculate an eye-opening degree indicating the state of opening of the eye in the picked-up image; and

a determination unit configured to determine the state of driver'"'"'s face and eye on the basis of the eye-opening degree and the specified shape parameter.

2. The determination apparatus according to claim 1, wherein

the determination unit determines whether or not the driver is in a state of gazing downward on the basis of the eye-opening degree and the specified shape parameter.

3. The determination apparatus according to claim 2, wherein

a state of changes in eye shape in the three-dimensional face model indicates changes from a standard state of the eyes to a downward-gaze state of the test subject,

the determination unit determines whether or not the specified shape parameter is within a range indicating the downward gaze when the eye-opening degree is within the range indicating that the eyes are closed, and determines that the driver is in a downward gaze state when the shape parameter is within the range indicating the downward gaze.

4. The determination apparatus according to claim 3, wherein

the determination unit determines that the driver is in an eye-closing state when the eye-opening degree is within the range indicating that the eyes are closed and when the specified shape parameter is out of the range indicating the downward gaze.

5. The determination apparatus according to claim 3 further comprising:

a processing unit configured to perform an action of alerting the driver when the driver is determined to be in the eye-closing state and when the driver is determined to be in the state of the downward gaze.

6. The determination apparatus according to claim 1, further comprising:

a measuring unit configured to obtain a three-dimensional face structure data including the three-dimensional face shape and the eye positions on the basis of the picked-up image of the test subject and the change information that maps each state of change in the eye shape to each shape parameter, and

a model creating unit configured to register the measured three-dimensional face structure data and the change information in the memory as the three-dimensional face model.

7. A determination method executed by the determination apparatus, the determination apparatus including:

a memory configured to memorize three-dimensional face models in which three-dimensional face shapes of a test subject, eye positions of the test subject, and change information that maps each state of change in eye shape to each shape parameter are registered, the determination method comprising;

matching a picked-up image of a driver picked up by an image pickup device with the three-dimensional face shape to specify the face direction, and specifying the eye position in the face direction,

changing the shape parameter to specify a shape parameter corresponding to the eye state in the picked-up image;

calculating an eye-opening degree indicating a state of opening of the eye in the picked-up image; and

determining states of driver'"'"'s face and eye on the basis of the eye-opening degree and the specified shape parameters.

8. An eye opening/closing determination apparatus comprising:

a memory configured to memorize three-dimensional face models in which three-dimensional face shapes of a test subject, positions of the eyes of the test subjects, and change information that maps each state of change of the eye shape from a standard state of the eyes to a state of squinted eyes in which the eyes are squinted in accordance with an expression change of the face to each shape parameter are registered,

a matching unit configured to match a picked-up image of a driver picked up by an image pickup device with the three-dimensional face shape to specify eye positions;

a shape parameter determination unit configured to change the shape parameter to specify a shape parameter corresponding to the eye state in the picked-up image;

an eye-opening degree calculating unit configured to calculate an eye-opening degree indicating a state of opening of the eye in the picked-up image; and

an eye opening/closing determination unit configured to determine opening and closing of the driver'"'"'s eyes on the basis of whether or not the specified shape parameter indicates a squinted eye state even when the eye-opening degree indicates the eye-closing state.

9. The determination apparatus according to claim 8, wherein

the state of squinted eyes includes a state of a smile, the eye opening/closing determination unit determines whether or not the eye-opening degree falls within a range indicating the eye-closing state, determines whether or not the specified shape parameter is within the range of a smile when the eye-opening degree falls within the range indicating the eye-closing state, and determines that the driver is in the eye-opening state when the shape parameter falls within the range indicating the smile.

10. The determination apparatus according to claim 9, wherein

the eye opening/closing determination unit determines that the driver is in the eye-closing state when the eye-opening degree is within the range indicating that the eyes are closed and when the specified shape parameter is out of the range indicating the smile.

11. The determination apparatus according to claim 10, further comprising:

a processing unit configured to perform an action of alerting the driver when the driver is determined to be in the eye-closing state.

12. The eye opening/closing determination apparatus according to claim 8, further comprising:

a measuring unit to obtain three-dimensional face structure data including the three-dimensional face shape and the eye positions on the basis of the picked-up image of the test subject and the change information that maps each state of the change in eye shape to each shape parameter, and

a model creating unit configured to register the measured three-dimensional face structure data and the change information in the memory as the three-dimensional face model.

13. An eye opening/closing determination method to be executed by an eye opening/closing determination apparatus, the eye opening/closing determination apparatus including:

a memory configured to memorize a three-dimensional face model in which a three-dimensional face shape of a test subject, eye positions of the test subject, and change information that maps each state of change of the eye shape from the standard state of the eyes to a state of squinted eyes in which the eyes are squinted in accordance with an expression change of the face to each shape parameters are registered, the determination method comprising;

matching a picked-up image of a driver picked up by an image pickup device with the three-dimensional face shape to specify the eye positions;

changing the shape parameter to specify the shape parameter corresponding to the eye state in the picked-up image;

calculating an eye-opening degree indicating the state of opening of the eye in the picked-up image; and

determining the opening and closing of the driver'"'"'s eyes on the basis of whether or not the specified shape parameter indicates a squinted eye state even when the eye-opening degree indicates the eye-closing state.