Apparatus for determining concentration of driver, system having the same, and method thereof

• US 10,657,397 B2
• Filed: 04/28/2017
• Issued: 05/19/2020
• Est. Priority Date: 11/08/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Ryu, Seong Sook, Kim, Jin Kwon, Lee, Byoung Joon, Kim, Sam Yong
• Primary Examiner(s): Rush, Eric
• Application Number:
  US15/581,070
• Publication Number:
  US 20180129891A1
• Time in Patent Office: 1,117 Days
• Field of Search: 382100, 382103, 382104, 382106, 382107, 382117, 382154, 382224, 382291, 382293, 382294, 340435, 340436, 340576, 340933, 340937, 340988, 348 77, 348 78, 348148, 348149, 351209, 351210, 701 1, 180272
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60Y 2302/03 Actuating a signal or alarm device
• CPC Class Codes: B60Y 2400/90 Driver alarms
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

An apparatus for determining a concentration of a driver includes: a region-of-interest definition processor configured to define a region-of-interest based on traffic-related information on front image information of a vehicle running; a driver eye-gaze detection processor configured to detect a plurality of eye-gaze points from driver image information of a driver in accordance with an eye-gaze movement of the driver; a driver concentration information calculation processor configured to calculate concentration information of each of the eye-gaze points; an image coordinate system processor configured to convert the region-of-interest to an image coordinate system and convert positions of the eye-gaze points to the image coordinate system; and a concentration determination processor configured to determine a concentration of the driver on the image coordinate system based on the concentration information.

Claims

1. An apparatus for determining a concentration of a driver, the apparatus comprising:

a region-of-interest definition processor configured to define a region-of-interest based on traffic-related information on front image information of a vehicle running;

a driver eye-gaze detection processor configured to detect a plurality of eye-gaze points from driver image information of a driver in accordance with an eye-gaze movement of the driver;

a driver concentration information calculation processor configured to calculate concentration information of each of the plurality of eye-gaze points;

an image coordinate system processor configured to convert the region-of-interest to an image coordinate system and convert positions of the plurality of eye-gaze points to the image coordinate system; and

a concentration determination processor configured to;

determine the concentration of the driver on the image coordinate system based on the concentration information,

calculate at least one or more of an average value of an eye-gaze movement speed, a dispersion value of the positions of the plurality of eye-gaze points, or an area value of the plurality of eye-gaze points from the concentration information of each of the plurality of eye-gaze points, and

decide a line connected between a first eye-gaze point and a last eye-gaze point in a monitoring unit pattern among the plurality of eye-gaze points as a reference line and calculate the dispersion value of the positions of the plurality of eye-gaze points using distances between the reference line and the plurality of eye-gaze points.

2. The apparatus of claim 1, further comprising a driver attention-guide control processor configured to provide at least one of a warning alarm, a guidance, and an attention-guide to the driver when the concentration of the driver is smaller than a reference concentration value.

3. The apparatus of claim 2, wherein the driver attention-guide control processor is configured to control a transfer ratio and a transfer time point of a driving control right, which enables the driver to control the vehicle, when the concentration of the driver is smaller than the reference concentration value and the vehicle is driven by a control support device.

4. The apparatus of claim 1, wherein the region-of-interest definition processor is configured to define the region-of-interest by adjusting a priority of information that the driver needs to check in accordance with a driving situation occurring in real time while driving.

5. The apparatus of claim 4, wherein the driving situation occurring in real time while driving comprises at least one of a variation in speed of a front-running vehicle in a driving lane, an interruption of a vehicle running in adjacent lane, a variation of safety distance in a direction of the vehicle, and a change of the driving lane.

6. The apparatus of claim 1, wherein the driver concentration information calculation processor is configured to calculate the positions of the plurality of eye-gaze points and the eye-gaze movement speed between the plurality of eye-gaze points as the concentration information.

7. The apparatus of claim 1, wherein the concentration determination processor is configured to calculate an area of a shape, which is defined by connecting the plurality of eye-gaze points to each other using a line, as the area value of the plurality of eye-gaze points.

8. The apparatus of claim 1, wherein the concentration determination processor is configured to calculate a relative distance between the region-of-interest and the plurality of eye-gaze points and determine a correlation degree between the region-of-interest and the plurality of eye-gaze points.

9. The apparatus of claim 8, wherein the concentration determination processor is configured to differentially quantify the concentration of the driver in accordance with the correlation degree between the region-of-interest and the plurality of eye-gaze points.

10. The apparatus of claim 1, wherein the concentration determination processor is configured to determine that the concentration of the driver is high or the driver gazes a point other than the region-of-interest when the area value of the plurality of eye-gaze points is smaller than a first reference value, the average value of the eye-gaze movement speed is smaller than a second reference value, or the dispersion value of the positions of the plurality of eye-gaze points is smaller than a third reference value.

11. A system for determining a concentration of a driver, the system comprising:

an image acquisition device configured to acquire a vehicle front image and a driver facial image;

a driver concentration determination processor configured to detect an eye-gaze movement of the driver from the driver facial image, calculate concentration information with respect to a plurality of eye-gaze points of the driver, and determine the concentration of the driver based on the concentration information; and

an output device configured to output a concentration guide warning alarm to the driver when the concentration of the driver is determined to be smaller than a reference value by the driver concentration determination processor,

wherein the driver concentration determination processor is further configured to calculate at least one or more of an average value of an eye-gaze movement speed, a dispersion value of positions of the plurality of eye-gaze points, or an area value of the plurality of eye-gaze points from the concentration information of the plurality of eye-gaze points, and

wherein the driver concentration determination processor is further configured to decide a line connected between a first eye-gaze point and a last eye-gaze point in a monitoring unit pattern among the plurality of eye-gaze points as a reference line and calculate the dispersion value of the positions of the plurality of eye-gaze points using distances between the reference line and the plurality of eye-gaze points.

12. The system of claim 11, wherein the output comprises:

a display configured to display at least one of a warning message, a warning light, and a warning screen of an augmented reality screen;

a speaker configured to output a warning sound; and

a vibrator configured to provide the driver with a vibration stimulus.

13. The system of claim 11, wherein the driver concentration determination processor comprises:

a region-of-interest definition processor configured to define a region-of-interest based on traffic-related information on front image information of a vehicle running;

a driver eye-gaze detection processor configured to detect the plurality of eye-gaze points from driver image information of a driver in accordance with an eye-gaze movement of the driver;

a driver concentration information calculation processor configured to calculate the concentration information of the plurality of eye-gaze points;

an image coordinate system processor configured to convert the region-of-interest to an image coordinate system and convert the positions of the plurality of eye-gaze points to the image coordinate system; and

a concentration determination processor configured to determine the concentration of the driver on the image coordinate system based on the concentration information.

14. The system of claim 13, wherein the driver concentration information calculation processor is configured to calculate positions of the plurality of eye-gaze points and the eye-gaze movement speed between the plurality of eye-gaze points as the concentration information.

15. A method for determining a concentration of a driver, the method comprising steps of:

defining a region-of-interest based on traffic-related information on front image information of a vehicle running;

detecting a plurality of eye-gaze points from driver image information of a driver in accordance with an eye-gaze movement of the driver;

calculating concentration information of each of the plurality of eye-gaze points;

converting the region-of-interest to an image coordinate system and converting positions of the plurality of eye-gaze points to the image coordinate system; and

determining the concentration of the driver on the image coordinate system based on the concentration information,

wherein the step of determining includes;

calculating at least one or more of an average value of an eye-gaze movement speed, a dispersion value of the positions of the plurality of eye-gaze points, or an area value of the plurality of eye-gaze points from the concentration information of the plurality of eye-gaze points; and

deciding a line connected between a first eye-gaze point and a last eye-gaze point in a monitoring unit pattern among the plurality of eye-gaze points as a reference line and calculating the dispersion value of the positions of the plurality of eye-gaze points using distances between the reference line and the plurality of eye-gaze points.

16. The method of claim 15, wherein the step of calculating includes calculating the positions of the plurality of eye-gaze points and the eye-gaze movement speed between the plurality of eye-gaze points as the concentration information.

17. The method of claim 15, wherein the step of determining further includes

comparing each of the average value of the eye-gaze movement speed, the dispersion value of the positions of the plurality of eye-gaze points, and the area value of the plurality of eye-gaze points with a corresponding reference value to determine the concentration of the driver.