Information notification apparatus and information notification method

• US 9,932,000 B2
• Filed: 12/20/2016
• Issued: 04/03/2018
• Est. Priority Date: 01/06/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Aoki, Yasuhiro, Mizutani, Masami
• Primary Examiner(s): McNally, Kerri
• Assistant Examiner(s): TRAN, THANG DUC
• Application Number:
  US15/384,657
• Publication Number:
  US 20170190306A1
• Time in Patent Office: 469 Days
• Field of Search: 340576
• CPC Class Codes: B60R 16/0231 Circuits relating to the driving or the functioning of the vehicle monitoring tyres B60C23/00; indicating overspeed B60K31/00; for dash boards B60K37/00, B60Q3/10; for indicating emergencies B60Q1/52; brake control systems B60T; registering or indicating the working of vehicles G07C5/00; measuring distance G01C, e.g. combinations of speed and distance G01C23/00; engine indicators G01L; measuring speed or acceleration G01P
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An information notification apparatus includes a camera; a memory; and a processor coupled to the memory and configured to perform line-of-sight detection on an image captured by the camera, calculate a forward gaze degree of a gaze directed to a predetermined area in accordance with a line-of-sight direction obtained in line-of-sight detection; execute a notification to an outside in a first information if the forward gaze degree is lower than a first threshold; and execute the notification in a second information which information is less than the first information if the forward gaze degree is higher than or equal to the first threshold.

Claims

1. An information notification apparatus comprising:

a camera configured to capture an image of a driver of a vehicle;

a memory; and

a processor coupled to the memory and configured to;

perform line-of-sight detection based on the image captured by the camera;

calculate a forward gaze degree of a gaze of the driver in accordance with a line-of-sight direction obtained in the line-of-sight detection, the forward gaze degree indicates whether the gaze of the driver is within a predetermined area of a direction in which the vehicle is traveling;

provide a notification to the driver in a first information when the forward gaze degree is lower than a first threshold; and

provide the notification to the driver in a second information when the forward gaze degree is higher than or equal to the first threshold, the second information includes less than the first information,

wherein the processor further executes the notification in the second information when the forward gaze degree is lower than a second threshold which is lower than the first threshold,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.

2. The information notification apparatus according to claim 1, further comprising a display,

wherein the notification is a display output directed to the display.

3. The information notification apparatus according to claim 2, wherein:

according to a result in the line-of-sight detection, the processor further decides whether the notification that had been output as the display output in the second information has been visually recognized; and

when the notification that had been output as the display output in the second information has not been visually recognized within a predetermined time, the processor switches the display output of the notification from the second information to the first information.

4. The information notification apparatus according to claim 1, wherein when a number of times at which the forward gaze degree higher than or equal to the first threshold is calculated is higher than or equal to a predetermined value, the processor further executes a circumference check notification that prompts the driver to check of a circumference of the vehicle.

5. The information notification apparatus according to claim 1, wherein when a number of times which the forward gaze degree lower than the second threshold is calculated is higher than or equal to a predetermined value, the processor further executes a warning notification that warns the driver of being inattentive.

6. The information notification apparatus according to claim 1, wherein the processor further sets a class value of the forward gaze degree to the second threshold, the class value being obtained at an intersection between a distribution curve of the forward gaze degree histogram related to the first group and a distribution curve of the forward gaze degree histogram related to the third group, and also sets a class value of the forward gaze degree to the first threshold, the class value being obtained at an intersection between a distribution curve of the forward gaze degree histogram related to the second group and a distribution curve of the forward gaze degree histogram related to the third group.

7. The information notification apparatus according to claim 1, wherein the processor is further configured to:

classify images obtained by the camera based on the calculated forward gaze degree for each image, each image being classified as a forward visually recognized image or not a forward visually recognized image;

determine a ratio of a number of the classified forward visually recognized images to a number of images obtained by the camera within a predetermined period; and

provide one of the notification that prompts the driver to check of a circumference of the vehicle based on the determined ratio.

8. An information notification method comprising:

capturing, with a camera, an image of a driver of a vehicle;

performing, with a processor, line-of-sight detection based on the image captured by the camera,

calculating, with the processor, a forward gaze degree of a gaze of the driver in accordance with a line-of-sight direction obtained in line-of-sight detection, the forward gaze degree indicates whether the gaze of the driver is within a predetermined area of a direction in which the vehicle is travelling;

providing a notification to the driver in a first information when the forward gaze degree is lower than a first threshold; and

providing the notification in a second information when the forward gaze degree is higher than or equal to the first threshold, the second information includes less information than the first information,

wherein the processor further executes the notification in the second information when the forward gaze degree is lower than a second threshold which is lower than the first threshold,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.

9. A vehicle-mounted apparatus to provide notifications to a driver, the vehicle-mounted apparatus comprising:

a notification device configured to present notifications to the driver of a vehicle in a plurality of forms based on a workload of the driver;

a memory;

a processor coupled to the memory and the at least one notification device, the processor configured to;

acquire a travelling direction of the vehicle;

perform line-of-sight detection to determine a line-of-sight for the driver;

calculate a gaze degree associated with the driver, the gaze degree indicating frequency of the determined line-of-sight of the driver included within a predetermined range of the travelling direction of the vehicle;

classify the workload of the driver as low when the calculated forward gaze degree is lower than a threshold;

classify the workload of the driver as high when the calculated forward gaze degree is higher than the threshold;

provide a notification to the at least one notification device in a first information when the workload of the driver is classified as low; and

provide the notification to the at least one notification device in a second information when the workload of the driver is classified as high, the second information includes less information than the first information,

wherein the line-of-sight detection determines a line-of-sight vector for the driver that interconnects a central position of a pupil of the eyeball and a center of the curvature of the cornea, and the processor calculates the gaze degree by determining whether an angle formed by the line-of-sight vector and a vector of the traveling direction of the vehicle is within a predetermined range,

wherein the processor further classifies the forward gaze degree as a first group when a position at a time when the forward gaze degree was calculated is a traffic intersection, as a second group when an inter-vehicle distance at a time when the forward gaze degree was calculated is smaller than a predetermined value, and as a third group when the forward gaze degree is not classified as either the first group or the second group, after which the processor creates a forward gaze degree histogram for each of the first group, the second group, and the third group and sets the first threshold and the second threshold according to three forward gaze degree histograms related to the first group, the second group, and the third group.