Adapting a warning output based on a driver's view

• US 9,262,924 B2
• Filed: 07/09/2014
• Issued: 02/16/2016
• Est. Priority Date: 07/09/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: McNew, John-Michael
• Primary Examiner(s): TRIEU, VAN THANH
• Application Number:
  US14/327,019
• Publication Number:
  US 20160009175A1
• Time in Patent Office: 587 Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3652 Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08B 5/22 using electric transmission; using electromagnetic transmission
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

A human machine interface (HMI) can communicate warnings and messages to a driver by selecting among a variety of warning devices based on information related to the driver'"'"'s viewing angle. The variety of warning devices can include a primary-visual warning device (e.g., a heads up display), secondary-visual warning devices (e.g., an instrument cluster or a navigational screen), a portable device (e.g., a tablet or a cellular phone), an audio system, or a haptic device. The HMI apparatus displays warnings on visual warning devices within the driver'"'"'s view, can signal an active portable device to output a warning. Visual warnings can include directional cues. Audio warnings may accompany the visual warnings, and can be output with greater intensity when no visual devices are within the driver'"'"'s view.

Claims

1. An apparatus comprising processing circuitry configured to:

determine which of a plurality of visual warning devices is within a first determined viewing angle of a driver; and

signal a first visual warning device, from among the plurality of visual warning devices, to output a first visual warning when the first visual warning device is determined to be within the first determined viewing angle of the driver.

2. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

determine the viewing angle of the driver based on one of an image of the driver, an image in the direction of a view of the driver, and positional data of the driver.

3. The apparatus according to claim 2, wherein the processing circuitry is further configured to:

determine whether none of the plurality of visual warning devices are within the first determined viewing angle of the driver; and

signal an at least one non-visual warning device to output a first non-visual warning at a first warning level when none of the plurality of visual warning devices are within the first determined viewing angle of the driver.

4. The apparatus according to claim 3, wherein the processing circuitry is further configured to:

determine an activity status of a portable device; and

signal the portable device to output a portable-device warning when the portable device is determined to be active based on the activity status of the portable device.

5. The apparatus according to claim 4, wherein the processing circuitry is further configured to:

determine whether the portable device is within the viewing angle of the driver; and

signal the portable device to output the portable-device warning when the portable device is determined to be recently active and the portable device is determined to be within the viewing angle of the driver.

6. The apparatus according to claim 4, wherein the processing circuitry is further configured to:

determine whether the driver is looking towards the portable device based on an image from the portable device; and

signal the portable device to output the portable-device warning when either there is a determination that the driver is looking towards the portable device or when the determination of whether the driver is looking towards the portable device is inconclusive.

7. The apparatus according to claim 4, wherein the processing circuitry is further configured to:

determine whether a second visual warning device of the plurality of visual warning devices is within a second determined viewing angle of the driver; and

signal the second visual warning device to output a second visual warning when the second visual warning device is within the second determined viewing angle of the driver.

8. The apparatus according to claim 7, wherein the processing circuitry is further configured to:

signal the at least one non-visual warning device to output an augmenting non-visual warning at a second warning level when at least one of the plurality of visual warning devices is outputting a first visual warning, wherein the second warning level is at a level that is lower than the first warning level.

9. The apparatus according to claim 8, wherein the processing circuitry is further configured to:

modify at least one of the first visual warning, the second visual warning, the first non-visual warning, the augmenting non-visual warning, and the portable-device warning to increase a level of the respective warning when the response of the driver does not satisfy a first predetermined criterion.

10. The apparatus according to claim 7, wherein the processing circuitry is further configured to:

determine the first visual warning device is within the first determined viewing angle of the driver according to the first determined viewing angle corresponding to a direct view of the driver; and

determine the second visual warning device is within the second determined viewing angle of the driver according to the second determined viewing angle corresponding to an extended view of the driver.

11. The apparatus according to claim 4, wherein the processing circuitry is further configured to:

obtain an activity log of the portable device;

compare the activity log to an active criterion and to a recently-active criterion;

determine the portable device is active when the activity log satisfies the active criterion;

determine the portable device is recently-active when the activity log satisfies the recently-active criterion and does not satisfy the active criterion; and

determine the portable device is inactive when the activity log does not satisfy the recently-active criterion and does not satisfy the active criterion.

12. The apparatus according to claim 4, wherein the at least one non-visual warning device includes at least one of a mechanical transducer in the brake pedal, a mechanical transducer in the driver'"'"'s seat, a heating ventilation and air conditioning system, a mechanical transducer that tightens the seat belt, an audio speaker, a haptic warning device, and a auditory warning device.

13. The apparatus according to claim 4, wherein the plurality of visual warning devices includes at least one of a heads-up display, a navigational screen, an instrument cluster, a side view mirror, and a rear view mirror.

14. The apparatus according to claim 4, wherein:

the at least one non-visual warning device includes a visual output;

the portable-device warning is at least one of a visual warning, an auditory warning, and a haptic warning; and

at least one of the plurality of visual warning devices includes a non-visual output, including an auditory output or a haptic output.

15. The apparatus according to claim 4, wherein at least one of the first visual warning, the first non-visual warning, and the portable-device warning provides directional cues that guide a view of the driver in a predetermined direction.

16. The apparatus according to claim 3, wherein the processing circuitry is further configured to:

signal at least one of the plurality of visual warning devices to output an augmenting visual warning when none of the plurality of visual warning devices are within the view of the driver.

17. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

signal a first non-visual warning device to output a non-visual warning;

monitor the driver'"'"'s grip on the steering wheel and monitor changes in the viewing angle of the driver to determine whether the driver is responsive to the warnings; and

when the driver is determined to not be responsive to the warnings, modify the warnings to better capture the driver'"'"'s attention.

18. A method comprising:

determining which of a plurality of visual warning devices is within a determined viewing angle of a driver;

signaling a first visual warning device, from among the plurality of visual warning devices, to output a first visual warning when the first visual warning device is determined to be within the determined viewing angle of the driver;

determining which, if any, of the plurality of visual warning devices are within view of the driver;

signaling a non-visual warning device to output a first non-visual warning at a first warning level when none of the plurality of visual warning devices are determined to be within the determined viewing angle of the driver;

signaling a portable device to output a portable-device warning when a state of the portable device is determined to be active;

monitoring a response of the driver to the first visual warning, the first non-visual warning, and the portable-device warning; and

augmenting one or more of the first visual warning, the first non-visual warning, and the portable-device warning when the response of the driver does not satisfy a predetermined criterion.

19. A non-transitory computer readable storage medium including executable instruction, wherein the instructions, when executed by circuitry, cause the circuitry to perform the method according to claim 18.

20. A system comprising:

a driver-monitor device configured to detect a viewing angle of a driver;

a plurality of warning devices, including a plurality of visual warning devices and at least one non-visual warning device;

control circuitry configured to;

determine which of the plurality of visual warning devices is within the viewing angle of a driver;

signal a first visual warning device, from among the plurality of visual warning devices, to output a first visual warning when the first visual warning device is determined to be within the determined viewing angle of the driver;

signal the non-visual warning device to output a first non-visual warning when no visual warning device is determined to be within the determined viewing angle of the driver;

determine whether the driver is responsive to the first visual warning or the first non-visual warning; and

signal a portable device to output a portable-device warning when the driver is determined to not be responsive to the first visual warning or the first non-visual warning.