System and method for monitoring and managing driver attention loads

• US 6,974,414 B2
• Filed: 02/19/2003
• Issued: 12/13/2005
• Est. Priority Date: 02/19/2002
• Status: Expired due to Fees

• Current Assignee: Volvo Technology Corporation
• Original Assignee: Volvo Technology Corporation
• Inventors: Victor, Trent
• Primary Examiner(s): Hindenburg, Max F.
• Assistant Examiner(s): Astorino, Michael C.
• Application Number:
  US10/248,798
• Publication Number:
  US 20030181822A1
• Time in Patent Office: 1,028 Days
• Field of Search: 600300-301, 600/558, 340574-576, 180/272, 180/220, 180/446, 701/1, 701/96, 128/898
• US Class Current: 600/300
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 2360/191 Highlight information
• CPC Class Codes: B60K 2360/338 Light strips
• CPC Class Codes: B60K 2370/191 Highlight information
• CPC Class Codes: B60K 2370/338 Light strips
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/086 using driver related features
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/08 Interaction between the driver and the control system
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08B 21/06 : indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

System and method for monitoring the physiological behavior of a driver that includes measuring a physiological variable of a driver, assessing a driver'"'"'s behavioral parameter on the basis of at least said measured physiological variable, and informing the driver of the assessed driver'"'"'s behavioral parameter. The measurement of the physiological variable can include measuring a driver'"'"'s eye movement, measuring a driver'"'"'s eye-gaze direction, measuring a driver'"'"'s eye-closure amount, measuring a driver'"'"'s blinking movement, measuring a driver'"'"'s head movement, measuring a driver'"'"'s head position, measuring a driver'"'"'s head orientation, measuring driver'"'"'s movable facial features, and measuring a driver'"'"'s facial temperature image.

Claims

1. A method for assessing a driver'"'"'s excessive distraction from driving control tasks due to the driver'"'"'s performance of secondary tasks and responsively stimulating the driver'"'"'s attention to the driving control tasks, said method comprising:

taking a series of non-bodily contact, on-board measurements of a driver'"'"'s head or eye movement and inputting the measurements to a computer processor for analysis;

analyzing the input measurements using the computer processor and determining therefrom whether an excessive amount of driver attention is being paid to a secondary task based on an assessment of driver workload; and

initiating stimulation of the driver'"'"'s attention to driving control tasks when excessive driver attention to a secondary task is assessed.

2. The method as recited in claim 1, further comprising:

detecting a driving control task performance decrement prior to stimulating the driver'"'"'s attention to driving control tasks.

3. The method as recited in claim 2, wherein said driving control task performance decrement is assessed based on inconsistent steering.

4. The method as recited in claim 2, wherein said driving control task performance decrement is assessed based on inconsistent lane keeping.

5. The method as recited in claim 2, wherein said driving control task performance decrement is assessed based on inconsistent vehicle speed.

6. The method as recited in claim 1, further comprising measuring the driver'"'"'s head movement in the assessment of driver workload.

7. The method as recited in claim 1, further comprising said driver'"'"'s eye movement being rotational movement of the eye.

8. The method as recited in claim 1, further comprising analyzing the driver'"'"'s glance behavior for the assessment of driver workload.

9. The method is recited in claim 1, further comprising:

comprising assessed level of driver workload to a predetermined acceptable level and visually presenting results of the comparison to the driver.

10. The method as recited in claim 1, further comprising:

utilizing an audible driver alertness stimulation mode to stimulate the driver'"'"'s attention to driving control tasks.

11. The method as recited in claim 10, wherein said audible driver alertness stimulation mode is a user-recorded message.

12. The method as recited in claim 1, further comprising:

utilizing a tactile driver alertness stimulation mode to stimulate the driver'"'"'s attention to driving control tasks.

13. The method as recited in claim 12, wherein said tactile driver alertness simulation mode is seat vibration.

14. The method as recited in claim 1, further comprising:

utilizing one of a plurality of selectable driver alertness stimulation modes to stimulate the driver'"'"'s attention to driving control tasks.

15. The method as recited in claim 14, further comprising selecting a particular driver alertness stimulation mode from the plurality of selectable driver alertness stimulation modes based on existing driver visual activity.

16. The method as recited in claim 14, further comprising enabling driver selection of the particular driver alertness stimulation mode to be utilized from the plurality of selectable driver alertness stimulation modes.

17. The method as recited in claim 16, further comprising enabling driver selection of up to three driver alertness stimulation modes from the plurality of selectable driver alertness stimulation modes.

18. The method as recited in claim 17, further comprising:

simultaneously activating up to three driver alertness stimulation modes selected by the driver when high workload is assessed.

19. The method as recited in claim 1, further comprising adapting a driver alertness stimulation mode utilized to stimulate the driver'"'"'s attention to driving control tasks based on existing driver visual activity.

20. The method as recited in claim 1, further comprising redirecting the driver'"'"'s visual attention in a different direction.

21. The method as recited in claim 20, wherein the different direction to which the driver'"'"'s visual attention is redirected is to the forward driving scene.

22. The method as recited in claim 20, wherein the redirection of the driver'"'"'s visual attention is affected by a wave of light.

23. The method as recited in claim 22, wherein the wave of light is subliminal to the driver.

24. The method as recited in claim 22, wherein the wave of light is constituted by a progressive illumination of a series of lights.

25. The method as recited in claim 22, wherein the wave of light initiates from a location approximately corresponding to a detected position of the driver'"'"'s gaze.

26. The method as recited in claim 22, wherein the wave of light terminates in a light area greater than the width of the light path of the light.

27. The method as recited in claim 22, wherein the wave of light first slowly flashes thereby permitting evaluation of the driver'"'"'s reaction.

28. The method as recited in claim 27, further comprising increasing a flash time of the wave of lights upon insufficient driver reaction being detected.

29. The method as recited in claim 27, further comprising increasing an intensity of the wave of lights upon insufficient driver reaction being detected.

30. The method as recited in claim 1, further comprising warning the driver and informing the driver of an assessed cause of the warning.

31. The method as recited in claim 30, wherein the assessed cause of the warning is inconsistent steering.

32. The method as recited in claim 31, wherein a visual depiction of inconsistent steering is displayed to the driver.

33. The method as recited in claim 31, wherein an audible message informing the driver that inconsistent steering is occurring is played to the driver.

34. The method as recited in claim 30, wherein the assessed cause of the warning is inconsistent lane keeping.

35. The method as recited in claim 34, wherein a visual depiction of inconsistent lane keeping is displayed to the driver.

36. The method as recited in claim 34, wherein an audible message informing the driver that inconsistent lane keeping is occurring is played to the driver.

37. The method as recited in claim 1, further comprising:

determining, based on existing visual activity, a probable source of an assessed high level of driver workload, and based thereupon, selecting a particular driver alertness stimulation mode from a plurality of possible stimulation modes.

38. The method as recited in claim 37, wherein said existing visual activity indicative of high workload is characterized by excessively frequent glancing toward substantially the same location.

39. The method as recited in claim 1, further comprising:

implementing workload management protocols based on an assessment of high workload.

40. The method as recited in claim 39, wherein said workload management protocols comprise a prioritization system that categorizes information and delays presentation of the information to the driver when high workload is assessed.

41. The method as recited in claim 40, wherein the information delayed to the driver is audible speech from an electronic device.

42. The method as recited in claim 41, wherein the audible speech is conversation emitting from an onboard telephone.

43. The method as recited in claim 41, wherein the audible speech is position information emitting from an onboard navigation system.

44. The method as recited in claim 41, wherein the audible speech is navigational instruction emitting from an onboard navigation system.

45. The method as recited in claim 41, wherein the audible speech is text-to-speech electronic translation.

46. The method as recited in claim 41, wherein the audible speech is text-to-speech electronic translation of email messages.

47. The method as recited in claim 40, further comprising:

resuming information delivery when acceptable driver characteristics resume.

48. The method as recited in claim 1, further comprising:

assessing an acceptable level of driver attentiveness to driving controls tasks, and based thereupon, delaying stimulation of the driver'"'"'s attention to driving control tasks.

49. The method as recited in claim 1, further comprising:

assessing an acceptable level of driver attentiveness to driving control tasks, and based thereupon, canceling stimulation of the driver'"'"'s attention to driving control tasks.

50. The method as recited in claim 1, further comprising:

classifying a specifically characterized driver visual activity as being related to the performance of driving control tasks that would otherwise be assessed as an unacceptable driver visual activity and preventing stimulation of the driver.

51. The method as recited in claim 50, wherein said driver visual activity is glancing toward rearview mirrors.

52. The method as recited in claim 1, further comprising:

adapting the performance of a forward-collision warning system.

53. The method as recited in claim 1, further comprising:

adapting the performance of a rear-collision warning system.

54. The method as recited in claim 1, further comprising:

adapting the performance of a lane-change-collision warning system.

55. The method as recited in claim 1, further comprising:

providing driver feedback in the form of a real-time representation of assessed driver workload.

56. The method as recited in claim 55, wherein said driver feedback is provided in the form of a light stack in which the number of illuminated lights in the stack correlates to an assessed level of driver workload.

57. The method as recited in claim 55, wherein said driver feedback is provided in the form of a light stack upon which an assessed level of driver workload is indicated along a length thereof.

58. The method as recited in claim 57, wherein trend information pertaining to recent assessed levels of driver workload are indicated in fade-out.

59. The method as recited in claim 55, wherein said driver feedback is provided in the form of an extending and retracting light bar, the illuminated length of which correlates to a real-time assessed workload.

60. The method as recited in claim 59, further comprising adjacently located, fixed length light bars, each demonstrating a historical statistical value of assessed workload.

61. The method as recited in claim 60, wherein said adjacently located, fixed length light bars have a more narrow width than said extending and retracting light bar.

62. The method as recited in claim 55, wherein said driver feedback is provided in a comparative format showing real-time assessed workload level relative to an optimum workload level.

63. The method as recited in claim 55, wherein said driver feedback is provided in the form of a histogram of assessed workload.

64. The method as recited in claim 1, further comprising:

logging said assessed workload and providing driver feedback in the form of a post-trip report.

65. The method as recited in claim 1, further comprising:

determining a probable source of a driver'"'"'s assessed high workload, and identifying said probable source to said driver upon demand.