Systems and methods for using an attention buffer to improve resource allocation management

• US 10,525,984 B2
• Filed: 08/21/2017
• Issued: 01/07/2020
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Seppelt, Bobbie Danielle, Lee, Joonbum, Angell, Linda Sala, Reimer, Bryan L., Mehler, Bruce L., Coughlin, Joseph F.
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Greene, Daniel L
• Application Number:
  US15/682,198
• Publication Number:
  US 20180072327A1
• Time in Patent Office: 869 Days
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60W 2040/0872 Driver physiology
• CPC Class Codes: B60W 2540/00 Input parameters relating to occupants
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Systems and methods for assessing resource allocation are provided. In some exemplary embodiments, the system uses an attention buffer to classify glances by a person and/or automated system, the buffer determining the impact the glances have on the person and/or automated system'"'"'s situation awareness level. The attention buffer calculates on a continuous basis a buffer value that is representative of the situation awareness level for the person and/or automated system at a particular moment in time. The calculated buffer values, referred to as moment-to-moment buffer values, among other names, can be used as data points, and/or they can also be used to direct action by the system and/or person to alter the situation awareness level of the person and/or automated system.

Claims

1. A method for real-time assessment of resource allocation by a resource allocation management system that includes one or more processors, the method comprising:

the one or more processors receiving information in real-time about a plurality of glances of a user, the plurality of glances occurring over a period of time;

the one or more processors classifying each of the plurality of glances as an on-target glance or an off-target glance;

the one or more processors calculating on a continuous basis a moment-to-moment buffer value of an attention buffer based on the classified glances of the plurality of glances, the attention buffer being a range of values corresponding to an awareness level of the user;

the one or more processors continuously outputting the calculated moment-to-moment buffer value such that the calculated moment-to-moment buffer value is known at any moment of time during performance of the method; and

at least one of;

the resource allocation management system providing instructions based on the continuously outputted calculated moment-to-moment buffer value; and

the resource allocation management system providing data to one or more databases, the data including at least some continuously outputted calculated moment-to-moment buffer values of the continuously outputted calculated moment-to-moment buffer values over a period of time.

2. The method of claim 1,

wherein the user is an operator of a vehicle,

wherein the method comprises the resource allocation management system providing instructions based on the continuously outputted calculated moment-to-moment buffer value, and

wherein providing instructions based on the continuously outputted calculated moment-to-moment buffer value further comprises providing instructions to at least one of the vehicle, the operator of the vehicle, and a receiver configured to receive the instructions, the instructions being based on the continuously outputted calculated moment-to-moment buffer value.

3. The method of claim 2, further comprises the resource allocation management system communicating with one or more components of the vehicle to at least one of continuously output the calculated moment-to-moment buffer value and provide instructions to at least one of the vehicle, the operator of the vehicle, and a receiver configured to receive the instructions based on the continuously outputted calculated moment-to-moment buffer value, the one or more components of the vehicle comprising:

an automation system configured to automate one or more actions associated with the vehicle;

an adaptive interface configured to respond to change at least one of;

one or more functions and content provided to the operator of the vehicle;

one or more modalities in which content is provided to the operator; and

a nature of one or more control inputs, to effectively assist the operator of the vehicle in attending to an environment in a way that promotes situation awareness;

a context-aware vehicle control system configured to adjust parameters of the vehicle in response to at least one of a surrounding environment of the vehicle, knowledge, and the awareness level of the operator of the vehicle; and

in-vehicle displays configured to provide to the operator of the vehicle at least one of information about the vehicle and control of aspects of the vehicle.

4. The method of claim 2, further comprising the resource allocation management system adapting the instructions provided to at least one of the vehicle, the operator of the vehicle, and the receiver configured to receive the instructions based on at least one of changes made by the operator of the vehicle during operation of the vehicle and changes to an environment surrounding the vehicle.

5. The method of claim 1,

wherein the method comprises the resource allocation management system providing instructions based on the continuously outputted calculated moment-to-moment buffer value, and

wherein providing instructions based on the continuously outputted calculated moment-to-moment buffer value further comprises at least one of:

(1) the resource allocation management system providing instructions configured to alter the awareness level of the user;

(2) the resource allocation management system providing instructions to an automated system to adjust one or more parameters thereof in response to the continuously outputted calculated moment-to-moment buffer value; and

(3) the resource allocation management system providing instructions to augment an exchange of information with the user.

6. The method of claim 5, wherein providing instructions to augment an exchange of information with the user further comprises the resource allocation management system providing instructions to suppress an exchange of information with the user.

7. The method of claim 1, wherein the information received by the one or more processors in real-time about a plurality of glances further comprises frequency, duration, and location.

8. The method of claim 1, wherein the one or more processors calculating on a continuous basis a moment-to-moment buffer value of an attention buffer based on the classified glances from the plurality of glances further comprises threading together the classified on-target glances and the classified off-target glances over a period of time, the classified on-target glances and the classified off-target glances comprising a duration for each such classified on-target and off-target glance.

9. The method of claim 1, wherein the one or more processors classifying each of the plurality of glances as an on-target glance or an off-target glance further comprises:

classifying each of the on-target glances as being a centrally-located glance or a peripherally-located glance; and

classifying each of the off-target glances as being a situation awareness relevant glance or a situation awareness irrelevant glance.

10. The method of claim 9,

wherein the user is an operator of a vehicle, and

wherein the situation awareness relevant glance comprises a glance in one of the following locations:

a rear view mirror, a side view mirror, a blind spot, a portion of an in-vehicle display on which information about at least one of vehicle performance, an automation state of the vehicle, a road and its conditions, environment data, surrounding data, and user data is displayed, and a location outside of the vehicle on which information about at least one of a road and its conditions, environment data, and surrounding data is displayed.

11. The method of claim 9,

wherein when the off-target glance is a situation awareness relevant glance, or a sequence of situation awareness relevant glances, the one or more processors calculating on a continuous basis a moment-to-moment buffer value of an attention buffer further comprises delaying application of a decrement rate of the attention buffer until after a delay period has expired and the situation awareness relevant glance is still the current classified glance.

12. The method of claim 9,

wherein when the on-target glance is a centrally-located glance, the one or more processors calculating on a continuous basis a moment-to-moment buffer value of an attention buffer further comprises applying a decrement rate of the attention buffer when a time for which the centrally-located glance occurs exceeds a threshold value.

13. The method of claim 1, further comprising:

the one or more processors comparing the continuously outputted calculated moment-to-moment buffer value to a threshold value,

wherein the method comprises the resource allocation management system providing instructions based on the continuously outputted calculated moment-to-moment buffer value, and wherein the provided instructions are selected based on the comparison of the calculated moment-to-moment buffer value and the threshold value.

14. The method of claim 13, further comprising the one or ore processors adjusting the threshold value in view of at least one of vehicle performance, an automation state of the vehicle, a road and its conditions, environment data, surrounding data, and user data.

15. The method of claim 1, wherein the attention buffer comprises an equation having a plurality of variables that impact the moment-to-moment buffer value, the plurality of variables being configured to cause a rate between moment-to-moment buffer values to increase or decrease at a variable rate.

16. The method of claim 1,

wherein the attention buffer comprises an equation having a plurality of variables that impact the moment-to-moment buffer value, the plurality of variables including at least one of:

(1) increment rate;

(2) decrement rate;

(3) initial value start;

(4) switching cost;

(5) existence of a glance rate decrement; and

(6) a location-based latency, and

the method further comprising the one or more processors selecting one or more variables of the plurality of variables to rely upon to perform the calculating of the moment-to-moment buffer value based on at least one of the following factors;

(1) broad situational demands;

(2) immediate task demands; and

(3) an information processing state.

17. The method of claim 16, wherein the equation of the attention buffer accounts for a duration of the on-target glances and a duration of the off-target glances.

18. The method of claim 16, wherein the equation of the attention buffer accounts for glance transitions.

19. The method of claim 1,

wherein the user is an operator of a vehicle,

wherein the method comprises the one or more processors providing data to one or more databases, the data including at least some continuously outputted calculated moment-to-moment buffer values of the continuously outputted calculated moment-to-moment buffer values over a period of time, and

wherein the provided data is configured for use to define an operator profile based on the received moment-to-moment buffer values.

20. The method of claim 1,

wherein the user is an operator of a vehicle,

wherein the method comprises the one or more processors providing data to one or more databases, the data including at least some continuously outputted calculated moment-to-moment buffer values of the continuously outputted calculated moment-to-moment buffer values over a period of time, and

wherein the provided data is configured for use to define a global profile based on at least one of a mean, mode, variability, moving average window, standard deviation, distribution summary, moment-to-moment aggregate plot, advanced metrics involving Fourier analysis of spectral width and power, and Symbolic Aggregate Approximation, that is calculated from at least some portion of the received moment-to-moment buffer values.

21. The method of claim 1, further comprising:

the one or more processors operating an artificial intelligence component to adjust the attention buffer based on learned information about at least one of the user and an environment impacting the user.

22. The method of claim 1,

wherein the user is an operator of a vehicle,

wherein the method comprises the one or more processors providing data to one or more databases, the data including at least some continuously outputted calculated moment-to-moment buffer values of the continuously outputted calculated moment-to-moment buffer values over a period of time, and

wherein the provided data includes information relevant for determining at least one of insurance credits and insurance rates.

23. A method for allocating resources by a resource allocation management system that includes one or more processors, the method comprising:

the one or more processors classifying a glance as an on-target glance or an off-target glance for use in quantifying a momentary buffer value;

if the glance is classified as an on-target glance, the method further comprising;

the one or more processors determining if an immediately preceding glance was an on-target glance or an off-target glance;

the one or more processors determining a momentary buffer value for the glance,

wherein, if the immediately preceding glance was an off-target glance, the resource allocation management system accounting for a transition associated with moving from the off-target glance to the on-target glance, and

if the glance is classified as an off-target glance, the method further comprising;

the one or more processors classifying the off-target glance as being a situation awareness relevant glance or a situation awareness irrelevant glance; and

the one or more processors determining a momentary buffer value for the glance,

wherein, if the off-target glance is a situation awareness relevance glance, determining a momentary buffer value for the glance further comprises the resource allocation management system accounting for a latency.

24. The method of claim 23, wherein if the glance is an on-target glance, the method further comprises:

the one or more processors classifying the on-target glance as being a centrally-located glance or a peripherally-located glance,

wherein, if the on-target glance is a centrally-located glance, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors accounting for an instance in which a duration of time for consecutive centrally-located glances exceeds a threshold time, and

wherein, if the on-target glance is a peripherally-located glance, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors determining the momentary buffer value for the glance based on at least one of an initially-determined momentary buffer value, a maximum buffer value, and an increment value associated with the on-target glance.

25. The method of claim 24, wherein, if the on-target glance is a centrally-located glance, the one or more processors determining a momentary buffer value for the glance further comprises:

(1) the one or more processors determining the duration of time for consecutive centrally-located glances;

(2) the one or more processors comparing the duration of time for the consecutive centrally-located glances to the threshold time;

(3) if the duration of time for the consecutive centrally-located glances is equal to or greater than the threshold time;

(a) the one or more processors comparing an initially-determined momentary buffer value to an on-target buffer decrement floor value;

(b) if the initially-determined momentary buffer value is greater than the on-target buffer decrement floor value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the initially-determined momentary buffer value less a decrement value associated with the on-target glance; and

(c) if the initially-determined momentary buffer value is less than or equal to the on-target buffer decrement floor value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the on-target buffer decrement floor value; and

(4) if the duration of time for the consecutive centrally-located glances is less than the threshold time, the one or more processors determining the momentary buffer value for the glance further comprises the one or more processors determining the momentary buffer value for the glance based on at least one of the initially-determined momentary buffer value, the maximum buffer value, and the increment value associated with the on-target glance.

26. The method of claim 23, wherein if the glance is an off-target glance, the one or more processors determining a momentary buffer value for the glance is based on at least one of an initially-determined momentary buffer value, a minimum buffer value, and a decrement value associated with the off-target glance.

27. The method of claim 26, wherein the one or more processors determining a momentary buffer value for the glance when the off-target glance is classified as a situation awareness relevant glance further comprises:

(1) the one or more processors determining a latency value associated with the momentary buffer value;

(2) the one or more processors comparing the determined latency value with a threshold latency value;

(3) if the determined latency value is less than or equal to the threshold latency value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the initially-determined momentary buffer value; and

(4) if the determined latency value is greater than the threshold latency value;

(a) the one or more processors comparing the initially-determined momentary buffer value to the minimum buffer value;

(b) if the initially-determined momentary buffer value is less than or equal to the minimum buffer value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the minimum buffer value; and

(c) if the initially-determined momentary buffer value is greater than the minimum buffer value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the initially-determined momentary buffer value less the decrement value associated with the off-target glance.

28. The method of claim 26, wherein the one or more processors determining a momentary buffer value for the glance when the off-target glance is classified as a situation awareness irrelevant glance further comprises:

the one or more processors comparing the initially-determined momentary buffer value to the minimum buffer value;

if the initially-determined momentary buffer value is less than or equal to the minimum buffer value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the minimum buffer value; and

if the initially-determined momentary buffer value is greater than the minimum buffer value, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors setting the momentary buffer value as the initially-determined momentary buffer value less the decrement value associated with the off-target glance.

29. The method of claim 23, wherein the one or more processors accounting for a transition associated with moving from the off-target glance to the on-target glance further comprises:

the one or more processors determining an information extraction latency value for the on-target glance;

the one or more processors comparing the information extraction latency value to a threshold information extraction latency value;

if the information extraction latency value is less than the threshold information extraction latency value, the one or more processors determining a momentary buffer value for the glance further comprises setting the momentary buffer value as the initially-determined momentary buffer value; and

if the information extraction latency value is greater than or equal to the threshold information extraction latency value, the method further comprises the one or more processors classifying the on-target glance as being a centrally-located glance or a peripherally-located glance,

wherein, if the on-target glance is a centrally-located glance, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors accounting for an instance in which a duration of time for consecutive centrally-located glances exceeds a threshold time, and

wherein, if the on-target glance is a peripherally-located glance, the one or more processors determining a momentary buffer value for the glance further comprises the one or more processors determining the momentary buffer value for the glance based on at least one of an initially-determined momentary buffer value, a maximum buffer value, and an increment value associated with the on-target glance.

30. The method of claim 23, further comprising:

the one or more processors storing the determined momentary buffer value;

the one or more processors exporting the determined momentary buffer value; and

repeating the steps of claim 23, to determine one or more subsequent momentary buffer values.

31. The method of claim 23, further comprising the one or more processors adjusting one or more threshold values used in conjunction with determining a momentary buffer value for the glance.

32. The method of claim 31, wherein the one or more threshold values comprises at least one of:

a threshold time associated with a duration of time for consecutive same-located glances;

a threshold latency value associated with one or more momentary buffer values; and

a threshold information extraction latency value associated with the glance.

33. The method of claim 23, wherein the glance that is classified is a glance that is performed while operating a vehicle.

34. The method of claim 33, further comprising:

the one or more processors providing the determined momentary buffer value to one or more databases configured to define an operator profile;

repeating the steps of claim 23 to determine one or more subsequent momentary buffer values; and

the one or more processors providing the determined one or more subsequent momentary buffer values to the one or more databases configured to define an operator profile, the operator profile being defined based on the provided determined momentary buffer value and the determined one or more subsequent momentary buffer values.

35. The method of claim 33, further comprising:

the one or more processors providing the determined momentary buffer value to one or more databases configured to define a global profile;

repeating the steps of claim 23 to determine one or more subsequent momentary buffer values; and

the one or more processors providing the determined one or more subsequent momentary buffer values to the one or more databases configured to define a global profile, the global profile being defined based on at least one of a mean, mode, variability, moving average window, standard deviation, distribution summary, moment-to-moment aggregate plot, advanced metrics involving Fourier analysis of spectral width and power, and Symbolic Aggregate Approximation, that is calculated from at least some portion of the determined momentary buffer value and the one or more subsequent momentary buffer values.

36. The method of claim 33, further comprising the one or more processors providing the determined momentary buffer value to one or more databases configured to be used for determining at least one of insurance credits and insurance rates.

37. The method of claim 23, further comprising:

the one or more processors operating an artificial intelligence component to adjust one or more threshold values used in conjunction with determining a momentary buffer value for the glance.

38. The method of claim 37, wherein adjustment of the one or more threshold values is based on learned information about at least one of a user and an environment impacting the user.