Apparatus and method for alerting machine operator responsive to the gaze zone

• US 8,599,027 B2
• Filed: 10/19/2010
• Issued: 12/03/2013
• Est. Priority Date: 10/19/2010
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Sanchez, Julian
• Primary Examiner(s): Lieu, Julie
• Application Number:
  US12/907,628
• Publication Number:
  US 20120092173A1
• Time in Patent Office: 1,141 Days
• Field of Search: None
• US Class Current: 340/576
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2756/10 Involving external transmission of data to or from the vehicle
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60Y 2200/20 Off-Road Vehicles
• CPC Class Codes: B60Y 2200/221 Tractors
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G01S 2013/9316 : combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 5/0072 : Transmission between mobile stations, e.g. anti-collision systems
• CPC Class Codes: G06F 3/013 : Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G08B 21/06 : indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08G 1/162 : event-triggered
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

The different illustrative embodiments provide a method, apparatus, and computer program product for generating alerts. Responsive to an occurrence of an event, a current gaze zone in a plurality of gaze zones associated with a current view of an operator is identified. A desired gaze zone in the plurality of gaze zones is identified for the event. The desired gaze zone contains a source location for the event. A determination is made whether the current gaze zone is inside the desired gaze zone. Responsive to a determination that the current gaze zone is outside the desired gaze zone, an alert is generated.

Claims

1. A method for generating alerts comprising:

responsive to an occurrence of an event, identifying a current gaze zone in a plurality of gaze zones associated with a current view of an operator;

identifying a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

determining whether the current gaze zone is inside the desired gaze zone; and

responsive to a determination that the current gaze zone is outside the desired gaze zone, generating an alert, wherein the alert comprises a prompt for user input to perform an action, and wherein the action is selected from stopping a first movement of a vehicle in which the operator is located, stopping a second movement of a follower vehicle following the vehicle, activating a first safety system on the vehicle, and activating a second safety system on the follower vehicle.

2. The method of claim 1, wherein the step of generating the alert comprises generating the alert within the desired gaze zone.

3. The method of claim 1 further comprising:

prior to generating the alert, identifying a priority for the alert based on the event.

4. The method of claim 2, wherein the priority for the alert increases with a risk of the event, and the priority for the alert decreases with the risk of the event.

5. The method of claim 2, wherein the alert is selected from a group comprising an audio alert, a visual alert, and a vibration alert.

6. The method of claim 1 further comprising:

prior to generating the alert, identifying a type for the alert based on the event, wherein the type for the alert is imaging data of the source location.

7. The method of claim 6, wherein the operator is located in a vehicle, and the imaging data of the source location is generated by an imaging system located on a follower vehicle of the vehicle.

8. The method of claim 7, wherein the occurrence of the event comprises an inconsistency developing in the follower vehicle.

9. The method of claim 7, wherein the occurrence of the event comprises the follower vehicle being located within a distance of a boundary for an area in which the follower vehicle is operating.

10. A method for generating alerts comprising:

responsive to an occurrence of an event, identifying a current gaze zone in a plurality of gaze zones associated with a current view of an operator;

identifying a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

determining whether the current gaze zone is inside the desired gaze zone;

responsive to a determination that the current gaze zone is outside the desired gaze zone, generating an alert;

prior to generating the alert, identifying a type for the alert based on the event, wherein the type for the alert is imaging data of the source location, wherein the operator is located in a vehicle, and the imaging data of the source location is generated by an imaging system located on a follower vehicle of the vehicle, and wherein the occurrence of the event comprises the follower vehicle completing an operation being performed by the follower vehicle.

11. A method for generating alerts comprising:

responsive to an occurrence of an event, identifying a current gaze zone in a plurality of gaze zones associated with a current view of an operator;

identifying a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

determining whether the current gaze zone is inside the desired gaze zone;

responsive to a determination that the current gaze zone is outside the desired gaze zone, generating an alert, wherein the step of generating the alert comprises presenting imaging data of the source location; and

prior to presenting the imaging data of the source location, identifying an imaging system from a collection of imaging systems that generates the imaging data for the source location.

12. The method of claim 11, wherein the imaging data is selected from an image and a video, and wherein the source location is a portion of the follower vehicle.

13. An apparatus comprising:

an event monitoring system configured to identify an occurrence of an event and identify a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

a gaze zone monitoring system configured to identify a current gaze zone in a plurality of gaze zones associated with a current view of an operator responsive to an occurrence of an event;

a controller configured to determine whether the current gaze zone is within the desired gaze zone; and

an alert system configured to generate an alert responsive to a determination that the current gaze zone is outside the desired gaze zone, wherein the alert system is further configured to generate the alert within the desired gaze zone.

14. The apparatus of claim 13 further comprising:

a location monitoring system configured to identify the source location for the event.

15. The apparatus of claim 14, wherein the location monitoring system is associated with a follower vehicle of a vehicle operated by the operator.

16. The apparatus of claim 15, wherein the source location is the location of the follower vehicle.

17. The apparatus of claim 13, wherein the alert system is further configured to identify a priority for the event.

18. The apparatus of claim 13, wherein the priority for the alert increases with a risk of the event, and the priority for the alert decreases with the risk of the event.

19. The apparatus of claim 13, wherein the alert is selected from a group comprising an audio alert, a visual alert, and a vibration alert.

20. The apparatus of claim 13, wherein the alert system is further configured to identify a type for the alert based on the event, and wherein the type for the alert is imaging data of the source location.

21. The apparatus of claim 13, wherein the operator is located in a vehicle, and further comprising:

an imaging system located on a follower vehicle of the vehicle configured to generate the imaging data of the source location.

22. The apparatus of claim 21, wherein the occurrence of the event comprises an inconsistency developing in the follower vehicle.

23. An apparatus comprising:

an event monitoring system configured to identify an occurrence of an event and identify a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

a gaze zone monitoring system configured to identify a current gaze zone in a plurality of gaze zones associated with a current view of an operator responsive to an occurrence of an event;

a controller configured to determine whether the current gaze zone is within the desired gaze zone;

an alert system configured to generate an alert responsive to a determination that the current gaze zone is outside the desired gaze zone; and

a location monitoring system configured to identify the source location for the event, wherein the location monitoring system is associated with a follower vehicle of a vehicle operated by the operator and the source location is the location of the follower vehicle;

wherein the location monitoring system comprises a first global positioning system receiver associated with the vehicle and a second global positioning system receiver associated with the follower vehicle.

24. An apparatus comprising:

an event monitoring system configured to identify an occurrence of an event and identify a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

a gaze zone monitoring system configured to identify a current gaze zone in a plurality of gaze zones associated with a current view of an operator responsive to an occurrence of an event;

a controller configured to determine whether the current gaze zone is within the desired gaze zone;

an alert system configured to generate an alert responsive to a determination that the current gaze zone is outside the desired gaze zone; and

a location monitoring system configured to identify the source location for the event, wherein the location monitoring system is associated with a follower vehicle of a vehicle operated by the operator and the source location is the location of the follower vehicle;

wherein the location monitoring system comprises a radio frequency identification receiver associated with the vehicle and a radio frequency identification tag associated with the follower vehicle.

25. A computer program product comprising:

computer readable program code, stored on a computer readable storage medium, for identifying a current gaze zone in a plurality of gaze zones associated with a current view of an operator responsive to an occurrence of an event;

computer readable program code, stored on the computer readable storage medium, for identifying a desired gaze zone in the plurality of gaze zones for the event, wherein the desired gaze zone contains a source location for the event;

computer readable program code, stored on the computer readable storage medium, for determining whether the current gaze zone is inside the desired gaze zone; and

computer readable program code, stored on the computer readable storage medium, for generating an alert responsive to a determination that the current gaze zone is outside the desired gaze zone, wherein the alert comprises a prompt for user input to perform an action, and wherein the action is selected from stopping a first movement of a vehicle in which the operator is located, stopping a second movement of a follower vehicle following the vehicle, activating a first safety system on the vehicle, and activating a second safety system on the follower vehicle.