Method and system for detecting operator alertness

• US 7,692,550 B2
• Filed: 02/06/2007
• Issued: 04/06/2010
• Est. Priority Date: 09/12/2006
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Bonefas, Zachary Thomas, Sanchez, Julian
• Primary Examiner(s): Bugg; George A
• Application Number:
  US11/671,659
• Publication Number:
  US 20080068186A1
• Time in Patent Office: 1,155 Days
• Field of Search: 340/575, 340/576, 340/439, 340/573.1, 340/540, 382/2, 382/117, 382/203, 382/291, 382/118, 434/236, 351/206, 351/209, 351/246
• US Class Current: 340/575
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A method and system for detecting operator alertness of an operator of a vehicle or machine comprises an image collection system. The image collection system collects reference position data and reference motion data associated with an operator, or a portion thereof, when the operator is in an alert state. An image, processor determines observed position data and observed motion data of one or more points of a three dimensional representation of the operator during a time interval. An analyzer sends an alert signal to alert the operator if a detected angular shift of one or more reference points of the representation exceeds at feast one of a displacement threshold and a motion threshold.

Claims

1. A method for detecting operator alertness of an operator of a vehicle or machine, the method comprising:

collecting reference position data associated with the operator when the operator is in an alert state;

determining observed position data of one or more reference points of a three dimensional representation of the operator, or a portion thereof, during a time interval;

collecting reference motion data associated with the operator when the operator is in the alert state;

determining observed motion data of the one or more reference points of the three dimensional representation of the operator during the time interval, wherein a change in position versus time is determined for the one or more reference points of the three dimensional representation of the operator; and

sending an alert signal to alert the operator if a detected angular shift of the one or more reference points of the three dimensional representation exceed at least one of a displacement threshold and a motion threshold, wherein the collecting reference position data, the determining observed position data, the collecting reference motion data, and the determining observed motion data comprises creating the three dimensional representation of the operator, or a portion thereof, via stereo vision processing.

2. The method according to claim 1 wherein the observed position data comprises three dimensional coordinates of the one or more reference points.

3. The method according to claim 1 wherein the observed position data comprises a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates.

4. The method according to claim 1 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more reference points.

5. The method according to claim 1 wherein the reference points define an observed axis and wherein the detected angular shift represents an inclination forwards, backwards or sideways of the observed axis with respect to a generally vertical axis by at least a critical angle.

6. The method according to claim 1 wherein the reference motion data comprises movement associated with a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates, when the operator is in an alert state.

7. The method according to claim 1 wherein the observed motion data comprises movement associated with three dimensional coordinates of the one or more reference points.

8. The method according to claim 1 wherein the observed motion data comprises movement associated with a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates.

9. The method according to claim 1 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more reference points during the time interval.

10. The method according to claim 1 wherein the reference points define an observed axis and wherein the detected angular shift represents an inclination forwards, backwards or sideways of the observed axis with respect to a generally vertical axis by at least a critical angle during the time interval.