METHOD AND SYSTEM FOR DETECTING OPERATOR ALERTNESS

• US 20080068187A1
• Filed: 02/07/2007
• Published: 03/20/2008
• Est. Priority Date: 09/12/2006
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Bonefas, Zachary Thomas, Sanchez, Julian
• Granted Patent: US 7,692,551 B2
• Time in Patent Office: Days
• US Class Current: 340/575
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A method and system for detecting operator alertness of an operator of a vehicle or machine comprises an image processor and an analyzer. The image processor determines whether at least one eye of an operator is visible and determines whether the operator is generally facing forward with respect to the vehicle. The analyzer applies at least one of position analysis and motion analysis to a three dimensional representation of an operator to determine alertness of the operator if at least one eye of operator is not visible or If the operator is generally facing forward.

Claims

1. A method for detecting operator alertness of an operator of a vehicle or machine, the method comprising:

determining whether at least one eye of an operator is visible;

determining whether the operator is generally facing forward with respect to the vehicle; and

applying at least one of position analysis and motion analysis to a three dimensional representation of an operator to determine alertness of the operator if at least one eye of the operator is not visible or if the operator is not facing forward.

2. The method according to claim 1 further comprising:

applying an eye monitoring algorithm to detect operator alertness if at least one eye of the operator is visible and if the operator is facing toward.

3. The method according to claim 1 wherein the applying further comprises:

collecting reference position data associated with an operator, or a portion thereof, when the operator is in an alert state;

determining observed position data of one or more points of a three dimensional representation of the operator during a time interval; and

sending an alert signal to alert the operator if a detected angular shift of one or more points of the representation exceed a displacement threshold.

4. The method according to claim 3 further comprising establishing the three dimensional representation of an operator via stereo vision processing.

5. The method according to claim 4 further comprising:

gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

combining the raw stereo scene images to produce a disparity map image that represents a disparity between the raw stereo scene images; and

creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

6. The method according to claim 3 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more points of the representation.

7. The method according to claim 1 wherein the applying further comprises:

collecting reference motion data associated with an operator representation when the operator is in an alert state;

determining observed motion data of one or more points of a three dimensional representation of the operator during a time interval; and

sending an alert signal to alert the operator if a detected angular shift of one or more points of the representation exceed a motion threshold.

8. The method according to claim 7 further comprising establishing the three dimensional representation of an operator via stereo vision processing.

9. The method according to claim 8 further comprising:

gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

combining the raw stereo scene images to produce a disparity map image that represents a disparity between the raw stereo scene images; and

creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

10. The method according to claim 7 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more points of the representation during an elapsed time period.

11. A system for detecting operator alertness of an operator of a vehicle or machine, the system comprising:

an image processor for determining whether at least one eye of an operator is visible and for determining whether the operator is generally facing forward with respect to the vehicle; and

an analyzer for applying at least one of position analysis and motion analysis to a three dimensional representation of an operator to determine alertness of the operator if at least one eye of operator is not visible or if the operator is generally facing forward.

12. The system according to claim 11 further comprising:

an evaluator for applying an eye monitoring algorithm to detect operator alertness if at least one eye of the operator is visible and if the operator is generally facing forward.

13. The system according to claim 11 wherein the applying further comprises;

an image collection system for collecting reference position data associated with an operator, or portion thereof, when the operator is in an alert state;

an object position estimator for determining observed position data of one or more points of a three dimensional representation of the operator during a time interval; and

the analyzer for sending an alert signal to alert the operator if a detected angular shift of one or more points of the representation exceed a displacement threshold.

14. The system according to claim 13 further comprising a three dimensional image former for establishing the three dimensional representation of an operator via stereo vision processing.

15. The system according to claim 13 further comprising:

a plurality of imaging units for gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

an image disparity module for combining the raw stereo scene images to produce a disparity map image that represents a disparity between the raw stereo scene images; and

a three dimensional image former for creating a three dimensional representation of the operator representation based on the disparity map and the raw stereo scene images.

16. The system according to claim 13 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more points of the representation.

17. The system according to claim 13 further comprising an object detection module for identifying an operator, or a portion thereof, in a cab or cockpit by analyzing one or more images in color space.

18. The system according to claim 11 wherein the applying further comprises:

an image collection system for collecting reference motion data associated with an operator, or a portion thereof, when the operator is in an alert state;

an object motion estimator for determining observed motion data of one or more points of a three dimensional representation of the operator during a time interval;

the analyzer for sending an alert signal to alert the operator if a detected angular shift of one or more points of the representation exceed a motion threshold.

19. The system according to claim 18 further comprising a three dimensional image former for establishing the three dimensional representation of an operator via stereo vision processing.

20. The system according to claim 18 further comprising:

a plurality of imaging units for gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

an image disparity module for combining the raw stereo scene images to produce a disparity map image that represents a disparity between the raw stereo scene images; and

a three dimensional image former for creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

21. The system according to claim 18 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more points of the representation during a time period.

22. The system according to claim 18 further comprising an object detection module for identifying an operator, or portion thereof, in a cab or cockpit by analyzing one or more images in color space.