METHOD AND SYSTEM FOR DETECTING OPERATOR ALERTNESS

• US 20080068184A1
• Filed: 02/05/2007
• Published: 03/20/2008
• Est. Priority Date: 09/12/2006
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Bonefas, Zachary Thomas, Sanchez, Julian
• Granted Patent: US 7,692,548 B2
• Time in Patent Office: Days
• US Class Current: 340/575
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A method and system for detecting operator alertness of an operator of a vehicle or machine comprises an image collection system. The image collection system collects reference motion data associated with an operator representation when the operator is in an alert state. An image processor determines observed motion data of one or more points of a three dimensional representation of the operator during a time interval An analyzer sends an alert signal to alert the operator if a detected angular shift of one or more points of the representation exceed a displacement threshold.

Claims

1. A method for detecting operator alertness of an operator of a vehicle or machine, the method comprising:

collecting reference position data associated with an operator when the operator is in an alert state;

determining observed position data of one or more reference points of a three dimensional representation of the operator, or a portion thereof, during a time interval; and

sending an alert signal to alert the operator if a detected angular shift of one or more reference points, associated with an operator, exceed a displacement threshold.

2. The method according to claim 1 wherein the collecting reference position data comprises creating a three dimensional representation of an operator, or a portion thereof, via stereo vision processing.

3. The method according to claim 1 wherein the determining observed position data comprises creating a three dimensional representation of an operator, or a portion thereof, via stereo vision processing.

4. The method according to claim 1 wherein the collecting reference position data further comprises:

gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

identifying respective operator-related image data in the gathered raw stereo scene images;

combining at least one of the raw stereo scene images and the operator-related image data to produce a disparity map image that represents a disparity between the raw stereo scene images; and

creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

5. The method according to claim 1 wherein the determining observed position data further comprises:

gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

identifying respective operator-related image data in the gathered raw stereo scene images;

combining at least one of the raw stereo scene images and the operator-related image data to produce a disparity map image that represents a disparity between the raw stereo scene images; and

creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

6. The method according to claim 1 wherein the reference position data comprises three dimensional coordinates of one or more reference points associated with an operator, or a portion thereof.

7. The method according to claim 1 wherein the reference position data comprises a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates, when the operator is in an alert state.

8. The method according to claim 1 wherein the observed position data comprises three dimensional coordinates of one or more reference points associated with an operator, or a portion thereof.

9. The method according to claim 1 wherein the observed position data comprises a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates.

10. The method according to claim 1 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more reference points.

11. The method according to claim 1 wherein the reference points define an observed axis and wherein the detected angular shift represents an inclination forwards, backwards or sideways of the observed axis with respect to a generally vertical axis by at least a critical angle.

12. A system for detecting operator alertness of an operator of a vehicle or machine, the system comprising:

an image collection system for collecting reference position data associated with an operator when the operator is in an alert state;

an image processor for determining observed position data of one or more reference points of a three, dimensional representation of the operator, or a portion thereof, during a time interval; and

an analyzer for sending an alert signal to alert the operator if a detected angular shift of one or more reference points, associated with an operator, exceed a displacement threshold.

13. The system according to claim 12 wherein the image processor further comprises an image former for creating a three dimensional representation of an operator, or a portion thereof.

14. The system according to claim 12 further comprising:

a plurality of imaging units for gathering a pair of raw stereo scene images of the operator, or a portion thereof, from spatially offset perspectives;

an image segmenter for identifying respective operator-related image data in the gathered raw stereo scene images;

an image disparity module for combining at least one of the raw stereo scene images and the operator-related image data to produce a disparity map image that represents a disparity between the raw stereo scene images; and

an image former for creating a three dimensional representation of the operator, or a portion thereof, based on the disparity map and the raw stereo scene images.

15. The system according to claim 12 wherein the reference position data comprises three dimensional coordinates of one or more reference points associated with an operator, or a portion thereof.

16. The system according to claim 12 wherein the observed position data comprises three dimensional coordinates of one or more reference points associated with an operator, or a portion thereof.

17. The system according to claim 12 wherein the reference position data comprises a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates, when the operator is in an alert state.

18. The system according to claim 12 wherein the observed position data comprises three dimensional coordinates of one or more reference points associated with an operator, or a portion thereof.

19. The system according to claim 12 wherein the observed position data comprises a reference axis defined by two reference points lying on a surface of the operator'"'"'s head, and their respective three dimensional coordinates.

20. The system according to claim 12 wherein the detected angular shift represents an inclination forwards, backwards, or sideways of one or more reference points.

21. The system according to claim 12 wherein the reference points define an observed axis and wherein the defected angular shift represents an inclination forwards, backwards or sideways of an observed axis with respect to a generally vertical axis by at least a critical angle.