System and method to determine awareness

• US 7,697,766 B2
• Filed: 03/17/2005
• Issued: 04/13/2010
• Est. Priority Date: 03/17/2005
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (Aptiv PLC)
• Inventors: Hammoud, Riad I., Wilhelm, Andrew L., Zhang, Harry, Witt, Gerald J.
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US11/082,608
• Publication Number:
  US 20060251297A1
• Time in Patent Office: 1,853 Days
• Field of Search: 382/225, 725/10, 708/542, 708/170, 708/205, 708/495, 712/222
• US Class Current: 382/225
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A system for measuring the awareness of a subject including an imaging device adapted to acquire data representative of an image of the subject, and an awareness processor connected to the imaging device, wherein the awareness processor is adapted to classify the awareness of the subject according to the position of the subject.

Claims

1. An awareness detection system, comprising:

an imaging device adapted to acquire data representative of an image of a subject; and

an awareness processor connected to said imaging device, wherein said awareness processor is adapted to determine a particular region within said image, size and establish said particular region, augment and normalize data of said image in said particular region with a distribution enhancement technique, represent said image within a fixed space with a vector relating technique that includes a reconstruction error determined in a fixed point system relative to a floating point system, which is used to classify an awareness state of the subject according to an orientation of the subject represented by the normalized data and said vector relating technique.

2. The system according to claim 1, wherein said classification is determined according to a facial orientation of the subject.

3. The system according to claim 2, wherein said awareness processor includes a pre-processing logic module that is adapted to identify a first eye and a second eye of the subject and a distance therebetween, and wherein said pre-processing logic module is adapted to generate said particular region according to said distance.

4. The system according to claim 1, further comprising a warning device connected to said awareness system that generates a sensory warning when said awareness processor classifies the subject state as distracted.

5. The system according to claim 1, wherein said awareness processor includes a pre-processing logic module that enhances said particular region with said distribution enhancement technique, which includes performing a linear intensity affine transformation firstly, and a histogram equalization secondly.

6. The system according to claim 1, wherein said awareness processor includes a classifying logic module that is adapted to vectorally project the particular region into at least two eigenfaces, wherein each projection includes said reconstruction error, and wherein said classifying logic module is adapted to classify said particular region according to a minimum reconstruction error.

7. The system according to claim 6, further comprising three eigenfaces, one of said eigenfaces representing the subject in an undistracted state, the other two of said eigenfaces representing the subject in a distracted right position and distracted left position.

8. The system according to claim 7, wherein said other two of said eigenfaces are arranged approximately greater than +/−

9 degrees from said undistracted eigenface.

9. The system according to claim 6 further comprising, a warning device connected to the classifying logic module that generates a sensory warning when the subject is distracted for a predefined period of time.

10. The system according to claim 9, wherein said predefined period of time is at or about 2 seconds.

11. The system according to claim 1, wherein said imaging device is a single imaging device, such that said awareness processor classifies such awareness state based upon said image received from said single imaging device.

12. The system according to claim 1, wherein coefficients of said vector relating technique are scaled to an integer value between −

128 and 127, such that said coefficients are represented and calculated using 8-bits and said fixed point system.

13. A method for determining awareness of a subject, comprising the steps of:

acquiring data representative of an image of a subject, wherein said image is acquired by an imaging device;

processing said data by an awareness processor, said processing including,

identifying a particular region of said image data; and

augmenting and normalizing the data in said particular region to a pre-defined size with a distribution enhancement technique;

representing said image within a fixed space with a vector relating technique that includes a reconstruction error determined in a fixed point system relative to a floating point system; and

classifying subject awareness based on the orientation of a portion of the subject represented by the normalized data in said particular region and said vector relating technique.

14. The method according to claim 13, wherein the step of augmenting and normalizing comprises the steps of:

performing a linear intensity affine transformation on said data in said particular region; and

performing a histogram equalization on said data in said particular region.

15. The method according to claim 14, wherein said performing a linear intensity affine transformation step is performed before said performing a histogram equalization step.

16. The method according to claim 13, wherein said identifying a particular region step further comprises the steps of:

identifying a first eye of the subject;

identifying a second eye of the subject;

calculating a distance between said first and second eyes of said subject; and

sizing said data in said particular region according to said distance between said first and second eyes of the subject.

17. The method according to claim 13, wherein said system is adapted to store a frontal eigenface that represents an undistracted subject and is referenced at 0 degrees, a right eigenface that represents a distracted subject facing right, and a left eigenface that represents a distracted subject facing left, and said processing said data step further includes,

representing said particular region vectorally;

projecting said particular region into each of said eigenfaces;

calculating said reconstruction error for each of said projections;

determining the minimum reconstruction error; and

classifying said subject as frontal or non-frontal according to said determining step.

18. The method according to claim 17, wherein said stored right and left eigenfaces are approximately greater than +/−

9 degrees from said frontal eigenface.

19. The method according to claim 13 further comprising the step of:

producing a warning when the subject is distracted for a predetermined period of time.

20. The method according to claim 19 wherein said pre-determined time is at or around two seconds.

21. The method according to claim 13, wherein coefficients of said vector relating technique are scaled to an integer value between −

128 and 127, such that said coefficients are represented and calculated using 8-bits and said fixed point system.