System and method of awareness detection

• US 7,940,962 B2
• Filed: 08/03/2007
• Issued: 05/10/2011
• Est. Priority Date: 08/03/2007
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (Aptiv PLC)
• Inventors: Hammoud, Riad I.
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US11/890,066
• Publication Number:
  US 20090034801A1
• Time in Patent Office: 1,376 Days
• Field of Search: 382/104
• US Class Current: 382/104
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

An awareness detection system and method are provided. The system includes an imaging device positioned to obtain a plurality of images of at least a portion of a subject'"'"'s head, and an awareness processor in communication with the imaging device. The awareness processor receives an image from the imaging device and performs the steps including conducting a two-point head pose analysis when two eyes of the subject are located, conducting a one-point head pose analysis when only one eye of the subject is located, and conducting a zero-point head pose analysis when zero eyes of the subject are located. The awareness processor further performs the steps of detecting at least one point of interest in the image, conducting a motion analysis of the image to estimate a head pose of the subject, and determining a state of awareness of the subject.

Claims

1. An awareness detection system comprising:

an imaging device positioned to obtain a plurality of images of at least a portion of a subject'"'"'s head; and

an awareness processor in communication with said imaging device, wherein said awareness processor receives said plurality of images from said imaging device and performs the steps comprising;

conducting a two-point head pose analysis of said subject and generating a head pose result when two eyes of said subject are located;

conducting a one-point head pose analysis of said subject and generating a head pose result when only one eye of said subject is located;

conducting a zero-point head pose analysis of said subject and generating a head pose result when zero said eyes of said subject are located;

detecting at least one point of interest in said plurality of images;

conducting a motion analysis of said plurality of images to estimate a head pose of said subject based upon said generated head pose result, wherein a motion vector for each said at least one point of interest is determined; and

determining a state of awareness of said subject based upon said motion analysis.

2. The awareness detection system of claim 1, wherein said awareness processor further performs the step comprising of storing head pose results from at least one of said two-point head pose analysis, said one-point head pose analysis, and said zero-point head pose analysis.

3. The awareness detection system of claim 1, wherein said step of conducting a motion analysis comprises clustering said motion vectors for each of said at least one point of interest into groups based upon at least one of an orientation of said motion vector, a magnitude of said motion vector, and a spatial position of said motion vector.

4. The awareness detection system of claim 3, wherein said step of conducting a motion analysis further comprises representing each said cluster with a dominate motion vector and forming a graph of each said dominate motion vector having a motion field, an orientation field, and a position field.

5. The awareness detection system of claim 4, wherein said step of conducting a motion analysis further comprises the step of matching said graph with at least one template that corresponds to graphs relating to prior information processed by said awareness processor.

6. The awareness detection system of claim 5, wherein said step of conducting a motion analysis further comprises the step of classifying at least one segmented area that corresponds to said dominate motion vector, and monitor said dominate motion vector across multiple said images for classifying said points of interest.

7. The awareness detection system of claim 1, wherein said step of conducting a zero-point head pose analysis comprises the steps of extracting at least one region of interest from said image and classifying said region.

8. The awareness detection system of claim 7, wherein said step of conducting a zero-point head pose analysis further comprises computing a vote total of said classifications for said at least one region of interest and determining said classification based upon said vote total for said classification of said at least one region of interest to classify said region.

9. The awareness detection system of claim 1, wherein said awareness detection system is used with a vehicle to detect said subject in said vehicle.

10. A method of detecting awareness of a subject, said method comprising the steps of:

obtaining a plurality of images of at least a portion of a subject;

conducting a two-point head pose analysis of said subject and generating a head pose result when two eyes of said subject are located;

conducting a one-point head pose analysis of said subject and generating a head pose result when only one said eye of said subject is located;

conducting a zero-point head pose analysis of said subject and generating a head pose result when zero of said eyes of said subject are located;

detecting at least one point of interest in said plurality of images;

conducting a motion analysis of said plurality of images to estimate a head pose of said subject based upon said head pose result, wherein a motion vector for each said at least one point of interest is determined; and

determining a state of awareness of said subject based upon said motion analysis.

11. The method of claim 10 further comprising the step of storing head pose results from at least one of said two-point head pose analysis, said one-point head pose analysis, and said zero-point head pose analysis.

12. The method of claim 10, wherein said step of conducting a motion analysis comprises clustering said motion vectors for each of said at least one point of interest into groups based upon at least one of an orientation of said motion vector, a magnitude of said motion vector, and a spatial position of said motion vector.

13. The method of claim 12, wherein said step of conducting a motion analysis further comprises representing each said cluster with a dominate motion vector and forming a graph of each said dominate motion vector having a motion field, an orientation field, and a position field.

14. The method of claim 13, wherein said step of conducting a motion analysis further comprises the step of matching said graph with at least one template that corresponds to graphs relating to prior information processed by an awareness processor.

15. The method of claim 14, wherein said step of conducting a motion analysis further comprises the step of classifying at least one segmented area that corresponds to said dominate motion vector, and monitor said dominate motion vector across multiple said images for classifying said points of interest.

16. The method of claim 10, wherein said step of conducting a zero-point head pose analysis comprises the steps of extracting at least one region of interest from said image and classifying said at least one region of interest.

17. The method of claim 16, wherein said step of conducting a zero-point head pose analysis further comprises computing a vote total of said classifications for said at least one region of interest and determining said classification based upon said vote total for said classification of said at least one region of interest to classify said region.

18. The method of claim 10, wherein said method detects awareness of said subject that is a driver of a vehicle.

19. A method of detecting awareness of a subject, said method comprising the steps of:

obtaining a plurality of images of at least a portion of a subject;

conducting a two-point head pose analysis of said subject and generating a head pose result when two eyes of said subject are located;

conducting a one-point head pose analysis of said subject and generating a head pose result when only one said eye of said subject is located;

conducting a zero-point head pose analysis of said subject and generating a head pose result when zero of said eyes of said subject are located;

detecting at least one point of interest in said plurality of images;

determining a motion vector for each said at least one point of interest;

clustering said motion vectors for each of said at least one point of interest into groups based upon at least one of an orientation of said motion vector, a magnitude of said motion vector, and a spatial position of said motion vector;

representing each said cluster with a dominate motion vector and forming a graph of each said dominate motion vector having a motion field, an orientation field, and a position field;

matching said graph with at least one template that corresponds to graphs relating to prior information processed by an awareness processor;

classifying at least one segmented area that corresponds to said dominate motion vector;

monitoring said dominate motion vector across multiple said images for classifying said at least one point of interest in order to estimate a head pose of said subject; and

determining a state of awareness of said subject that is a driver of a vehicle based upon said classification.

20. The method of claim 19 further comprising the step of storing head pose results from at least one of said two-point head pose analysis, said one-point head pose analysis, and said zero-point head pose analysis.

21. The method of claim 19 further comprising the steps of conducting a zero-point head pose analysis comprises the steps of extracting at least one region of interest from said plurality of images, classifying said at least one region of interest, computing a vote total of said classifications for said at least one region of interest, and determining said classification based upon said vote total for said classification of said at least one region of interest to classify said at least one region of interest.