Method and apparatus for detection of drowsiness and quantitative control of biological processes

• US 7,336,804 B2
• Filed: 10/23/2003
• Issued: 02/26/2008
• Est. Priority Date: 10/28/2002
• Status: Expired due to Fees

• Current Assignee: Morris Steffin
• Original Assignee: Morris Steffin
• Inventors: Steffin, Morris
• Primary Examiner(s): Ahmed; Samir
• Assistant Examiner(s): Wang; Claire X
• Application Number:
  US10/692,834
• Publication Number:
  US 20040234103A1
• Time in Patent Office: 1,587 Days
• Field of Search: 382/118, 382/115, 382/117, 382/190, 382/272, 382/274, 382/104, 382/203, 382/207
• US Class Current: 382/104
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

The present invention is directed to a real-time automated video cognizer that functions as a facial video processor for detecting drowsiness in operators of motorized vehicles, including the use of a video cognizer to provide pattern recognition and control signal generation during monitoring of macroscopic or microscopic biological processes. More specifically, the present invention accepts input from a video monitoring system that continuously captures the operator'"'"'s facial images; employing three sequential means of processing the digitized video information to extract the position and configuration of drowsy-relevant facial features and numerically processes this information to yield a quantitative estimate of drowsiness probability in each epoch of monitoring. The means of the present invention are noninvasive, do not restrict driver movement or performance, provide increased measurement reliability for actual driver behavior, and include the capability of generating or triggering suitable alarms when drowsiness occurs.

Claims

1. A method for monitoring a biological process, said method comprising:

receiving image data of an object having one or more features, said image data corresponding to frames comprising a plurality of pixels;

dividing each frame into one or more regions based on the one or more features and each region into one or more subregions;

filtering pixels in each subregion according to a pixel intensity range to provide a filtered output of pixels;

defining for each frame a first area within each subregion, said first area defined by one or more predetermined shape equations according to a shape of a feature to be monitored in the subregion;

evaluating for each frame the number of filtered output of pixels in the first area for each subregion to transform the image data to a scalar feature signal for each subregion; and

transforming each scalar feature signal to produce a behavior indicating output signal for each subregion, said step of transforming comprising setting a value of the behavior indicating output signal to a value of a scalar feature signal at a first time if a value of a scalar feature signal at the first time is equal to or greater than the value of the behavior indicating output signal at a time preceding the first time and reducing the value of the behavior indicating output signal by a fraction if the value of the scalar feature signal at the first time is less than the value of the behavior indicating output signal at a time preceding the first time.

2. The method of claim 1 further comprising:

defining for each frame a second area within each subregion, said second area adjacent to said first area; and

wherein

said step of evaluating comprises determining for each frame a difference between a number of filtered output pixels in the first area and a number of filtered output pixels in the second area and outputting a value of the difference to produce the scalar feature signal for each subregion.

3. The method of claim 1 further comprising:

detecting one or more changes in a feature in the first area of a subregion; and

modifying parameters of said shape equations defining said first area to correspond to the detected changes.

4. The method of claim 1 wherein said fraction being a function of time elapsed from the first time and wherein parameters of said function being determined by the biological process.

5. The method of claim 1 further comprising:

combining the behavior indicating output signals for each subregion to obtain a composite behavior indicating output signal to monitor the biological process.

6. The method of claim 5 wherein said step of combining comprises linearly combining the behavior indicating output signals.

7. The method of claim 5 further comprising:

correlating the composite behavior output signal with one or more independent measures of the biological process so as to increase the accuracy with which the composite behavior output signal monitors the biological process.

8. The method of claim 5 further comprising:

modifying the composite behavior indicating output signal pursuant to an algorithm under program control to produce a composite measure of the biological process; and

determining whether the composite measure is below a threshold.

9. The method of claim 8 further comprising:

sounding an alarm when the composite measure is below the threshold.

10. The method of claim 8 further comprising:

generating electrical control signals pursuant to an algorithm when the composite measure is below the threshold.

11. The method of claim 1 further comprising

acquiring image data via computerized microscopy;

and

wherein the biological process is microscopic at the tissue, cellular or subcellular level.

12. The method of claim 1 wherein the biological process is macroscopic.

13. The method of claim 12 wherein the macroscopic biological process is drowsiness.

14. The method of claim 13 wherein the step of receiving image data includes receiving data of facial images of an operator.

15. The method of claim 14 wherein the one or more regions comprises an eye region, a mouth region and a facial boundary region.

16. The method of claim 1 further comprising:

acquiring said image data by a video unit.

17. The method of claim 1 wherein said step of filtering comprises:

determining whether a video intensity level of each pixel is within the pixel intensity range; and

setting the video intensity level to a predetermined value if the video intensity level is within the range and to another predetermined value if the video intensity level is outside the range to provide the filtered output.

18. An apparatus for monitoring a biological process, said apparatus comprising a processor programmed to perform a method, said method comprising:

receiving image data of an object having one or more features, said image data corresponding to frames comprising a plurality of pixels;

dividing each frame into one or more regions based on the one or more features and each region into one or more subregions;

filtering pixels in each subregion according to a pixel intensity range to provide a filtered output of pixels;

defining for each frame a first area within each subregion, said first area defined by one or more predetermined shape equations according to a shape of a feature to be monitored in the subregion;

evaluating for each frame the number of filtered output of pixels in the first area for each subregion to transform the image data to a scalar feature signal for each subregion; and

transforming each scalar feature signal to produce a behavior indicating output signal for each subregion, said step of transforming comprising setting a value of the behavior indicating output signal to a value of a scalar feature signal at a first time if a value of a scalar feature signal at the first time is equal to or greater than the value of the behavior indicating output signal at a time preceding the first time and reducing the value of the behavior indicating output signal by a fraction if the value of the scalar feature signal at the first time is less than the value of the behavior indicating output signal at a time preceding the first time.

19. The apparatus of claim 18 wherein the processor is programmed to perform the method further comprising:

defining for each frame a second area within each subregion, said second area adjacent to said first area; and

wherein

said step of evaluating comprises determining for each frame a difference between a number of filtered output pixels in the first area and a number of filtered output pixels in the second area and outputting a value of the difference to produce the scalar feature signal for each subregion.

20. The apparatus of claim 18 wherein the processor is programmed to perform the method further comprising:

detecting one or more changes in a feature in the first area in a subregion; and

modifying parameters of said shape equations defining said first area to correspond to the detected changes.

21. The apparatus of claim 18 wherein said fraction being a function of time elapsed from the first time and wherein parameters of said function being determined by the biological process.

22. The apparatus of claim 18 wherein the processor is programmed to perform the method further comprising:

combining the behavior indicating output signals for each subregion to obtain a composite behavior indicating output signal to monitor the biological process.

23. The apparatus of claim 22 wherein the processor is programmed to perform the method wherein said step of combining comprises linearly combining the behavior indicating output signals.

24. The apparatus of claim 22 wherein the processor is programmed to perform the method further comprising:

correlating the composite behavior output signal with one or more independent measures of the biological process so as to increase the accuracy with which the composite behavior output signal monitors the biological process.

25. The apparatus of claim 22 wherein the processor is programmed to perform the method further comprising:

modifying the composite behavior indicating output signal pursuant to an algorithm under program control to produce a composite measure of the biological process; and

determining whether the composite measure is below a threshold.

26. The apparatus of claim 25 wherein the processor is programmed to perform the method further comprising:

sounding an alarm when the composite measure is below the threshold.

27. The apparatus of claim 25 wherein the processor is programmed to perform the method further comprising:

generating electrical control signals pursuant to an algorithm when the composite measure is below the threshold.

28. The apparatus of claim 18 wherein the processor is programmed to perform the method further comprising:

acquiring image data via computerized microscopy; and

wherein the biological process is microscopic at the tissue, cellular or subcellular level.

29. The apparatus of claim 18 wherein the biological process is macroscopic.

30. The apparatus of claim 29 wherein the macroscopic biological process is drowsiness.

31. The apparatus of claim 29 wherein the processor is programmed to perform the method wherein the step of receiving image data includes receiving data of facial images of an operator.

32. The apparatus of claim 31 wherein the one or more regions comprises an eye region, a mouth region and a facial boundary region.

33. The apparatus of claim 18 a video unit for acquiring said image data.

34. The apparatus of claim 18 wherein the processor is programmed to perform the method wherein said step of filtering comprises:

determining whether a video intensity level of each pixel is within the pixel intensity range; and

setting the video intensity level to a predetermined value if the video intensity level is within the range and to another predetermined value if the video intensity level is outside the range to provide the filtered output.