Eye position detecting apparatus

• US 5,218,387 A
• Filed: 05/20/1991
• Issued: 06/08/1993
• Est. Priority Date: 05/21/1990
• Status: Expired due to Term

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Ueno, Hiroshi, Yoshida, Kazuhiko
• Primary Examiner(s): Arnold, Bruce Y.
• Assistant Examiner(s): Parsons, David
• Application Number:
  US07/703,114
• Time in Patent Office: 750 Days
• Field of Search: 351/209, 351/210, 351/246, 356/214, 356/380, 359/201, 359/202, 364/922, 364/922.2, 364/922.3, 364/922.4, 364/DIG. 2, 364/413.02
• US Class Current: 351/210
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

An image of a face including eyes is converted into a binary coded image. The binary coded image is processed to detect first and second rectangular regions including image portions representing the eyes, respectively. For this purpose, the binary coded image is scanned in a series of horizontal scan lines to detect the X and Y coordinates of initial and final pixels of a line segment representing the longest width of the face. The X coordinates of each of the first and second rectangular regions are calculated based upon the detected X coordinates of the initial and final pixels. An image portion above the line segment is scanned to detect the Y coordinate of a lowermost one of pixels forming black image portions. The Y coordinates of the first and second rectangular regions are calculated based upon the detected Y coordinate of the lowermost pixel.

Claims

1. An eye position detecting apparatus comprising:

image input means for inputting an image of a face including eyes; and

a microcomputer, the microcomputer including;

image converting means for converting the inputted image into a binary coded image formed by an array of pixels, each of the pixels having a first value indicating a black pixel or a second value indicating a white pixel, and each being located at a position specified by coordinates (X,Y); and

detecting means for processing the binary coded image to detect first and second rectangular regions including image portions representing the eyes, respectively, each of the first and second rectangular regions being specified by two X coordinates, the detecting means including;

first means for scanning the binary coded image in a series of horizontal scan lines to detect the X and Y coordinates of initial and final pixels of a line segment representing the longest width of the face;

second means for calculating the X coordinates of each of the first and second rectangular regions based upon the detected X coordinates of the initial and final pixels;

third means for scanning an image portion above the line segment to detect the Y coordinate of a lowermost one of pixels forming black image portions; and

fourth means for calculating the Y coordinates of the first and second rectangular regions based upon the detected Y coordinate of the lowermost pixel.

2. The eye position detecting apparatus as claimed in claim 1, wherein the first means includes means for detecting the X and Y coordinates of the initial and final pixels of a longest white line segment formed by a greatest number of consecutive white pixels arranged along one of the horizontal scan lines.

3. The eye position detecting apparatus as claimed in claim 2, wherein the first means includes means for producing an inhibitory signal when the number of consecutive white pixels forming a line segment is greater than a predetermined value, and means responsive to the inhibitory signal for preventing the line segment from being detected as the longest line segment.

4. The eye position detecting apparatus as claimed in claim 1, wherein the third means scans the binary coded image upward in a series of vertical scan lines over a range of X1≦

X≦

X2 and 0≦

Y≦

YL where X1 and X2 are the calculated X coordinates of each of the first and second rectangular regions and YL is the detected Y coordinate of the line segment.

5. The eye position detecting apparatus as claimed in claim 4, wherein each vertical scan line is arranged for a predetermined number of pixels.

6. The eye position detecting apparatus as claimed in claim 4, wherein the third means includes means for detecting the Y coordinate of a lowermost one of black pixels forming a lower black image portion and the Y coordinate of a lowermost one of black pixels forming an upper black area image portion, means for detecting an image portion representing a spectacle frame, and wherein the fourth means includes means for calculating the Y coordinates of the first and second rectangular regions based upon the detected Y coordinate of the lowermost pixel of the upper black image portion.

7. The eye position detecting apparatus as claimed in claim 6, wherein the means for detecting an image portion representing a spectacle frame includes means for scanning the binary coded image in horizontal scan lines between Y coordinates of the lowermost pixels of the upper and lower black image portions to detect black pixels, and means for detecting the spectacle frame when a predetermined number of black pixels are detected.

8. The eye position detecting means as claimed in claim 6, wherein the third means includes for detecting one of the lower and upper black image portions when at least two consecutive black pixels exist on one of the vertical scan lines.

9. The eye position detecting apparatus according to claim 1, wherein said image input means includes a source of light intermittently irradiating said face, a timing circuit controlling actuation of said source of light, a video camera providing a video signal of said irradiated face, and an analog-to-digital converter receiving said video signal and converting it into digital form.

10. An eye position detecting apparatus comprising:

image input means for inputting an image of a face including eyes;

image converting means for converting the inputted image into a binary coded image formed by an array of pixels, each of the pixels having a first value indicating a black pixel or a second value indicating a white pixel, and each being located at a position specified by coordinates (X,Y); and

a microcomputer including detecting means for processing the binary coded image to detect first and second rectangular regions including image portions representing the eyes, respectively, each of the first and second rectangular regions being specified by two X coordinates and two Y coordinates, the detecting means including;

first means for scanning the binary coded image in a series of horizontal scan lines to detect the X and Y coordinates of initial and final pixels of a line segment representing the longest width of the face;

second means for calculating the X coordinates of each of the first and second rectangular regions based upon the detected X coordinates of the initial and final pixels;

third means for scanning an image portion above the line segment to detect the Y coordinate of a lowermost one of pixels forming black image portions; and

fourth means for calculating the Y coordinates of the first and second rectangular regions based upon the detected Y coordinate of the lowermost pixel.

11. The eye position detecting apparatus as claimed in claim 10, wherein the first means includes means for detecting the X and Y coordinates of the initial and final pixels of a longest white line segment formed by a greatest number of consecutive white pixels arranged along one of the horizontal scan lines.

12. The eye position detecting apparatus as claimed in claim 11, wherein the first means includes means for producing an inhibitory signal when the number of consecutive white pixels forming a line segment is greater than a predetermined value, and means responsive to the inhibitory signal for preventing the line signal from being detected as the longest line segment.

13. The eye position detecting apparatus as claimed in claim 10, wherein the third means scans the binary coded image upward in a series of vertical scan lines over a range of X1≦

X≦

X2 and 0≦

Y≦

YL where X1 and X2 are the calculated X coordinates of each of the first and second rectangular regions and YL is the detected Y coordinate of the line segment.

14. The eye position detecting apparatus as claimed in claim 13, wherein each vertical scan line is arranged for a predetermined number of pixels.

15. The eye position detecting apparatus as claimed in claim 13, wherein the third means includes means for detecting the Y coordinate of a lowermost one of black pixels forming a lower black image portion and the Y coordinate of a lowermost one of black pixels forming an upper black area image portion, means for detecting an image portion representing a spectacle/frame, and wherein the fourth means includes means for calculating the Y coordinates of the first and second rectangular regions based upon the detected Y coordinate of the lowermost pixel of the upper black image portion.

16. The eye position detecting apparatus as claimed in claim 15, wherein the means for detecting an image portion representing a spectacle frame includes means for scanning the binary coded image in horizontal scan lines between Y coordinates of the lowermost pixels of the upper and lower black image portions to detect black pixels, and means for detecting the spectacle frame when a predetermined number of black pixels are detected.

17. The eye position detecting means as claimed in claim 15, wherein the third means includes means for detecting one of the lower and upper black image portions when at least two consecutive black pixels exist on one of the vertical scan lines.

18. The eye position detecting apparatus according to claim 10, wherein said image input means includes a source of light intermittently irradiating said face, a timing circuit controlling actuation of said source of light, a video camera providing a video signal of said irradiated face, and an analog-to-digital converter receiving said video signal and converting it into digital form.

19. An eye position detecting apparatus comprising:

image input means for inputting an image of a face including eyes; and

a microcomputer, the microcomputer including;

image converting means for converting the inputted image into a binary coded image formed by an array of pixels, each of the pixels having a first value indicating a black pixel or a second value indicating a white pixel, and each being located at a position specified by coordinates (X,Y); and

detecting means for processing the binary coded image to detect a rectangular region including an image portion representing one of the eyes, the rectangular region being specified by two X coordinates and two Y coordinates, the detecting means including;

first means for scanning the binary coded image in a series of horizontal scan lines to detect the X and Y coordinates of initial and final pixels of a line segment representing the longest width of the face;

second means for calculating the X coordinates of the rectangular region based upon the detected X coordinates of the initial and final pixels;

third means for scanning an image portion above the line segment to detect the Y coordinate of a lowermost one of pixels forming black image portions; and

fourth means for calculating the Y coordinates of the rectangular region based upon the detected Y coordinate of the lowermost pixel.

20. The eye position detecting apparatus as claimed in claim 19, wherein the first means includes means for detecting the X and Y coordinates of the initial and final pixels of a longest white line segment formed by a greatest number of consecutive white pixels arranged along one of the horizontal scan lines.

21. The eye position detecting apparatus as claimed in claim 20, wherein the first means includes means for producing an inhibitory signal when the number of consecutive white pixels forming a line segment is greater than a predetermined value, and means responsive to the inhibitory signal for preventing the line segment from being detected as the longest line segment.

22. The eye position detecting apparatus as claimed in claim 19, wherein the third means scans the binary coded image upward in a series of vertical scan lines over a range of X1≦

X≦

X2 and 0≦

Y≦

YL where X1 and X2 are the calculated X coordinates of the rectangular region and YL is the detected Y coordinate of the line segment.

23. The eye position detecting apparatus as claimed in claim 22, wherein each vertical scan line is arranged for a predetermined number of pixels.

24. The eye position detecting apparatus as claimed in claim 22, wherein the third means includes means for detecting the Y coordinate of a lowermost one of black pixels forming a lower black image portion and the Y coordinate of a lowermost one of black pixels forming an upper black area image portion, means for detecting an image portion representing a spectacle frame, and wherein the fourth means includes means for calculating the Y coordinate of the rectangular region based upon the detected Y coordinate of the lowermost pixel of the upper black image portion.

25. The eye position detecting apparatus as claimed in claim 24, wherein the means for detecting an image portion representing a spectacle frame includes means for scanning the binary coded image in horizontal scan lines between Y coordinates of the lowermost pixels of the upper and lower black image portions to detect black pixels, and means for detecting the spectacle frame when a predetermined number of black pixels are detected.

26. The eye position detecting means as claimed in claim 24, wherein the third means includes means for detecting one of the lower and upper black image portions when at least two consecutive black pixels exist on one of the vertical scan lines.

27. The eye position detecting apparatus according to claim 19, wherein said image input means includes a source of light intermittently irradiating said face, a timing circuit controlling actuation of said source of light, a video camera providing a video signal of said irradiated face, and an analog-to-digital converter receiving said video signal and converting it into digital form.

28. An eye position detecting apparatus comprising:

image input means for inputting an image of a face including eyes;

image converting means for converting the inputted image into a binary coded image formed by an array of pixels, each of the pixels having a first value indicating a black pixel or a second value indicating a white pixel, and each being located at a position specified by coordinates (X,Y); and

a microcomputer including detecting means for processing the binary coded image to detect a rectangular region including image portions representing one of the eyes, the rectangular region being specified by two X coordinates and two Y coordinates, the detecting means including;

first means for scanning the binary coded image in a series of horizontal scan lines to detect the X and Y coordinates of initial and final pixels of a line segment representing the longest width of the face;

second means for calculating the X coordinates of the rectangular region based upon the detected X coordinates of the initial and final pixels;

third means for scanning an image portion above the line segment to detect the Y coordinates of a lowermost one of pixels forming black image portions; and

fourth means for calculating the Y coordinates of the rectangular region based upon the detected Y coordinate of the lowermost pixel.

29. The eye position detecting apparatus as claimed in claim 28, wherein the first means includes means for detecting the X and Y coordinates of the initial and final pixels of a longest white line segment formed by a greatest number of consecutive white pixels arranged along one of the horizontal scan lines.

30. The eye position detecting apparatus as claimed in claim 29, wherein the first means includes means for producing an inhibitory signal when the number of consecutive white pixels forming a line segment is greater than a predetermined value, and means responsive to the inhibitory signal for preventing the line segment from being detected as the longest line segment.

31. The eye position detecting apparatus as claimed in claim 28, wherein the third means scans the binary coded image upward in a series of vertical scan lines over a range of X1≦

X≦

X2 and 0≦

Y≦

YL where X1 and X2 are the calculated X coordinates of the rectangular region and YL is the detected Y coordinate of the line segment.

32. The eye position detecting apparatus as claimed in claim 31, wherein each vertical scan line is arranged for a predetermined number of pixels.

33. The eye position detecting apparatus as claimed in claim 31, wherein the third means includes means for detecting the Y coordinate of a lowermost one of black pixels forming a lower black image portion and the Y coordinate of a lowermost one of black pixels forming an upper black area image portion, means for detecting an image portion representing a spectacle frame, and wherein the fourth means includes means for calculating the Y coordinate of the rectangular region based upon the detected Y coordinate of the lowermost pixel of the upper black image portion.

34. The eye position detecting apparatus as claimed in claim 33, wherein the means for detecting an image portion representing a spectacle frame includes means for scanning the binary coded image in horizontal scan lines between Y coordinates of the lowermost pixels of the upper and lower black image portions to detect black pixels, and means for detecting the spectacle frame when a predetermined number of black pixels are detected.

35. The eye position detecting means as claimed in claim 33, wherein the third means includes means for detecting one of the lower and upper black image portions when at least two consecutive black pixels exist on one of the vertical scan lines.

36. The eye position detecting apparatus according to claim 28, wherein said image input means includes a source of light intermittently irradiating said face, a timing circuit controlling actuation of said source of light, a video camera providing a video signal of said irradiated face, and an analog-to-digital converter receiving said video signal and converting it into digital form.