Method and apparatus for detecting eye location in an image

• US 6,381,345 B1
• Filed: 06/03/1997
• Issued: 04/30/2002
• Est. Priority Date: 06/03/1997
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Swain, Cassandra Turner
• Primary Examiner(s): Patel, Jayanti K.
• Application Number:
  US08/867,728
• Time in Patent Office: 1,792 Days
• Field of Search: 382/100, 382/115, 382/117, 382/118, 382/171, 382/181, 382/190, 382/254, 382/255, 382/256, 382/260, 382/261, 382/266, 382/274, 382/324, 404/16, 404/25, 348/78, 351/200, 351/206, 351/209, 351/210-215, 396/18, 396/51, 396/158, 424/571, 434/271, 446/389, 446/392, 600/318, 600/558, 606/5
• US Class Current: 382/117
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

A simple method for segmenting eyes and extracting parameters enables further processing of the image to enable a person to appear to be making eye contact with another via a video conferencing system. This method is a first step for eye synthesis and gaze detection because it can automatically extract select eye parameters useful to these processes. Its advantage is that no a priori information is necessary to segment eyes, unlike modeling and neural network methods. The method of the present invention first blurs the image to make it easier to determine the location of the two eye regions in the image. The eyebrows are then eliminated based on the located eye regions. The eyes are then segmented and the eye parameters are extracted from the resulting image According to the present invention, the process applies a Gaussian filter, h(x, y), where g(x, y) is the resulting image and ƒ(x, y) is the original image.

Claims

1. A method for locating eyes in an image, comprising the steps of:

a) spatially blurring the image with a Gaussian filter to form a blurred image;

b) determining a location of two eye regions in the blurred image;

c) eliminating eyebrows in the two eye regions in the original image based on the location of the two eye regions obtained in step b) from the blurred image, thereby forming a first processed image;

d) segmenting the eyes in the first processed image; and

e) extracting an iris location from the first processed image.

2. The method according to claim 1, wherein said Gaussian filter is a function, h(x, y), such that $g$

(x,y)=1∑

 



h

(x,y)

∑

x

 



∑

y

 



f

(x,y)

h

(x,y)

where g(x, y) is the resulting image and ƒ

(x, y) is the original image.

3. The method according to claim 2, wherein the gaussian filter, h(x,y), is defined as follows:

$h<br><br>\left(x,y\right)=\left[\begin{array}{ccccccccccccccc}2& 2& 3& 4& 5& 5& 6& 6& 6& 5& 5& 4& 3& 2& 2\\ 2& 3& 4& 5& 7& 7& 8& 8& 8& 7& 7& 5& 4& 3& 2\\ 3& 4& 6& 7& 9& 10& 10& 11& 10& 10& 9& 7& 6& 4& 3\\ 4& 5& 7& 9& 10& 12& 13& 13& 13& 12& 10& 9& 7& 5& 4\\ 5& 7& 9& 11& 13& 14& 15& 16& 15& 14& 13& 11& 9& 7& 5\\ 5& 7& 10& 12& 14& 16& 17& 18& 17& 16& 14& 13& 10& 7& 5\\ 6& 8& 10& 13& 15& 17& 19& 19& 19& 17& 15& 13& 10& 8& 6\\ 6& 8& 11& 13& 16& 18& 19& 20& 19& 18& 16& 13& 11& 8& 6\\ 6& 8& 10& 13& 15& 17& 19& 19& 19& 17& 15& 13& 10& 8& 6\\ 5& 7& 10& 12& 14& 16& 17& 18& 17& 16& 14& 13& 10& 7& 5\\ 5& 7& 9& 11& 13& 14& 15& 16& 15& 14& 13& 11& 9& 7& 5\\ 4& 5& 7& 9& 10& 12& 13& 13& 13& 12& 10& 9& 7& 5& 4\\ 3& 4& 6& 7& 9& 10& 10& 11& 10& 10& 9& 7& 6& 4& 3\\ 2& 3& 4& 5& 7& 7& 8& 8& 8& 7& 7& 5& 4& 3& 2\\ 2& 2& 3& 4& 5& 5& 6& 6& 6& 5& 5& 4& 3& 2& 2\end{array}\right].$

4. The method according to claim 1, wherein the step b) of determining further comprises limiting a search for the location of the eye regions to a central region of the original image.

5. The method according to claim 1, wherein the step b) of determining further comprises detecting a contrast between dark and light areas to identify eye regions.

6. The method according to claim 1, wherein the step c) of eliminating further comprises removing dark pixels at a top section of the eye regions.

7. The method according to claim 1, wherein the step of segmenting further comprises segmenting the eyes into three parts:

the iris, the corners, and the whites.

8. The method according to claim 1, wherein the step of segmenting further comprises segmenting the eyes based on intensity.

9. The method according to claim 8, wherein the step of segmenting further comprises segmenting the eyes according to the following function:

$s<br><br>\left(x,y\right)=\left\{\begin{array}{cc}\mathrm{eye}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{white}& \mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>g<br><br>\left(x,y\right)><br><br>T\\ \mathrm{iris},\mathrm{corner}& \mathrm{otherwise}\end{array}\right\}$

where T is a preset threshold and s(x, y) is the segmented image.

10. The method according to claim 9, wherein the threshold T is set high enough to segment all iris colors, but low enough to segment all white areas.

11. The method according to claim 9, wherein the step of segmenting further comprises reversing the intensity of the eye corners, irises and whites at intensity.

12. The method according to claim 1, wherein the step of extracting further comprises:

(1) scanning the eye regions from left to right, and from top to bottom;

(2) identifying the corners as the left-most and right-most white pixels;

(3) identifying the upper and lower lids as the top- and bottom-most white pixels in each eye region.

13. The method according to claim 12, wherein the step of extracting further comprises:

(1) calculating a width and height of white pixels that are separated from the corners by black pixels, which represent the eye whites;

(2) determining a center of the iris from the width and height; and

(3) determining the iris radius by dividing the width in half.

14. An apparatus for locating eyes in an image, comprising:

a) a digital camera capturing the image and converting the image into a plurality of pixels;

b) a processor being coupled to the digital camera and performing the steps of;

(1) spatially blurring the image with a Gaussian filter to form a blurred image;

(2) determining a location of two eye regions in the blurred image;

(3) eliminating eyebrows in the two eye regions in the original image based on the location of the two eye regions obtained in step (2) from the blurred image, thereby forming a first processed image;

(4) segmenting the eyes in the first processed image; and

(5) extracting an iris location from the first processed image.

15. The apparatus according to claim 14, wherein said Gaussian filter, h(x, y), operates according to the following equation:

$g<br><br>\left(x,y\right)=\frac{1}{\sum <br><br>\text{\hspace{1em}<br><br>}<br><br>h<br><br>\left(x,y\right)}<br><br>\underset{x}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{y}{\sum <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>f<br><br>\left(x,y\right)<br><br>h<br><br>\left(x,y\right)$

where g(x, y) is the resulting image and ƒ

(x, y) is the original image.

16. The apparatus according to claim 14, wherein the processor limits its scan to a central region of the image in its search for the location of the eye regions.

17. The apparatus according to claim 14, wherein the processor segments the eyes using the following function:

$s<br><br>\left(x,y\right)=\left\{\begin{array}{cc}\mathrm{eye}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{white}& \mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>g<br><br>\left(x,y\right)><br><br>T\\ \mathrm{iris},\mathrm{corner}& \mathrm{otherwise}\end{array}\right\}$

where T is a preset threshold and s(x, y) is the segmented image.

18. The apparatus according to claim 14, wherein the processor extracts the parameters by:

(1) scanning the eye regions from left to right, and from top to bottom;

(2) identifying the corners as the left-most and right-most white pixels;

(3) identifying, the upper and lower lids as the top- and bottom-most white pixels in each eye region.

19. The apparatus according to claim 18, wherein the processor further extracts the parameters by:

(1) calculating a width and height of white pixels that are separated from the corners by black pixels, which represent the eye whites;

(2) determining a center of the iris from the width and height; and

(3) determining the iris radius by dividing the width in half.

20. An apparatus for detecting eyes in an image, comprising:

a) means for capturing the image;

b) means for converting the image into a plurality of pixels being coupled to the capturing means;

c) a processor being coupled to the means for converting and receiving the plurality of pixels, and including;

(1) means for spatially blurring the image with a Gaussian filter to form a blurred image;

(2) means for determining a location of two eye regions in the blurred image;

(3) means for eliminating eyebrows in the two eye regions in the original image based on the location of the two eye regions obtained by said determining means, thereby forming a first processed image;

(4) means for segmenting the eyes in the first processed image, and

(5) means for extracting an iris location from the first processed image.

21. The apparatus according to claim 20 wherein the means for capturing comprises a camera and the means for converting comprises a digital to analog converter.

22. A method for locating eyes in an image, comprising:

spatially blurring the image with a Gaussian filter to form a blurred image;

predicting a probable location of iris regions in the blurred image, based on the location of other facial features;

within a window surrounding the probable location of iris regions, identifying light-to-dark transitions; and

extracting an iris location based on the identified transitions.