Method and apparatus for determining eye contact
First Claim
1. A method for determining whether a subject is looking at a target area, the subject having a face including an eye with a pupil, the method comprising the operations of:
- if distance between the subject and the target area is less than a predetermined range, performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria; and
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
otherwise, if distance between the subject and the target area is less than the predetermined range, performing operations comprising;
receiving a machine-readable input representing three-dimensional positions of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area.
4 Assignments
0 Petitions
Accused Products
Abstract
A method and apparatus determine when a subject is looking at a specific target area by estimating a divergence angle between (1) the direction in which the subject is looking and (2) the direction from the subject directly to the target area. This technique accesses whether the subject is looking at a particular area. The invention may further condition this determination according to the subject'"'"'s distance from the target area, because there is less tolerance for divergent angles when the subject is farther away. In one embodiment, the divergence angle is estimated using the position of a glint of light in the subject'"'"'s pupil. The glint is created by a light source located in the target area. If the glint is sufficiently central to the pupil, with the camera and light source being near the target area, the subject is looking at the target area. At long distances, when the glint is not sufficiently discernable from the pupil, another technique may be employed to estimate divergence angle. Namely, the plane of the subject'"'"'s face is computed, and analyzed with respect to a vector between the subject'"'"'s face and the target area. If the plane is substantially normal to the vector, the subject is looking at the target area.
180 Citations
22 Claims
-
1. A method for determining whether a subject is looking at a target area, the subject having a face including an eye with a pupil, the method comprising the operations of:
-
if distance between the subject and the target area is less than a predetermined range, performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria; and
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
otherwise, if distance between the subject and the target area is less than the predetermined range, performing operations comprising;
receiving a machine-readable input representing three-dimensional positions of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area. - View Dependent Claims (2, 3)
-
-
4. A method for conditionally processing voice input, comprising:
-
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria;
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
5. A method for conditionally processing voice input comprising:
-
receiving a machine-readable input representing three-dimensional position of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area;
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
6. A signal-bearing medium tangibly embodying a program of machine-readable instructions executable by a digital data processing apparatus to perform operations for determining whether a subject is looking at a target area, the subject having a face including an eye with a pupil, the operations comprising:
-
if distance between the subject and the target area is less than a predetermined range, performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria; and
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
otherwise, if distance between the subject and the target area is less than the predetermined range, performing operations comprising;
receiving a machine-readable input representing three-dimensional positions of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area. - View Dependent Claims (7, 8)
-
-
9. A signal-bearing medium tangibly embodying a program of machine-readable instructions executable by a digital data processing apparatus to perform operations for conditionally processing voice input according to whether a subject having a face is looking at a target area, the operations comprising:
-
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria;
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
10. A signal-bearing medium tangibly embodying a program of machine-readable instructions executable by a digital data processing apparatus to perform operations for conditionally processing voice input according to whether a subject having a face is looking at a target area, the operations comprising:
-
receiving a machine-readable input representing three-dimensional position of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area;
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
11. A logic circuit configured to perform operations for determining whether a subject is looking at a target area, the subject having a face including an eye with a pupil, the operations comprising:
-
if distance between the subject and the target area is less than a predetermined range, performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria; and
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
otherwise, if distance between the subject and the target area is less than the predetermined range, performing operations comprising;
receiving a machine-readable input representing three-dimensional position of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area.
-
-
12. A logic circuit configured to perform operations for conditionally processing voice input according to whether a subject having a face is looking at a target area, the operations comprising:
-
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria;
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
13. A logic circuit configured to perform operations for conditionally processing voice input according to whether a subject having a face is looking at a target area, the operations comprising:
-
receiving a machine-readable input representing three-dimensional position of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area;
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input.
-
-
14. A computer-driven eye contact sensing system, comprising:
-
a light source;
a light sensor; and
an eye contact estimating apparatus, coupled to the light sensor and configured to perform operations for determining whether a subject is looking at a target area, the subject having a face including an eye with a pupil, the operations comprising;
if distance between the subject and the target area is less than a predetermined range, performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria; and
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
otherwise, if distance between the subject and the target area is less than the predetermined range, performing operations comprising;
receiving a machine-readable input representing three-dimensional positions of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets, predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area.
-
-
15. A computer-driven eye contact sensing system, comprising:
-
a light source;
a light sensor; and
a data processing apparatus, coupled to the light sensor and programmed to conditionally process voice input according to whether a subject having a face is looking at a target area by performing operations comprising;
operating a light source to create a glint upon the subject'"'"'s eye;
using a light sensor to detect the glint and measure glint position;
receiving a machine-readable representation of the pupil;
analyzing relative position of the glint with respect to the pupil to determine whether the relative position satisfies predetermined criteria;
only if the relative position satisfies the predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input. - View Dependent Claims (16, 17, 18)
-
-
19. A computer-driven eye contact sensing system, comprising:
-
a light source;
a light sensor; and
a data processing apparatus, coupled to the light sensor and programmed to conditionally process voice input according to whether a subject having a face is looking at a target area by performing operations comprising;
receiving a machine-readable input representing three-dimensional position of multiple points on the face;
using the points to compute a face plane describing an average orientation of the face;
determining alignment between the face plane and the target area; and
if the alignment between the face plane and the target area meets predetermined criteria, providing an eye contact output declaring that the subject is looking at the target area;
responsive to issuance of voice input, consulting the eye contact output to determine whether the subject is looking in the target area; and
ignoring the voice input unless the subject is looking in the target area when issuing the voice input. - View Dependent Claims (20, 21, 22)
-
Specification