EYE-TRACKING METHOD AND SYSTEM FOR IMPLEMENTING THE SAME

• US 20100103375A1
• Filed: 07/20/2009
• Published: 04/29/2010
• Est. Priority Date: 10/27/2008
• Status: Active Grant

• Current Assignee: Utechzone Co., Ltd.
• Original Assignee: Utechzone Co., Ltd.
• Inventors: CHAO, Shin-Min
• Granted Patent: US 7,832,866 B2
• Time in Patent Office: Days
• US Class Current: 351/210
• CPC Class Codes: A61B 3/113 for determining or recording eye movement

Abstract

An eye-tracking method includes acquiring an image of an eye of a user, identifying a pupil of and a reflected light spot on the eye from the acquired image, determining a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil, determining a gazing point based on the distance and the angle and a pre-established set of data, and performing an operation associated with the gazing point. A system for implementing the eye-tracking method is also disclosed.

Claims

1. An eye-tracking method adapted to be implemented by a system, said eye-tracking method comprising:

A) while a user is gazing at a screen module and a light source emits light toward an eye of the user, configuring the system to acquire an image of the eye of the user captured by an image-capturing module;

B) configuring the system to identify a pupil of and a reflected light spot on the eye from the image acquired in step A);

C) configuring the system to determine a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil;

D) configuring the system to determine a gazing point based on the distance and the angle determined in step C) and a pre-established set of data obtained based on predefined regions on the screen module; and

E) configuring the system to perform an operation associated with the gazing point.

2. The eye-tracking method as claimed in claim 1, wherein the pre-established set of data includes a plurality of candidate points, each of which corresponds to a respective one of the predefined regions and is defined by coordinates, and step D) includes the sub-steps of:

d1) configuring the system to define a point, the coordinates of which are the distance and the angle determined in step C);

d2) configuring the system to find one of the candidate points in the pre-established set of data that has a minimum distance from the point defined in sub-step d1); and

d3) configuring the system to regard the candidate point found in sub-step d2) as the gazing point.

3. The eye-tracking method as claimed in claim 2, wherein sub-step d2) is performed using a minimum object function, which is defined as

w₁*|x₁′

−

x₂′

|+w₂*|y₁′

−

y₂′

|

where x₁′ and

y₁′

are the coordinates of one of the candidate points in the pre-established set of data, x₂′ and

y₂′

are the coordinates of the point defined in sub-step d1), w₁ is a weight assigned to |x₁′

−

x₂′

|, and w₂ is a weight assigned to |y₁−

y₂′

|.

4. The eye-tracking method as claimed in claim 2, further comprising F) obtaining the pre-established set of data prior to step A).

5. The eye-tracking method as claimed in claim 4, wherein step F) includes the sub-steps of:

f1) configuring the system to partition the screen module into the predefined regions;

f2) configuring the system to control the screen module to indicate one of the predefined regions;

f3) while the user is gazing at the predefined region indicated in sub-step f2) and the light source emits light toward the eye of the user, configuring the system to acquire an image of the eye of the user captured by the image-capturing module;

f4) configuring the system to identify a pupil of and a reflected light spot on the eye from the image acquired in sub-step f3);

f5) configuring the system to determine a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil;

f6) configuring the system to obtain a candidate point, the coordinates of which are the distance and the angle determined in sub-step f5), whereby the candidate point corresponds to the predefined region indicated in sub-step f2); and

f7) configuring the system to repeat sub-steps f2) to f6) until the candidate points corresponding to all of the predefined regions are obtained.

6. The eye-tracking method as claimed in claim 1, wherein step D) is performed using an unwarping technique.

7. The eye-tracking method as claimed in claim 1, wherein the pre-established set of data includes a plurality of coefficients, and step D) is performed using an unwarping technique, which calculates coordinates (u′

, v′

) of the gazing point using equations

u′

=a₀+a₁x′

+a₂y′

+a₃x′

²+a₄x′

y′

+a₅y′

²,and

v′

=b₀+b₁x′

+b₂y′

+b₃x′

²+b₄x′

y′

+b₅y′

²

where a₀ to a₅ and b₀ to b₅ are the coefficients in the pre-established set of data, and x′ and

y′

are the distance and the angle determined in step C).

8. The eye-tracking method as claimed in claim 6, further comprising F) obtaining the pre-established set of data prior to step A).

9. The eye-tracking method as claimed in claim 8, wherein step F) includes the sub-steps of:

f1) configuring the system to partition the screen module into the predefined regions;

f2) configuring the system to define coordinates corresponding to each of the predefined regions;

f3) configuring the system to control the screen module to indicate one of the predefined regions;

f4) while the user is gazing at the predefined region indicated in sub-step f3) and the light source emits light toward the eye of the user, configuring the system to acquire an image of the eye of the user captured by the image-capturing module;

f5) configuring the system to identify a pupil of and a reflected light spot on the eye from the image acquired in sub-step f4);

f6) configuring the system to determine a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil, whereby the distance and the angle correspond to the predefined region indicated in sub-step f3);

f7) configuring the system to repeat sub-steps f3) to f6) until the distances and the angles corresponding to all of the predefined regions are determined; and

f8) configuring the system to obtain coefficients based on the coordinates corresponding to the predefined regions defined in sub-step f2) and the distances and the angles corresponding to the predefined regions.

10. The eye-tracking method as claimed in claim 9, wherein, in sub-step f8), the coefficients a₀ to a₅ and b₀ to b₅ are obtained using equations

u=a₀+a₁x+a₂y+a₃x²+a₄xy+a₅y²,and

v=b₀+b₁x+b₂y+b₃x²+b₄xy+b₅y²

where u and v are the coordinates of one of the predefined regions defined in sub-step f2), and x and y are the distance and the angle determined in sub-step f6).

11. The eye-tracking method as claimed in claim 1, wherein step E) includes the sub-step of moving a cursor on the screen module to the gazing point.

12. A system adapted to be coupled to an image-capturing module and a screen module, comprising:

a computing module operable

to acquire an image of an eye of a user captured by the image-capturing module while the user is gazing at the screen module and a light source emits light toward the eye of the user,

to identify a pupil of and a reflected light spot on the eye from the image acquired from the image-capturing module,

to determine a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil, and

to determine a gazing point based on the distance and the angle determined thereby and a pre-established set of data obtained based on predefined regions on the screen module; and

a processing module coupled to said computing module and operable to perform an operation associated with the gazing point.

13. The system as claimed in claim 12, wherein the pre-established set of data includes a plurality of candidate points, each of which corresponds to a respective one of the predefined regions and is defined by coordinates, said computing module determining the gazing point by

defining a point, the coordinates of which are the distance and the angle determined thereby,

finding one of the candidate points in the pre-established set of data that has a minimum distance from the point defined thereby, and

regarding the candidate point found thereby as the gazing point.

14. The system as claimed in claim 13, wherein said computing module finds said one of the candidate points in the pre-established set of data using a minimum object function, which is defined as

w₁*|x₁′

−

x₂′

|+w₂*|y₁′

−

y₂′

|

where x₁′ and

y₁′

are the coordinates of one of the candidate points in the pre-established set of data, x₂′ and

y₂′

are the coordinates of the point defined by said computing module, w₁ is a weight assigned to |x₁′

−

x₂′

|, and w₂ is a weight assigned to |y₁′

−

y₂′

|.

15. The system as claimed in claim 13, wherein said computing module is further operable so as to obtain the pre-established set of data.

16. The system as claimed in claim 15, wherein said computing module obtains the pre-established set of data by obtaining the candidate points corresponding to all of the predefined regions, said computing module obtaining each of the candidate points by

partitioning the screen module into the predefined regions,

controlling the screen module to indicate one of the predefined regions,

acquiring an image of the eye of the user captured by the image-capturing module while the user is gazing at the predefined region indicated by the screen module and the light source emits light toward the eye of the user,

identifying a pupil of and a reflected light spot on the eye from the image acquired from the image-capturing module,

determining a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil, and

obtaining a candidate point, the coordinates of which are the distance and the angle thus determined, whereby the candidate point corresponds to the predefined region indicated by the screen module.

17. The system as claimed in claim 12, wherein said computing module determines the gazing point using an unwarping technique.

18. The system as claimed in claim 12, wherein the pre-established set of data includes a plurality of coefficients, and said computing module determines the gazing point using an unwarping technique, which calculates coordinates (u′

, v′

) of the gazing point using equations

u′

=a₀+a₁x′

+a₂y′

+a₃x′

²+a₄x′

y′

+a₅y′

²,and

v′

=b₀+b₁x′

+b₂y′

+b₃x′

²+b₄x′

y′

+b₅y′

²,

where a₀ to a₅ and b₀ to b₅ are the coefficients in the pre-established set of data, and x′ and

y′

are the distance and the angle determined by said computing module.

19. The system as claimed in claim 17, wherein said computing module is further operable so as to obtain the pre-established set of data.

20. The system as claimed in claim 19, wherein:

said computing module obtains the pre-established set of data by

defining coordinates corresponding to each of the predefined regions,

determining a distance and an angle corresponding to each of the predefined regions, and

obtaining the coefficients based on the coordinates defined thereby and the distances and the angles determined thereby; and

said computing module determines the distance and the angle corresponding to each of the predefined regions by

partitioning the screen module into the predefined regions,

controlling the screen module to indicate one of the predefined regions,

acquiring an image of the eye of the user captured by the image-capturing module while the user is gazing at the predefined region indicated by the screen module and the light source emits light toward the eye of the user,

identifying a pupil of and a reflected light spot on the eye from the image acquired from the image-capturing module, and

determining a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil, whereby the distance and the angle correspond to the predefined region indicated by the screen module.

21. The system as claimed in claim 20, wherein said computing module obtains the coefficients a₀ to a₅ and b₀ to b₅ using equations

u=a₀+a₁x+a₂y+a₃x²+a₄xy+a₅y²,and

v=b₀+b₁x+b₂y+b₃x²+b₄xy+b₅y²,

where u and v are the coordinates corresponding to one of the predefined regions defined by said computing module, and x and y are the distance and the angle determined by said computing module.

22. The system as claimed in claim 12, wherein the operation associated with the gazing point includes moving a cursor on the screen module to the gazing point.

23. A computer program product, comprising:

a machine readable storage medium having program instructions stored therein which when executed cause a system to perform a set of operations including;

A) while a user is gazing at a screen module and a light source emits light toward an eye of the user, acquiring an image of the eye of the user captured by an image-capturing module;

B) identifying a pupil of and a reflected light spot on the eye from the image acquired in step A);

C) determining a distance of the reflected light spot from a reference point on the pupil and an angle of the reflected light spot from an axis passing through the reference point on the pupil;

D) determining a gazing point based on the distance and the angle determined in step C) and a pre-established set of data obtained based on predefined regions on the screen module; and

E) performing an operation associated with the gazing point.