Sight-line detection method and device, and three-dimensional view-point measurement device

• US 7,533,989 B2
• Filed: 12/24/2004
• Issued: 05/19/2009
• Est. Priority Date: 12/25/2003
• Status: Active Grant

• Current Assignee: National University Corporation Shizuoka University
• Original Assignee: National University Corporation Shizuoka University
• Inventors: Ebisawa, Yoshinobu
• Primary Examiner(s): HASAN, MOHAMMED A
• Application Number:
  US10/584,635
• Publication Number:
  US 20070279590A1
• Time in Patent Office: 1,607 Days
• Field of Search: 351200-245
• US Class Current: 351/208
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Method and device for detecting a line-of-sight of a subject and a three-dimensional view-point measurement device. The method uses a first camera, that measures the position of a pupil relative to a coordinate system, a second camera having a light source arranged at a known position in the coordinate system, and forming a corneal reflection center to obtain data of a size of vector r from the corneal reflection center to the pupil center and an angle φ of the vector r relative to a coordinate axis of the coordinate system, and a calculation means for calculating a line-of-sight direction based on information from each camera. In determining a relational formula, a subject is made to gaze at a known point to perform measurement and a relational formula is determined. In a line-of-sight determining stage, the subject is measured again using the relational formula. A three-dimensional view-point measurement device can be configured so as to simultaneously measure the lines-of-sights of both eyes with two cameras and two light sources.

Claims

1. A line-of-sight detection method of a subject using:

a first camera for measuring the position of a pupil relative to a coordinate system;

a second camera having a light source arranged at a known position in the coordinate system and forming a corneal reflection center to obtain data of a size of vector r from the corneal reflection center to a pupil center and an angle φ

of the vector r relative to a coordinate axis of the coordinate system; and

a calculation means for calculating the line-of-sight direction for executing steps below based on information from each of the cameras, comprises the stages of;

determining a relational formula, including the steps of;

obtaining data on a coordinate point O of the position of a pupil of a subject with the first camera by making the subject gaze at a known point G in the coordinate system;

obtaining, in the state of the subject, data of the corneal reflection center, a size of vector r from the reflection center to a pupil center P, and an inclination φ

of the vector r relative to the coordinate axis with the second camera;

calculating an angle θ

between a line connecting a reference position of the second camera and the pupil center and a line-of-sight of the subject by the calculation means; and

calculating a formula θ

=f(r*) showing a relationship between r* related to r and θ

based on the measured values and calculated value; and

determining a line-of-sight, including the steps of;

obtaining data on a coordinate point O′

of the pupil position of the subject with the first camera by making the subject gaze at an unknown point G′

in the coordinate system;

obtaining data of the corneal reflection center, a size of vector r′

from the reflection center to the pupil center P, and an inclination φ

′

of the vector r′

relative to the coordinate axis with the second camera; and

calculating θ

=f(r*′

) by using the relational formula to obtain the unknown point G′

from the inclination φ

′ and

θ

′

.

2. The line-of-sight detection method of the subject according to claim 1, wherein r* is r itself or a corrected value of r based on OP, and r*′

is r′

itself or a corrected value of r′

based on OP′

.

3. The line-of-sight detection method of the subject according to claim 1, wherein the first camera is a stereo camera arranged by aligning a baseline in a horizontal axis direction of the coordinate system, and a light source of the second camera is constructed so as to provide an optical axis that is substantially aligned with that of the second camera.

4. The line-of-sight detection method of the subject according to claim 1, wherein the first camera is a stereo camera, and a light source of the second camera is constructed so as to provide an optical axis that is substantially aligned with that of the second camera.

5. The line-of-sight detection method of the subject according to claim 1, wherein the formula θ

=f(r*) showing the relationship between r* and θ

is given by θ

=k×

r* (where k is a constant).

6. A line-of-sight detection method of the subject according to claim 1, wherein the pupil is one of pupils of the subject.

7. An line-of-sight detection device of the subject, comprising:

a first camera for measuring a position P of a pupil relative to the coordinate system;

a second camera having a light source arranged at a known position in the coordinate system to obtain data of a size of vector r from a corneal reflection center to a pupil center illuminated by the light source and an angle φ

of r relative to the coordinate axis; and

a calculation means for executing the steps of;

obtaining data on a coordinate point P of the position of a pupil of a subject with the first camera by making the subject gaze at a known point G in the coordinate system;

obtaining, in the state of the subject, data of the corneal reflection center, a size of vector r from the reflection center to a pupil center P, and an inclination φ

of the vector r relative to the coordinate axis with the second camera;

calculating an angle φ

between a line connecting a reference position of the second camera and the pupil center and the line-of-sight of the subject and calculating a formula θ

=f(r*) showing a relationship between r* related to r and θ

;

obtaining data on a coordinate point O′

of the pupil position of the subject with the first camera by making the subject gaze at an unknown point G′

in the coordinate system;

obtaining data of the corneal reflection center, a size of vector r′

from the reflection center to the pupil center P, and an inclination φ

′

of the vector r′

relative to the coordinate axis with the second camera; and

calculating θ

=f(r*′

) from r*′

related to r′

by using the relational formula to further obtain the unknown point G′

from φ

′ and

θ

′

.

8. A three-dimensional view-point measurement device, comprising:

two cameras, a first light source arranged near one of the two cameras, a second light source arranged near another of the two cameras, a control means for controlling ON/OFF of the first light source and the second light source and obtaining an image signal in sync with ON/OFF, and a calculation means for extracting a pupil and corneal reflection from the obtained image signal.

9. The three-dimensional view-point measurement device according to claim 8, wherein the first light source and the second light source are configured to have an approximately identical emission wavelength.

10. A three-dimensional view-point measurement device, comprising:

two cameras, a first light source arranged near one of the two cameras, a second light source arranged near another of the two cameras, a control means for controlling ON/OFF of the first light source and the second light source and obtaining an image signal in sync with ON/OFF, and a calculation means for extracting a pupil and corneal reflection from the obtained image signal and calculating a line of-sight vector from these positions.

11. The three-dimensional view-point measurement device according to claim 10, wherein the first light source and the second light source are configured to have an approximately identical emission wavelength.

12. A three-dimensional view-point measurement device, comprising:

two cameras, a first light source arranged near one of the two cameras, a second light source arranged near another of the two cameras, a control means for controlling ON/OFF of the first light source and the second light source and obtaining an image signal in sync with ON/OFF, and a calculation means for extracting a pupil and corneal reflection from the obtained image signal and calculating a three-dimensional position of the pupil from these positions.

13. The three-dimensional view-point measurement device according to claim 12, wherein the first light source and the second light source are configured to have an approximately identical emission wavelength.