METHOD AND APPARATUS FOR TRACKING GAZE POSITION

• US 20080137909A1
• Filed: 12/06/2007
• Published: 06/12/2008
• Est. Priority Date: 12/06/2006
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: JUNG, Young Giu, LIM, Joa Sang, PARK, Kang Ryoung, LEE, Eui Chul, LEE, Jaeseon, CHO, Yongjoo, HAN, Mun Sung, HWANG, Min Cheol, PARK, Jun Seok
• Application Number:
  US11/951,813
• Publication Number:
  US 20080137909A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A gaze position tracking method and apparatus for simply mapping one'"'"'s gaze position on a monitor screen are provided. The gaze position tracking apparatus includes an image capturing module, and an image processing module. The image capturing module illuminates infrared rays to a user'"'"'s eyes, reflects an illuminated eye image at 45°, and captures the 45° reflected eye image. The image processing module obtains a pupil center point of the illuminated eye image by performing a predetermined algorithm, and maps the pupil center point on a display plane of a display device through a predetermined transform function.

Claims

1. A gaze position tracking apparatus for detecting a user'"'"'s gaze position form a terminal having a display device, comprising:

an image capturing module for illuminating infrared rays to an user'"'"'s eyes, reflecting an user'"'"'s eye image illuminated by infrared rays (hereinafter, infrared eye image) at 45°

, and capturing the 45°

reflected user'"'"'s eye image; and

an image processing module for obtaining a pupil center point of the infrared eye image by performing a predetermined algorithm, and mapping the pupil center point on a display plane of a display device through a predetermined transform function.

2. The gaze position tracking apparatus according to claim 1, wherein the image capturing module includes:

an infrared ray lighting unit for illuminating an infrared ray to the user'"'"'s eyes;

an infrared ray reflector for reflecting the infrared eye image at 45°

; and

an miniature camera for capturing the 45°

reflected user'"'"'s eye image.

3. The gaze position tracing apparatus according to claim 2, wherein the infrared lighting unit includes at least one of LED (light emitting diode), a halogen lamp, a xenon lamp, and an incandescent electric lamp.

4. The gaze position tracking apparatus according to claim 2, wherein the miniature camera includes:

a lens for receiving the 45°

reflected user'"'"'s eye image through the infrared reflector;

an image sensor formed of charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) for receiving the user'"'"'s eye image inputted to the lens; and

an infrared ray pass filter mounted on the entire surface of the lens or the image sensor for passing a infrared ray wavelength only.

5. The gaze position tracking apparatus according to claim 1, wherein the image capturing module is mounted at least one of glasses, goggles, a helmet, and a fixable supporting member.

6. The gaze position tracking apparatus according to claim 1, wherein the image capturing module is embodied in the terminal in software manner.

7. The gaze position tracking apparatus according to claim 2, wherein the image capturing module further includes an interface unit connected to the terminal in a PnP (plug and play) manner, supplying power provided from the terminal to the infrared ray lighting unit and the miniature camera, and providing every image frame captured through the miniature camera to the terminal.

8. The gaze position tracking apparatus according to claim 7, wherein the interface unit is connected to the terminal in at least one of a USB (universal serial bus) type, an analog type, a SD (secure digital) type, and a CD (compact disc) type.

9. The gaze position tracking apparatus according to claim 1, wherein the image processing module obtains the pupil center point from the eye image by performing at least one of a circle detection algorithm and a local binarization scheme, wherein the circle detection algorithm detects a pupil region included in the eye image by shifting a circle template to the eye image and obtains the pupil center point from the detected pupil region and the local binarization scheme performs binarization on predetermined region from the pupil region and detects the center of gravity of a dark region as the pupil center point.

10. The gaze position tracking apparatus according to claim 1, wherein the transform function is at least one of a linear interpolation transform function, a geometric transform function, and a cross ration transform function.

11. The gaze position tracking apparatus according to claim 1, wherein the image processing module calibrates a display plane position corresponding to a user'"'"'s pupil center point by performing a user calibration process in a system initialization period, and performs a mapping process on the display plane for the every captured user'"'"'s eye image.

12. The gaze position tracking apparatus according to claim 11, wherein the image processing module performs the user calibration process by calibrating the display plane position from an image of eyes gazing at a right upper corner and a left lower corner or from an image of eyes gazing at a right lower corner and a left upper corner using the linear interpolation transform function, or by calibrating the display plane position from images of eyes gazing at four corners of the display plane using the geometric transform function and the cross ration transform function.

13. A gaze position tracking method for detecting a user'"'"'s gaze position for a terminal having a display device, comprising:

illuminating an infrared ray to a user'"'"'s eyes gazing a display plane of the display device;

reflecting an eye image illuminated by infrared rays (hereinafter, infrared eye image) at 45° and

capturing a 45°

reflected eye image through a miniature camera;

obtaining a pupil center point of the eye image by performing the predetermined algorithm; and

mapping the pupil center point to the display plane using the predetermined transform function.

14. The gaze position tracking method according to claim 13, wherein in the step of illuminating the infrared ray, the user'"'"'s eyes are illuminated using at least one of LED (light emitting diode), a halogen lamp, a xenon lamp, and an incandescent electric lamp.

15. The gaze position tracking detection method according to claim 13, wherein in the step of obtaining the user'"'"'s eye image through the miniature camera, the 45°

reflected infrared eye image is passed only by mounting an infrared ray passing filter on the entire surface of the miniature camera lens or an image sensor.

16. The gaze position tracking method according to claim 13, wherein in the step of capturing the 45°

reflected user'"'"'s eye image, the user'"'"'s eye image is captured a miniature camera, an infrared ray lighting unit, and an infrared ray reflector mounted on at least one of glasses, goggles, a helmet, and a fixable supporting member.

17. The gaze position tracking method according to claim 13, wherein in the step of obtaining the pupil center point, the eye image captured through the miniature camera is provided to the terminal in a PnP manner, and the pupil center point is obtained through the predetermined algorithm equipped with the terminal.

18. The gaze position tracking method according to claim 17, wherein in the step of providing the eye image to a terminal in the PnP manner, the eye image is provided to the terminal in at least one of an USB type, an analog type, a SD type, and a CD type.

19. The gaze position tracking method according to claim 13, wherein the step of obtaining the pupil center point includes:

obtaining the pupil center point from the eye image by performing at least one of a circle detection algorithm and a local binarization scheme, wherein the circle detection algorithm detects a pupil region included in the eye image by shifting a circle template to the eye image and obtains the pupil center point from the detected pupil region and the local binarization scheme performs binarization on predetermined region from the pupil region and detects the center of gravity of a dark region as the pupil center point.

20. The gaze position tracking method according to claim 13, wherein in the step for mapping the pupil center point on a display plane, the pupil center point is mapped to the display plane through at least one of a linear interpolation transform function, a geometric transform function, and a cross ration transform function.

21. The gaze position tracking method according to claim 13, further comprising a step of calibrating a display plane position corresponding a user'"'"'s pupil center point by performing a user calibration process in a system initialization period.

22. The gaze position tracking method according to claim 21, wherein in the step of calibrating the display plane position corresponding the user'"'"'s pupil center point, the display plane position is calibrated from an image of eyes gazing at a right upper corner and a left lower corner or from an images of eyes gazing at a right lower corner and a left upper corner using the linear interpolation transform function, or the display plane position is calibrated from images of eyes gazing at four corners of the display plane using the geometric transform function and the cross ration transform function.