Device and method for determining gaze direction

• US 8,494,229 B2
• Filed: 02/14/2008
• Issued: 07/23/2013
• Est. Priority Date: 02/14/2008
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Nokia Corporation
• Inventors: Levola, Tapani, Jrvenp, Toni
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): GILLIARD, DELOMIA L
• Application Number:
  US12/866,427
• Publication Number:
  US 20110019874A1
• Time in Patent Office: 1,986 Days
• Field of Search: 382/103, 382/117, 351/206, 351/221
• US Class Current: 382/117
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

An eye tracker device (200) comprises a diffractive beam expander (207) to provide two substantially collimated illuminating light beams (B11, B12). The collimated light beams (B11, B12) provide two reflection spots (G1, G2) appearing in the image of the eye. The gaze direction (GZD) is calculated from the positions of the reflection spots (G1, G2) with respect to the pupil (P) of the eye (E1). The two illuminating beams (B11, B12) are provided by splitting an infrared laser beam (B4) into two in-coupled beams (B5, B6), which propagate in different directions in the substrate (7) of the beams expander. The in-coupled beams (B5, B6) are expanded and their light is subsequently coupled out of the substrate (7) by an out-coupling grating (230) to illuminate the eye (E1). The same substrate (7) may also be used to implement a virtual display device (100) for displaying virtual images to said eye (E1).

Claims

1. An apparatus comprising:

a first imaging unit configured to acquire an image of an eye;

a substantially planar waveguiding substrate;

a light source configured to provide a first light beam, said first light beam being substantially collimated;

an in-coupling grating configured to diffract light of said light beam into said substrate and to form a first in-coupled beam and a second in-coupled beam propagating in different directions within said substrate;

a first expanding grating portion configured to provide a first expanded internal beam by diffracting light of said first in-coupled beam;

a second expanding grating portion configured to provide a second expanded internal beam by diffracting light of said second in-coupled beam;

a first out-coupling grating portion configured to form a first substantially collimated illuminating beam by diffracting light of said first internal beam out of said substrate;

a second out-coupling grating portion configured to form a second substantially collimated illuminating beam by diffracting light of said second internal beam out of said substrate, said illuminating beams having different directions such that said first illuminating beam provides a first reflection spot when light is reflected from the surface of the eye and that said second illuminating beam provides a second reflection spot when light is reflected from the surface of said eye, said reflection spots appearing in said image; and

a data processing unit configured to determine the gaze direction of the eye with respect to said apparatus based on the position of said first reflection spot in said image, on the position of said second reflection spot in said image, on the position of the pupil and/or iris of the eye in said image and on the directions of the illuminating light beams.

2. The apparatus of claim 1 wherein said first in-coupled beam substantially corresponds to the reflective or transmissive diffraction order −

1 and said second in-coupled beam substantially corresponds to the reflective or transmissive diffraction order +1.

3. The apparatus of claim 2 wherein said in-coupling grating is further configured to diffract light of said light beam in the diffraction order 2 or −

2 in order to form a third in-coupled beam propagating within said substrate.

4. The apparatus according to claim 1 wherein diffractive features of said first out-coupling grating portion are substantially parallel to diffractive features of said second out-coupling grating portion.

5. The apparatus of claim 4 wherein said first out-coupling grating portion overlaps said second out-coupling grating portion.

6. The apparatus of claim 1 further comprising a position detecting unit to determine the position of said apparatus with respect to at least one external reference.

7. The apparatus of claim 1 being adapted to associate a real object or a location with a gaze direction.

8. The apparatus of claim 1 further comprising an optical engine configured to provide at least one light beam, and a diffractive beam expander to expand said at least one light beam such that a virtual image is visually observable through a viewing aperture of said diffractive beam expander.

9. The apparatus of claim 8 wherein an output grating portion of said diffractive beam expander overlaps said first out-coupling portion.

10. The apparatus of claim 9 wherein an output grating of said diffractive beam expander is substantially perpendicular to said first out-coupling portion.

11. The apparatus of claim 8 being adapted to associate a virtual object with a gaze direction.

12. A method comprising:

acquiring an image of an eye by using a first imaging unit;

providing a substantially collimated first light beam by using a light source;

diffracting light of said first light beam by using an in-coupling grating in order to form a first in-coupled beam and a second in-coupled beam propagating in different directions within a substantially planar waveguiding substrate;

diffracting light of said first in-coupled beam by a first expanding grating portion to provide a first expanded internal beam propagating within said substrate;

diffracting light of said second in-coupled beam by a second expanding grating portion to provide a second expanded internal beam propagating within said substrate;

diffracting light of said first expanded internal beam by a first out-coupling grating portion out of said substrate to form a first substantially collimated illuminating beam;

diffracting light of said second expanded internal beam out of said substrate to form a second substantially collimated illuminating beam, said illuminating beams having different directions such that said first illuminating beam provides a first reflection spot when light is reflected from the surface of the eye and that said second illuminating beam provides a second reflection spot when light is reflected from the surface of said eye, said reflection spots appearing in said image; and

determining the gaze direction of the eye based on the position of said first reflection spot in said image, on the position of said second reflection spot in said image, on the position of the pupil and/or iris of the eye in said image and on the directions of the illuminating light beams.

13. The method of claim 12 wherein diffractive features of said first out-coupling grating portion are substantially parallel to diffractive features of said second out-coupling grating portion.

14. The method of claim 13 wherein said first out-coupling grating portion overlaps said second out-coupling grating portion.

15. The method according to claim 12 further comprising:

providing at least one light beam by an using optical engine; and

displaying a virtual image by expanding said at least one light beam by using a diffractive beam expander, said virtual image being observable through a viewing aperture of said diffractive beam expander.

16. The method of claim 15 wherein an output grating of said diffractive beam expander is substantially perpendicular to said first out-coupling portion.

17. A portable apparatus comprising:

a first imaging unit to acquire an image of an eye;

a substantially planar waveguiding substrate;

a light source configured to provide a first light beam, said first light beam being substantially collimated;

an in-coupling grating configured to diffract light of said light beam into said substrate and to form a first in-coupled beam and a second in-coupled beam propagating in different directions within said substrate;

a first expanding grating portion configured to provide a first expanded internal beam by diffracting light of said first in-coupled beam;

a second expanding grating portion configured to provide a second expanded internal beam by diffracting light of said second in-coupled beam;

a first out-coupling grating portion configured to form a first substantially collimated illuminating beam by diffracting light of said first internal beam out of said substrate;

a second out-coupling grating portion configured to form a second substantially collimated illuminating beam by diffracting light of said second internal beam out of said substrate, said illuminating beams having different directions such that said first illuminating beam provides a first reflection spot when light is reflected from the surface of the eye and that said second illuminating beam provides a second reflection spot when light is reflected from the surface of said eye, said reflection spots appearing in said image;

a data processing unit configured to determine the gaze direction of the eye with respect to said apparatus based on the position of said first reflection spot in said image, on the position of said second reflection spot in said image, on the position of the pupil and/or iris of the eye in said image and on the directions of the illuminating light beams;

an optical engine configured to provide at least one light beam; and

a diffractive beam expander configured to expand said at least one light beam such that a virtual image is visually observable through a viewing aperture of said diffractive beam expander.

18. The apparatus of claim 17 wherein an output grating of said diffractive beam expander is substantially perpendicular to said first out-coupling portion.

19. An apparatus comprising:

a first imaging means for acquiring an image of said eye;

a substantially planar waveguiding substrate;

a light source means for providing a first light beam, said first light beam being substantially collimated;

an in-coupling grating for diffracting light of said light beam into said substrate and to form a first in-coupled beam and a second in-coupled beam propagating in different directions within said substrate;

a first expanding grating portion for providing a first expanded internal beam by diffracting light of said first in-coupled beam;

a second expanding grating portion for providing a second expanded internal beam by diffracting light of said second in-coupled beam;

a first out-coupling grating portion for forming a first substantially collimated illuminating beam by diffracting light of said first internal beam out of said substrate;

a second out-coupling grating portion for forming a second substantially collimated illuminating beam by diffracting light of said second internal beam out of said substrate, said illuminating beams having different directions such that said first illuminating beam provides a first reflection spot when light is reflected from the surface of the eye and that said second illuminating beam provides a second reflection spot when light is reflected from the surface of said eye, said reflection spots appearing in said image; and

a data processing means for determining the gaze direction of the eye with respect to said gaze direction determining means based on the position of said first reflection spot in said image, on the position of said second reflection spot in said image, on the position of the pupil and/or iris of the eye in said image, and on the directions of the illuminating light beams.

20. The apparatus of claim 19 further comprising an image forming means for providing at least one light beam, and a diffractive expanding means to expand said at least one light beam such that a virtual image is visually observable through a viewing aperture of said diffractive expanding means.