Eye Tracking With Mobile Device In A Head-Mounted Display

• US 20160180591A1
• Filed: 05/13/2015
• Published: 06/23/2016
• Est. Priority Date: 12/23/2014
• Status: Active Grant

• Current Assignee: Communication Advances LLC (IP Edge LLC)
• Original Assignee: MediaTek, Inc.
• Inventors: Chang, Kai-Mau, Shiu, Da-Shan, Liu, Tsu-Ming
• Granted Patent: US 9,791,924 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04M 1/05 specially adapted for use on head, throat or breast

Abstract

A head-mount display (HMD) and variations thereof are described. An HMD may include a mobile device having a display unit, a camera, a light source and a processing unit that controls operations of the display unit, camera and light source. The processing unit receives data associated with one or more optical images captured by the camera, and renders a visual image displayable by the display unit. The HMD may also include an eyewear piece having a holder, one or more lenses and a reflective unit. The holder is wearable by a user and holds the mobile device in front of eyes of the user. The user can view the display unit through the one or more lenses. The reflective unit reflects an image of at least an eye of the user. The camera is oriented to capture the reflected image of the eye through the reflective unit.

Claims

1. A head-mounted display, comprising:

an eyewear piece comprising;

a holder that is wearable by a user and configured to hold the mobile device in front of eyes of the user;

one or more lenses through which the display unit of the mobile device is viewable by the user; and

a reflective unit configured to reflect an image of at least a first eye of the eyes of the user.

2. The head-mounted display of claim 1, wherein the holder comprises a pair of goggles configured to seal off a space between the eyewear piece and a face of the user to prevent an ambient light from entering the space.

3. The head-mounted display of claim 1, wherein the reflective unit is configured to allow an infrared light to pass through and reflect a visible light or alternatively configured to allow the visible light to pass through and reflect the infrared light.

4. The head-mounted display of claim 1, wherein the reflective unit comprises one or more mirrors.

5. The head-mounted display of claim 1, wherein the reflective unit comprises a first mirror and a second mirror, the first mirror configured to reflect a first image of the first eye to the camera, the second mirror configured to reflect a second image of the first eye to the camera.

6. The head-mounted display of claim 1, further comprising:

a mobile device comprising;

a display unit;

a camera configured to capture one or more optical images, the camera oriented to capture the image of the first eye reflected by the reflective unit;

a light source configured to emit a light; and

a processing unit configured to control operations of the display unit, the camera and the light source, the processing unit also configured to receive data associated with the captured one or more optical images from the camera, the processing unit further configured to render a visual image displayable by the display unit.

7. The head-mounted display of claim 6, wherein the light emitted by the light source comprises a visible light or an infrared light.

8. The head-mounted display of claim 6, wherein the mobile device further comprises a communication unit configured to at least transmit data to a remote computing device, and wherein the processing unit is further configured to provide data associated with the reflected image of the first eye to the remote computing device through the communication unit.

9. The head-mounted display of claim 6, wherein the mobile device further comprises a communication unit configured to at least transmit data to a remote computing device, and wherein the processing unit is further configured to perform operations comprising:

analyzing the reflected image of the first eye to determine a gaze direction of the first eye; and

providing data associated with the gaze direction to the remote computing device through the communication unit.

10. The head-mounted display of claim 6, wherein the processing unit is further configured to perform operations comprising:

analyzing the reflected image of the first eye to determine a gaze direction of the first eye; and

based on the gaze direction, rendering a revised visual image displayable by the display unit.

11. The head-mounted display of claim 6, wherein the processing unit is further configured to perform operations comprising:

analyzing the reflected image of the first eye to determine a gaze direction of the first eye; and

based on the gaze direction, performing foveated rendering.

12. The head-mounted display of claim 6, wherein the processing unit is further configured to perform operations comprising:

analyzing the reflected image of the first eye to determine a behavior of the first eye which indicates a command for execution; and

executing the command.

13. The head-mounted display of claim 12, wherein, in executing the command, the processing unit is configured to execute the command in a context of virtual reality (VR), and wherein the visual image comprises a VR image.

14. The head-mounted display of claim 12, wherein the behavior of the first eye comprises fixation, saccades and blinking.

15. The head-mounted display of claim 6, wherein the mobile device comprises a smartphone, a tablet computer, a phablet, or a portable computing device.

16. The head-mounted display of claim 6, further comprising:

a wide angle lens disposed in front of the camera such that an angle of a field of view of the camera through the wide angle lens is at least enough to cover an observation target of interest.

17. The head-mounted display of claim 6, further comprising:

a light guide coupled to the light source, the light guide configured to provide a point light in front of the at least one of the eyes of the user to produce a Purkinje image.

18. The head-mounted display of claim 17, wherein the camera is configured to sense the Purkinje image, and wherein the processing unit is configured to determine a location of a pupil of the first eye by tracking a movement of the Purkinje image.

19. The head-mounted display of claim 6, further comprising:

an optical element disposed between the camera and the one or more lenses and between the camera and the reflective unit, the optical element configured to reflect an infrared light while allowing a visible light to pass through.

20. The head-mounted display of claim 19, wherein the optical element comprises a hot mirror.

21. The head-mounted display of claim 19, wherein the camera is configured to sense visible and near infrared reflections.