Eye mounted displays and systems, with eye tracker and head tracker

• US 9,858,901 B2
• Filed: 09/30/2016
• Issued: 01/02/2018
• Est. Priority Date: 01/23/2008
• Status: Active Grant

• Current Assignee: Tectus Corporation
• Original Assignee: Spy Eye, LLC
• Inventors: Deering, Michael Frank, Huang, Alan
• Primary Examiner(s): Bukowski, Kenneth
• Application Number:
  US15/281,654
• Publication Number:
  US 20170019661A1
• Time in Patent Office: 459 Days
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 7/04 Contact lenses for the eyes disinfection or sterilisation of contact lenses A61L12/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/0435 Change or adaptation of the frame rate of the video stream
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/02 by tracing or scanning a light beam on a screen
• CPC Class Codes: G09G 5/003 : Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: G09G 5/391 : Resolution modifying circuits, e.g. variable screen formats
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 : for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/0117 : involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/0127 : by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 9/31 : Projection devices for colour picture display , e.g. using electronic spatial light modulators [ESLM]
• CPC Class Codes: H04N 9/3188 : Scale or resolution adjustment scaling in general G06T3/40; resolution modifying circuits for displays in general G09G5/391
• CPC Class Codes: H04N 9/3191 : Testing thereof testing of displays in general G09G3/006

Abstract

A display device is mounted on and/or inside the eye. The eye mounted display contains multiple sub-displays, each of which projects light to different retinal positions within a portion of the retina corresponding to the sub-display. The projected light propagates through the pupil but does not fill the entire pupil. In this way, multiple sub-displays can project their light onto the relevant portion of the retina. Moving from the pupil to the cornea, the projection of the pupil onto the cornea will be referred to as the corneal aperture. The projected light propagates through less than the full corneal aperture. The sub-displays use spatial multiplexing at the corneal surface. Various electronic devices interface to the eye mounted display.

Claims

1. An eye mounted display system, comprising:

an eye mounted display mountable on an eye of a human user for forming a visual sensation of an image that spans a field of view of the user'"'"'s eye, the eye mounted display comprising;

a contact lens; and

a plurality of sub-displays in the contact lens, the eye mounted display moving with the user'"'"'s eye as the user'"'"'s eye rotates in its socket, each of the sub-displays comprising;

a plurality of display pixels displaying a different portion of the image for each sub-display, and

display optics projecting light from the display pixels to a portion of the retina corresponding to the portion of the image displayed by that sub-display, wherein the portion of the retina is fixed as the user'"'"'s eye rotates in its socket and thereby forming a visual sensation of the portion of the image displayed by that sub-display, each such projection of light propagating through a partial corneal aperture for that sub-display, different sub-displays projecting light to different portions of the retina, and the sub-displays in aggregate projecting light to portions of the retina that in aggregate span the field of view;

an eye tracker that detects an orientation of the eye; and

a device that generates images from an image input, the images generated based on the orientation of the eye and generated in an output format suitable for use in the eye mounted display, the images in the output format including pixels of different resolutions within each image, and the eye mounted display projecting the images onto a retina of the eye.

2. The eye mounted display system of claim 1 wherein the images projected onto the retina of the eye appear to be stationary with respect to the user'"'"'s surrounding environment.

3. The eye mounted display system of claim 2 wherein:

the image input comprises images in an input format with pixels of constant resolution within each image; and

the device converts the images from the input format to the output format, the output format including pixels of different resolutions within each image, the pixels of different resolutions including higher resolution pixels and lower resolution pixels, wherein which pixels in the input format are converted to higher resolution pixels or to lower resolution pixels in the output format depends on the orientation of the eye.

4. The eye mounted display system of claim 1 wherein the images in the output format include pseudo cone pixels, and the device generates the pseudo cone pixels from the image input.

5. The eye mounted display system of claim 1 wherein the eye tracker detects the orientation of the eye by tracking the eye directly.

6. The eye mounted display system of claim 1 wherein the eye tracker detects the orientation of the eye by tracking the eye mounted display.

7. The eye mounted display system of claim 1 wherein the device determines at least one of the following eye motion types based on the detected orientations of the eye over time:

saccades, drifts, micro saccades, vergence motion and tremor motion.

8. The eye mounted display system of claim 1 wherein the device further simulates retinal motion blur based on the detected orientations of the eye over time;

and the device generates images from the image input taking into account the simulated retinal motion blur.

9. The eye mounted display system of claim 1 wherein the device further detects eye blinks and/or saccades, and no images are projected onto the retina of the eye during detected eye blinks and saccades.

10. The eye mounted display system of claim 1 wherein the device further predicts a future orientation of the eye based on past detected orientations of the eye, and the device generates images from the image input based on the future predicted orientation of the eye.

11. The eye mounted display system of claim 1 wherein the eye tracker comprises a camera mounted outside the eye, the camera capturing images of the eye, the orientation of the eye determined based on the images captured by the camera.

12. The eye mounted display system of claim 1 wherein the eye tracker comprises a camera mounted on the eye, the camera capturing images from the eye looking outward, the orientation of the eye determined based on the images captured by the camera.

13. The eye mounted display system of claim 1 further comprising:

a head tracker that detects a position and/or orientation of a head of the human user, wherein the images are generated further based on the position and/or orientation of the head.

14. The eye mounted display system of claim 13 wherein the head tracker detects the position and/or orientation of the head by tracking the head directly.

15. The eye mounted display system of claim 13 wherein the head tracker detects the position and/or orientation of the head by tracking a component mounted to the head.

16. The eye mounted display system of claim 13 wherein the eye tracker and the head tracker together detect the orientation of the eye in at least two degrees of freedom and the position of the eye in at least three degrees of freedom.

17. The eye mounted display system of claim 13 wherein the head tracker detects the position of the head with an accuracy of 100 microns or better.

18. The eye mounted display system of claim 13 wherein the eye tracker and head tracker together determine a view frustum for the eye, and the device generates images from the image input based on the view frustum.

19. The eye mounted display system of claim 13 wherein the device further predicts a future position and/or orientation of the head based on past detected positions and/or orientations of the head;

and the device generates images from the image input based on the future predicted positions and/or orientations of the head.

20. The eye mounted display system of claim 1 wherein the eye tracker comprises a camera mounted outside the eye, the camera capturing images of a tracker component mounted on the eye, the orientation of the eye determined based on the images captured by the camera.