Systems using eye mounted displays

• US 8,786,675 B2
• Filed: 01/26/2009
• Issued: 07/22/2014
• Est. Priority Date: 01/23/2008
• Status: Active Grant

• Current Assignee: Tectus Corporation
• Original Assignee: Michael F. Deering
• Inventors: Deering, Michael F.
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US12/359,951
• Publication Number:
  US 20090189974A1
• Time in Patent Office: 2,003 Days
• Field of Search: 348/46, 348/51, 348/E13.074, 348/8, 348/36, 348/E13.075, 345/8
• US Class Current: 348/46
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0147 comprising a device modifying the resolution of the displayed image
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G02C 7/04 Contact lenses for the eyes disinfection or sterilisation of contact lenses A61L12/00
• CPC Class Codes: G09G 3/02 by tracing or scanning a light beam on a screen
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

A display device is mounted on and/or inside the eye. The eye mounted display contains multiple sub-displays, each of which projects light to different retinal positions within a portion of the retina corresponding to the sub-display. The projected light propagates through the pupil but does not fill the entire pupil. In this way, multiple sub-displays can project their light onto the relevant portion of the retina. Moving from the pupil to the cornea, the projection of the pupil onto the cornea will be referred to as the corneal aperture. The projected light propagates through less than the full corneal aperture. The sub-displays use spatial multiplexing at the corneal surface. Various electronic devices interface to the eye mounted display.

Claims

1. An eye mounted display (EMD)-aware device comprising:

an electronic device; and

an interface between the electronic device and the eye mounted display (EMD), for transmitting image data from the electronic device to the EMD, wherein the EMD is attachable to a user'"'"'s eye, and the EMD projects light onto a user'"'"'s retina to form a visual sensation of an image, the EMD comprising a plurality of sub-displays attached to the user'"'"'s eye, each sub-display projecting light to different retinal positions within a portion of the retina corresponding to the sub-display, each such projection of light propagating through a partial corneal aperture for that retinal position, and sub-displays that project light to portions of the retina closer to the fovea project light through partial corneal apertures that are larger than the partial corneal apertures through which sub-displays project light to portions of the retina farther away from the fovea.

2. The EMD-aware device of claim 1 wherein the EMD has a variable resolution and the electronic device generates image data according to the variable resolution.

3. The EMD-aware device of claim 1 wherein the EMD has an effective horizontal field of view of at least 100 degrees and the electronic device generates image data according to the EMD'"'"'s field of view.

4. The EMD-aware device of claim 1 wherein the EMD supports a stereo display and the electronic device generates image data according to the stereo display.

5. The EMD-aware device of claim 1 wherein the EMD is coupled to a head tracker and the electronic device generates image data according to the head tracker.

6. The EMD-aware device of claim 1 wherein the EMD is partly transparent and displays an augmented reality to the user, and the electronic device generates image data according to the augmented reality.

7. The EMD-aware device of claim 1 wherein the EMD is opaque and displays a virtual reality to the user, and the electronic device generates image data according to the virtual reality.

8. The EMD-aware device of claim 1 wherein the EMD is coupled to an eye tracker and the electronic device generates image data according to the eye tracker.

9. The EMD-aware device of claim 1 wherein the EMD is coupled to an object tracker and the electronic device generates image data according to the object tracker.

10. The EMD-aware device of claim 1 wherein the image data includes pseudo cone pixel data stream.

11. The EMD-aware device of claim 1 wherein the electronic device is a still picture camera or a motion picture camera.

12. The EMD-aware device of claim 11 wherein the camera is a stereo camera.

13. The EMD-aware device of claim 1 wherein the electronic device is a cell phone or a PDA.

14. The EMD-aware device of claim 1 wherein the electronic device is a heads up display.

15. The EMD-aware device of claim 1 wherein the electronic device is a video kiosk or digital signage.

16. The EMD-aware device of claim 1 wherein the electronic device is a personal computer, including desktop, laptop and palm-top computers.

17. The EMD-aware device of claim 1 wherein the electronic device is a wearable computer.

18. The EMD-aware device of claim 1 wherein the electronic device is a TV.

19. The EMD-aware device of claim 1 wherein the EMD produces a motion picture display.

20. The EMD-aware device of claim 1 wherein the EMD produces a 3D display.

21. The EMD-aware device of claim 1 wherein the EMD produces a large screen format display.

22. The EMD-aware device of claim 1 wherein the EMD produces a sports display.

23. The EMD-aware device of claim 1 wherein the EMD produces a virtual reality display.

24. The EMD-aware device of claim 1 wherein the electronic device is a game platform.

25. The EMD-aware device of claim 1 wherein the electronic device is part of a simulation system.

26. The EMD-aware device of claim 1 wherein the electronic device controls a physical system.

27. The EMD-aware device of claim 1 wherein the electronic device is part of a command, control and communications center.

28. The EMD-aware device of claim 1 wherein the EMD produces an industrial design display.

29. The EMD-aware device of claim 1 wherein the EMD produces a display within an office environment.

30. The EMD-aware device of claim 1 wherein the EMD produces a telepresence display.

31. The EMD-aware device of claim 1 wherein the EMD is part of an eye mounted display system, the eye mounted display system further comprising:

an eye tracker that tracks an orientation of the eye; and

a scaler coupled to the EMD and to the eye tracker, the scaler receiving video input via the interface to the electronic device and converting the video input, based in part on the orientation of the eye received from the eye tracker, to a format suitable for projection by the EMD.

32. The EMD-aware device of claim 31 wherein the eye mounted display system further comprises:

a headpiece worn by the user, on which is mounted a first portion of a head tracker, a first portion of the eye tracker, and a data link component communicatively coupling the scaler to the EMD, the data link component receiving the converted video input from the scaler and wirelessly transmitting the converted video input to the EMD;

a second portion of the head tracker positioned in a frame of reference, the first and second portions of the head tracker cooperating to track the user'"'"'s head, the scaler converting the video input based in part on tracking of the user'"'"'s head; and

the EMD containing a second portion of the eye tracker, the first and second portions of the eye tracker cooperating to track the orientation of the eye.

86. The EMD-aware device of claim 1 wherein the EMD further comprises:

a sclera contact lens mountable on the eye; and

a display capsule having an anterior shell and a posterior shell and an interior, the display capsule mounted in the sclera contact lens so that the anterior shell of the display capsule is flush to an anterior surface of the sclera contact lens, the plurality of sub-displays comprising a plurality of femto projectors located in the interior of the display capsule, the femto projectors projecting light through partial corneal apertures that are substantially non-overlapping.

33. An eye mounted display (EMD)-aware device comprising:

an electronic device; and

an interface between the electronic device and the eye mounted display (EMD), for transmitting image data from the electronic device to the EMD, wherein the EMD is attachable to a user'"'"'s eye, and the EMD projects light onto a user'"'"'s retina to form a visual sensation of an image, the EMD comprising;

a plurality of sub-displays attached to the user'"'"'s eye, each sub-display projecting light to different retinal positions within a portion of the retina corresponding to the sub-display, each such projection of light propagating through a partial corneal aperture for that retinal position;

a sclera contact lens mountable on the eye; and

a display capsule having an anterior shell and a posterior shell and an interior, the display capsule mounted in the sclera contact lens so that the anterior shell of the display capsule is flush to an anterior surface of the sclera contact lens, the plurality of sub-displays comprising a plurality of femto projectors located in the interior of the display capsule, the femto projectors projecting light through partial corneal apertures that are substantially non-overlapping.

34. The EMD-aware device of claim 33 wherein the EMD has a variable resolution and the electronic device generates image data according to the variable resolution.

35. The EMD-aware device of claim 33 wherein the EMD has an effective horizontal field of view of at least 100 degrees and the electronic device generates image data according to the EMD'"'"'s field of view.

36. The EMD-aware device of claim 33 wherein the EMD supports a stereo display and the electronic device generates image data according to the stereo display.

37. The EMD-aware device of claim 33 wherein the EMD is coupled to a head tracker and the electronic device generates image data according to the head tracker.

38. The EMD-aware device of claim 33 wherein the EMD is partly transparent and displays an augmented reality to the user, and the electronic device generates image data according to the augmented reality.

39. The EMD-aware device of claim 33 wherein the EMD is opaque and displays a virtual reality to the user, and the electronic device generates image data according to the virtual reality.

40. The EMD-aware device of claim 33 wherein the EMD is coupled to an eye tracker and the electronic device generates image data according to the eye tracker.

41. The EMD-aware device of claim 33 wherein the EMD is coupled to an object tracker and the electronic device generates image data according to the object tracker.

42. The EMD-aware device of claim 33 wherein the image data includes pseudo cone pixel data stream.

43. The EMD-aware device of claim 33 wherein the electronic device is a still picture camera or a motion picture camera.

44. The EMD-aware device of claim 43 wherein the camera is a stereo camera.

45. The EMD-aware device of claim 33 wherein the electronic device is a cell phone or a PDA.

46. The EMD-aware device of claim 33 wherein the electronic device is a heads up display.

47. The EMD-aware device of claim 33 wherein the electronic device is a video kiosk or digital signage.

48. The EMD-aware device of claim 33 wherein the electronic device is a personal computer, including desktop, laptop and palm-top computers.

49. The EMD-aware device of claim 33 wherein the electronic device is a wearable computer.

50. The EMD-aware device of claim 33 wherein the electronic device is a TV.

51. The EMD-aware device of claim 33 wherein sub-displays that project light to portions of the retina closer to the fovea project light through partial corneal apertures that are larger than the partial corneal apertures through which sub-displays project light to portions of the retina farther away from the fovea.

52. The EMD-aware device of claim 33 wherein the EMD is part of an eye mounted display system, the eye mounted display system further comprising:

an eye tracker that tracks an orientation of the eye; and

a scaler coupled to the EMD and to the eye tracker, the scaler receiving video input via the interface to the electronic device and converting the video input, based in part on the orientation of the eye received from the eye tracker, to a format suitable for projection by the EMD.

53. The EMD-aware device of claim 52 wherein the eye mounted display system further comprises:

a headpiece worn by the user, on which is mounted a first portion of a head tracker, a first portion of the eye tracker, and a data link component communicatively coupling the scaler to the EMD, the data link component receiving the converted video input from the scaler and wirelessly transmitting the converted video input to the EMD;

a second portion of the head tracker positioned in a frame of reference, the first and second portions of the head tracker cooperating to track the user'"'"'s head, the scaler converting the video input based in part on tracking of the user'"'"'s head; and

the EMD containing a second portion of the eye tracker, the first and second portions of the eye tracker cooperating to track the orientation of the eye.

54. An eye mounted display (EMD)-aware device comprising:

an electronic device; and

an interface between the electronic device and the eye mounted display (EMD), for transmitting image data from the electronic device to the EMD, wherein the EMD is attachable to a user'"'"'s eye and is part of an eye mounted display system, the eye mounted display system further comprising;

an eye tracker that tracks an orientation of the eye,

a scaler coupled to the EMD and to the eye tracker, the scaler receiving video input via the interface to the electronic device and converting the video input, based in part on the orientation of the eye received from the eye tracker, to a format suitable for projection by the EMD;

a headpiece worn by the user, on which is mounted a first portion of a head tracker, a first portion of the eye tracker, and a data link component communicatively coupling the scaler to the EMD, the data link component receiving the converted video input from the scaler and wirelessly transmitting the converted video input to the EMD;

a second portion of the head tracker positioned in a frame of reference, the first and second portions of the head tracker cooperating to track the user'"'"'s head, the scaler converting the video input based in part on tracking of the user'"'"'s head; and

the EMD containing a second portion of the eye tracker, the first and second portions of the eye tracker cooperating to track the orientation of the eye.

55. The EMD-aware device of claim 54 wherein the EMD has a variable resolution and the electronic device generates image data according to the variable resolution.

56. The EMD-aware device of claim 54 wherein the EMD has an effective horizontal field of view of at least 100 degrees and the electronic device generates image data according to the EMD'"'"'s field of view.

57. The EMD-aware device of claim 54 wherein the EMD supports a stereo display and the electronic device generates image data according to the stereo display.

58. The EMD-aware device of claim 54 wherein the EMD is partly transparent and displays an augmented reality to the user, and the electronic device generates image data according to the augmented reality.

59. The EMD-aware device of claim 54 wherein the EMD is opaque and displays a virtual reality to the user, and the electronic device generates image data according to the virtual reality.

60. The EMD-aware device of claim 54 wherein the electronic device generates image data according to the eye tracker.

61. The EMD-aware device of claim 54 wherein the EMD is coupled to an object tracker and the electronic device generates image data according to the object tracker.

62. The EMD-aware device of claim 54 wherein the image data includes pseudo cone pixel data stream.

63. The EMD-aware device of claim 54 wherein the electronic device is a still picture camera or a motion picture camera.

64. The EMD-aware device of claim 63 wherein the camera is a stereo camera.

65. The EMD-aware device of claim 54 wherein the electronic device is a cell phone or a PDA.

66. The EMD-aware device of claim 54 wherein the electronic device is a heads up display.

67. The EMD-aware device of claim 54 wherein the electronic device is a video kiosk or digital signage.

68. The EMD-aware device of claim 54 wherein the electronic device is a personal computer, including desktop, laptop and palm-top computers.

69. The EMD-aware device of claim 54 wherein the electronic device is a wearable computer.

70. The EMD-aware device of claim 54 wherein the electronic device is a TV.

71. The EMD-aware device of claim 54 wherein the EMD produces a motion picture display.

72. The EMD-aware device of claim 54 wherein the EMD produces a 3D display.

73. The EMD-aware device of claim 54 wherein the EMD produces a large screen format display.

74. The EMD-aware device of claim 54 wherein the EMD produces a sports display.

75. The EMD-aware device of claim 54 wherein the EMD produces a virtual reality display.

76. The EMD-aware device of claim 54 wherein the electronic device is a game platform.

77. The EMD-aware device of claim 54 wherein the electronic device is part of a simulation system.

78. The EMD-aware device of claim 54 wherein the electronic device controls a physical system.

79. The EMD-aware device of claim 54 wherein the electronic device is part of a command, control and communications center.

80. The EMD-aware device of claim 54 wherein the EMD produces an industrial design display.

81. The EMD-aware device of claim 54 wherein the EMD produces a display within an office environment.

82. The EMD-aware device of claim 54 wherein the EMD produces a telepresence display.

83. The EMD-aware device of claim 54 wherein the EMD projects light onto a user'"'"'s retina to form a visual sensation of an image, the EMD comprising a plurality of sub-displays attached to the user'"'"'s eye, each sub-display projecting light to different retinal positions within a portion of the retina corresponding to the sub-display, each such projection of light propagating through a partial corneal aperture for that retinal position.

84. The EMD-aware device of claim 83 wherein the EMD further comprises:

a sclera contact lens mountable on the eye; and

a display capsule having an anterior shell and a posterior shell and an interior, the display capsule mounted in the sclera contact lens so that the anterior shell of the display capsule is flush to an anterior surface of the sclera contact lens, the plurality of sub-displays comprising a plurality of femto projectors located in the interior of the display capsule, the femto projectors projecting light through partial corneal apertures that are substantially non-overlapping.

85. The EMD-aware device of claim 83 wherein sub-displays that project light to portions of the retina closer to the fovea project light through partial corneal apertures that are larger than the partial corneal apertures through which sub-displays project light to portions of the retina farther away from the fovea.