Wearable device and method for outputting virtual image

• US 10,802,283 B2
• Filed: 01/02/2019
• Issued: 10/13/2020
• Est. Priority Date: 11/18/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Seong-Woon, Bae, Jae-Cheol, An, Kwang-Dek, Na, In-Hak
• Primary Examiner(s): Alexander, William R
• Assistant Examiner(s): Duong, Henry A
• Application Number:
  US16/238,058
• Publication Number:
  US 20190137764A1
• Time in Patent Office: 650 Days
• Field of Search: 359630
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/005 Constructional features of the display
• CPC Class Codes: A61B 3/103 for determining refraction, e.g. refractometers, skiascopes
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 3/14 Arrangements specially adapted for eye photography apparatus or arrangements for taking photographs per se G03B
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 3/0081 having one or more elements with analytic function to create variable power variable magnification in general G02B15/00
• CPC Class Codes: G02B 6/00 Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence

Abstract

A wearable device and a method of outputting a virtual image by the wearable device area provided. The wearable device includes a projection type display unit that includes a variable lens and configured to project a light forming an image, a first sensor configured to detect a light reflected from an eye of a user, and a processor configured to control to display the virtual image on the projection type display unit by controlling one of a location of the variable lens and refractive power of the variable lens based on retina image information corresponding to the detected light.

Claims

1. A method of outputting a virtual image by a wearable device, the method comprising:

radiating a first light to an eye of a user wearing the wearable device;

acquiring eye information related to a viewing point of the eye by detecting a second light which is reflected from the eye based on the first light;

acquiring tilt information on a tilt of the wearable device by using a sensor of the wearable device;

identifying a viewing distance of the user based on the eye information and the tilt information; and

displaying the virtual image on an actual view by using a projection type display based on the viewing distance of the user.

2. The method of claim 1, wherein the displaying of the virtual image on the actual view comprises displaying the virtual image at a point of the actual view corresponding to the viewing point of the eye.

3. The method of claim 1, further comprising changing a table comprising the eye information related to the viewing point of the eye.

4. The method of claim 1, further comprising:

radiating a third light to the eye of the user wearing the wearable device;

changing the eye information related to the viewing point of the eye by detecting a fourth light which is reflected from the eye; and

displaying another virtual image at another point of the actual view different from a point at which the virtual image is displayed by using the projection type display based on the changed eye information.

5. The method of claim 4, further comprising adjusting a location or refractive power of a variable lens within the projection type display.

6. The method of claim 1, wherein the eye information comprises information regarding to one piece of pixel on an image signal processor (ISP) sensor, size information of the eye, area information of the eye, location information of the eye, pattern information of the eye, or brightness information of the eye.

7. The method of claim 1,

wherein the viewing distance is identified based on focal length information on a crystalline lens of the eye, retina reflection surface size information on the crystalline lens, magnification information on the crystalline lens, refractive power information on the crystalline lens, curvature information on the crystalline lens, or thickness information on the crystalline lens.

8. The method of claim 1, further comprising:

detecting ambient illumination through an illumination sensor;

identifying a pupil size based on the ambient illumination, and

identifying a focal length of a crystalline lens of the eye based on the pupil size and the eye information.

19. At least one non-transitory computer readable storage medium for storing a computer program comprising instructions configured to be readable by at least one processor for configuring the at least one processor to execute a computer process for performing the method of claim 1.

9. A wearable device comprising:

a projection type display comprising a variable lens, and configured to project a light forming an image;

a first sensor configured to detect a light reflected from an eye of a user wearing the wearable device based on the projected light;

a second sensor configured to detect a tilt of the wearable device; and

at least one processor configured to;

acquire eye information related to a viewing point of the eye based on the light detected by using the first sensor,

acquire tilt information on the tilt of the wearable device by using the second sensor,

identify a viewing distance of the user based on the eye information and the tilt information, and

control the projection type display to display a virtual image by controlling one of a location of the variable lens or a refractive power of the variable lens based on the viewing distance.

10. The wearable device of claim 9, wherein, to control the projection type display to display the virtual image, the at least one processor is further configured to control the projection type display to display the virtual image at a point of an actual view corresponding to the viewing point of the eye.

11. The wearable device of claim 9, wherein the at least one processor is further configured to change a table comprising the eye information related to the viewing point of the eye.

12. The wearable device of claim 9,

wherein the first sensor is further configured to detect another light reflected from the eye, and

wherein the at least one processor is further configured to:

change the eye information related to the viewing point of the eye based on the other light detected from the first sensor, and

control the projection type display to display another virtual image at another point different from a point at which the virtual image is displayed based on the changed eye information.

13. The wearable device of claim 9, wherein the eye information comprises information regarding to one piece of pixel on an image signal processor (ISP) sensor, size information of the eye, area information of the eye, location information of the eye, pattern information of the eye, or brightness information of the eye.

14. The wearable device of claim 9,

wherein the viewing distance is identified based on focal length information on a crystalline lens of the eye, retina reflection surface size information on the crystalline lens, magnification information on the crystalline lens, refractive power information on the crystalline lens, curvature information on the crystalline lens, or thickness information on the crystalline lens.

15. The wearable device of claim 9, wherein the projection type display further comprises:

a display element configured to output a first light forming the virtual image, the variable lens being disposed on a progress path of the first light, and configured to adjust a location or a refractive power according to a control signal;

a light source configured to output an illumination light;

a light receiver configured to detect the reflected light; and

a light guide element configured to;

project the first light output from the display element onto a retina,

project the illumination light output from the light source onto the retina, and

output the reflected light to the light receiver.

16. The wearable device of claim 15,

wherein the projection type display further comprises a lens configured to form an image of the reflected light, and

wherein the light receiver is configured to detect the image of the reflected light.

17. The wearable device of claim 16, wherein the projection type display further comprises a first beam splitter configured to:

output the illumination light to the light guide element, and

output the reflected light received from the light guide element to the lens.

18. The wearable device of claim 15,

wherein the light guide element comprises:

a second beam splitter; and

a third beam splitter,

wherein the second beam splitter is configured to;

output the first light output from the display element and the illumination light output from the light source to the third beam splitter, and

output the reflected light input from the third beam splitter to the light receiver, and

wherein the third beam splitter is configured to;

project the first light and the illumination light received from the second beam splitter on the retina,

project a second light forming an actual view onto the retina, and

output the reflected light to the second beam splitter.