Wearable device and method for outputting virtual image

• US 10,175,485 B2
• Filed: 11/13/2015
• Issued: 01/08/2019
• Est. Priority Date: 11/18/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Seong-Woon, Bae, Jae-Cheol, An, Kwang-Dek, Na, In-Hak
• Primary Examiner(s): Alexander, William R
• Assistant Examiner(s): Duong, Henry
• Application Number:
  US14/941,053
• Publication Number:
  US 20160139411A1
• Time in Patent Office: 1,152 Days
• Field of Search: 359630
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/005 Constructional features of the display
• CPC Class Codes: A61B 3/103 for determining refraction, e.g. refractometers, skiascopes
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 3/14 Arrangements specially adapted for eye photography apparatus or arrangements for taking photographs per se G03B
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 3/0081 having one or more elements with analytic function to create variable power variable magnification in general G02B15/00
• CPC Class Codes: G02B 6/00 Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence

Abstract

A wearable device and a method of outputting a virtual image by the wearable device area provided. The wearable device includes a projection type display unit that includes a variable lens and configured to project a light forming an image, a first sensor configured to detect a light reflected from an eye of a user, and a processor configured to control to display the virtual image on the projection type display unit by controlling one of a location of the variable lens and refractive power of the variable lens based on retina image information corresponding to the detected light.

Claims

1. A method of outputting a virtual image by a wearable device, the method comprising:

radiating a light to an eye of a user;

acquiring retina image information by receiving a light which is reflected from a retina;

calculating a focal length of a crystalline lens based on the acquired retina image information; and

determining viewing distance information of the user based on the calculated focal length.

2. The method of claim 1, further comprising displaying the virtual image on an actual view by using a projection type display to place the virtual image at a distance corresponding to the viewing distance information.

3. The method of claim 1, wherein the retina image information comprises at least one piece of pixel information on an image signal processor (ISP) sensor, size information, area information, location information, pattern information, or brightness information, on a retina image.

4. The method of claim 1, wherein the viewing distance information comprises at least one piece of distance measurement information based on a sensor, object distance information calculated based on focal length information on the crystalline lens, retina reflection surface size information calculated based on the retina image information, magnification information, refractive power information, curvature information, or thickness information on the crystalline lens.

5. The method of claim 1, further comprising changing a database indicating a correspondence relationship between the retina image information and the viewing distance information based on the retina image information.

6. The method of claim 1, wherein the viewing distance information corresponding to the retina image information is determined based on a database.

7. The method of claim 1, wherein control information of a variable lens corresponding to the viewing distance information is determined based on a database.

8. The method of claim 1, further comprising:

detecting ambient illumination through an illumination sensor; and

determining a pupil size based on the ambient illumination,

wherein the focal length of the crystalline lens is calculated based on the retina image information and the pupil size.

20. At least one non-transitory computer readable storage medium for storing a computer program comprising instructions configured to be readable by at least one processor for configuring the at least one processor to execute a computer process for performing the method of claim 1.

9. A wearable device comprising:

a projection type display comprising a variable lens, and configured to project a light forming an image;

a first sensor configured to detect a light reflected from a retina of a user; and

at least one processor configured to;

acquire retina image information from the first sensor,

determine viewing distance information of the user based on focal length of a crystalline lens and the acquired retina image information, and

control the projection type display to display a virtual image by controlling one of a location of the variable lens or a refractive power of the variable lens based on the viewing distance information.

10. The wearable device of claim 9, further comprising an illumination sensor configured to detect ambient illumination,

wherein the at least one processor is further configured to:

determine a pupil size based on the ambient illumination, and

calculate the focal length of the crystalline lens based on the retina image information and the pupil size.

11. The wearable device of claim 9, wherein the at least one processor is further configured to:

acquire viewing distance information related to a viewing point of the retina based on the retina image information, and

display the virtual image on an actual view through the projection type display to place the virtual image at a distance corresponding to the viewing distance information.

12. The wearable device of claim 11,

wherein the retina image information comprises at least one of pixel information on an image signal processor (ISP) sensor, size information, area information, location information, pattern information, or brightness information on a retina image, and

wherein the viewing distance information comprises at least one piece of distance measurement information based on a sensor, object distance information calculated based on focal length information on a crystalline lens, retina reflection surface size information calculated based on the retina image information, magnification information, refractive power information, curvature information, or thickness information on the crystalline lens.

13. The wearable device of claim 11, wherein the at least one processor is further configured to change a database pre-stored in the wearable device, which indicates a correspondence relationship between the retina image information and the viewing distance information, based on the retina image information with respect to the detected light.

14. The wearable device of claim 11, wherein the at least one processor is further configured to determine the viewing distance information corresponding to the retina image information based on a database pre-stored in the wearable device.

15. The wearable device of claim 11, wherein the at least one processor is further configured to determine control information of the variable lens corresponding to the viewing distance information based on a database pre-stored in the wearable device.

16. The wearable device of claim 9, wherein the projection type display further comprises:

a display element configured to output a first light forming the virtual image, the variable lens being disposed on a progress path of the first light, and configured to adjust a location or refractive power according to a control signal;

a light source configured to detected an illumination light;

a light receiver configured to detect a reflected light; and

a light guide element configured to;

project the first light output from the display element onto the retina,

project the illumination light output from the light source onto the retina, and

output the reflected light to the light receiver.

17. The wearable device of claim 16,

wherein the projection type display further comprises a lens configured to form an image of the reflected light, and

wherein the light receiver is configured to detect the image of the reflected light.

18. The wearable device of claim 17, wherein the projection type display further comprises a first beam splitter configured to:

output the illumination light to the light guide element, and

output the reflected light received from the light guide element to the lens.

19. The wearable device of claim 16,

wherein the light guide element comprises:

a third beam splitter; and

a second beam splitter configured to;

output the first light output from the display element and the illumination light output from the light source to the third beam splitter, and

output the reflected light input from the third beam splitter to the light receiver, and

wherein the third beam splitter is configured to;

project the first light and the illumination light received from the second beam splitter on the retina,

project the second light forming an actual view onto the retina, and

output the reflected light to the second beam splitter.