DISPLAY INSTRUMENT AND IMAGE DISPLAY METHOD

• US 20170069140A1
• Filed: 11/16/2016
• Published: 03/09/2017
• Est. Priority Date: 05/09/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Mukawa, Hiroshi
• Granted Patent: US 9,972,135 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A head-mounted display device including an image display apparatus configured to display a captured image of a portion of an environment viewable through the head-mounted display device; and a dimmer configured to, while the captured image is displayed, allow a portion of ambient light from the environment to pass through the dimmer. Also, a method of displaying information on a head-mounted display device. The method may include displaying a captured image of a portion of an environment viewable through the display device; and dimming ambient light received through the head-mounted display device from the environment while displaying the captured image.

Claims

1. A head-mounted display device comprising:

an image sensor configured to capture an image of a portion of an environment viewable through the head-mounted display device;

a display configured to display an image captured by the image sensor of the portion of the environment viewable through the head-mounted display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

a light shutter configured to allow an amount of ambient light from the environment to pass through the light shutter when the head-mounted display is configured in a first mode of operation, and further configured to decrease the amount of ambient light from the environment that is allowed to pass through the light shutter when the head-mounted display device is configured in a second mode of operation; and

a control circuit configured to selectively place the head-mounted display in either of the first mode of operation or the second mode of operation, and further configured to process the image captured by the image sensor to extract feature points therefrom and to compare the extracted feature points with information identifying feature points of various objects to determine whether the extracted feature points conform to the feature points identified by the information;

wherein the display is further configured to, for at least one of the first or second modes of operation of the head-mounted display device, display the display data together with the captured image when the control circuit determines that the extracted feature points conform to the feature points identified by the information.

2. The head-mounted display device of claim 1, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

3. The head-mounted display device of claim 1, wherein:

the display is configured to display the display data when the head-mounted display device is configured in the first mode of operation.

4. The head-mounted display device of claim 3, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the head-mounted display device.

5. The head-mounted display device of claim 3, wherein the display data comprises at least one image corresponding at least one subject of the environment viewable through the head-mounted display device.

6. The head-mounted display device of claim 3, further comprising a user interface configured to allow a user of the head-mounted display device to input an indication to cause the controller to switch the head-mounted display from the second mode of operation to the first mode of operation.

7. The head-mounted display device of claim 6, wherein the user interface comprises a microphone configured to receive a voice command from the user to switch from the second mode of operation to the first mode of operation.

8. The head-mounted display device of claim 6, wherein the user interface comprises an infrared receiver circuit configured to receive an eye-blinking command from the user to switch from the second mode of operation to the first of operation.

9. The head-mounted display of claim 3, wherein the display is configured so as not to display the display data when the head-mounted display is configured in the second mode of operation.

10. The head-mounted display of claim 9, wherein the display is configured so as not to display the captured image data when the head-mounted display is configured in the first mode of operation.

11. The head-mounted display of claim 3, wherein the display is configured so as not to display the captured image data when the head-mounted display is configured in the first mode of operation.

12. The head-mounted display of claim 3, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

13. The head-mounted display of claim 1, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

14. The head-mounted display of claim 3, wherein the display is configured to display the captured image data when the head-mounted display is configured in the first mode of operation.

15. The head-mounted display of claim 14, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

16. The head-mounted display of claim 14, wherein the display is configured so as not to display the display data when the head-mounted display is configured in the second mode of operation.

17. A method of displaying information on a head-mounted display device, the method comprising:

with an image sensor of the head-mounted display device, capturing an image of a portion of an environment viewable through the display device;

displaying on a display an image captured by the image sensor of the portion of the environment viewable through the display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

when the head-mounted display is configured in a first mode of operation, controlling a light shutter so as to allow an amount of ambient light from the environment to pass through the light shutter;

when the head-mounted display is configured in a second mode of operation, controlling the light shutter so as to decrease the amount ambient light from the environment that is allowed to pass through the light shutter;

processing the captured image to extract feature points therefrom;

comparing the extracted feature points with information that identifies feature points of various objects to determine whether the extracted feature points conform to the feature points identified by the information;

when the extracted feature points are determined to conform to the feature points identified by the information, identifying display data that corresponds to the information; and

at least one of (a) displaying the identified display data together with the captured image when the head-mounted display device is configured in the first mode of operation, or (b) displaying the identified display data together with the captured image when the head-mounted display device is configured in the second mode of operation.

18. The method of claim 17, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

19. The method of claim 17, further comprising:

in response to receiving an indication to switch from the second mode of operation to the first mode of operation;

switching from displaying the captured image to displaying the display data; and

switching from controlling the light shutter to decrease the amount of ambient light from the environment that is allowed to pass through the light shutter to controlling the light shutter to allow the amount of ambient light from the environment to pass through the shutter.

20. The method of claim 19, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the display device.

21. The method of claim 19, wherein the display data comprises at least one image corresponding to at least one subject of the environment viewable through the display device.

22. The method of claim 19, wherein the indication to switch from the second mode of operation to the first mode of operation is received from a user of the head-mounted display device.

23. The method of claim 22, wherein the indication from the user is a voice command or an eye-blinking command.