Display instrument and image display method

• US 9,972,135 B2
• Filed: 11/16/2016
• Issued: 05/15/2018
• Est. Priority Date: 05/09/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Mukawa, Hiroshi
• Primary Examiner(s): AMIN, JWALANT B
• Application Number:
  US15/353,110
• Publication Number:
  US 20170069140A1
• Time in Patent Office: 545 Days
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A head-mounted display device including an image display apparatus configured to display a captured image of a portion of an environment viewable through the head-mounted display device; and a dimmer configured to, while the captured image is displayed, allow a portion of ambient light from the environment to pass through the dimmer. Also, a method of displaying information on a head-mounted display device. The method may include displaying a captured image of a portion of an environment viewable through the display device; and dimming ambient light received through the head-mounted display device from the environment while displaying the captured image.

Claims

1. A head-mounted display device comprising:

an image sensor configured to capture an image of a portion of an environment viewable through the head-mounted display device;

a display configured to display an image captured by the image sensor of the portion of the environment viewable through the head-mounted display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

a light shutter configured to allow an amount of ambient light from the environment to pass through the light shutter when the head-mounted display is configured in a first mode of operation, and to decrease the amount of ambient light from the environment allowed to pass through the light shutter when the head-mounted display device is configured in a second mode of operation; and

a control circuit configured to;

selectively place the head-mounted display in either of the first mode of operation or the second mode of operation,

process the image captured by the image sensor to extract feature points therefrom and to compare the extracted feature points with information identifying feature points of various objects to determine whether the extracted feature points conform to the feature points identified by the information, and

in response to determining that the extracted feature points conform to the feature points identified by the information, cause the light shutter to decrease the amount of ambient light that is allowed to pass the light shutter and cause the display to present display data together with the captured image.

2. The head-mounted display device of claim 1, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

3. The head-mounted display device of claim 1, wherein:

the display is configured so as not to display the display data when the head-mounted display device is configured in the first mode of operation.

4. The head-mounted display device of claim 3, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the head-mounted display device.

5. The head-mounted display device of claim 3, wherein the display data comprises at least one image corresponding at least one subject of the environment viewable through the head-mounted display device.

6. The head-mounted display device of claim 3, further comprising a user interface configured to allow a user of the head-mounted display device to input an indication to cause the controller to switch the head-mounted display from the second mode of operation to the first mode of operation.

7. The head-mounted display device of claim 6, wherein the user interface comprises a microphone configured to receive a voice command from the user to switch from the second mode of operation to the first mode of operation.

8. The head-mounted display device of claim 6, wherein the user interface comprises an infrared receiver circuit configured to receive an eye-blinking command from the user to switch from the second mode of operation to the first of operation.

9. The head-mounted display of claim 3, wherein the display is configured so as not to display the captured image data when the head-mounted display is configured in the first mode of operation.

10. The head-mounted display of claim 3, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

11. The head-mounted display of claim 3, wherein the display is configured to display the captured image data when the head-mounted display is configured in the first mode of operation.

12. The head-mounted display of claim 11, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

13. The head-mounted display of claim 11, wherein the display is configured so as not to display the display data when the head-mounted display is configured in the second mode of operation.

14. The head-mounted display of claim 1, wherein the display is configured to display the display data when the head-mounted display is configured in the second mode of operation.

15. A method of displaying information on a head-mounted display device, the method comprising:

with an image sensor of the head-mounted display device, capturing an image of a portion of an environment viewable through the display device;

displaying on a display an image captured by the image sensor of the portion of the environment viewable through the display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

controlling a light shutter so as to allow an amount of ambient light from the environment to pass through the light shutter in a first mode of operation and to decrease the amount of ambient light from the environment that is allowed to pass through the light shutter in a second mode of operation;

processing the captured image to extract feature points therefrom;

comparing the extracted feature points with information that identifies feature points of various objects to determine whether the extracted feature points conform to the feature points identified by the information; and

in response to determining that the extracted feature points conform to the feature points identified by the information, controlling the light shutter to decrease the amount of ambient light that is allowed to pass the light shutter and displaying, on the display, display data corresponding to the information.

16. The method of claim 15, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

17. The method of claim 15, further comprising:

in response to receiving an indication to switch from the second mode of operation to the first mode of operation;

switching from displaying the captured image to displaying the display data; and

switching from controlling the light shutter to decrease the amount of ambient light from the environment that is allowed to pass through the light shutter to controlling the light shutter to allow the amount of ambient light from the environment to pass through the shutter.

18. The method of claim 17, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the display device.

19. The method of claim 17, wherein the display data comprises at least one image corresponding to at least one subject of the environment viewable through the display device.

20. The method of claim 17, wherein the indication to switch from the second mode of operation to the first mode of operation is received from a user of the head-mounted display device.

21. The method of claim 20, wherein the indication from the user is a voice command or an eye-blinking command.