Display instrument and image display method

• US 9,558,540 B2
• Filed: 05/02/2013
• Issued: 01/31/2017
• Est. Priority Date: 05/09/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Mukawa, Hiroshi
• Primary Examiner(s): AMIN, JWALANT B
• Application Number:
  US13/875,593
• Publication Number:
  US 20130300766A1
• Time in Patent Office: 1,370 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A head-mounted display device including an image display apparatus configured to display a captured image of a portion of an environment viewable through the head-mounted display device; and a dimmer configured to, while the captured image is displayed, allow a portion of ambient light from the environment to pass through the dimmer. Also, a method of displaying information on a head-mounted display device. The method may include displaying a captured image of a portion of an environment viewable through the display device; and dimming ambient light received through the head-mounted display device from the environment while displaying the captured image.

Claims

1. A head-mounted display device comprising:

an image sensor configured to capture an image of a portion of an environment viewable through the head-mounted display device;

a display configured to display an image captured by the image sensor of the portion of the environment viewable through the head-mounted display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

a light shutter configured to, while the captured image is displayed, block ambient light from the environment from passing through any portion of the light shutter when the head-mounted display device is configured in a first mode of operation and further configured to allow a portion of ambient light from the environment to pass through the light shutter when the head-mounted display is configured in a second mode of operation;

a control circuit configured to selectively place the head-mounted display in either of the first mode of operation or the second mode of operation; and

at least one storage device configured to store display data corresponding to stored information that identifies feature points of various objects;

wherein the control circuit is further configured to process the image captured by the image sensor to extract feature points therefrom and to compare the extracted feature points with the information stored in the at least one storage device to determine whether the extracted feature points conform to the feature points identified by the stored information;

wherein the display is further configured to, for at least one of the first or second modes of operation of the head-mounted display device, display the display data together with the captured image when the control circuit determines that the extracted feature points conform to the feature points identified by the stored information.

2. The head-mounted display device of claim 1, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

3. The head-mounted display device of claim 1, wherein:

the display is configured to display the display data when the head-mounted display device is configured in the second mode of operation.

4. The head-mounted display device of claim 3, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the head-mounted display device.

5. The head-mounted display device of claim 3, wherein the display data comprises at least one image corresponding at least one subject of the environment viewable through the head-mounted display device.

6. The head-mounted display device of claim 3, further comprising a user interface configured to allow a user of the head-mounted display device to input an indication to cause the controller to switch the head-mounted display from the first mode of operation to the second mode of operation.

7. The head-mounted display device of claim 6, wherein the user interface comprises a microphone configured to receive a voice command from the user to switch from the first mode of operation to the second mode of operation.

8. The head-mounted display device of claim 6, wherein the user interface comprises an infrared receiver circuit configured to receive an eye-blinking command from the user to switch from the first mode of operation to the second of operation.

9. The head-mounted display of claim 3, wherein the display is configured so as not to display the display data when the head-mounted display is configured in the first mode of operation.

10. The head-mounted display of claim 9, wherein the display is configured so as not to display the captured image data when the head-mounted display is configured in the second mode of operation.

11. The head-mounted display of claim 3, wherein the display is configured so as not to display the captured image data when the head-mounted display is configured in the second mode of operation.

12. The head-mounted display of claim 3, wherein the display is configured to display the display data when the head-mounted display is configured in the first mode of operation.

13. The head-mounted display of claim 3, wherein the display is configured to display the captured image data when the head-mounted display is configured in the second mode of operation.

14. The head-mounted display of claim 13, wherein the display is configured to display the display data when the head-mounted display is configured in the first mode of operation.

15. The head-mounted display of claim 13, wherein the display is configured so as not to display the display data when the head-mounted display is configured in the first mode of operation.

16. The head-mounted display of claim 1, wherein the display is configured to display the display data when the head-mounted display is configured in the first mode of operation.

17. A method of displaying information on a head-mounted display device, the method comprising:

with an image sensor of the head-mounted display device, capturing an image of a portion of an environment viewable through the display device;

displaying on a display an image captured by the image sensor of the portion of the environment viewable through the display device, the display comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band; and

when the head-mounted display is configured in a first mode of operation, controlling a light shutter so as to block ambient light from the environment from passing through any portion the shutter while displaying the captured image;

when the head-mounted display is configured in a second mode of operation, controlling the light shutter so as to allow a portion of ambient light from the environment to pass through the light shutter;

processing the captured image to extract feature points therefrom;

comparing the extracted feature points with stored information that identifies feature points of various objects to determine whether the extracted feature points conform to the feature points identified by the stored information;

when the extracted feature points are determined to conform to the feature points identified by the stored information, identifying stored display data that corresponds to the stored information; and

at least one of (a) displaying the identified display data together with the captured image when the head-mounted display device is configured in the first mode of operation, or (b) displaying the identified display data together with the captured image when the head-mounted display device is configured in the second mode of operation.

18. The method of claim 17, wherein the display data is selected from the group consisting of a character, a word, a sentence, a figure, an illustration, a still image and a moving image.

19. The method of claim 17, further comprising:

in response to receiving an indication to switch from the first mode of operation to the second mode of operation;

switching from displaying the captured image to displaying the display data; and

switching from controlling the light shutter so as to block the ambient light from the environment from passing through any portion of the light shutter to controlling the light shutter so as to allow a portion of the ambient light from the environment to pass through the shutter.

20. The method of claim 19, wherein the display data comprises at least one word describing at least one subject of the environment viewable through the display device.

21. The method of claim 19, wherein the display data comprises at least one image corresponding to at least one subject of the environment viewable through the display device.

22. The method of claim 19, wherein the indication to switch from the first mode of operation to the second mode of operation is received from a user of the head-mounted display device.

23. The method of claim 22, wherein the indication from the user is a voice command or an eye-blinking command.

24. A head-mounted display device comprising:

image capturing means for capturing an image of a portion of an environment viewable through the display device;

display means for displaying an image captured by the image capturing means of the portion of the environment viewable through the display device, the display means comprising at least one diffraction grating member comprising a plurality of laminated diffraction grating layers each causing diffractive reflection of light beams of a different wavelength band;

dimming means for blocking ambient light received from the environment from passing through any portion of the dimming means while displaying the captured image when the head-mounted display is configured in a first mode of operation, and for allowing a portion of ambient light from the environment to pass through the dimming means when the head-mounted display is configured in a second mode of operation;

at least one storage device configured to store display data corresponding to stored information that identifies feature points of various objects;

a control circuit configured to selectively place the head-mounted display in either of the first mode of operation or the second mode of operation, wherein the control circuit is further configured to process the image captured by the image sensor to extract feature points therefrom and to compare the extracted feature points with the information stored in the at least one storage device to determine whether the extracted feature points conform to the feature points identified by the stored information; and

means for at least one of (a) displaying display data stored in at least one storage device together with the captured image when the head-mounted display device is configured in the first mode of operation and the control circuit determines that the extracted feature points conform to the feature points identified by the stored information, or (b) displaying display data stored in at least one storage device together with the captured image when the head-mounted display device is configured in the second mode of operation and the control circuit determines that the extracted feature points conform to the feature points identified by the stored information.