HEAD MOUNTED DISPLAY AND CONTROL METHOD FOR HEAD MOUNTED DISPLAY

• US 20150363979A1
• Filed: 02/04/2014
• Published: 12/17/2015
• Est. Priority Date: 02/14/2013
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Takano, Masahide, TSUDA, Atsunari
• Granted Patent: US 9,916,691 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 6/461 Displaying means of special interest
• CPC Class Codes: A61B 6/462 characterised by constructional features of the display
• CPC Class Codes: A61B 6/5229 combining image data of a patient, e.g. combining a functional image with an anatomical image
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 5/00 Image enhancement or restoration

Abstract

A head mounted display which allows a user to visually recognize a virtual image and external scenery, and includes an image display unit that forms the virtual image which is visually recognized by the user, and a superimposition processing unit that causes the image display unit to form the virtual image based on superimposition information for superimposing invisible information which is not shown in an appearance of an object on the object included in the external scenery.

Claims

1. A head mounted display which allows a user to visually recognize a virtual image and external scenery, comprising:

an image display unit that forms the virtual image which is visually recognized by the user; and

a superimposition processing unit that causes the image display unit to form the virtual image based on superimposition information for superimposing invisible information which is not shown in an appearance of an object on the object included in the external scenery.

2. The head mounted display according to claim 1,

wherein the superimposition processing unit causes the image display unit to repeatedly form the virtual image based on the superimposition information corresponding to the object included in the external scenery every predetermined time.

3. The head mounted display according to claim 1, further comprising:

a storage unit that stores the invisible information; and

an image acquisition unit that acquires an image of external scenery in a visual field direction of the user when the user wears the head mounted display,

wherein the superimposition processing unit extracts a feature of the object included in the external scenery through image recognition on the image in the visual field direction acquired by the image acquisition unit, and corrects the invisible information of the object stored in the storage unit on the basis of the extracted feature so as to generate the superimposition information.

4. The head mounted display according to claim 3,

wherein the invisible information is an image indicating a structure of the object.

5. The head mounted display according to claim 3,

wherein the feature is an edge of the object.

6. The head mounted display according to claim 3,

wherein the feature is a marker attached to the object.

7. The head mounted display according to claim 3,

wherein the correction is performed by performing at least one of enlargement, reduction, rotation, inversion, trimming, distortion, and noise removal on an image indicating the invisible information.

8. The head mounted display according to claim 3,

wherein the correction is performed by dividing an image indicating the invisible information into a plurality of regions and performing at least one of enlargement, reduction, trimming, distortion, and noise removal on each of the plurality of regions.

9. The head mounted display according to claim 8,

wherein the superimposition processing unit measures a distance between the user and each part of the object by using triangulation, and performs the division on the basis of a result of the measurement.

10. The head mounted display according to claim 3,

wherein the superimposition processing unit changes accuracy of the correction on the basis of an information accepting characteristic indicating an extent of visual capability within a visual field of the user.

11. The head mounted display according to claim 3, further comprising:

a visual line detection unit that detects a visual line of the user,

wherein the image display unit forms both the virtual image corresponding to the right eye of the user and the virtual image corresponding to the left eye of the user,

wherein the superimposition processing unit further obtains a steady gaze point distance which is a distance between the user and a steady gaze point at which the user steadily gazes, from the visual line detected by the visual line detection unit, and generates the superimposition information for the right eye and the superimposition information for the left eye to which a parallax corresponding to the obtained steady gaze point distance is given.

12. The head mounted display according to claim 11, further comprising:

parallax information that correlates a plurality of distances between the user and the virtual image visually recognized by the user with parallaxes which respectively correspond to the plurality of distances,

wherein the superimposition processing unit gives a parallax corresponding to the steady gaze point distance by using the parallax information.

13. The head mounted display according to claim 12,

wherein the parallax information is provided in a plurality of pieces, and

wherein the superimposition processing unit selects one parallax information piece from the plurality of parallax information pieces in accordance with the user.

14. The head mounted display according to claim 3, further comprising:

a position detection unit that detects a position of the user relative to the object,

wherein the storage unit stores a plurality of invisible information pieces regarding a single object,

wherein the superimposition processing unit selects one invisible information piece corresponding to the position detected by the position detection unit from the plurality of invisible information pieces stored in the storage unit, and corrects the selected invisible information.

15. The head mounted display according to claim 3,

wherein the superimposition processing unit further generates enlargement information in which at least a part of the object included in the external scenery is extracted and enlarged, from the image in the visual field direction acquired by the image acquisition unit, and

wherein the image display unit allows the user to visually recognize the virtual image based on the superimposition information and the enlargement information.

16. The head mounted display according to claim 1,

wherein the superimposition processing unit does not display the virtual image in response to a request from the user.

17. The head mounted display according to claim 1, further comprising:

a procedure management unit that generates procedure information which is information regarding content which is to be performed by the user in a current procedure,

wherein the image display unit allows the user to visually recognize the virtual image based on the superimposition information and the procedure information.

18. The head mounted display according to claim 17,

wherein the procedure information includes at least one of an image of a scene in which a third party exemplarily performs the procedure and an image indicating content to be performed by the user when the individual specific procedure is performed.

19. The head mounted display according to claim 1,

wherein the superimposition processing unit further generates auxiliary information which is additional information for assisting work of the user, and

wherein the image display unit allows the user to visually recognize the virtual image based on the superimposition information and the auxiliary information.

20. The head mounted display according to claim 3,

wherein the superimposition processing unit further performs a color conversion process for improving visibility of the generated superimposition information on the basis of the image in the visual field direction acquired by the image acquisition unit.

21. The head mounted display according to claim 1,

wherein the superimposition processing unit withdraws the superimposition information which is displayed as the virtual image to a position which does not hinder the visual field of the user, in response to a request from the user.

22. The head mounted display according to claim 3,

wherein the superimposition processing unit generates the superimposition information corresponding to an overlap region in which a display region of the virtual image overlaps with an acquisition region of the image acquisition unit in the image acquired by the image acquisition unit.

23. The head mounted display according to claim 16,

wherein the procedure management unit generates the procedure information corresponding to an overlap region in which a display region of the virtual image overlaps with an acquisition region of the image acquisition unit in the image acquired by the image acquisition unit.

24. A control method for a head mounted display which allows a user to visually recognize a virtual image and external scenery, the method comprising:

(a) forming the virtual image which is visually recognized by the user; and

(b) forming the virtual image based on superimposition information for superimposing invisible information which is not shown in an appearance of an object on the object included in the external scenery.