HEAD-MOUNTED DISPLAY DEVICE, CONTROL METHOD FOR HEAD-MOUNTED DISPLAY DEVICE, AND WORK SUPPORTING SYSTEM

• US 20140168266A1
• Filed: 12/03/2013
• Published: 06/19/2014
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Kimura, Fusashi
• Granted Patent: US 9,448,407 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A head-mounted display device for enabling a user to simultaneously visually recognize a virtual image and an outside scene includes a warning-information generating unit configured to generate warning information, which is an image for calling the user'"'"'s attention and an image display unit configured to cause the user to visually recognize the warning information as the virtual image.

Claims

1. A head-mounted display device for enabling a user to simultaneously visually recognize a virtual image and an outside scene, the head-mounted display device comprising:

an image acquiring unit configured to acquire an image in a visual field direction of the user in a wearing state of the head-mounted display device;

a warning-information generating unit configured to generate warning information, which is an image for calling the user'"'"'s attention, on the basis of the image acquired by the image acquiring unit; and

an image display unit configured to cause the user to visually recognize the warning information as the virtual image.

2. The head-mounted display device according to claim 1, wherein the warning-information generating unit specifies, from the image acquired by the image acquiring unit, a place where the user needs to be careful and generates warning information in which a predetermined image is arranged near the specified place.

3. The head-mounted display device according to claim 2, wherein

the image acquiring unit detects infrared light radiated from an object and acquires a temperature distribution image representing a distribution of temperature, and

the warning-information generating unit specifies a place where the temperature is equal to or higher than a first threshold in the temperature distribution image acquired by the image acquiring unit and sets the place as the place where the user needs to be careful.

4. The head-mounted display device according to claim 2, wherein

the image acquiring unit detects infrared light radiated from an object and acquires a temperature distribution image representing a distribution of temperature, and

the warning-information generating unit specifies a place where the temperature is equal to or lower than a second threshold in the temperature distribution image acquired by the image acquiring unit and sets the place as the place where the user needs to be careful.

5. The head-mounted display device according to claim 2, wherein

the image acquiring unit acquires an image representing the shape of an object from visible light radiated from the object, and

the warning-information generating unit recognizes the image acquired by the image acquiring unit, specifies a part coinciding with a pattern stored in advance, and sets the coinciding part as the place where the user needs to be careful.

6. The head-mounted display device according to claim 5, wherein the pattern includes an acute angle part.

7. The head-mounted display device according to claim 2, wherein the predetermined image is an image formed by a character, a pattern, or a figure that reminds the user of a warning or a combination of the character, the pattern, and the figure.

8. The head-mounted display device according to claim 2, wherein the predetermined image is an image formed by a figure having a shape corresponding to a shape of the specified place.

9. The head-mounted display device according to claim 3, wherein the predetermined image is an image obtained by extracting, from the temperature distribution image acquired by the image acquiring unit, a place where temperature is equal to or higher than the first threshold or a place where temperature is equal to or lower than the second threshold.

10. The head-mounted display device according to claim 1, wherein the warning-information generating unit generates the warning information for an overlapping area where a display area of the virtual image and an acquisition area of the image acquiring unit overlap each other in the image acquired by the image acquiring unit.

11. The head-mounted display device according to claim 1, further comprising a sounding body, wherein

the warning-information generating unit causes, on the basis of the image acquired by the image acquiring unit, the sounding body to output sound for calling the user'"'"'s attention.

12. The head-mounted display device according to claim 1, further comprising a tactile stimulation unit configured to apply stimulation through a tactile sense to the user, wherein

the warning-information generating unit actuates, on the basis of the image acquired by the image acquiring unit, the tactile stimulation unit in order to call the user'"'"'s attention.

29. A work supporting system, comprising:

the head-mounted display device according to claim 5; and

a server, wherein

the server includes;

a first table configured to store association of the information representing the procedure included in the work and the information representing content that the user should carry out in the procedure;

a second table configured to store association of the information representing the procedure included in the work and the information concerning at least one of an accident in the past and a near miss in the past in the procedure; and

a transmitting unit configured to transmit,

according to a request from the head-mounted display device, at least one of the information stored in the first table and the information stored in the second table.

13. A control method for a head-mounted display device for enabling a user to simultaneously visually recognize a virtual image and an outside scene, the control method comprising:

acquiring an image in a visual field direction of the user in a wearing state of the head-mounted display device;

generating warning information, which is an image for calling the user'"'"'s attention, on the basis of the acquired image; and

causing the user to visually recognize the warning information as the virtual image.

14. A head-mounted display device for enabling a user to simultaneously visually recognize a virtual image and an outside scene, the head-mounted display device comprising:

a procedure managing unit configured to generate procedure information, which is an image representing content that the user should carry out, concerning a present procedure;

a warning-information generating unit configured to generate warning information, which is an image for calling the user'"'"'s attention, in the present procedure; and

an image display unit configured to cause the user to visually recognize the procedure information and the warning information as the virtual image.

15. The head-mounted display device according to claim 14, wherein the warning-information generating unit generates the warning information when warning point data stored in advance is present in the present procedure.

16. The head-mounted display device according to claim 15, further comprising an image acquiring unit configured to acquire an image in a visual field direction of the user in a wearing state of the head-mounted display device, wherein

the procedure managing unit specifies, on the basis of the image acquired by the image acquiring unit, the present procedure from a plurality of procedures included in work.

17. The head-mounted display device according to claim 16, wherein the procedure managing unit generates the procedure information corresponding to the present procedure using association of information representing the procedure included in the work and information representing content that the user should carry out in the procedure.

18. The head-mounted display device according to claim 16, wherein

the warning point data stored in advance is data associating information representing the procedure included in the work and information concerning at least one of an accident in the past and a near miss in the past in the procedure, and

the warning-information generating unit generates, using the warning point data, the warning information corresponding to the present procedure.

19. The head-mounted display device according to claim 14, wherein the procedure managing unit limits generation of the procedure information corresponding to a next procedure until the procedure managing unit succeeds in predetermined check processing after generating the procedure information corresponding to the present procedure.

20. The head-mounted display device according to claim 19, wherein the predetermined check processing is acquisition of an indication of intension for check from the user.

21. The head-mounted display device according to claim 16, wherein the predetermined check processing is processing for recognizing the image acquired by the image acquiring unit and checking whether a result of the image recognition satisfies a condition set in advance.

22. The head-mounted display device according to claim 21, wherein the condition set in advance varies according to characteristics of the user including at least any one of age, sex, physique, and proficiency in work of the user.

23. The head-mounted display device according to claim 14, wherein the warning-information generating unit generates different kinds of the warning information when an accident occurred in the past in the present procedure and when a near miss occurred in the past in the present procedure.

24. The head-mounted display device according to claim 14, wherein the warning-information generating unit generates different kinds of the warning information according to characteristics of the user including at least any one of age, sex, physique, and proficiency in work of the user.

25. The head-mounted display device according to claim 16, wherein

the procedure managing unit generates the procedure information for an overlapping area where a display area of the virtual image and an acquisition area of the image acquiring unit overlap each other in the image acquired by the image acquiring unit, and

the warning-information generating unit generates the warning information for the overlapping area.

26. The head-mounted display device according to claim 18, further comprising a sounding body, wherein

the warning-information generating unit causes, on the basis of the information concerning at least one of an accident in the past and a near miss in the past corresponding to the present procedure, the sounding body to output sound for calling the user'"'"'s attention.

27. The head-mounted display device according to claim 18, further comprising a tactile stimulation unit configured to apply stimulation through a tactile sense to the user, wherein

the warning-information generating unit actuates, on the basis of the information concerning at least one of an accident in the past and a near miss in the past corresponding to the present procedure, the tactile stimulation unit in order to call the user'"'"'s attention.

28. A control method for a head-mounted display device for enabling a user to simultaneously visually recognize a virtual image and an outside scene, the control method comprising:

generating procedure information, which is an image representing content that the user should carry out, concerning a present procedure;

generating warning information, which is an image for calling the user'"'"'s attention, in the present procedure; and

causing the user to visually recognize the procedure information and the warning information as the virtual image.