Head mounted display and image display system

• US 9,454,006 B2
• Filed: 02/21/2013
• Issued: 09/27/2016
• Est. Priority Date: 02/28/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Kobayashi, Shinichi
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): BRAY, STEPHEN A
• Application Number:
  US13/772,552
• Publication Number:
  US 20130222215A1
• Time in Patent Office: 1,314 Days
• Field of Search: 345/8
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves

Abstract

A head mounted display includes an external world information acquisition unit that acquires external world information, a visual effect unit that generates an effect image for visual effect in response to the acquired external world information, and an image display unit that superimposes and displays an outside view on the generated effect image.

Claims

1. A head mounted display comprising:

an external world information acquisition unit that acquires external world information comprising a pose of an actor in the external world;

an analysis unit that analyzes the acquired external world information comprising the pose of the actor in the external world to estimate an action of the actor in the external world;

a detection unit that detects information on a motion of a head of a user of the head mounted display;

a touch sensation stimulation unit that applies a stimulus to the head of the user of the head mounted display through touch sensation;

a visual effect unit that generates an effect image for visual effect based on the estimated action of the actor in the external world and the information on the motion of the head of the user of the head mounted display detected by the detection unit;

a touch effect unit that activates the touch sensation stimulation unit to apply the stimulus to the head of the user of the head mounted display based on the estimated action of the actor in the external world and the information on the motion of the head of the user of the head mounted display detected by the detection unit;

an image display unit that superimposes and displays the effect image for visual effect generated by the visual effect unit on a view of the external world.

2. The head mounted display according to claim 1, wherein

the external world information acquisition unit includes an imaging device, and

the external world information is an image taken by the imaging device.

3. The head mounted display according to claim 1, wherein the external world information comprising the pose of the actor in the external world is information on a motion of the actor in the external world received from an external device.

4. The head mounted display according to claim 3, wherein the information on the motion of the actor in the external world includes at least one of acceleration information, angular velocity information, and a heart rate of the actor in the external world.

5. The head mounted display according to claim 1, wherein the detection unit includes at least one of

a head acceleration information detection part provided in the image display unit for detecting acceleration information of the head of the user of the head mounted display,

a head angular velocity information detection part provided in the image display unit for detecting angular velocity information of the head of the user of the head mounted display, and

a heart rate meter provided in the image display unit for measuring a heart rate of the user of the head mounted display.

6. An image display system comprising a plurality of the head mounted displays according to claim 3, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device,

wherein the external world information acquisition unit of the head mounted display

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the another head mounted display, and

acquires, from the another head mounted display, information on motion of a user of the another head mounted display detected by the another head mounted display as the external world information.

7. An image display system comprising a plurality of the head mounted displays according to claim 4, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device,

wherein the external world information acquisition unit of the head mounted display

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the another head mounted display, and

acquires, from the another head mounted display, information on motion of a user of the another head mounted display detected by the another head mounted display as the external world information.

8. An image display system comprising a plurality of the head mounted displays according to claim 5, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device,

wherein the external world information acquisition unit of the head mounted display

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the another head mounted display, and

acquires, from the another head mounted display, information on motion of a user of the another head mounted display detected by the another head mounted display as the external world information.

9. An image display system comprising:

a plurality of the head mounted displays according to claim 3, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device; and

a server as the external device,

wherein the external world information acquisition unit of one of the head mounted displays

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the server, and

acquires, from the server, information transmitted by the another head mounted display as information on motion of a user of the another head mounted display detected by the head mounted display as the external world information.

10. An image display system comprising:

a plurality of the head mounted displays according to claim 4, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device; and

a server as the external device,

wherein the external world information acquisition unit of one of the head mounted displays

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the server, and

acquires, from the server, information transmitted by the another head mounted display as information on motion of a user of the another head mounted display detected by the head mounted display as the external world information.

11. An image display system comprising:

a plurality of the head mounted displays according to claim 5, the plurality of the head mounted displays including the head mounted display and another head mounted display that operates as the external device; and

a server as the external device,

wherein the external world information acquisition unit of one of the head mounted displays

transmits the information on the motion of the head of the user of the head mounted display detected by the detection unit of the head mounted display to the server, and

acquires, from the server, information transmitted by the another head mounted display as information on motion of a user of the another head mounted display detected by the head mounted display as the external world information.

12. The head mounted display according to claim 1, further comprising:

an effect image memory part including effect image data representative of a plurality of predetermined effect images associated with a plurality of actions,

wherein the visual effect unit generates the effect image for visual effect by;

searching the effect memory part to identify, as the effect image for visual effect, a predetermined effect image associated with the estimated action of the actor in the external world, and

calculating an effect image output location based at least in part on the information on the motion of the head of the user of the head mounted display detected by the detection unit, and

wherein the image display unit displays the effect image for visual effect generated by the visual effect unit at the calculated effect image output location.

13. The head mounted display according to claim 12, wherein:

the effect image memory part further includes touch effect data representative of a plurality of predetermined touch sensations associated with the plurality of estimated actions; and

the touch effect unit activates the touch sensation stimulation unit to apply the stimulus to the head of the user of the head mounted display by;

searching the effect image memory part to identify a predetermined touch sensation associated with the estimated action of the actor in the external world,

correcting the identified predetermined touch sensation based on the information on the motion of the head of the user of the head mounted display, and

activating the touch sensation stimulation unit to apply, as the stimulus to the head of the user of the head mounted display, the corrected predetermined touch sensation.

14. The head mounted display according to claim 12, wherein the information on the motion of the head of the user of the head mounted display comprises at least one of acceleration information and angular velocity information of the head of the user of the head mounted display.

15. The head mounted display according to claim 1, further comprising:

one or more speakers; and

a hearing effect unit that causes the one or more speakers to output an effect sound based on the estimated action of the actor in the external world and the information on the motion of the head of the user of the head mounted display detected by the detection unit.