HEAD-MOUNT TYPE DISPLAY DEVICE, CONTROL SYSTEM, METHOD OF CONTROLLING HEAD-MOUNT TYPE DISPLAY DEVICE, AND COMPUTER PROGRAM

• US 20150317518A1
• Filed: 04/09/2015
• Published: 11/05/2015
• Est. Priority Date: 05/01/2014
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: FUJIMAKI, Yutaka, TAKANO, Masahide
• Granted Patent: US 9,766,715 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G05B 2219/23148 Helmet display, mounted on head of operator
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/30 User interface
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device

Abstract

A transmissive head-mount type display device includes an image display section adapted to display a virtual image, and capable of transmitting an external sight, an object acquisition section adapted to obtain a selectable object located in a predetermined distance range from the image display section, and a position of a specific object included in the external sight, and a control section adapted to display an object-correspondence virtual image associated with the object obtained as the virtual image using the image display section, identify a change in the position of the specific object based on the position of the specific object obtained, select the object based on a relationship between the change in the position of the specific object identified and a position of the object obtained, and display a specific check image associated with the object selected as the virtual image using the image display section.

Claims

1. A transmissive head-mount type display device comprising:

an image display section adapted to display a virtual image, and capable of transmitting an external sight;

an object acquisition section adapted to obtain a selectable object located in a predetermined distance range from the image display section, and a position of a specific object included in the external sight; and

a control section adapted to display an object-correspondence virtual image associated with the object obtained as the virtual image using the image display section, identify a change in the position of the specific object based on the position of the specific object obtained, select the object based on a relationship between the change in the position of the specific object identified and a position of the object obtained, and display a specific check image associated with the object selected as the virtual image using the image display section.

2. The head-mount type display device according to claim 1, wherein

the object acquisition section includes an imaging section adapted to image the external sight, and an image detection section adapted to detect and obtain the position of the specific object and the object included in the external sight imaged, and

the control section determines control of the object selected, and then executes the control of the object determined.

3. The head-mount type display device according to claim 2, wherein

the control section displays the object-correspondence virtual image associated in advance with a combination of the position of the specific object obtained and the object obtained as the virtual image using the image display section.

4. The head-mount type display device according to claim 3, wherein

the control section displays the virtual image representing the change in the position of the specific object necessary for executing the control of the object as the object-correspondence virtual image associated in advance with the combination of the position of the specific object obtained and the object obtained using the image display section.

5. The head-mount type display device according to claim 2, wherein

the control section displays the virtual image representing a content of the control of the object to be executed as the object-correspondence virtual image associated in advance with the object obtained using the image display section.

6. The head-mount type display device according to claim 2, wherein

in a case in which the position of the specific object having been changed and the object overlap each other in the external sight imaged, the control section determines the control of the object associated in advance with the object overlapping the position of the specific object having been changed.

7. The head-mount type display device according to claim 2, further comprising:

a distance identification section adapted to identify a distance between the object obtained and the image display section,

wherein the control section sets the object, which has been obtained, and has the distance identified equal to or shorter than a threshold value, as the selectable object.

8. The head-mount type display device according to claim 2, further comprising:

a sound acquisition section adapted to obtain an external sound,

wherein the control section determines the control of the object based on a combination of the change in the position of the specific object identified and the sound obtained.

9. The head-mount type display device according to claim 1, wherein

the object acquisition section obtains position information of at least one the objects from another device.

10. The head-mount type display device according to claim 9, wherein

in a case in which the object obtained fails to be included in the external sight, the control section displays a positional relationship between the image display section and the object obtained as the virtual image using the image display section.

11. The head-mount type display device according to claim 1, wherein

the control section detects an acceleration of the specific object, and identifies the change in the position of the specific object based on the acceleration of the specific object detected.

12. The head-mount type display device according to claim 11, further comprising:

a sound acquisition section adapted to obtain an external sound,

wherein the control section determines the control of the object selected, based on the external sound obtained and a combination of the change in the position of the specific object identified.

13. The head-mount type display device according to claim 1, wherein

the control section executes control of a control device associated with the object selected, and in a case in which the object selected and the control device are obtained in the external sight, the control section displays, as the virtual image using the image display section, that the object selected and the control device are in a correspondence relationship.

14. The head-mount type display device according to claim 13, wherein

the control section displays virtual images having identical shapes and identical colors using the image display section as the virtual image representing that the object selected and the control device are in the correspondence relationship.

15. The head-mount type display device according to claim 1, further comprising:

an identifying section adapted to identify an attribute of a user of the image display section,

wherein the control section displays at least one of a virtual image associated with the object and the specific check image as the virtual image using the image display section so as to correspond to the attribute identified.

16. A control system comprising:

the head-mount type display device according to claim 2;

an object identification section adapted to identify the object;

a first communication section adapted to transmit information of the object identified; and

a control execution section,

wherein the head-mount type display device includes a second communication section adapted to receive the information of the object transmitted, and then transmit information of the control of the object determined,

the first communication section receives the information of the object transmitted, and

the control execution section executes control based on the control of the object received by the first communication section.

17. A control system comprising:

the head-mount type display device according to claim 3;

an object identification section adapted to identify the object;

a first communication section adapted to transmit information of the object identified; and

a control execution section,

wherein the head-mount type display device includes a second communication section adapted to receive the information of the object transmitted, and then transmit information of the control of the object determined,

the first communication section receives the information of the object transmitted, and

the control execution section executes control based on the control of the object received by the first communication section.

18. A control system comprising:

the head-mount type display device according to claim 4;

an object identification section adapted to identify the object;

a first communication section adapted to transmit information of the object identified; and

a control execution section,

wherein the head-mount type display device includes a second communication section adapted to receive the information of the object transmitted, and then transmit information of the control of the object determined,

the first communication section receives the information of the object transmitted, and

the control execution section executes control based on the control of the object received by the first communication section.

19. A method of controlling a transmissive head-mount type display device including an image display section adapted to display a virtual image and capable of transmitting an external sight, the method comprising:

obtaining a selectable object located in a predetermined distance range from the image display section, and a position of a specific object included in the external sight; and

displaying an object-correspondence virtual image associated with the object obtained as the virtual image using the image display section, identifying a change in the position of the specific object based on the position of the specific object obtained, selecting the object based on a relationship between the change in the position of the specific object identified and a position of the object obtained, and displaying a specific check image associated with the object selected as the virtual image using the image display section.

20. A computer program used with a transmissive head-mount type display device including an image display section adapted to display a virtual image and capable of transmitting an external sight, the computer program making a computer realize functions comprising:

obtaining a selectable object located in a predetermined distance range from the image display section, and a position of a specific object included in the external sight; and

displaying an object-correspondence virtual image associated with the object obtained as the virtual image using the image display section, identifying a change in the position of the specific object based on the position of the specific object obtained, selecting the object based on a relationship between the change in the position of the specific object identified and a position of the object obtained, and displaying a specific check image associated with the object selected as the virtual image using the image display section.