Head-mounted display device, control method therefor, and computer program

• US 10,310,502 B2
• Filed: 01/05/2018
• Issued: 06/04/2019
• Est. Priority Date: 01/10/2017
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Kobayashi, Shinichi
• Primary Examiner(s): Wilson, Nicholas R
• Application Number:
  US15/863,445
• Publication Number:
  US 20180196425A1
• Time in Patent Office: 515 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04B 7/18506 Communications with or from aircraft, i.e. aeronautical mobile service

Abstract

A transmission-type head-mounted display device includes an image display, a control section configured to control the image display section, a position detecting section, a visual-field detecting section configured to detect a visual field of the user, and a wireless communication section configured to perform wireless communication with an external apparatus. The control section includes a state-information acquiring section configured to acquire, via the wireless communication section, state information including a position of an own aircraft, which is a mobile body set as an operation target object of the user, and a support-information creating section configured to create, on the basis of a relation between the position and the visual field of the user and the position of the own aircraft, a support image including support information for operation of the own aircraft and cause the image display section to display the support image as the virtual image.

Claims

1. A transmission-type head-mounted display device comprising:

an image display configured to, in a state in which the head-mounted display device is mounted on a head of a user, transmit an outside scene to cause the user to visually recognize the outside scene and superimpose a virtual image on the outside scene to cause the user to visually recognize the virtual image;

a wireless communication transceiver configured to perform wireless communication with an own aircraft, which is a mobile body set as an operation target object of the user; and

a processor,

wherein the processor is configured to;

control the image display;

detect a position of the user wearing the image display;

detect a visual field of the user;

acquire, via the wireless communication transceiver, state information including a position of the own aircraft;

calculate a positional relation of the position of the own aircraft with respect to the visual field of the user;

determine whether the own aircraft is absent or present in the outside scene corresponding to the visual field of the user, based on the positional relation; and

when the own aircraft is present in the outside scene corresponding to the visual field of the user, create a first support image including support information for operation of the own aircraft and cause the image display to display the first support image as the virtual image.

2. The head-mounted display device according to claim 1, wherein the processor acquires, via the wireless communication transceiver, the state information transmitted from the own aircraft to a remote control device used for controlling the own aircraft.

3. The head-mounted display device according to claim 2, wherein the processor further acquires, via the wireless communication transceiver, state information of other aircrafts, which are one or more mobile bodies other than the own aircraft, transmitted from the other aircrafts to each of one or more remote control devices used for controlling each of the other aircrafts.

4. The head-mounted display device according to claim 3, wherein the processor further causes the image display to display, to the own aircraft included in the visual field of the user, as the virtual image, moving direction information indicating a moving direction included in the state information of the own aircraft and causes the image display to display, to the other aircrafts included in the visual field of the user, as the virtual image, moving direction information indicating a moving direction included in the state information of the other aircrafts.

5. The head-mounted display device according to claim 3, wherein, when another aircraft satisfying a first approach condition is present, the processor further causes, on a basis of the position of the own aircraft and a predicted position after elapse of predetermined time included in the state information of the own aircraft and a position of the other aircraft and a predicted position after the elapse of the predetermined time included in state information of the other aircraft satisfying the first approach condition, the image display to display, as the virtual image, information indicating an approach state of the other aircraft when the own aircraft and the other aircraft satisfy a second approach condition.

6. The head-mounted display device according to claim 2, wherein the processor further causes the image display to display, to the own aircraft included in the visual field of the user, as the virtual image, moving direction information indicating a moving direction included in the state information of the own aircraft.

7. The head-mounted display device according to claim 1, wherein, when the own aircraft is absent in the outside scene corresponding to the visual field of the user, the processor causes the image display to display, as the virtual image, a second support image indicating a direction in which the own aircraft is located.

8. The head-mounted display device according to claim 1, wherein the first support image is a marker image for distinguishing the own aircraft.

9. The head-mounted display device according to claim 1, wherein the processor changes a state of the support image according to a state of the outside scene to make it easy to distinguish the support image from the outside scene.

10. A control method for a transmission-type head-mounted display device including an image display configured to, in a state in which the head-mounted display device is mounted on a head of a user, transmit an outside scene to cause the user to visually recognize the outside scene and superimpose a virtual image on the outside scene to cause the user to visually recognize the virtual image, the control method comprising:

acquiring state information including a position of an own aircraft, which is a mobile body set as an operation target object of the user;

calculating a positional relation of the position of the own aircraft with respect to a visual field of the user;

determining whether the own aircraft is absent or present in the outside scene corresponding to the visual field of the user, based on the positional relation; and

when the own aircraft is present in the outside scene corresponding to the visual field of the user, creating a first support image including support information for operation of the own aircraft and causing the image display to display the first support image as the virtual image.

11. A non-transitory computer readable medium with instructions stored thereon for controlling a transmission-type head-mounted display device including an image display configured to, in a state in which the head-mounted display device is mounted on a head of a user, transmit an outside scene to cause the user to visually recognize the outside scene and superimpose a virtual image on the outside scene to cause the user to visually recognize the virtual image, that when executed by computer, perform the steps comprising:

acquiring state information including a position of an own aircraft, which is a mobile body set as an operation target object of the user;

calculating a positional relation of the position of the own aircraft with respect to a visual field of the user;

determining whether the own aircraft is absent or present in the outside scene corresponding to the visual field of the user, based on the positional relation; and

when the own aircraft is present in the outside scene corresponding to the visual field of the user, creating a first support image including support information for operation of the own aircraft and causing the image display to display the first support image as the virtual image.