DISPLAY SYSTEM AND METHOD FOR CONTROLLING DISPLAY SYSTEM

• US 20200133002A1
• Filed: 10/28/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: YAMAZAKI, Akio, TAKANO, Masahide
• Granted Patent: US 11,131,855 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 13/00 Controls for manipulators programme controls B25J9/16; control in general G05
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0181 Adaptation to the pilot/driver
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G05B 2219/40161 Visual display of machining, operation, remote viewing
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A display system includes an HMD and a PC. The HMD displays an image on a scene in a real space in an overlapped manner. The PC includes a first control unit. The first control unit causes the HMD to display a guide image indicating a direction set based on a robot to correspond to a robot arranged in the real space. On a tool of the robot, a tool coordinate system is set based on the tool. The first control unit displays the guide image indicating the tool coordinate system in accordance with a direction of the tool. The guide image includes an X-axis image, a Y-axis image, and a Z-axis image.

Claims

1. A display system, comprising:

a head-mounted display unit configured to display an image on a scene in a real space in an overlapped manner; and

a first control unit configured to cause the display unit to display a guide image indicating a direction set based on a control target object to correspond to the control target object disposed in the real space,

wherein on a target location of the control target object, a coordinate system based on the target location is set, and

the first control unit is configured to display the guide image indicating the coordinate system corresponding to a direction of the target location.

2. The display system according to claim 1,

wherein the control target object includes a motion mechanism configured to at least move or rotate the target location with respect to the display unit.

3. The display system according to claim 1,

wherein a plurality of the target locations and the coordinate system based on each of the target locations are set for the control target object, and

the first control unit is configured to identify a location and a direction of each of the target locations, and display the guide image corresponding to a position and a direction of each of the target locations.

4. The display system according to claim 1,

wherein the display unit includes an imaging unit, and

the first control unit is configured to identify a location and a direction of the target location with respect to the display unit, based on a captured image captured by the imaging unit, and display the guide image corresponding to the position and the direction of the target location.

5. The display system according to claim 4,

wherein the first control unit is configured to detect, from the captured image, an image marker provided at the target location of the control target object to identify the location and the direction of the target location with respect to the display unit.

6. The display system according to claim 5,

wherein a plurality of the target locations are set on the control target object, and

the first control unit is configured to detect, from the captured image, the image marker provided at each of the target locations of the control target object to identify the location and the direction of each of the target locations with respect to the display unit.

7. The display system according to claim 5,

wherein the first control unit is configured to identify, based on a line-of-sight of a user wearing the display unit on a head of the user, one of the image markers provided respectively at the target locations and display the guide image corresponding to the one image marker.

8. The display system according to claim 2,

wherein the first control unit is configured to determine a size of the guide image in accordance with a distance between the display unit and the target location of the control target object.

9. The display system according to claim 1, comprising:

a display device having the display unit; and

a control device communicably coupled to the control target object and the display device, and including the first control unit.

10. The display system according to claim 1, comprising:

an input unit configured to receive an input regarding an operation of the control target object; and

a second control unit configured to cause the control target object to operate corresponding to the input,

wherein the first control unit is configured to cause the display unit to update display of the guide image corresponding to the operation of the control target object.

11. The display system according to claim 1,

wherein the first control unit is configured to cause the display unit to display a state image indicating a state of the control target object to correspond to the control target object installed in the real space.

12. The display system according to claim 1,

wherein the first control unit is configured to cause the display unit to display a region guide image for guiding a region in which the control target object moves to correspond to the control target object installed in the real space.

13. The display system according to claim 1,

wherein the control target object is a robot having at least a plurality of joints or a plurality of shafts.

14. A control method for a display system including a head-mounted display unit configured to display an image on a scene in a real space in an overlapped manner, the method comprising:

a display step for causing the display unit to display a guide image indicating a direction set based on a control target object to correspond to the control target object disposed in the real space.