DISPLAY SYSTEM, CONTROL METHOD FOR DISPLAY DEVICE, AND COMPUTER PROGRAM

• US 20160196693A1
• Filed: 12/21/2015
• Published: 07/07/2016
• Est. Priority Date: 01/06/2015
• Status: Abandoned Application

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: KOBAYASHI, Shinichi, TAKANO, Masahide
• Application Number:
  US14/976,761
• Publication Number:
  US 20160196693A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A display system includes a head-mounted display device including a display section worn on the head of a user and an operation device configured separately from the display section. The head-mounted display device performs, according to the operation device, with an image display section, AR display related to an outside scene transmitted through the image display section.

Claims

1. A display system comprising:

a display device worn on a head of a user and including a display section of a transmission type configured to transmit an outside scene and cause the user to visually recognize the outside scene; and

an operation target object configured separately from the display section, wherein

the display device includes a control section configured to perform, according to the operation target object, with the display section, AR display related to the outside scene transmitted through the display section.

2. The display system according to claim 1, wherein the display device displays, with the display section, in association with the operation target object, a control target associated image associated with the operation target object.

3. The display system according to claim 1, wherein the operation target object is an operation device including:

an operation section configured to receive operation by the user; and

a communication section configured to transmit data indicating the operation received by the operation section to the display device, and

the control section of the display device performs the AR display with the display section on the basis of operation of the operation device.

4. The display system according to claim 3, wherein the control section of the display device performs, on the basis of the operation of the operation device, in a form associated with the operation section, with the display section, the AR display related to the outside scene transmitted through the display section.

5. The display system according to claim 3, wherein the control section detects the operation of the operation section on the basis of data transmitted from the operation device and changes, according to the detected operation, the AR display being displayed on the display section.

6. The display system according to claim 3, wherein the control section is capable of setting allocation of the AR display to the operation section.

7. The display system according to claim 6, wherein the control section is capable of executing processing for updating the allocation of the AR display to the operation section.

8. The display system according to claim 3, wherein the control section of the display device calculates relative positions of the operation device and the display device and performs the AR display with the display section on the basis of the calculated relative positions.

9. The display system according to claim 3, wherein

the operation section includes a touch operation section configured to detect contact operation,

the touch operation section includes a plurality of operation regions, and

the control section displays, with the display section, the AR display allocated to the operation region operated by the touch operation section.

10. The display system according to claim 3, wherein the operation device includes an image pickup section and transmits, with the communication section, data including a picked-up image of the image pickup section to the display device.

23. The display system according to claim 1, wherein the operation device includes a wearing section worn on the body of the user.

24. The display system according to claim 1, wherein, when detecting the operation target object, the control section included in the display device displays, according to the operation target object, on the display section, a control target associated image associated with the operation target object.

25. The display system according to claim 24, wherein, when detecting operation on the operation target object for which the control target associated image is displayed, the control section included in the display device executes processing corresponding to operation content and the control target associated image.

26. The display system according to claim 24, wherein the display device includes a detecting section configured to detect operation on the operation target object.

27. The display system according to claim 24, wherein

the display device includes an image pickup section, and

the control section detects the operation target object on the basis of a picked-up image of the image pickup section and detects operation on the operation target object.

28. The display system according to claim 1, wherein

the display device includes a data storing section configured to store data for the AR display for performing the AR display of an image for operation corresponding to the operation section of the operation target object, and

the control section is capable of changing the data for the AR display stored by the data storing section.

29. The display system according to claim 28, wherein

the data for the AR display stored by the data storing section includes data for associating the image for operation and the operation section of the operation target object, and

the control section is capable of changing association of the image for operation and the operation section of the operation target object in the data for the AR display stored by the data storing section.

11. A display system comprising:

a display device worn on a head of a user and including a display section of a transmission type configured to transmit an outside scene and cause the user to visually recognize the outside scene; and

an operation target object configured separately from the display section, wherein

the display device includes a control section configured to perform, on the basis of operation to the operation target object, display control including processing for transitioning a screen displayed by the display section.

12. The display system according to claim 11, wherein the operation target object is an operation device including an operation section configured to receive operation by the user.

13. The display system according to claim 12, wherein

the operation device includes a communication section configured to transmit data indicating the operation received by the operation section to the display device, and

the control section of the display device detects operation of the operation device on the basis of the data transmitted from the operation device.

14. The display system according to claim 12, wherein the operation section of the operation device detects operation involving rotation, and

when data indicating the operation involving the rotation is transmitted from the operation device, the control section of the display device transitions, according to a rotating direction of the operation, a screen displayed by the display section.

15. The display system according to claim 12, wherein the control section of the display device performs, on the basis of data transmitted from the operation device, display control including any one of enlargement, reduction, and rotation of the image displayed by the display section.

16. The display system according to claim 12, wherein

the display device transmits data for an output to the operation device, and

the operation device receives, with the communication section, the data for output transmitted by the display device and outputs the received data for output.

17. The display system according to claim 12, wherein the operation device includes an image pickup section and transmits, with the communication section, data including a picked-up image of the image pickup section to the display device.

18. The display system according to claim 17, wherein the control section of the display device receives the data including the picked-up image from the operation device and calculates, on the basis of the received data, relative positions of the operation device and the display device.

19. The display system according to claim 11, wherein the operation section of the operation device detects a movement of the operation device as operation.

20. The display system according to claim 11, wherein the control section of the display device calculates relative positions of the operation device and the display device and controls display of the display section on the basis of the calculated relative positions.

21. The display system according to claim 20, wherein

the display device includes an image pickup section, and

the control section calculates the relative positions of the operation device and the display device on the basis of a picked-up image of the image pickup section.

22. The display system according to claim 11, wherein the display section is a display section of a transmission type configured to transmit an outside scene and cause the user to visually recognize the outside scene.

30. A control method for controlling a display device, which includes a display section worn on a head of a user, using an operation target object configured separately from the display section, the control method comprising performing, with the display device, according to the operation target object, AR display related to an outside scene transmitted through the display section.

31. A control method for controlling a display device, which includes a display section worn on a head of a user, using an operation target object configured separately from the display section, the control method comprising performing, with the display device, on the basis of operation on the operation target object, display control including processing for transitioning a screen displayed by the display section.

32. A computer program executable by a control section that controls a display device worn on a head of a user and including a display section of a transmission type configured to transmit an outside scene and cause the user to visually recognize the outside scene, the computer program causing the control section to:

detect an operation target object configured separately from the display section;

detect an operation section included in the operation target section;

display an image for operation in a position corresponding to the detected operation section of the operation target object;

detect operation concerning the image for operation; and

execute processing corresponding to the image for operation for which the operation is detected.