Display system and operation input method

• US 9,678,663 B2
• Filed: 11/27/2012
• Issued: 06/13/2017
• Est. Priority Date: 11/28/2011
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Kobayashi, Shinichi
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Sugent, James F
• Application Number:
  US13/686,256
• Publication Number:
  US 20130139093A1
• Time in Patent Office: 1,659 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A display system includes a sensor unit that detects the position of a pointing object in contact with a predetermined object, a control apparatus that sets a key layout on the object and recognizes a key operated by the pointing object based on the position of the pointing object relative to the key layout, and a display apparatus that receives operational input from the operated key.

Claims

1. A display system comprising:

a sensor unit that detects the position of a pointing object in contact with a predetermined object, the sensor unit being directly disposed on the predetermined object;

a control apparatus that sets a key layout on the predetermined object and recognizes a key operated by the pointing object based on the position of the pointing object relative to the key layout; and

a display apparatus that is wearable around the head of a user and receives operational input from the operated key, the display apparatus including an image pickup section disposed on the display apparatus in a viewing path of an eye of the user, the image pickup section including a triangular prism and having an image pickup area, wherein

the image pickup section presents predetermined image light visible to the user in the image pickup area, and the image pickup area transmits externally incident light,

wherein the sensor unit comprises;

an oscillation detector configured to detect a magnitude of oscillation applied to the predetermined object by the pointing object and to output a signal corresponding to the magnitude of oscillation to a position detector; and

an oscillator configured to oscillate the predetermined object based on the signal, the oscillator oscillating at the magnitude of oscillation controlled by the position detector, and

the magnitude of oscillation physically oscillates the predetermined object in order to provide haptic feedback when the user operates the operated key in the key layout.

2. The display system according to claim 1,

wherein the display apparatus superimposes a key image showing the key layout contained in the predetermined image light on an optical image of the predetermined object having passed through the image pickup area based on the position of the display apparatus relative to the predetermined object and displays the resultant image.

3. The display system according to claim 2,

wherein the display apparatus displays the key image after adjusting the size thereof.

4. The display system according to claim 1,

wherein the display apparatus displays another image in the image pickup area in such a way that the other image does not overlap with the key image showing the key layout.

5. The display system according to claim 1,

wherein the control apparatus sets the key layout based on the position of the pointing object having come into contact with the predetermined object.

6. The display system according to claim 1,

wherein the control apparatus sets the key layout based on user'"'"'s selection.

7. The display system according to claim 1,

wherein the sensor unit captures an image of the pointing object and detects the position of the pointing object based on the captured image of the pointing object.

8. The display system according to claim 1,

wherein the display apparatus includes

an image output section that outputs the predetermined image light,

a projection optical system that projects the predetermined image light outputted from the image output section, and

a light guide section that guides the predetermined image light projected through the projection optical system to the image pickup area.

9. The display system according to claim 1, wherein the control apparatus sets a key layout on the predetermined object according to a calibration operation in which the key layout is set based on a relationship of at least two keys on the predetermined object.

10. The display system according to claim 1, wherein the image pickup section including the prism is disposed in a display portion of the display apparatus.

11. The display system according to claim 1, further comprising a light guide section configured to house the image pickup section, wherein the image pickup section including the prism is disposed in a display portion of the light guide section.

12. The display system according to claim 1, wherein the position detector is configured to detect the position of the pointing object in contact with the predetermined object relative to the sensor unit.

13. The display system according to claim 12, wherein the position detector is configured to position the predetermined object by determining reference coordinates on the predetermined object corresponding to the detected position of the pointing object in contact with the predetermined object.

14. The display system according to claim 12,

wherein the position detector projects at least one of an infrared light flux and an ultrasonic wave and detects the position of the pointing object based on reflection from the pointing object.

15. The display system according to claim 13, wherein the position detector is further configured to determine at least one subsequent position of the pointing object relative to the reference coordinates based on the position of the predetermined object and to output the at least one subsequent position to a communication unit.

16. An operation input method used with a display system, the method comprising:

setting a key layout on a predetermined object by using a control apparatus;

detecting the position of a pointing object having come into contact with the predetermined object by using a sensor unit, the sensor unit being directly disposed on the predetermined object;

detecting a magnitude of oscillation applied to the predetermined object by the pointing object;

outputting a signal corresponding to the magnitude of oscillation to a position detector;

oscillating the predetermined object based on the signal at the magnitude of oscillation controlled by the position detector;

recognizing a key operated by the pointing object based on the position of the pointing object relative to the key layout by using the control apparatus; and

receiving operational input from the operated key by using a display apparatus wearable around the head of a user, the display apparatus including an image pickup section disposed on the display apparatus in a viewing path of an eye of the user, the image pickup section including a triangular prism and having an image pickup area,

wherein the image pickup section presents predetermined image light visible to the user in the image pickup area, and the image pickup area transmits externally incident light, and

the magnitude of oscillation physically oscillates the predetermined object in order to provide haptic feedback when the user operates the operated key in the key layout.

17. The operation input method according to claim 16, wherein the key layout is set according to a calibration operation in which the key layout is set based on a relationship of at least two keys on the predetermined object.

18. The operation input method according to claim 16, further comprising:

positioning the predetermined object by determining reference coordinates on the predetermined object corresponding to the detected position of the pointing object in contact with the predetermined object;

after positioning the predetermined object, determining at least one subsequent position of the pointing object relative to the reference coordinates based on the position of the predetermined object; and

outputting the at least one subsequent position to a communication unit.