Providing input signals

• US 20040046747A1
• Filed: 08/22/2003
• Published: 03/11/2004
• Est. Priority Date: 09/26/2000
• Status: Abandoned Application

• Current Assignee: Eugenio Bustamante
• Original Assignee: Eugenio Bustamante
• Inventors: Bustamante, Eugenio
• Application Number:
  US10/381,640
• Publication Number:
  US 20040046747A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Apparatus for providing input signals to a device such as a computer (1) to be controlled by a n operator comprises means (3) for generating a virtual image (2) of an interface tot he device to be controlled, means (5) for detecting the position of an operating member (4) actuated by the operator, and means (6, 7, 3) for generating a virtual image (8) of the operating member (4), the arrangement being such that movement of the operating member image (8) in accordance with movement of the operating member (4) interacts with the interface image (2) to provide the input signals for the device. A method of providing the input signals is also disclosed. The detecting means (5) preferably comprises a stereoscopic image generator, while the means for generating the virtual images includes a virtual environment processor (3). The advantage of the apparatus and method lies in its ease of use, as the operating member can be held in a comfortable position.

Claims

1. Apparatus for providing input signals to a device (1) to be controlled by an operator comprises means (3) for generating a virtual image (2) of an interface to the device to be controlled, means (5) for detecting the position of an operating member (4) actuated by the operator, and means (6, 7, 3) for generating a virtual image (8) of the operating member, the arrangement being such that movement of the operating member image (8) in accordance with movement of the operating member (4) interacts with the interface image (2) to provide the input signals for the device.

2. Apparatus as claimed in claim 1, in which the means for generating the virtual image of the interface comprises a virtual environment processor means (3).

3. Apparatus as claimed in claim 1 or claim 2, in which the operating member (4) comprises the operator'"'"'s hand.

4. Apparatus as claimed in claim 1 or claim 2, in which the operating member (4) comprises a tool such as a stylus or pointer adapted to be held in the hand or other part of the operator'"'"'s body.

5. Apparatus as claimed in any preceding claim, in which the means for detecting the position of the operating member comprises a stereoscopic image generator (5).

6. Apparatus as claimed in claim 5, in which the stereoscopic image generator (5) comprises a pair of pinhole cameras whose optical properties are known or can be calculated.

7. Apparatus as claimed in claim 5 or claim 6, in which the means for generating the virtual image of the operating member includes an image preparation means (6), to which the data from the stereoscopic pairs of images are passed to manipulate the stereo data.

8. Apparatus as claimed in claim 7, in which the means for generating the virtual image of the operating member includes a signal generator (7), to which data from the image preparation means (6) is passed to generate signals indicating the three-dimensional co-ordinates of the operating member (4) using the stereo data.

9. Apparatus as claimed in claim 8, in which the virtual environment processor means (3) receives the three-dimensional co-ordinate signals from the signal generator (7), to complete the generation of the virtual image (8) of the operating member (4).

10. Apparatus as claimed in claim 9, in which the virtual environment processor means (3) combines the image (2) of the interface with the image (8) of the operating member for the user.

11. Apparatus as claimed in any of claims 2 to 10, in which the virtual environment processor means (3) provides input signals for the device in accordance with the interaction of the operating member image (8) with the interface image (2).

12. Apparatus as claimed is any preceding claim including an application programming interface to allow application programs to operate on the device.

13. Apparatus as claimed in any of claims 2 to 12, in which the virtual environment processor means (3) provides output signals for an output display such as a monitor, TV or head-up display.

14. Apparatus as claimed in any of claims 2 to 12, in which the virtual environment processor means (3) generates output signals to control an external device.

15. A method of providing input signals to a device to be controlled by an operator comprising:

generating a virtual image (2) of an interface to the device to be controlled;

detecting the position of an operating member (4) actuated by the operator;

generating a virtual image (8) of the operating member (4);

relating the virtual image (8) of the operating member to the virtual image (2) of the interface such that movement of the operating member image (8) in accordance with movement of the operating member (4) interacts with the interface image (2); and

providing input signals for the device from the interaction of the operating member image and the interface image.

16. A method as claimed in claim 15, in which generation of the interface image (2) is performed by a virtual environment processor means (3).

17. A method as claimed in claim 15 or claim 16, in which detection of the position of the operating member is performed by a stereoscopic image generator (5).

18. A method as claimed in any of claims 15 to 17, in which generation of the operating member image (8) is performed by manipulation of the data by an image preparation means (6), by a signal generator (7), which generates signals indicating the three-dimensional co-ordinates of the operating member from the stereo data, and by the virtual environment processor means (3), using the three-dimensional co-ordinates.

19. A method as claimed in any of claims 15 to 18, in which relating the operating member image to the interface image is performed by the virtual environment processor means (3).

20. A method as claimed in any of claims 15 to 19, in which production of the input signals for the device is performed by the virtual environment processor means (3).

21. A method as claimed in any of claims 15 to 20, in which the virtual environment processor means (3) accepts signals from external devices or programs to enable these to be operated in the interface image (2).

22. A method as claimed in any of claims 15 to 21, in which the virtual environment processor means (3) provides output signals for the output display or to control external devices.

23. Apparatus for providing input signals to a device (1) to be controlled by an operator comprises means (5) for optically detecting the position of an operating member (4) actuated by the operator in a control space corresponding to an input device of the device to be controlled, movement of the operating member (4) in the control space being represented on an output device (20) and providing input signals for the device, characterised in that the means for detecting the position of the operating member comprises a stereoscopic image generator (5).

24. Apparatus as claimed in claim 23, in which the operating member (4) comprises the operator'"'"'s hand.

25. Apparatus as claimed in claim 23, in which the operating member (4) comprises a tool such as a stylus or pointer adapted to be held in the hand or other part of the operator'"'"'s body.

26. Apparatus as claimed in any of claims 23 to 25, in which the stereoscopic image generator (5) comprises a pair of pinhole cameras whose optical properties are known or can be calculated.

27. Apparatus as claimed in any of claims 23 to 26, in which the data from the stereoscopic pairs of images are passed to an image preparation means (6) to manipulate the stereo data.

28. Apparatus as claimed in claim 27, in which the image preparation means (6) passes the data to a signal generator (7), which generates signals indicating the three-dimensional co-ordinates of the operating member (4) using the stereo data.

29. Apparatus as claimed in claim 28, in which the signals from the signal generator (7) provide the input signals for the device to be controlled.

30. Apparatus as claimed in any of claims 23 to 28, in which movement of the operating member (4) in the control space provides the input signals to the device directly.

31. Apparatus as claimed in any of claims 23 to 30, in which the stereoscopic image generator (5), the image preparation means (6) and the signal generator (7) are combined in a single unit (21), separate from the device to be controlled.

32. Apparatus as claimed in claim 31, in which the input signals are relayed from the unit (21) to the device being controlled by wires (22), or infra-red or other appropriate wireless protocols.

33. Apparatus as claimed in any of claims 23 to 30, in which the stereoscopic image generator (5), the image preparation means (6) and the signal generator (7) are incorporated into the device to be controlled.

34. A method of providing input signals to a device to be controlled by an operator comprising:

defining a control space corresponding to an input device of the device to be controlled;

optically detecting by means of a stereoscopic image generator (5) the position of an operating member (4) actuated by the operator in the control space;

providing input signals for the device from the movement of the operating member (4) in the control space; and

representing movement of the operating member (4) in the control space on an output device (20) of the device to be controlled.

35. A method as claimed in claim 34, in which providing the input signals is performed by manipulation of the data by an image preparation means (5), and by a signal generator (6), which generates signals indicating the three-dimensional co-ordinates of the operating member from the stereo data.

36. A method as claimed in claim 34 or claim 35, in which representing the movement of the operating member (4) on the output device is performed as movement of a cursor on a display screen.