Virtual input environment

• US 6,611,253 B1
• Filed: 09/19/2000
• Issued: 08/26/2003
• Est. Priority Date: 09/19/2000
• Status: Expired due to Fees

• Current Assignee: FTK Technologies Limited
• Original Assignee: Harel Cohen
• Inventors: Cohen, Harel
• Primary Examiner(s): WU, XIAO MIN
• Application Number:
  US09/664,753
• Time in Patent Office: 1,071 Days
• Field of Search: 345/156, 345/157, 345/168, 345/169, 345/173, 345/7, 345/8, 345/9, 345/158, 341/20, 341/22, 382/107, 382/108
• US Class Current: 345/168
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

The invention presents a method and a system for virtual input environment, and the creation of an input unit with a changeable keys display on an electronic device. The method enables generating a virtual representation through real-time simulation of a user'"'"'s hands and of an input device (e.g. a keyboard), defining the position and the movements of the hands over the real input device, and displaying a virtual representation of the input device and of the user'"'"'s hands in virtual space on an output device. The method further allows creating changeable keys on the input device, and assigning functions to the keys from various sources. The system includes virtual reality enabling hardware and software, and a virtual input device in which key displays are fully changeable. The method and system of the present invention enable natural typing action while seeing both hands and input device on a screen in front of the user.

Claims

1. A method for inputting data into an electronic system, comprising the steps of:

(a) providing a physical keyboard connected to the electronic system;

(b) sensing a calibration image of said physical keyboard, and recognizing, at least in part from said calibration image, said physical keyboard as one of a plurality of keyboard types;

(c) generating a simulated image of said physical keyboard on a display of the electronic system, said simulated image functioning as a virtual keyboard;

(d) assigning at least one symbol to at least one key of said virtual keyboard;

(e) sensing real-time images of at least one hand of a user and processing said real-time images to generate a dynamic display including simulated images of at least one hand in spatial relationship to said virtual keyboard, said simulated images functioning coordinately with the hand of the user such that said simulated images function as a virtual hand of the user; and

(f) inputting data via said physical keyboard with the at least one hand.

2. The method of claim 1, wherein said step of inputting further includes positioning said at least one virtual hand over said virtual keyboard, and influencing said at least one key to generate an input signal into the system.

3. The method of claim 1, wherein said steps of simulating further comprise running a calibration procedure for said at least one virtual hand and said virtual keyboard.

4. The method of claim 1, wherein said steps of sensing a calibration image and sensing real-time images are performed by at least one video camera.

5. The method of claim 1, wherein said step of assigning further comprises:

(i) providing a changeable keys software to said system; and

(ii) instructing said changeable keys software to assign said at least one symbol.

6. The method of claim 5, wherein said step of instructing is performed manually.

7. The method of claim 5, wherein step of said providing includes supplying an exchange file, and said step of instructing includes receiving inputs from external sources.

8. The method of claim 5, wherein said step of instructing further includes choosing said at least one symbol selected from the group consisting of color, size, language, function, letters, words, sentences, animation, digital display, video images and voice.

9. The method of claim 7, wherein said external sources include applications software and Internet sites.

10. In an electronic system, a virtual input environment system comprising:

(i) at least one imaging device for obtaining real-time images of real-space objects, said real-space objects including at least one hand and a keyboard, said at least one imaging device and said keyboard connected to said system;

(ii) a simulation software for processing said images obtained from said at least one imaging device so as to;

(a) recognize, at least in part from at least one of said images, said keyboard as one of a plurality of types of keyboard and to generate a simulated image of said keyboard, said simulated image functioning as a virtual keyboard; and

(b) to generate from said images a dynamic display including simulated images of said at least one hand in spatial relationship to said virtual keyboard;

(iii) a changeable keys software for assigning virtual keys to said virtual keyboard;

(iv) a display for displaying said simulated images of said at least one hand and said keyboard;

wherein

said at least one hand can be used to input data through said keyboard into said electronic system.

11. The virtual input environment of claim 10, wherein said display is selected from the group consisting of computer monitor, eye-mounted display, television screen, head-up display and eye oriented display.

12. The virtual input environment system of claim 10, wherein said simulated image of said keyboard is configurable in a plurality of configurations.

13. The virtual input environment system of claim 10, wherein said keyboard includes a blank keyboard.

14. The virtual input environment system of claim 10, wherein said keyboard includes a flat substrate.

15. The virtual input environment system of claim 10, wherein said keyboard includes a sensing plate.

16. The virtual input environment of claim 15, wherein said sensing plate includes sensing elements chosen from one of the group consisting of pressure, heat, and sound.

17. The virtual input environment of claim 10, wherein said keyboard comprises two parts.

18. The virtual input environment system of claim 12, wherein said plurality of configurations includes changeable characters selected from the group consisting of color, size, language, function, letters, words, sentences, animation, digital display, video images and voice.