Virtual reality keyboard system and method

• US 6,600,480 B2
• Filed: 05/13/2002
• Issued: 07/29/2003
• Est. Priority Date: 12/31/1998
• Status: Expired due to Fees

• Current Assignee: Anthony James Francis Natoli
• Original Assignee: Anthony James Francis Natoli
• Inventors: Natoli, Anthony James Francis
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): LESPERANCE, JEAN E
• Application Number:
  US10/144,404
• Publication Number:
  US 20020130844A1
• Time in Patent Office: 442 Days
• Field of Search: 345/168, 345/169, 345/170, 345/171, 345/172, 345/173, 345/146, 345/156, 345/158, 345/7, 345/8, 345/9
• US Class Current: 345/168
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G10H 1/0008 Associated control or indicating means

Abstract

A system and method implement a virtual reality (VR) keyboard generated in a display device viewable to a user, with the VR keyboard simulating a physical keyboard with keys and simulating the depression of a physical key on the physical keyboard indicating a key input. The system and method address numerous difficulties of physical and/or hardware-based input devices.

Claims

1. A system for implementing a virtual reality (VR) keyboard, the system comprising:

a display device associated with a user, the display device being responsive to first image data for generating and displaying a first image simulating a physical keyboard having a plurality of physical keys and appearing to the user to be in space independent of actual physical objects, with the first image representing the VR keyboard to the user through the display device, with the VR keyboard having a first appearance corresponding to the first image data;

an input device for detecting a motion associated with the user, the input device being responsive to the detected motion for generating motion-representative input signals corresponding to the motion; and

a processor connected to the display device and operating a VR-keyboard-generating program for providing the first image data to the display device, the processor being responsive to the motion-representative input signals from the input device, for generating motion image data corresponding to the motion; and

wherein the display device is responsive to the motion image data for generating a second image simulating the physical keyboard having the plurality of physical keys to represent the VR keyboard having a second appearance simulating actuation of at least one of the plurality of physical keys corresponding to the motion.

2. The system of claim 1, wherein the input device is responsive to a motion of a finger of the user corresponding to keystrokes to generate the input signals;

wherein the processor, responsive to the keystroke-corresponding input signals, generates finger-motion image data; and

wherein the display device is responsive to the finger-motion image data for generating the VR keyboard having the second appearance representing depressed VR keys on the VR keyboard, thereby displaying depressed VR keys in response to VR keystrokes.

3. The system of claim 2, wherein the processor, responsive to the input signals from the input device, generates corresponding input data to be output to external devices with the input data corresponding to the VR keystrokes.

4. The system of claim 3, wherein the input data corresponds to at least one of a command, text, and a graphic user interface signal.

5. The system of claim 3, wherein the external device is a graphic user interface responsive to the input data corresponding to mouse operations and text.

6. The system of claim 1, wherein the processor generates VR world data;

and

wherein the display device generates a VR world corresponding to the VR world data, with the VR keyboard displayed in the VR world.

7. The system of claim 1, wherein the processor, responsive to a selected mapping chosen from a plurality of predetermined mappings, generates the first image data corresponding to the VR keyboard having VR keys corresponding to the selected mapping;

and

wherein the display device, responsive to the first image data, displays the VR keyboard having VR keys corresponding to the selected mapping.

8. The system of claim 7, wherein the predetermined mappings include specifications for VR keys, each specification corresponding to a respective language.

9. The system of claim 7, wherein the predetermined mappings include specifications for VR keys corresponding to different pre-defined computer commands.

10. The system of claim 7, wherein the predetermined mappings include specifications for positioning a plurality of keys having different positions on a common keyboard shape.

11. The system of claim 7, wherein the predetermined mappings include specifications for defining different VR keyboard shapes and key orientations displayed in virtual reality.

12. The system of claim 2, wherein the input device includes force-feedback means for selectively generating pressure to the fingers of the user during motion of the fingers;

wherein the processor, responsive to the input data from the input device, controls the force-feedback means.

13. The system of claim 2, wherein the processor includes:

a neural network, responsive to the input signals, for learning a custom three-dimensional orientation of at least one hand and at least one finger of a predetermined user during use of the VR keyboard by the predetermined user; and

wherein the processor, responsive to the input signals applied to the trained neural network, predicts the VR keys to be depressed corresponding to the motion of the keystroke-corresponding input signals, and generates the corresponding finger-motion image data.

14. The system of claim 13, wherein the trained neural network, responsive to the input signals, authenticates a current user as the predetermined user.

15. The system of claim 1, wherein the display device includes:

an orientation sensor, responsive to a vertical orientation of the head of the user wearing the display device, for generating an orientation signal; and

wherein the processor, responsive to the orientation signal, for generating the first and second image data in response to the vertical orientation being within a predetermined vertical range, and for not generating any of the first and second image data in response to the vertical orientation being outside of the predetermined vertical range.

16. The system of claim 1, further comprising:

a switch for switching input signals to the processor either from the input device or from a physical keyboard.

17. The system of claim 2, the processor generates hand image data corresponding to the motion of the finger;

and

wherein the display device, responsive to the hand image data, generates images of at least one VR hand having at least one VR finger positioned substantially adjacent to the VR keyboard, with the at least one VR finger appearing to depress the depressed VR keys corresponding to the motion.

18. A system for implementing an interactive virtual reality (VR) keyboard, the system comprising:

a display device associated with a user, with the display device including at least one liquid crystal display (LCD) viewable by the user, with the display device being responsive to first image data for generating and displaying a first image simulating a physical keyboard having a plurality of physical keys and appearing to the user to be in space independent of actual physical objects, with the first image representing the VR keyboard having a plurality of VR keys to the user through the at least one LCD, with the VR keyboard having a first appearance corresponding to the first image data;

an input device for detecting a motion associated with the user corresponding to at least one physical keystroke, the input device being responsive to the detected motion for generating keystroke-corresponding input signals corresponding to the motion;

a processor connected to the display device and operating a VR-keyboard-generating program for providing the first image data to the display device, the processor being responsive to the keystroke-corresponding input signals for generating finger-motion image data; and

wherein the display device is responsive to the finger-motion image data for generating a second image simulating the physical keyboard having the plurality of physical keys to represent the VR keyboard having a second appearance representing a depressed VR key on the VR keyboard simulating actuation of at least one of the plurality of physical keys, thereby displaying at least one depressed VR key in response to the at least one physical keystroke.

19. A method for providing an interactive virtual reality (VR) keyboard, the method comprising the steps of:

operating a VR keyboard-generating-program using a processor;

outputting first image data to a display device;

displaying with a display device a first image simulating a physical keyboard with a plurality of physical keys and appearing to the user to be in space independent of actual physical objects, with the first image representing the VR keyboard having a plurality of VR keys on the display to the user, with the VR keyboard having a first appearance corresponding to the first image data;

detecting motion at an input device, with the motion associated with the user corresponding to at least one physical keystroke;

generating keystroke-corresponding input signals at the input device corresponding to the detected motion;

generating finger-motion image data using the processor corresponding to the input signals; and

generating, using the finger-motion data, a second image simulating the physical keyboard having the plurality of physical keys to represent the VR keyboard viewable by the display device, with the VR keyboard having a second appearance representing a depressed VR key on the VR keyboard simulating actuation of at least one of the plurality of physical keys, thereby displaying at least one depressed VR key in response to the at least one physical keystroke.

20. The method of claim 19, wherein the step of displaying the VR keyboard using the display device to the user includes the step of:

displaying a selected keyboard image from a plurality of keyboard images.