Virtual workstation

• US 20040128012A1
• Filed: 11/05/2003
• Published: 07/01/2004
• Est. Priority Date: 11/06/2002
• Status: Active Grant

• Current Assignee: Julius Lin
• Original Assignee: Julius Lin
• Inventors: Lin, Julius
• Granted Patent: US 7,337,410 B2
• Time in Patent Office: Days
• US Class Current: 700/100
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves

Abstract

In accordance with the present invention there is provided devices and methods for controlling a microprocessor controlled device in a virtual environment through the use of at least one sensor disposed at least one user'"'"'s finger and a scanner, said sensor and scanner being utilized to input position data, the method comprising the steps of: loading an operating system in a computing environment; displaying a virtual keyboard and a virtual pointing device on a display device; initializing coordinates defining individual keys of said keyboard; initializing coordinates defining a location of said pointing device in relation to the keyboard; monitoring the position of at least one sensor disposed on a user'"'"'s finger; determining if sensor movement correlates to depression of a key on the keyboard and providing feedback to the user to indicate that a key was depressed on the keyboard; transmitting data correlating to the depression of the key; and returning to the monitoring step.

Claims

1. A method for controlling a microprocessor based system in a virtual environment, said method comprising:

loading a computer program into a memory space;

transmitting at least one signal from at least one transmitter;

displaying virtual input devices on a display device, wherein said virtual input devices initially have pre-determined coordinates;

receiving data generated from movement of at least one sensor;

calculating coordinates of said sensor movement;

comparing calculated coordinates to the pre-determined coordinates of each virtual input device;

calculating desired input from coordinates generated by said sensor movement; and

displaying input on said display device and transmitting said input to said operating system.

2. The method according to claim 1, wherein said display device is a head mounted display device.

3. The method according to claim 2, wherein coordinates calculated from said sensor are x,y, and z coordinates.

4. The method according to claim 3, wherein the virtual input devices displayed on said display device are a keyboard and a pointing device.

5. The method according to claim 1 wherein said transmitter transmits and electromagnetic signal.

6. The method according to claim 5, wherein said transmitter transmits at least two different signals, one signal being electromagnetic, the second being chosen from the group consisting of:

electromagnetic, optical, acoustical, and inertial.

7. The method according to claim 4 further including the steps of changing the location of said virtual input devices by grabbing the virtual input devices in the virtual environment and dragging the virtual devices to a new location.

8. The method according to claim 4, wherein said sensor movement is translated to define key presses on said virtual keyboard or movement of said virtual pointing device.

9. The method according to claim 4, wherein the physical layout of said virtual keyboard may be manipulated by a user through movement of said sensors.

10. A method for controlling a microprocessor controlled device in a virtual environment, said method comprising:

loading a computer program into a memory space;

loading an operating system into a memory space;

transmitting at least one signal from a transmitting device displaying a virtual keyboard and a virtual input device on a display device;

initializing coordinates defining individual keys of said keyboard;

initializing coordinates defining a location of said input device in relation to said keyboard;

receiving data at least one sensor wherein said data received is converted into coordinate information and stored in a memory space;

determining if coordinated derived from movement correlates to a key location of the virtual keyboard or movement of the virtual input device; and

displaying sensor movement on said display device.

11. The method according to claim 10, further including the step of;

determining if sensor movement correlates to movement of said virtual input device, and providing feedback to the user to indicate movement of said virtual input device on said display device.

12. The method according to claim 10, wherein said coordinates defining said virtual keyboard are user definable.

13. The method according to claim 10, wherein a user may switch between the virtual keyboard and the virtual mouse through defined sensor movement.

14. The method according to claim 13, wherein transmission rate of said transmitter is reduced by certain percent proportion to the distance between sensor and virtual object when it is determined that at least one sensor moves beyond a predetermined coordinate location in said virtual environment.

15. The method according to claim 10, wherein said display device is configured to be wearable by the user.

16. The method according to claim 10, wherein determining step further comprises:

adding a time stamp and sensor identification tag to each data set;

placing said coordinate date, time stamp data and sensor identification data into a memory location;

comparing transmitted sensor data to the known coordinates of the virtual keyboard and virtual input device and determining if the sensor date correlates to a location on the virtual keyboard or a location on the virtual input device; and

using known threshold values to determine if data from sensor movement correlates to a desired input chosen from the group consisting of;

a key press, a key release, a keyhold, movement of the virtual input device, a button press on the virtual input device or a double button press, or a button release.

17. The method according to claim 10, further including the step of providing said feedback in relation to sensor movement.

18. The method according to claim 17, wherein said feedback is audible feedback.

19. The method according to claim 17, wherein said feedback is visual and displayed on the display device.

20. The method according to claim 17, wherein said feedback is tactile.

21. The method according to claim 17, wherein said feedback includes visual and tactile feedback.

22. A method of generating and controlling a virtual workstation, said method comprising:

initializing an operating system and a controller system in a microprocessor based computer system;

displaying virtual hands, a virtual keyboard, a virtual pointing device and a virtual workstation environment on a display device;

monitoring movement of sensors disposed on a user'"'"'s hands for movement;

displaying movement of said virtual hands in response to movement of said sensors; and

determining if movement of at least one sensor passes a user defined threshold.

23. A method of generating a virtual gaming system, said method comprising:

initializing an operating system and a controller system in a microprocessor based computer system;

loading a game program into a memory space;

displaying a virtual player on a display device;

monitoring movement of sensors disposed on a user for movement; and

displaying movement of said virtual player in response to movement of said sensors.

24. A system for virtually controlling a microprocessor based system, said system comprising:

a microprocessor based computer system having an operating system configured to be run thereon;

a display device,

at least one sensor;

at least one transmitter;

a system electronic unit in association with said sensor and said transmitter, said system electronic unit being configured to configured to track movement of said sensor and determine coordinates of said sensor and time components of sensor movement; and

a software component, wherein said software is stored in a computer readable medium, wherein said software is in communication system electronic unit, wherein said software determines vector movement and acceleration of said sensor and displays said sensor movement on said display device.

25. In a computer readable medium, stored programming for controlling hardware and software components in a virtual environment, comprising:

(a) programming for controlling a tracker system;

(b) programming for storing sensor location data into a memory location; and

(c) programming configured to generate a visual user interface that displays virtual input devices on a virtual display device.

26. A system for implementing a virtual reality environment, the system comprising:

a display device associated with a user, the display device being responsive to image data for generating and displaying an image simulating a physical computer system, including a physical keyboard, a physical input device, and physical representation of the user'"'"'s hands, wherein each of the simulated components appear to the user to be in space independent of actual physical objects;

an output device for transmitting a signal;

an input device for generating data in response to interaction with said signal;

a processor connected to said input and output device and said display device and operating a virtual environment manager program and a multi-dimensional basic input and output program for generating a virtual keyboard, a virtual input device, and virtual hands, the processor being responsive to data generated from said input device, for generating motion image data corresponding to the input device data; and

wherein the display device is responsive to motion image data for generating a second image simulating physical motion of at least one virtual component.

27. A system for implementing a virtual reality (VR) computing environment, the system comprising:

a VR display device including at least one display and worn by a user the one display viewable by the user, with the VR display, responsive to first image data, for generating and displaying a first VR image simulating a physical computer system including a virtual keyboard having a plurality of physical keys, a virtual mouse having at least one physical button, with the first VR image representing the VR keyboard and VR mouse, the VR keyboard and VR mouse having a first appearance corresponding to the first image data;

an input and an output device for generating motion-representative data corresponding to motion of a user'"'"'s body part; and

a processor connected to the VR display device and operating a virtual environment manager (VEM) and multi-dimensional basic input output (MD-BIOS) program, wherein the VEM and MD-BIOS provide the first image data to the VR display device, the processor being responsive to the motion-representative data generated from the input device, for generating motion image data corresponding to the motion; and

wherein the VR display device is responsive to the motion image data for generating a second VR image simulating motion corresponding to the motion of the portion of the body of the user.

28. The system of claim 27, wherein the input device is responsive to the motion of the fingers of the user to generate the input signals;

wherein the processor, responsive to the input signals, generates motion image data; and

wherein the VR display device is responsive to the motion image data for generating the VR keyboard having the second appearance representing depressed VR keys on the VR keyboard, thereby displaying depressed VR keys in response to VR keystrokes.

29. The system of claim 28, wherein the processor, responsive to the input signals from the input device, generates corresponding input data output to external devices with the input date corresponding to the VR keystrokes.

30. The system of claim 29, wherein the input data corresponds to at least one of a command, text, and a graphic user interface signal.

31. The system of claim 27, wherein the processor generates VR world data;

and

wherein the VR display device generates a VR world corresponding to the VR world data, with the VR keyboard, VR mouse and VR hands displayed in the VR world.

32. The system of claim 27, wherein the processor, responsive to a selected mapping chosen from a plurality of predetermined mappings, generates the first image data corresponding to the VR keyboard having VR keys corresponding to the selected mapping;

and

wherein the VR display device, responsive to the first image data, displays the VR keyboard having VR keys corresponding to the selected mapping.

33. The system of claim 32, wherein the predetermined mappings include specifications for defining different VR keyboard shapes and key orientations displayed in virtual reality.

34. The system of claim 28, wherein the input device includes force-feedback means for generating feedback to the user during motion of the fingers;

wherein the processor, responsive to the input data from the input device, controls the force-feedback means.

35. The system of claim 27, wherein the input device is responsive to the motion of the fingers of the user to generate the input signals;

wherein the processor, responsive to the input signals, generates motion image data; and

wherein the VR display device is responsive to the motion image data for generating the VR mouse having the second appearance representing cursor movement, thereby displaying mouse movement in response to said motion image data.