System and Method for Virtual Engineering

• US 20130257904A1
• Filed: 04/03/2012
• Published: 10/03/2013
• Est. Priority Date: 04/03/2012
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Roth, Andrew S.
• Granted Patent: US 9,170,648 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A method and system for generating a virtual image of a virtual environment is provided. A virtual reality manager receives hand position data for at least one hand of a user from a hand system. The virtual reality manager receives head position data for a head of the user from a head-mounted system. The virtual reality manager identifies image-based position data and a current frame of reference for a current time using a target image corresponding to the current time. The virtual reality manager generates virtual image control data for the current time using the hand position data, the head position data, the image-based position data, and the current frame of reference. The virtual image control data is configured for use by a virtual image application.

Claims

1. A method for generating a virtual image of a virtual environment, the method comprising:

receiving, at a virtual reality manager, hand position data for at least one hand of a user from a hand system;

receiving, at the virtual reality manager, head position data for a head of the user from a head-mounted system;

identifying, by the virtual reality manager, image-based position data and a current frame of reference for a current time using a target image corresponding to the current time; and

generating, by the virtual reality manager, virtual image control data for the current time using the hand position data, the head position data, the image-based position data, and the current frame of reference, wherein the virtual image control data is configured for use by a virtual image application.

2. The method of claim 1 further comprising:

sending, by the virtual reality manager, the virtual image control data to the virtual image application; and

generating, by the virtual image application, a current virtual image of the virtual environment for the current time using the virtual image control data for the current time.

3. The method of claim 2, wherein the step of generating the current virtual image of the virtual environment for the current time using the virtual image control data for the current time comprises:

positioning a virtual left hand and a virtual right hand in the current virtual image based on the virtual image control data, wherein the virtual left hand is an image representing a left hand of the user and the virtual right hand is an image representing a right hand of the user.

4. The method of claim 2 further comprising:

displaying, by the virtual reality manager, the current virtual image of the virtual environment to the user on a display device associated with the head-mounted system, wherein the current virtual image replaces a previous virtual image displayed on the display device.

5. The method of claim 1, wherein the step of generating the virtual image control data for the current time using the hand position data, the head position data, the image-based position data, and the current frame of reference comprises:

synchronizing, by the virtual reality manager, the hand position data and the head position data with respect to time.

6. The method of claim 1 further comprising:

generating left hand position data using a left glove configured to substantially conform to a left hand of the user; and

generating right hand position data using a right glove configured to substantially conform to a right hand of the user, wherein the left hand position data and the right hand position data form the hand position data.

7. The method of claim 1, wherein the step of generating the virtual image control data for the current time comprises:

identifying modified hand position data using left hand position data in the hand position data, right hand position data in the right hand position data, and the image-based position data;

identifying modified head position data using the head position data and the current frame of reference; and

generating the virtual image control data using the modified hand position data, the modified head position data, and a set of constraints.

8. The method of claim 7, wherein the step of generating the virtual image control data for the current time further comprises:

identifying the set of constraints using user data, wherein the user data is based on a set of user images.

9. The method of claim 7, wherein the step of generating the virtual image control data using the modified hand position data, the modified head position data, and the set of constraints comprises:

identifying a finger position error, a relative hand position error, and a head position error using a feedback controller, the modified hand position data, the modified head position data, and the set of constraints; and

generating the virtual image control data with a desired level of accuracy using the feedback controller, the finger position error, the relative hand position error, and the head position error.

10. The method of claim 1, wherein the virtual environment is a simulation of an engineering environment selected from one of a design environment, a manufacturing environment, a computer environment, a testing environment, a data management environment, an inspection environment, and an operations environment.

11. A virtual reality system comprising:

a head-mounted system configured to be worn relative to a head of a user; and

a virtual reality manager associated with the head-mounted system, wherein the virtual reality manager is configured to;

receive hand position data for at least one hand of the user from a hand system;

receive head position data for the head of the user from a sensor system in the head-mounted system;

identify image-based position data and a current frame of reference for a current time using a target image corresponding to the current time; and

generate virtual image control data for the current time using the hand position data, the head position data, the image-based position data, and the current frame of reference, wherein the virtual image control data is configured for use by a virtual image application.

12. The virtual reality system of claim 11, wherein the virtual reality manager is configured to send the virtual image control data to the virtual image application and receive a current virtual image of a virtual environment for the current time from the virtual image application.

13. The virtual reality system of claim 11 further comprising:

a display device associated with the head-mounted system, wherein the virtual reality manager is further configured to display the current virtual image of the virtual environment to the user on the display device.

14. The virtual reality system of claim 12, wherein the current virtual image includes a virtual left hand and a virtual right hand in which the virtual left hand is an image representing a left hand of the user and the virtual right hand is an image representing a right hand of the user.

15. The virtual reality system of claim 11, wherein the virtual reality manager comprises:

a data coordinator configured to synchronize the hand position data and the head position data with respect to time, wherein the data coordinator comprises;

a hand data modulator configured to identify modified hand position data using left hand position data in the hand position data, right hand position data in the hand position data, and the image-based position data;

a head data modulator configured to identify modified head position data using the head position data and the current frame of reference;

a control data generator configured to generate the virtual image control data using the modified hand position data, the modified head position data, and a set of constraints; and

a constraint identifier configured to identify the set of constraints using user data, wherein the user data is based on a set of user images.

16. The virtual reality system of claim 15, wherein the data coordinator further comprises:

a feedback controller configured to identify a finger position error, a relative hand position error, and a head position error using the modified hand position data, the modified head position data, and the set of constraints, wherein the control data generator is configured to use the finger position error, the relative hand position error, and the head position error to generate the virtual image control data.

17. The virtual reality system of claim 11, wherein the virtual reality system further comprises:

the hand system, wherein the hand system comprises;

a left glove configured to generate left hand position data, wherein the left glove is configured to substantially conform to a left hand of the user; and

a right glove configured to generate right hand position data, wherein the right glove is configured to substantially conform to a right hand of the user.

18. The virtual reality system of claim 11, wherein the virtual image control data is used to control a virtual image of a virtual environment in which the virtual environment is a simulation of an engineering environment selected from one of a design environment, a manufacturing environment, a computer environment, a testing environment, a data management environment, an inspection environment, and an operations environment.

19. A computer comprising:

a bus;

a non-transitory storage device connected to the bus, wherein the non-transitory storage device includes program code; and

a processor unit connected to the bus, wherein the processor unit is configured to execute the program code to;

receive hand position data for at least one hand of a user from a hand system;

receive head position data for a head of the user from a head-mounted system;

identify image-based position data and a current frame of reference for a current time using a target image corresponding to the current time; and

generate virtual image control data for the current time using the hand position data, the head position data, the image-based position data, and the current frame of reference, wherein the virtual image control data is configured for use by a virtual image application.

20. The computer of claim 19, wherein the processor unit is further configured to execute the program code to display a current virtual image for the current time, received from the virtual image application, on a display device associated with the head-mounted system.