Portable Immersive Environment Using Motion Capture and Head Mounted Display

• US 20090213114A1
• Filed: 01/17/2009
• Published: 08/27/2009
• Est. Priority Date: 01/18/2008
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Lovell, Scott, Barron, Michael K., Harbor, Anthony Ray SR., Rondot, Pascale, Dobbins, Michael K., Yokell, Michael R., Shone, Eric D., Abshire, Kevin J.
• Granted Patent: US 8,624,924 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Embodiments of the present invention provide a collaborative visualization system, which integrates motion capture and virtual reality, along with kinematics and computer-aided design (CAD), for the purpose of, for example, virtual training on a task. A portable motion capture system tracks the movements of one or more trainers and records full-body motion capture data for one or more trainers performing one or more tasks. A virtual reality simulator receives the recorded motion capture data and animates scaled avatars within a three-dimensional virtual reality simulation responsive to recorded motion capture data. An immersive observation system displays the virtual reality simulation, including the animated avatars, as a three-dimensional image that appears to surround one or more trainees using one or more head mounted displays so that the one or more trainees can analyze the one or more tasks performed.

Claims

1. A system for simulating one or more tasks using virtual reality, the system comprising:

a portable motion capture system positioned to track movements of one or more users and to record full-body motion capture data for one or more users performing one or more tasks;

a virtual reality simulator being positioned to receive the motion capture data and capable of animating one or more avatars within a three-dimensional virtual reality simulation, responsive to the motion capture data; and

an immersive observation system to display the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation.

2. A system of claim 1, wherein the virtual reality simulator is further positioned to scale each of the one or more animated avatars independently so that an avatar of a first size is responsive to motion capture data from a user of a second size, the second size different than the first size.

3. A system of claim 1,

wherein the portable motion capture system is located at a first location;

and

wherein the virtual reality simulator and immersive observation system are located at a second location, the second remote from the first location.

4. A system of claim 3,

wherein the portable motion capture system at the first location is a first motion capture system;

and

further comprising a second motion capture system at the second location positioned to track the movements of one or more users at the second location and to record full-body motion capture data for one or more users at the second location interacting with the virtual reality simulation.

5. A system of claim 1, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras installed on a tripod positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.

6. A system of claim 1, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras clamped on a rigid structure positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.

7. A system of claim 1, wherein the virtual reality simulation further comprises an aircraft, and wherein the one or more tasks are associated with an aircraft.

8. A system of claim 1, wherein the virtual reality simulation further comprises a space system, and wherein the one or more tasks are associated with a space system.

9. A system of claim 1, wherein the virtual reality simulation further comprises a ship, and wherein the one or more tasks are associated with a ship.

10. A system of claim 1, wherein the virtual reality simulation further comprises a missile system, and wherein the one or more tasks are associated with a missile system.

13. A computer program product of claim 8, wherein the program product further operates to perform the following instructions:

detecting a collision between an avatar animated by a user and a simulated object in the virtual reality simulation by the virtual reality simulator; and

altering an appearance of the simulated object in the virtual reality simulation by the virtual reality simulator to provide feedback for the detected collision.

11. A computer program product, stored in one or more tangible computer readable media and readable by a computer so that the computer program product operates to perform the following instructions when read by the computer:

recording full-body motion capture data for one or more users performing one or more tasks by a portable motion capture system;

animating one or more avatars within a virtual reality simulation by a virtual reality simulator responsive to motion capture data for the one or more users so that each of the one or more users corresponds to one of the one or more avatars;

displaying the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation.

12. A computer program product of claim 11, wherein the program product further operates to perform the following instruction:

scaling each of the one or more animated avatars independently by the virtual reality simulator so that an avatar of a first size is responsive to motion capture data from a user of a second size, the second size different than the first size.

14. A computer program product of claim 11, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras installed on a tripod positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.

15. A computer program product of claim 11, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras clamped on a rigid structure positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.

16. A method of simulating a task, the method comprising:

recording full-body motion capture data for one or more users performing one or more tasks by a portable motion capture system;

animating one or more avatars within a virtual reality simulation by a virtual reality simulator responsive to motion capture data for the one or more users so that each of the one or more users corresponds to one of the one or more avatars;

displaying the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation.

17. A system of claim 16,

wherein the portable motion capture system is located at a first location;

and

wherein the virtual reality simulator and immersive observation system are located at a second location, the second remote from the first location.

18. A method of claim 17, wherein the virtual reality simulation further comprises an environment of the first location.

19. A method of claim 16, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras installed on a tripod positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.

20. A method of claim 16, wherein the portable motion capture system comprises:

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras clamped on a rigid structure positioned to record images that capture movements of the one or more trainers wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras.