Portable immersive environment using motion capture and head mounted display

• US 8,624,924 B2
• Filed: 01/17/2009
• Issued: 01/07/2014
• Est. Priority Date: 01/18/2008
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Dobbins, Michael K., Rondot, Pascale, Shone, Eric D., Yokell, Michael R., Abshire, Kevin J., Harbor, Anthony Ray Sr., Lovell, Scott, Barron, Michael K.
• Primary Examiner(s): CHOW, JEFFREY J
• Application Number:
  US12/355,768
• Publication Number:
  US 20090213114A1
• Time in Patent Office: 1,816 Days
• Field of Search: 345/633
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Embodiments of the present invention provide a collaborative visualization system, which integrates motion capture and virtual reality, along with kinematics and computer-aided design (CAD), for the purpose of, for example, virtual training on a task. A portable motion capture system tracks the movements of one or more trainers and records full-body motion capture data for one or more trainers performing one or more tasks. A virtual reality simulator receives the recorded motion capture data and animates scaled avatars within a three-dimensional virtual reality simulation responsive to recorded motion capture data. An immersive observation system displays the virtual reality simulation, including the animated avatars, as a three-dimensional image that appears to surround one or more trainees using one or more head mounted displays so that the one or more trainees can analyze the one or more tasks performed.

Claims

1. A system for simulating one or more tasks using virtual reality, the system comprising:

a portable motion capture system at a location positioned to track movements of one or more users and to record full-body motion capture data for the one or more users performing one or more tasks;

wherein the portable motion capture system comprising;

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras positioned to record images that capture movements of the one or more users wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras;

a remote camera at a location remote from the users that records an image of the location;

a virtual reality simulator being positioned to receive the motion capture data and capable of animating one or more avatars within the three-dimensional virtual reality simulation, responsive to the motion capture data of the motion capture systems and image recorded by the remote camera; and

an immersive observation system to display the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation and that selectively switches between the virtual reality simulation and the image recorded by the remote camera.

2. A system of claim 1, wherein the virtual reality simulator is further positioned to scale each of the one or more animated avatars independently so that an avatar of a first size is responsive to motion capture data from a user of a second size, the second size different than the first size.

3. A system of claim 1, wherein a second virtual reality simulator and a second immersive observation system are located at a second location.

4. A system of claim 1, wherein the plurality of cameras of the portable motion capture system are installed on a tripod.

5. A system of claim 1, wherein the plurality of cameras of the portable motion capture system are clamped on a portable rigid structure.

6. A system of claim 1, wherein the virtual reality simulation further comprises an aircraft, and wherein the one or more tasks are associated with an aircraft.

7. A system of claim 1, wherein the virtual reality simulation further comprises a space system, and wherein the one or more tasks are associated with a space system.

8. A system of claim 1, wherein the virtual reality simulation further comprises a ship, and wherein the one or more tasks are associated with a ship.

9. A system of claim 1, wherein the virtual reality simulation further comprises a missile system, and wherein the one or more tasks are associated with a missile system.

10. A computer program product, stored in one or more non-transitory tangible computer readable media and readable by a computer so that the computer program product operates to perform the following instructions when read by the computer:

recording full-body motion capture data for a set of one or more users performing one or more tasks by a portable motion capture system;

recording an image at a location remote to the users with a remote camera to obtain a telepresence view

wherein the portable motion capture systems comprises;

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras positioned to record images that capture movements of the one or more users wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras;

animating one or more avatars within a the virtual reality simulation by a virtual reality simulator responsive to motion capture data for the one or more users of the first and second motion capture systems so that each of the one or more users corresponds to one of the one or more avatars;

displaying the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation and selectively displaying the telepresence view.

11. A computer program product of claim 10, wherein the program product further operates to perform the following instruction:

scaling each of the one or more animated avatars independently by the virtual reality simulator so that an avatar of a first size is responsive to motion capture data from a user of a second size, the second size different than the first size.

12. A method of simulating a task, the method comprising:

recording full-body motion capture data for a set of one or more users performing one or more tasks by a portable motion capture system;

recording a telepresence view from a location remote to the users

wherein the portable motion capture systems each comprises;

a motion capture bodysuit having markers at predetermined locations;

a plurality of cameras positioned to record images that capture movements of the one or more users wearing the motion capture bodysuit; and

one or more computers to digitally record locations of the markers associated with the bodysuit, responsive to the recorded images from the plurality of cameras;

animating one or more avatars within a the virtual reality simulation by a virtual reality simulator responsive to motion capture data for the one or more users of the motion capture systems so that each of the one or more users corresponds to one of the one or more avatars;

displaying the virtual reality simulation, including the one or more animated avatars, as a three-dimensional image using one or more head mounted displays so that each of the one or more head mounted displays can provide a different perspective of the virtual reality simulation; and

selectively displaying the telepresence view to validate the simulation.

13. A method of claim 12, wherein a second virtual reality simulator and second immersive observation system are located at a second location.

14. A method of claim 13, wherein the virtual reality simulation further comprises an environment of the first location.