Automated trainee monitoring and performance evaluation system
First Claim
Patent Images
1. A system for monitoring a training environment, comprising:
- at least one first sensor for monitoring a training facility, the at least one first sensor being configured to measure geometric and reflective properties of arbitrary surfaces in the training facility and to compensate for color, texture, and form factor of the surfaces of the training facility by component projectors;
at least one second sensor, coupled to at least one trainee, for monitoring activities of the at least one trainee;
a control system, coupled to the at least one first sensor and the at least one second sensor for receiving monitoring data from the at least one first sensor and for receiving monitoring data from the at least one second sensor, respectively, the control system being configured to create two and three dimensional scriptable synthetic objects that comprise static and animated two and three-dimensional images of targets, opponents, and obstacles corresponding to objectives of training exercises, the control system being further configured to providing interaction of the two and three dimensional scriptable synthetic objects to the at least one trainee within the training facility, said control system being further configured to analyze the activities of the at least one trainee by evaluating performance criteria comprising movements, poses and positions of a head, torso, legs and arms of the at least one trainee, and relative movements, poses and positions of the at least one trainee to the two and three dimensional scriptable synthetic objects and positions of real objects based on the monitoring data received from the at least one first sensor and the at least one second sensor, said control system being further configured to process the at least one pose using projective warping so as to produce at least one warped image, the at least one warped image being applied to an occlusion computation to create augmented imagery within a scene from a point of view of the trainee, the augmented imagery being processed to form left and right eye views; and
the at least one component projector for projecting onto the training facility synthetic, the two and three dimensional scriptable synthetic objects to construct a seamless mixed reality scene comprising both real, live world and projected synthetic world within which the at least one trainee is immersed on a real-time basis, the two and three dimensional scriptable synthetic objects changing in response to the performance of the at least one trainee in the training facility and based on the monitoring data received from the at least one first sensor and the at least one second sensor.
2 Assignments
0 Petitions
Accused Products
Abstract
A method and system for performing automated training environment monitoring and evaluation. The training environment may include a mixed reality elements to enhance a training experience.
-
Citations
17 Claims
-
1. A system for monitoring a training environment, comprising:
-
at least one first sensor for monitoring a training facility, the at least one first sensor being configured to measure geometric and reflective properties of arbitrary surfaces in the training facility and to compensate for color, texture, and form factor of the surfaces of the training facility by component projectors; at least one second sensor, coupled to at least one trainee, for monitoring activities of the at least one trainee; a control system, coupled to the at least one first sensor and the at least one second sensor for receiving monitoring data from the at least one first sensor and for receiving monitoring data from the at least one second sensor, respectively, the control system being configured to create two and three dimensional scriptable synthetic objects that comprise static and animated two and three-dimensional images of targets, opponents, and obstacles corresponding to objectives of training exercises, the control system being further configured to providing interaction of the two and three dimensional scriptable synthetic objects to the at least one trainee within the training facility, said control system being further configured to analyze the activities of the at least one trainee by evaluating performance criteria comprising movements, poses and positions of a head, torso, legs and arms of the at least one trainee, and relative movements, poses and positions of the at least one trainee to the two and three dimensional scriptable synthetic objects and positions of real objects based on the monitoring data received from the at least one first sensor and the at least one second sensor, said control system being further configured to process the at least one pose using projective warping so as to produce at least one warped image, the at least one warped image being applied to an occlusion computation to create augmented imagery within a scene from a point of view of the trainee, the augmented imagery being processed to form left and right eye views; and the at least one component projector for projecting onto the training facility synthetic, the two and three dimensional scriptable synthetic objects to construct a seamless mixed reality scene comprising both real, live world and projected synthetic world within which the at least one trainee is immersed on a real-time basis, the two and three dimensional scriptable synthetic objects changing in response to the performance of the at least one trainee in the training facility and based on the monitoring data received from the at least one first sensor and the at least one second sensor. - View Dependent Claims (2, 3, 4, 5, 13, 14, 15, 16, 17)
-
-
6. A computer implemented method of monitoring a training environment, comprising:
-
providing at least one first sensor for monitoring a training facility, the at least one first sensor being configured to measure geometric and reflective properties of arbitrary surfaces in the training facility and to compensate for color, texture, and form factor of the surfaces of the training facility by component projectors; providing at least one second sensor, coupled to at least one trainee, for monitoring activities of the at least one trainee; monitoring the training facility and the activities of the at least one trainee, using a computer processor to perform the steps of; receiving monitoring data from the at least one first sensor and receiving monitoring data from the at least one second sensor; creating two and three dimensional scriptable synthetic objects that comprise static and animated images of targets, opponents, and obstacles corresponding to objectives of training exercises; providing interaction between the two and three dimensional scriptable synthetic objects and the at least one trainee within the training facility; analyzing the activities of the at least one trainee by evaluating performance criteria comprising movements, poses and positions of a head, torso, legs and arms of the at least one trainee, and relative movements, poses and positions of the at least one trainee to the two and three dimensional scriptable synthetic objects and positions of real objects based on the monitoring data received from the at least one first sensor and the at least one second sensor; processing the at least one pose using projective warping so as to produce at least one warped image, the at least one warped image being applied to an occlusion computation to create augmented imagery within a scene from a point of view of the trainee, the augmented imagery being processed to form left and right eye views; and using the at least one component project to perform the steps of; projecting onto the training facility the two and three dimensional scriptable synthetic objects to construct a seamless mixed reality scene comprising both real, live world and a projected synthetic world within which the at least one trainee is immersed on a real-time basis, the two and three dimensional scriptable synthetic objects changing in response to the performance of the at least one trainee in the training facility and based on the monitoring data received from the at least one first sensor and the at least one second sensor. - View Dependent Claims (7, 8, 9, 10, 11, 12)
-
Specification