SYNCHRONOUS MULTI-MEDIA RECORDING AND PLAYBACK WITH END USER CONTROL OF TIME, DATA, AND EVENT VISUALIZATION FOR PLAYBACK CONTROL OVER A NETWORK

• US 20080145830A1
• Filed: 12/15/2006
• Published: 06/19/2008
• Est. Priority Date: 12/15/2006
• Status: Active Grant

• Current Assignee: Atellis, Inc.
• Original Assignee: Atellis, Inc.
• Inventors: Huang, Lucas K., Kazoun, Chafic A.
• Granted Patent: US 8,113,844 B2
• Time in Patent Office: Days
• US Class Current: 434/336
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

A system and system are provided for providing synchronous multimedia recording and playback. Aspects the exemplary embodiment include in response to a training session is being conducted, synchronously recording in real-time simulator data from a simulator captured by a simulator capture tool, and video of the training session captured by a plurality of A/V sources; encoding each of the videos captured by the plurality of A/V sources as respective digital media files formatted as streaming media; and transmitting both the simulator data and the video media files from a server to a client over a network, such that when the client receives the simulator data and the stream, the respective videos are synchronously played back with the simulator data on the client

Claims

1. A method for providing synchronous multimedia recording and playback, comprising:

in response to a training session is being conducted, synchronously recording in real-time simulator data from a simulator captured by a simulator capture tool, and video of the training session captured by a plurality of A/V sources;

encoding each of the videos captured by the plurality of A/V sources as respective digital media files formatted as streaming media; and

transmitting both the simulator data and the video media files from a server to a client over a network, such that when the client receives the simulator data and the stream, the respective videos are synchronously played back with the simulator data on the client.

2. The method of claim 1 wherein the respective videos are played back in a single interface.

3. The method of claim 2 wherein the single interface is divided into respective windows for each video.

4. The method of claim 2 wherein the streaming further comprises dynamically generating a single media file that references as sources the respective streaming media files, and streaming the single media file.

5. The method of claim 1 further comprising recording the simulator data as captured video of the simulator itself showing an output of the simulator.

6. The method of claim 5 wherein the video of the simulator data is also encoded as a streaming media file, and both the video of the simulator data and the respective videos are played back in the single interface.

7. The method of claim 1 further comprising recording the simulator data as telemetry values.

8. The method of claim 7 further comprising playing back the simulator data as a trend of the telemetry values along a timeline.

9. The method of claim 8 further comprising identifying transition points of the telemetry values during playback.

10. The method of claim 8 further comprising allowing an end-user to selectively control visualization of the telemetry information.

11. The method of claim 8 further comprising displaying points of interest along the timeline.

12. The method of claim 7 further comprising playing back the videos on the client in a streaming media player, and playing back the simulator data in a flash player.

13. The method of claim 1 further comprising allowing an end-user of the client to advance or return to any point in time in the synchronous play back of the videos.

14. The method of claim 1 further comprising streaming the synchronous playback in real-time during the training session.

15. The method of claim 1 further comprising storing the simulator data and digital media files, and transmitting the synchronous playback subsequent to the training session.

16. The method of claim 15 further comprising prior to the synchronous playback, allowing an end-user of the client to select which ones of the videos are played back.

17. The method of claim 1 further comprising enabling an end-user to enter annotation and assessment data while the videos and simulation data are synchronously played back.

18. The method of claim 17 further comprising during the synchronous playback, displaying a checklist of training tasks to be completed by a trainee during the trading session, and allowing an end-user to indicate through the interface which tasks were completed during the synchronous playback for real-time annotation and assessment of the trainee.

19. The method of claim 18 wherein the stream is received and played back on multiple clients to provide multiple simultaneous local and remote assessments by multiple end-users.

37. The system of claim 15 wherein prior to the synchronous playback, an end-user of the client is allowed to select which ones of the videos are played back.

39. The system of claim 17 wherein the stream is received and played back on multiple clients to provide multiple simultaneous local and remote assessments by multiple end-users.

20. A system for providing synchronous multimedia recording and playback, comprising:

a plurality of A/V sources for capturing video of a training session, the training session including a use of a simulator;

a simulation capture tool for capturing real-time simulator data from the simulator, wherein the simulator data and the video of the training session are recorded synchronously;

one or more encoders for encoding each of the videos captured by the plurality of A/V sources as respective digital media files formatted as streaming media; and

means for transmitting both the simulator data and the video media files from a server to a client over a network, such that when the client receives the simulator data and the stream, the respective videos are synchronously played back with the simulator data on the client.

21. The system of claim 20 wherein the means for streaming include a skills assessment tool and a video on demand server.

22. The system of claim 21 wherein the means skills assessment tool includes:

an annotation and assessment tool for enabling an end-user to enter annotation and assessment data of a trainee'"'"'s performance while the videos and simulation data are synchronously played back; and

a debrief tool for allowing the end-user to select which recorded training session to view for enabling automated debrief sessions.

23. The system of claim 20 wherein the respective videos are played back in a single interface.

24. The system of claim 23 wherein the single interface is divided into respective windows for each video.

25. The system of claim 24 wherein the streaming further comprises dynamically generating a single media file that references as sources the respective streaming media files, and streaming the single media file.

26. The system of claim 20 wherein the simulator data is recorded as captured video of the simulator itself showing an output of the simulator.

27. The system of claim 26 wherein the video of the simulator data is also encoded as a streaming media file, and both the video of the simulator data and the respective videos are played back in the single interface.

28. The system of claim 20 wherein the simulator data is recorded as telemetry values.

29. The system of claim 28 wherein the simulator data is played back as a trend of the telemetry values along a timeline.

30. The system of claim 29 wherein transition points of the telemetry values are identified during playback.

31. The system of claim 29 wherein an end-user is allowed to selectively control visualization of the telemetry information.

32. The system of claim 29 wherein points of interest are displayed along the timeline.

33. The system of claim 28 wherein the videos are played back on the client in a streaming media player, and playing back the simulator data in a flash player.

34. The system of claim 20 wherein an end-user of the client is allowed to advance or return to any point in time in the synchronous play back of the videos.

35. The system of claim 20 wherein the synchronous playback is streamed in real-time during the training session.

36. The system of claim 20 wherein the simulator data is stored as streaming media files, and streaming the synchronous playback subsequent to the training session.

38. The system of claim 20 wherein during the synchronous playback, a checklist of training tasks to be completed is displayed by a trainee during the trading session, and an end-user is allowed to indicate through the interface which tasks were completed during the synchronous playback for real-time annotation and assessment of the trainee.

40. An executable software product stored on a computer-readable medium containing program instructions for providing synchronous multimedia recording and playback, wherein in response to a training session is being conducted, video of the training session is captured by a plurality of A/V sources, the program instructions for:

synchronously recording with the video, real-time simulator data from a simulator captured by a simulator capture tool;

encoding each of the videos captured by the plurality of A/V sources as respective digital media files formatted as streaming media; and

transmitting both the simulator data and the video media files from a server to a client over a network, such that when the client receives the simulator data and the stream, the respective videos are synchronously played back with the simulator data on the client.