Real-time data acquisition and recording system viewer

• US 10,410,441 B2
• Filed: 05/15/2017
• Issued: 09/10/2019
• Est. Priority Date: 05/16/2016
• Status: Active Grant

• Current Assignee: Wi-Tronix, LLC
• Original Assignee: Wi-Tronix, LLC
• Inventors: Jordan, Lawrence B., Patel, Savankumar V., Weaver, Bryan D.
• Primary Examiner(s): Camby, Richard M
• Application Number:
  US15/595,689
• Publication Number:
  US 20170330398A1
• Time in Patent Office: 848 Days
• Field of Search: None
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0858 wherein the data carrier is removable
• CPC Class Codes: G07C 5/0866 the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/9201 involving the multiplexing of an additional signal and the video signal
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

A data acquisition and recording system (DARS) and viewer for mobile assets that includes a data encoder, onboard data manager, and at least one local memory module. DARS processes video data from at least one 360 degree camera and stores a compressed record of the data at least once per second in the local memory module. DARS is designed to run in near real-time mode, storing a full record comprising five minutes of data to a remote memory module every five minutes, and in real-time mode, streaming video data to the remote memory module by uploading a record of data at least once per second and up to once every tenth of a second. Remotely located users can view video, audio, and data in various view modes through a web browser or virtual reality device, which provides for quicker emergency response, validate the effectiveness of repairs and rerouting, and monitor crew performance and safety.

Claims

1. A method for processing, storing, and transmitting data from at least one mobile asset comprising:

receiving, using a multimedia management system onboard the mobile asset, data based on at least one data signal from at least one of;

at least one 360 degree camera;

at least one fixed camera; and

at least one microphone;

receiving, using a data recorder onboard the mobile asset, the data;

encoding, using a data encoder of the data recorder, video and audio data based on a predetermined amount of data;

storing, using an onboard data manager of the data recorder, at least one of the data and the encoded data at a configurable first predetermined rate in at least one local memory component of the data recorder;

autonomously sending, using the onboard data manager, the encoded data to a remote data manager via a wireless data link at a configurable second predetermined rate, wherein the second predetermining rate is in the range of and including zero and one second; and

storing, using the remote data manager, the encoded data to a remote data repository.

2. The method of claim 1, wherein the data includes at least one of video image information and audio information.

3. The method of claim 1, wherein the at least one 360 degree camera is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset, wherein the at least one fixed camera is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset, and wherein the at least one microphone is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset.

4. The method of claim 1, further comprising:

storing, using the management system, at least one of the data and the encoded data at a configurable second predetermined rate in the at least one local memory component of the data recorder.

5. The method of claim 1, further comprising:

appending, using the onboard data manager, the encoded data to a data segment; and

storing, using the onboard data manager, the data segment at the first predetermined rate in the at least one local memory component of the data recorder.

6. The method of claim 1, wherein the encoded data is stored in one of at least 2 temporary storage locations of the at least one local memory component.

7. The method of claim 1, wherein the at least one local memory component comprises at least one of a crash hardened memory module and a non-crash hardened removable storage device.

9. The method of claim 1, wherein the data recorder receives the data via at least one of a wireless data link and a wired data link.

10. The method of claim 1, further comprising:

time synchronizing, using the data encoder, the encoded data.

11. The method of claim 5, further comprising:

sending, using the onboard data manager, the data segment to a remote data manager via a wireless data link at a configurable second predetermined rate, wherein the second predetermining rate is in the range of and including zero and five minutes; and

storing, using the remote data manager, the data segment to a remote data repository.

18. The method of claim 1, wherein the encoded data comprises a predetermined amount of encoded data, and wherein the predetermined amount of encoded data comprises up to five minutes of encoded data.

38. The method of claim 1, further comprising:

receiving, using a web server in wireless communication with the remote data repository, a request comprising specified multimedia data of the at least one mobile asset and a specified view mode;

receiving, using the web server, the specified multimedia data of the at least one mobile asset from the remote data repository; and

displaying, using a display device, the specified multimedia data of the at least one mobile asset in the specified view.

39. The method of claim 38, wherein the request is one of initiated by a remote user and initiated by an incident occurring with the at least one mobile asset.

40. The method of claim 39, wherein the incident is at least one of an operator of the mobile asset initiating an emergency stop request, emergency braking activity, rapid acceleration of the mobile asset in any axis, rapid deceleration of the mobile asset in any axis, and loss of input power to a data recorder onboard the mobile asset.

41. The method of claim 38, wherein the request is received by the web server via one of a wireless network connection and a wired network connection.

42. The method of claim 39, further comprising:

receiving, using a web client, a command from the user for the request; and

sending, using the web client, the command for the request to the web server.

43. The method of claim 38, wherein the display device is one of a web client and a virtual reality device.

44. The method of claim 38, wherein the specified view is one of fisheye view, dewarped view, panorama view, double panorama view, and quad view.

8. A method for processing, storing, and transmitting data from at least one mobile asset comprising:

receiving, using a multimedia management system onboard the mobile asset, data based on at least one data signal from at least one of;

at least one 360 degree camera;

at least one fixed camera; and

at least one microphone;

receiving, using a data recorder onboard the mobile asset, the data;

encoding, using a data encoder of the data recorder, encoded data based on a predetermined amount of data;

storing, using an onboard data manager of the data recorder, at least one of the data and the encoded data at a configurable first predetermined rate in at least one local memory component of the data recorder;

storing the encoded data in a first temporary storage location of the at least one local memory component on a condition that a previous data segment was stored in a second temporary storage location of the at least one local memory component; and

storing the encoded data in the second temporary storage location of the at least one local memory component on a condition that the previous data segment was stored in the first temporary storage location of the at least one local memory component.

12. A method for processing, storing, and transmitting data from at least one mobile asset comprising:

receiving, using a multimedia management system onboard the mobile asset, data based on at least one data signal from at least one of;

at least one 360 degree camera;

at least one fixed camera; and

at least one microphone;

receiving, using a data recorder onboard the mobile asset, the data;

encoding, using a data encoder of the data recorder, encoded data based on a predetermined amount of data;

storing, using an onboard data manager of the data recorder, at least one of the data and the encoded data at a configurable first predetermined rate in at least one local memory component of the data recorder;

appending, using the onboard data manager, the encoded data to an interim data segment;

storing, using the onboard data manager, the interim data segment at the first predetermined rate in the at least one local memory component of the data recorder;

appending the interim data segment to a data segment; and

storing, using the onboard data manager, the data segment to the at least one local memory component on a condition that the data segment includes a predetermined amount of interim data segments comprising at least one interim data segment.

13. The method of claim 12, further comprising:

sending, using the onboard data manager, the data segment to a remote data manager via a wireless data link at a configurable second predetermined rate.

14. The method of claim 13, wherein the second predetermined rate is in the range of and including zero to five minutes.

15. The method of claim 13, further comprising:

storing, using the remote data manager, the data segment in a remote data repository.

16. The method of claim 13, wherein the data segment is sent to the remote data manager via the wireless data link at the configurable second predetermined rate, wherein the second predetermined rate is in the range of and including zero to one second on a condition that the data recorder is in real-time mode.

17. The method of claim 13, wherein the data segment is sent to the remote data manager via the wireless data link at the configurable second predetermined rate, wherein the second predetermined rate is in the range of and including zero to five minutes on a condition that the data recorder is in near real-time mode.

19. A system for processing, storing, and transmitting data from at least one mobile asset comprising:

at least one of at least one 360 degree camera, at least one fixed camera, and at least one microphone;

a multimedia management system onboard the mobile asset configured to receive data based on at least one data signal from the at least one of the at least one 360 degree camera, at least one fixed camera, and at least one microphone;

a data recorder onboard the mobile asset comprising at least one local memory component, an onboard data manager, and a data encoder, the data recorder configured to receive the data from the multimedia management system;

the data encoder configured to encode encoded data based on the data;

the onboard data manager configured to store at least one of the data and the encoded data at a configurable first predetermined rate in the at least one local memory component;

a remote data manager remote from the mobile asset, the remote data manager configured to autonomously receive a data segment from the onboard data manager via a wireless data link at a configurable second predetermined rate, wherein the second predetermined rate is in the range of and including zero and one second; and

a remote data repository remote from the mobile asset, the remote data repository configured to store the data segment received from the remote data manager.

20. The system of claim 19, wherein the at least one 360 degree camera is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset, wherein the at least one fixed camera is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset, and wherein the at least one microphone is located in at least one of in the mobile asset, on the mobile asset, and in the vicinity of the mobile asset.

21. The system of claim 19, wherein the data includes at least one of video image information and audio information.

22. The system of claim 19, wherein the multimedia management system is configured to store the data in the at least one local memory component.

23. The system of claim 19, wherein the at least one local memory component comprises at least one of a crash hardened memory module and a non-crash hardened removable storage device.

24. The system of claim 19, wherein the onboard data manager is configured to:

append the encoded data to a data segment;

store the data segment at the first predetermined rate in the at least one local memory component.

25. The system of claim 19, wherein the at least one local memory component includes at least two temporary storage locations.

26. The system of claim 25, further comprising:

a first temporary storage location configured to store the data segment on a condition that a previous data segment was stored in a second temporary storage location; and

the second temporary storage location configured to store the data segment on a condition the previous data segment was stored in the first temporary storage location.

27. The system of claim 19, further comprising:

at least one of a wireless data link and a wired data link, the wireless data link and the wired data link configured to send the data from the multimedia management system to the data recorder.

28. The system of claim 24, further comprising:

a remote data manager remote from the mobile asset, the remote data manager configured to receive the data segment from the onboard data manager via a wireless data link at a configurable second predetermined rate, wherein the second predetermined rate is in the range of and including zero and five minutes; and

a remote data repository remote from the mobile asset, the remote data repository configured to store the data segment received from the remote data manager.

32. The system of claim 19, wherein the encoded data comprises a predetermined amount of encoded data and wherein the predetermined amount of encoded data comprises up to and including five minutes of data.

33. The system of claim 19, wherein the encoded data is stored in the at least one local memory component at least once a second.

34. The system of claim 19, wherein the encoded data is stored in the at least one local memory component every tenth of a second.

35. The system of claim 19, further comprising:

a web client comprising a display device;

a web server in wireless communication with the web client, the web server configured to receive a request comprising specified multimedia data relating to the at least one mobile asset and a specified view mode;

the web server in wireless communication with the remote data repository, the web server configured to receive the specified multimedia data relating to the at least one mobile asset; and

the display device configured to display the specified multimedia data relating to the at least one mobile asset in the specified view.

36. The system of claim 35, further comprising:

a virtual reality device configured to receive and display the specified multimedia data relating to the at least one mobile asset in the specified view.

37. The system of claim 35, wherein the specified view is one of fisheye view, dewarped view, panorama view, double panorama view, and quad view.

29. A system for processing, storing, and transmitting data from at least one mobile asset comprising:

at least one of at least one 360 degree camera, at least one fixed camera, and at least one microphone;

a multimedia management system onboard the mobile asset configured to receive data based on at least one data signal from the at least one of the at least one 360 degree camera, at least one fixed camera, and at least one microphone;

a data recorder onboard the mobile asset comprising at least one local memory component, an onboard data manager, and a data encoder, the data recorder configured to receive the data from the multimedia management system;

the data encoder configured to encode encoded data based on the data;

the onboard data manager configured to store at least one of the data and the encoded data at a configurable first predetermined rate in the at least one local memory component;

append the encoded data to an interim data segment;

store the interim data segment at the first predetermined rate in the at least one local memory component of the data recorder;

append the interim data segment to a data segment; and

store the data segment to the at least one local memory component on a condition that the data segment includes a predetermined amount of interim data segments comprising at least one interim data segment.

30. The system of claim 29, further comprising:

a remote data manager remote from the mobile asset, the remote data manager configured to receive the data segment from the onboard data manager via a wireless data link at a configurable second predetermined rate, the data segment comprising at least one data segment;

a remote data repository remote from the mobile asset, the remote data repository configured to store the data segment received from the remote data manager.

31. The system of claim 30, wherein the second predetermined rate is configurable in the range of up to and including zero and one second when the data recorder is in real-time mode and wherein the second predetermined rate is configurable in the range of up to and including zero and five minutes when the data recorder is in near real-time mode.