Data logging in aerial platform

• US 10,643,406 B2
• Filed: 04/18/2017
• Issued: 05/05/2020
• Est. Priority Date: 04/20/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Arya, Ashwani, Fong, Gary
• Primary Examiner(s): Whittington, Jess
• Application Number:
  US15/490,876
• Publication Number:
  US 20170309088A1
• Time in Patent Office: 1,113 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/293 Foldable or collapsible rotors or rotor supports
• CPC Class Codes: G01R 31/382 Arrangements for monitoring battery or accumulator variables, e.g. SoC
• CPC Class Codes: G04R 20/02 the radio signal being sent by a satellite, e.g. GPS
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/085 using electronic data carriers
• CPC Class Codes: H04L 1/00 Arrangements for detecting or preventing errors in the information received correcting synchronisation H04L7/00
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/143 Termination or inactivation of sessions, e.g. event-controlled end of session
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/14 Multichannel or multilink protocols
• CPC Class Codes: H04L 69/16 : Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 : Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 69/40 : for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control
• CPC Class Codes: H04W 4/38 : for collecting sensor information

Abstract

An unmanned aerial vehicle manages storage of data and transfer between other connected devices. The unmanned aerial vehicle captures sensor data from sensors on the unmanned aerial vehicle. The unmanned aerial vehicle transfers the captured sensor data from the unmanned aerial vehicle to a remote controller via a wireless interface. The captured data may be transferred via a TCP link, a UDP link, or a combination thereof. If a loss of link is detected, the captured sensor data is stored to a buffer and a battery level of the unmanned aerial vehicle and a flight status of the unmanned aerial vehicle is monitored. The stored sensor data is transferred from the buffer to a non-volatile storage responsive to the battery level dropping below a predefined threshold or detecting that the unmanned aerial vehicle is stationary and a shutdown may be imminent.

Claims

1. A method for managing data in an unmanned aerial vehicle, the method comprising:

capturing sensor data from sensors on the unmanned aerial vehicle;

transferring the captured sensor data from the unmanned aerial vehicle to a remote controller via a wireless interface;

detecting a loss of a link to a remote controller via the wireless interface;

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a flight status of the unmanned aerial vehicle, the flight status indicating whether the aerial vehicle is in motion or stationary; and

transferring the stored sensor data from the buffer to a non-volatile storage responsive to detecting that the unmanned aerial vehicle is stationary following the loss of the link.

2. The method of claim 1, further comprising:

capturing system state information relating to a state of the unmanned aerial vehicle;

transferring the system state information from the unmanned aerial vehicle to the remote controller via the wireless interface;

responsive to detecting the loss of the link, storing the captured system state information to the buffer; and

transferring the stored system state information from the buffer to the non-volatile storage responsive to the battery level dropping below the predefined threshold.

3. The method of claim 1, further comprising:

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a battery level of the unmanned aerial vehicle; and

transferring the stored sensor data from the buffer to the non-volatile storage responsive to the battery level dropping below a predefined threshold.

4. The method of claim 1, further comprising:

responsive to detecting the loss of the link, monitoring a flight status of the unmanned aerial vehicle, the flight status indicating whether the aerial vehicle is in motion or stationary;

responsive to the unmanned aerial vehicle being in motion following the loss of the link, determining if the buffer is full of untransferred data; and

transferring the untransferred data to the non-volatile storage responsive to the buffer being full.

5. The method of claim 1, further comprising:

determining an available bandwidth for transferring data between the unmanned aerial vehicle and the remote controller;

selecting, based on the available bandwidth, a wireless interface for transferring a data packet to a remote controller, the wireless interface selected between a TCP interface and a UDP interface; and

transferring the sensor data from the buffer to the remote controller via the selected wireless interface.

6. The method of claim 5, further comprising:

responsive to detecting that the available bandwidth exceeds a first threshold, transferring the data packet in duplicate over both the TCP interface and the UDP interface.

7. The method of claim 5, further comprising:

responsive to detecting that the available bandwidth does not exceed a first threshold, transferring the data packet over the UDP interface only.

8. The method of claim 1, wherein capturing the sensor data comprises:

determining an importance factor for the sensor data;

selectively compressing the sensor data based on a compression level dependent on the importance factor.

9. The method of claim 1, further comprising:

synchronizing a local clock of the unmanned aerial vehicle; and

timestamping the sensor data using the local clock when storing the sensor data to the buffer.

10. The method of claim 9, wherein synchronizing the clock comprises:

synchronizing the local clock to a GPS clock from a GPS device integrated in the unmanned aerial vehicle when a GPS connection is available to the GPS device;

responsive to the GPS connection being unavailable, synchronizing the local clock to a controller clock on the remote controller.

11. A non-transitory computer readable storage medium storing instructions for managing data in an unmanned aerial vehicle, the instructions when executed by a processor causing the processor to perform steps including:

capturing sensor data from sensors on the unmanned aerial vehicle;

transferring the captured sensor data from the unmanned aerial vehicle to a remote controller via a wireless interface;

detecting a loss of a link to a remote controller via the wireless interface;

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a flight status of the unmanned aerial vehicle, the flight status indicating whether the aerial vehicle is in motion or stationary; and

transferring the stored sensor data from the buffer to a non-volatile storage responsive to detecting that the unmanned aerial vehicle is stationary following the loss of the link.

12. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

capturing system state information relating to a state of the unmanned aerial vehicle;

transferring the system state information from the unmanned aerial vehicle to the remote controller via the wireless interface;

responsive to detecting the loss of the link, storing the captured system state information to the buffer; and

transferring the stored system state information from the buffer to the non-volatile storage responsive to the battery level dropping below the predefined threshold.

13. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a battery level of the unmanned aerial vehicle; and

transferring the stored sensor data from the buffer to the non-volatile storage responsive to the battery level dropping below a predefined threshold.

14. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

responsive to detecting the loss of the link, monitoring a flight status of the unmanned aerial vehicle, the flight status indicating whether the aerial vehicle is in motion or stationary;

responsive to the unmanned aerial vehicle being in motion following the loss of the link, determining if the buffer is full of untransferred data; and

transferring the untransferred data to the non-volatile storage responsive to the buffer being full.

15. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

determining an available bandwidth for transferring data between the unmanned aerial vehicle and the remote controller;

selecting, based on the available bandwidth, a wireless interface for transferring a data packet to a remote controller, the wireless interface selected between a TCP interface and a UDP interface; and

transferring the sensor data from the buffer to the remote controller via the selected wireless interface.

16. The non-transitory computer readable storage medium of claim 15, wherein the instructions when executed further cause the processor to perform steps including:

responsive to detecting that the available bandwidth exceeds a first threshold, transferring the data packet in duplicate over both the TCP interface and the UDP interface.

17. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

responsive to detecting that the available bandwidth does not exceed a first threshold, transferring the data packet over the UDP interface only.

18. The non-transitory computer readable storage medium of claim 11, wherein the instructions when executed further cause the processor to perform steps including:

synchronizing a local clock of the unmanned aerial vehicle; and

timestamping the sensor data using the local clock when storing the sensor data to the buffer.

19. An aerial vehicle system comprising:

a processor; and

a non-transitory computer readable storage medium storing instructions for managing data in the aerial vehicle system, the instructions when executed by a processor causing the processor to perform steps including;

capturing sensor data from sensors on the unmanned aerial vehicle;

transferring the captured sensor data from the unmanned aerial vehicle to a remote controller via a wireless interface;

detecting a loss of a link to a remote controller via the wireless interface;

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a flight status of the unmanned aerial vehicle, the flight status indicating whether the aerial vehicle is in motion or stationary; and

transferring the stored sensor data from the buffer to a non-volatile storage responsive to detecting that the unmanned aerial vehicle is stationary following the loss of the link.

20. The aerial vehicle system of claim 19, wherein the instructions when executed further cause the processor to perform steps including:

responsive to detecting the loss of the link, storing the captured sensor data to a buffer and monitoring a battery level of the unmanned aerial vehicle; and

transferring the stored sensor data from the buffer to the non-volatile storage responsive to the battery level dropping below a predefined threshold.