System for recording and synchronizing audio and video associated with a UAV flight

• US 10,535,372 B2
• Filed: 07/26/2016
• Issued: 01/14/2020
• Est. Priority Date: 07/27/2015
• Status: Active Grant

• Current Assignee: Vantage Robotics, LLC
• Original Assignee: Vantage Robotics, LLC
• Inventors: Fisher, Tobin, Van Niekerk, Johannes Becker
• Primary Examiner(s): Hunter, Mishawn N.
• Application Number:
  US15/220,263
• Publication Number:
  US 20170053674A1
• Time in Patent Office: 1,267 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10546 specifically adapted for audio data
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/3036 Time code signal
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A method for synchronizing an audio track that is being recorded at a first location with a video track that is being recorded at a second location, such as on a flying UAV, is disclosed. First, the audio and video recorders being to record sound and video. Then, a GPS receive is used to pick up the very accurate GPS clock signal. A real-time clock is also used locally to generate a real-time value. At a predetermined time, the GPS time data is interrogated and a time-stamp is generated. A video encoder is then used to embed the time-stamp, either during recording, or shortly thereafter. A networked audio recording device records audio with a time stamp acquired from the network. These respective time-stamps are then used in post-processing to accurately synchronize the audio and video tracks.

Claims

1. A method for encoding time stamp information within a video recorded using an unmanned aerial vehicle (UAV), the method including:

receiving GPS (global positioning system) time information using a GPS receiver on the UAV;

providing the GPS information to a real-time clock on the UAV to trigger the real-time clock;

initializing video recording via a camera supported by the UAV; and

encoding time data from the real-time clock into the video recording using a video encoder on the UAV.

2. The method of claim 1, wherein the time data includes the start time of the UAV video recording, wherein the start time is synchronized with an absolute time reference from the GPS time information.

3. The method of claim 1, wherein triggering the real-time clock comprises:

receiving a GPS pulse signal; and

extracting and storing the GPS time and date information in memory.

4. The method of claim 3, wherein receiving a GPS pulse signal comprises receiving a GPS pulse signal via a GPIO port.

5. The method of claim 3, wherein encoding time data from the real-time clock into the video recording comprises:

generating at least one time stamp using the real-time clock based on the stored GPS time and date information in response to initializing video recording; and

transmitting the at least one time stamp to the video encoder on the UAV.

6. The method of claim 5, wherein initializing video recording comprises sending a request to the real-time clock via a GPIO port.

7. The method of claim 1, wherein encoding time data from the real-time clock into the video recording comprises including time stamp data in a metadata field of the video recording.

8. The method of claim 1, additionally comprising:

receiving an audio recording made at least partially simultaneously with the video recording, wherein the audio recording is synchronized with an absolute time reference; and

merging the video recording with at least a portion of the audio recording using the video encoder on the UAV to generate a video recording including synchronized audio.

9. The method of claim 8, wherein merging the video recording with at least a portion of the audio recording comprises merging the video recording with at least a portion of the audio recording using the video encoder on the UAV.

10. The method of claim 8, additionally comprising initializing the audio recording, wherein the audio recording occurs at a location different than a location of the UAV.

11. The method of claim 10, wherein initializing the video recording via a camera supported by the UAV occurs when the UAV is airborne.

12. The method of claim 8, wherein the audio recording is an audio recording recorded by a device including a GPS receiver, and wherein the audio recording comprises encoded time data extracted from the GPS receiver of the audio recording device, to synchronize the audio recording with the absolute time reference.

13. The method of claim 1, wherein the real-time clock comprises a 16-bit counter and a 32.768 kHz oscillator.