Multi-camera time synchronization

• US 9,848,132 B2
• Filed: 11/24/2015
• Issued: 12/19/2017
• Est. Priority Date: 11/24/2015
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Newman, David A., Johnston, II, Phil
• Primary Examiner(s): Shibru, Helen
• Application Number:
  US14/951,482
• Publication Number:
  US 20170150236A1
• Time in Patent Office: 756 Days
• Field of Search: 386201, 386223, 386279, 386281
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

Video clips and images captured by one device (e.g., a camera) are associated with one or more synchronization labels such as synchronization device labels and synchronization time labels determined by the device. Synchronization device labels can be used to identify devices that are synchronized. Synchronization time labels indicate relative timing between the devices that are synchronized. When a device is on, it transmits a synchronization signal and receives another synchronization signal transmitted by another device in response to receiving the synchronization signal it has transmitted. The two devices each calculate a synchronization device label and a synchronization time label using the synchronization signals and associate the synchronization device label and synchronization time label with video frames and images captured. Video clips and images can subsequently be aligned using the associated synchronization device labels and the synchronization time labels.

Claims

1. A computer-implemented method of editing video clips, comprising:

receiving a first video clip and a second video clip, the first video clip and the second video clip each comprising a series of video frames, individual video frame associated with a synchronization device label and a synchronization time label;

identifying a first set of video frames from the first video clip and a second set of video frames from the second video clip, the first set of video frames and the second set of video frames associated with the same synchronization device label;

selecting a first subset of video frames from the first set of video frames;

identifying a first synchronization time label associated with the first subset of video frames;

selecting a second subset of video frames from the second set of video frames based on the first synchronization time label and a second synchronization time label associated with the second subset of video frames, the first synchronization time label and the second synchronization time label representative of a time difference between a first time corresponding to a first camera that captured the first video clip and a second time corresponding to a second camera that captured the second video clip; and

combining the first subset of video frames and the second subset of video frames to produce an aligned combined video.

2. The computer-implemented method of claim 1, further comprising aligning all video frames of the first video clip and the second video clip that are captured at the same time.

3. The computer-implemented method of claim 1, wherein selecting the second subset of video frames comprises:

determining a first local time point of a first video frame of the first subset of video frames and a second local time point of a second video frame of the second subset of video frames;

converting the first local time point of the first video frame to a corresponding local time point of the second camera based on the first synchronization time label associated with the first video frame; and

identifying the second video frame of the second subset of video frames according to the converted corresponding local time point and the second time local time point of the second camera, the converted corresponding local time point and the second local time point being the same.

4. The computer-implemented method of claim 1, wherein the synchronization device label is generated based on a first identifier identifying the first camera and a second identifier identifying the second camera, the synchronization device label representative of the first identifier and the second identifier.

5. The computer-implemented method of claim 4, wherein the synchronization device label is generated further based on:

a first value determined based on the first identifier;

a second value determined based on the second identifier; and

a combination of the first value and the second value.

6. The computer-implemented method of claim 5, wherein the first value is a first hash value of the first identifier and the second value is a second hash value of the second identifier.

7. The computer-implemented method of claim 5, wherein the first identifier and the second identifier cannot be reversely determined from the synchronization device label.

8. The computer-implemented method of claim 4, wherein the first identifier is conveyed in a first synchronization signal transmitted by the first camera and the second identifier is conveyed in a second synchronization signal transmitted by the second camera, the second synchronization signal transmitted by the second camera in response to the second camera receiving the first synchronization signal.

9. The computer-implemented method of claim 1, wherein the first synchronization time label is generated based on a difference between the first time and the second time.

10. The computer-implemented method of claim 1, wherein the individual video frame is associated with the synchronization device label and the synchronization time label based on the synchronization device label and the synchronization time label being included within metadata of the individual video frame.

11. A system that edits video clips, comprising:

one or more physical processors configured by machine-readable instructions to;

receive a first video clip and a second video clip, the first video clip and the second video clip each comprising a series of video frames, individual video frame associated with a synchronization device label and a synchronization time label;

identify a first set of video frames from the first video clip and a second set of video frames from the second video clip, the first set of video frames and the second set of video frames associated with the same synchronization device label;

select a first subset of video frames from the first set of video frames;

identify a first synchronization time label associated with the first subset of video frames;

select a second subset of video frames from the second set of video frames based on the first synchronization time label and a second synchronization time label associated with the second subset of video frames, the first synchronization time label and the second synchronization time label representative of a time difference between a first time corresponding to a first camera that captured the first video clip and a second time corresponding to a second camera that captured the second video clip; and

combine the first subset of video frames and the second subset of video frames to produce an aligned combined video.

12. The system of claim 11, wherein the one or more physical processors are further configured by the machine-readable instructions to align all video frames of the first video clip and the second video clip that are captured at the same time.

13. The system of claim 11, wherein the one or more physical processors are, to select the second subset of video frames, further configured by the machine-readable instructions to:

determine a first local time point of a first video frame of the first subset of video frames and a second local time point of a second video frame of the second subset of video frames;

convert the first local time point of the first video frame to a corresponding local time point of the second camera based on the first synchronization time label associated with the first video frame; and

identify the second video frame of the second subset of video frames according to the converted corresponding local time point and the second time local time point of the second camera, the converted corresponding local time point and the second local time point being the same.

14. The system of claim 11, wherein the synchronization device label is generated based on a first identifier identifying the first camera and a second identifier identifying the second camera, the synchronization device label representative of the first identifier and the second identifier.

15. The system of claim 14, wherein the synchronization device label is generated further based on:

a first value determined based on the first identifier;

a second value determined based on the second identifier; and

a combination of the first value and the second value.

16. The system of claim 15, wherein the first value is a first hash value of the first identifier and the second value is a second hash value of the second identifier.

17. The system of claim 15, wherein the first identifier and the second identifier cannot be reversely determined from the synchronization device label.

18. The system of claim 14, wherein the first identifier is conveyed in a first synchronization signal transmitted by the first camera and the second identifier is conveyed in a second synchronization signal transmitted by the second camera, the second synchronization signal transmitted by the second camera in response to the second camera receiving the first synchronization signal.

19. The system of claim 11, wherein the first synchronization time label is generated based on a difference between the first time and the second time.

20. The system of claim 11, wherein the individual video frame is associated with the synchronization device label and the synchronization time label based on the synchronization device label and the synchronization time label being included within metadata of the individual video frame.