AUTOMATIC SYNCHRONIZATION OF MEDIA CLIPS

• US 20120198317A1
• Filed: 02/02/2011
• Published: 08/02/2012
• Est. Priority Date: 02/02/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Eppolito, Aaron M., Orife, Iroro F.
• Granted Patent: US 8,621,355 B2
• Time in Patent Office: Days
• US Class Current: 715/202
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

For media clips captured by several media recording devices, a novel method for organizing the captured media clips is presented. The method arranges the media clips into sequences according to content data associated with the media clips. The method further analyzes content data of the media clips in each sequence to identify a best candidate media clip from each sequence. The method performs one or more algorithms to determine whether the candidate media clips are related. The algorithms are also used to determine a time offset between the candidate clips. The method then synchronizes the sequences based on the time offset determined by the algorithms.

Claims

1. A method of organizing media clips captured by two media recording devices, the method comprising:

arranging the media clips into sequences according to content data associated with the media clips;

analyzing content data of the media clips in each sequence to identify a best candidate media clip in each sequence to have time overlap;

analyzing the candidate media clips to determine a time offset between the candidate clips; and

synchronizing the sequences according to the determined time offset.

2. The method of claim 1, wherein arranging the media clips into sequences comprises sorting the media clips into the sequences based on recording device identifier data associated with the media clips.

3. The method of claim 2, wherein arranging the media clips into sequences further comprises putting the media clips of each of the sequences into chronological order according to time code data associated with the media clips.

4. The method of claim 2, wherein arranging the media clips into sequences further comprises putting the media clips of each of the sequences into chronological order according to file creation time data associated with the media clips.

5. The method of claim 1, wherein analyzing the candidate media clips comprises correlating audio contents of the candidate media clips, wherein determining the time offset comprises identifying a value that corresponds to a peak value of a correlation function.

6. The method of claim 5, wherein correlating comprises performing a frequency domain phase correlation to normalize out relative amplitudes of the candidate media clips to produce a more accurate result.

7. The method of claim 1, wherein synchronizing the sequences comprises aligning the sequences according to the determined time offset, wherein the time offset corresponds to the peak value of the correlation function when the peak value of the correlation function satisfies a threshold value, wherein the threshold value is based on a statistical analysis of the correlation function performed on the candidate media clips.

8. The method of claim 1, wherein identifying the best candidate media clip comprises identifying a media clip that includes a marker placed by a user to indicate a match.

9. The method of claim 1, wherein identifying the best candidate media clip comprises identifying candidate media clips that include time code data indicating that the candidate media clips were recorded during an overlapping time period.

10. The method of claim 1 further comprising performing a pre-processing on the media clips, the pre-processing comprising filtering out noise, applying equalization to target specific frequencies, and applying normalization and/or dynamics processing to balance levels.

11. The method of claim 1 further comprising performing a data reduction on the media clips, the data reduction comprising reducing the sample rate, skipping silent sections, reducing quiet sections to silence, and picking clip subsections around a marker or a point of likely alignment

12. A method of organizing media clips, the method comprising:

importing a plurality of media clips captured by a plurality of media recording devices during a particular timeframe; and

performing an automated process comprising;

analyzing content data of the media clips to group media clips that temporally overlap; and

placing the groups of media clips in an order indicative of a sequence by which the media clips were captured.

13. The method of claim 12, wherein performing the automated process further comprises aligning the media clips included in each group of media clips according to audio content of the media clips.

14. The method of claim 12, wherein analyzing content data of the media clips comprises:

determining recording device identifier data and temporal data associated with the media clips;

sorting the media clips into sequences based on recording device identifier data; and

putting the media clips of each of the sequences into chronological order according to temporal data associated with the media clips.

15. The method of claim 14, wherein analyzing content data of the media clips further comprises:

determining a time offset between the two sorted sequences by (i) correlating audio contents of a media clip from each of the two sorted sequences to produce a correlation function and (ii) identifying a resulting peak value of the correlation function that corresponds to the time offset when the peak value of the correlation function satisfies a threshold value, wherein the threshold value is based on a statistical analysis of the correlation function between the audio contents of the media clips; and

aligning the two sorted sequences based on the determined time offset.

16. The method of claim 12, wherein grouping media clips that temporally overlap comprises:

identifying media clips from a first sequence and a second sequence that are time-coincident media clips, the two media clips being time-coincident media clips when subsections of the two media clips record the same event during a same time period; and

placing the two media clips into a single contents structure.

17. The method of claim 12, wherein the particular timeframe is a user determined timeframe representing a duration of a recording session.

18. The method of claim 12, wherein each of a plurality of media clips to be imported and automatically processed is determined by detecting a recording session from a plurality of recording sessions from which the plurality of media clips captured by a plurality of media recording devices during a particular timeframe originate.

19. The method of claim 12, wherein when media clips that temporally overlap are grouped, the grouped media clips are formed into multi-clips that represent time-coincident media clips of a same recorded event.

20. A method of organizing media clips, the method comprising:

arranging a plurality of media clips captured by a plurality of media recording devices into sequences according to metadata associated with each of the media clips;

determining a time offset between a first sequence and a second sequence by comparing audio content of media clips from the first sequence to media clips from the second sequence;

aligning the first sequence and the second sequence based on the determined time offset; and

merging media clips from the first and second sequences determined to be temporally overlapping.

21. The method of claim 20 further comprising placing the merged media clips in an order indicative of a sequence by which the media clips were captured.

22. The method of claim 20 further comprising temporally aligning the merged media clips according to audio content of the merged media clips.

23. The method of claim 22, wherein temporally aligning the merged media clips comprises aligning the merged media based on a time offset determined by correlating audio contents of the merged media clips.

24. The method of claim 23, wherein correlating audio contents of the merged media clips comprises performing a frequency domain phase correlation of the merged clips to generate a correlation function, wherein the time offset is determined as a value corresponding to a resulting peak value of the correlation function when the peak value satisfies a threshold value, wherein the threshold value is based on a statistical analysis of the correlation function performed on the candidate media clips.

25. The method of claim 20 further comprising adjusting the alignment of the first sequence and the second sequence to compensate for time drift.

26. The method of claim 25, wherein adjusting the alignment comprises:

calculating a difference between a first offset of a first pair of overlapping media clips and a second offset of a second pair of overlapping media clips; and

adjusting a temporal position of the media clips on the first sequence and the second sequence based on the calculated difference of the first and second offsets.

27. The method of claim 20 further comprising adjusting a first media clip of the merged media clips to compensate for time drift.

28. The method of claim 27, wherein the first media clip of the merged media clips is from the first sequence, wherein adjusting the alignment comprises:

calculating a difference between a first offset of a first subsection of each of the first media clip and a second media clip and a second offset of a second subsection of each of the first media clip and the second media clip, the second media clip being of the merged media clips and from the second sequence; and

adjusting content data of the first media clip based on the calculated difference of the first and second offsets.