Post-capture generation of synchronization points for audio to synchronize video portions captured at multiple cameras

• US 8,111,326 B1
• Filed: 05/23/2007
• Issued: 02/07/2012
• Est. Priority Date: 05/23/2007
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Talwar, Abhishek
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): Haiem, Sean
• Application Number:
  US11/805,830
• Time in Patent Office: 1,721 Days
• Field of Search: None
• US Class Current: 348/500
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus

Abstract

Embodiments of the invention relate generally to computing devices and systems, software, computer programs, applications, and user interfaces, and more particularly, to synchronizing portions of video as a function of the post-capture generation of synchronization points for audio, the portions of video being captured at multiple cameras.

Claims

1. A non-transitory computer readable medium storing executable instructions which, when executed by a computer, cause the computer to:

generate a post-capture synchronization point as a function of a subject audio, the post-capture synchronization point specifying equivalent portions of the subject audio, wherein to generate the post-capture synchronization point the executable instructions generate a candidate synchronization point with which to establish the post-capture synchronization point, and to generate the candidate synchronization point the executable instructions;

analyze an attribute of sound associated with multiple audio tracks of the subject audio; and

determine whether the attribute of sound for the multiple audio tracks of the subject audio is substantially equivalent; and

synchronize at least two portions of video substantially at the post-capture synchronization point,

wherein the two portions of video are each captured at different multiple capture devices.

2. The non-transitory computer readable medium of claim 1, wherein the executable instructions to generate the post-capture synchronization point are configured to execute after the two portions of video are captured.

3. The non-transitory computer readable medium of claim 1, wherein to synchronize the at least two portions of video excludes implementing external synchronization information.

4. The non-transitory computer readable medium of claim 1, wherein to determine whether the attribute of sound for the multiple audio tracks of the subject audio is substantially equivalent, the executable instructions:

compare waveform amplitudes of subsets of the multiple audio tracks of the subject audio; and

match the waveform amplitudes to form at least the candidate synchronization point.

5. The non-transitory computer readable medium of claim 1, wherein to determine whether the attribute of sound for the multiple audio tracks of the subject audio is substantially equivalent, the executable instructions:

compare waveform frequencies of subsets of the multiple audio tracks of the subject audio; and

match waveform frequencies to form at least the candidate synchronization point.

6. The non-transitory computer readable medium of claim 1, further comprising executable instructions which, when executed by the computer, cause the computer to:

certify that the candidate synchronization point is useable as the post-capture synchronization point to synchronize the at least two portions of video.

7. The non-transitory computer readable medium of claim 6, wherein to certify, the executable instructions:

identify additional subsets of the multiple audio tracks of the subject audio for each of the two portions of video, the additional subsets defining points of reference;

select other subsets of the multiple audio tracks of the subject audio at approximately the same amount of time units from the additional subsets;

analyze an attribute of sound associated with the other subsets of the multiple audio tracks of the subject audio; and

determine whether the attribute of sound for the other subsets of the multiple audio tracks of the subject audio is substantially equivalent.

8. The non-transitory computer readable medium of claim 1, wherein to synchronize the two portions of video excludes implementing either a synchronization signal or an artificial sound, or both.

9. A non-transitory computer readable medium storing executable instructions which, when executed by a computer, cause the computer to:

extract a reference audio and a specimen audio from content files in a repository;

characterize a portion of the reference audio in terms of an audio attribute to form a characterized portion of the reference audio;

determine whether the specimen audio includes the characterized portion of the reference audio to form a synchronization point; and

synchronize video clips associated with the reference audio and the specimen audio substantially at the synchronization point.

10. The non-transitory computer readable medium of claim 9, wherein the reference audio and the specimen audio include subject audio.

11. The non-transitory computer readable medium of claim 9, further comprising executable instructions which, when executed by the computer, cause the computer to:

edit the video clips associated with the reference audio and the specimen audio to form a video; and

implement as audible sound either the reference audio or the specimen audio, or both, in association with the video.

12. The non-transitory computer readable medium of claim 9, further comprising executable instructions which, when executed by the computer, cause the computer to:

synchronize the video clips based on the synchronization point for the reference audio and the specimen audio.

13. The non-transitory computer readable medium of claim 9, further comprising executable instructions which, when executed by the computer, cause the computer to:

designate a content file of a plurality of content files as including the reference audio as a function of an amount of data associated with the content file.

14. The non-transitory computer readable medium of claim 9, wherein to determine whether the specimen audio includes the characterized portion of the reference audio, the executable instructions:

characterize other portions of the reference audio to form other characterized portions of the reference audio; and

confirm that the specimen audio includes the other characterized portions of the reference audio.

15. The non-transitory computer readable medium of claim 9, further comprising executable instructions which, when executed by a computer, cause the computer to:

define a portion of the specimen audio as a vicinity range;

determine an alignment point for the portion of the reference audio; and

align the vicinity range at the alignment point.

16. The non-transitory computer readable medium of claim 9, wherein to determine whether the specimen audio includes the characterized portion of the reference audio, the executable instructions:

search for a pattern of the audio attribute associated with the characterized portion of the reference audio within a portion of the specimen audio;

establish a tolerance range for the pattern within which the portion of the specimen audio matches the pattern; and

generate a candidate synchronization point if at least a part of the portion of the specimen audio is within the tolerance range.

17. A device comprising:

a panel generator configured to generate one or more panels depicting audio-based synchronization points for synchronizing video; and

an audio synchronization point generator comprising;

a sound attribute analyzer configured to;

analyze waveform amplitudes associated with a first audio track and a second audio track that include a subject audio, and

a candidate synchronization point detector configured to;

detect that a portion of the first audio track and a portion of the second audio track includes substantially equivalent patterns of the waveform amplitudes.

18. The device of claim 17 wherein the audio synchronization point generator is configured to:

synchronize video clips based on the first audio track and the second audio track without synchronization information external to the subject audio.

19. The device of claim 17 wherein the panel generator is further configured to:

present an interactive panel configured to accept synchronization point generation parameters for governing the operation of the audio synchronization point generator.

20. A system comprising:

a memory configured to store instructions and data associated with storing a reference audio and a specimen audio from content files; and

logic configured to;

characterize a portion of the reference audio in terms of an audio attribute to form a characterized portion of the reference audio;

determine whether the specimen audio includes the characterized portion of the reference audio to form a synchronization point; and

synchronize video clips associated with the reference audio and the specimen audio substantially at the synchronization point.

21. A system, comprising:

a memory configured to store instructions and data associated with generating a candidate synchronization point as a function of a subject audio; and

a processor configured to;

generate a post-capture synchronization point as a function of a subject audio, the post-capture synchronization point specifying equivalent portions of the subject audio, wherein to generate the post-capture synchronization point the processor is configured to generate a candidate synchronization point with which to establish the post-capture synchronization point, and to generate the candidate synchronization point the processor is configured to;

analyze an attribute of sound associated with multiple audio tracks of the subject audio; and

determine whether the attribute of sound for the multiple audio tracks of the subject audio is substantially equivalent; and

synchronize at least two portions of video substantially at the post-capture synchronization point,

wherein the two portions of video are each captured at different multiple capture devices.

22. A method comprising:

extracting a reference audio and a specimen audio from content files in a repository;

characterizing a portion of the reference audio in terms of an audio attribute to form a characterized portion of the reference audio;

determining that the specimen audio includes the characterized portion of the reference audio to form a synchronization point; and

synchronizing video clips associated with the reference audio and the specimen audio substantially at the synchronization point.

23. A method, comprising:

generating a post-capture synchronization point as a function of a subject audio, the post-capture synchronization point specifying equivalent portions of the subject audio, wherein generating the post-capture synchronization point comprises generating a candidate synchronization point with which to establish the post-capture synchronization point;

certifying that the candidate synchronization point is useable as the post-capture synchronization point to synchronize the at least two portions of video; and

synchronizing at least two portions of video substantially at the post-capture synchronization point,

wherein the two portions of video are each captured at different multiple capture devices.