Systems and methods for audio track selection in video editing with audio mixing option

• US 9,838,731 B1
• Filed: 04/07/2016
• Issued: 12/05/2017
• Est. Priority Date: 04/07/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Matias, Joven
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Smith, Stephen
• Application Number:
  US15/093,701
• Time in Patent Office: 607 Days
• Field of Search: 386285, 386207, 386278, 386282, 715769, 715716, 715723, 84625
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally

Abstract

Video clips may be automatically edited to be synchronized for accompaniment by audio tracks. A preliminary version of a video clip may be made up from stored video content. Occurrences of video events within the preliminary version may be determined. A first audio track may include audio event markers. A first revised version of the video clip may be synchronized so that moments within the video clip corresponding to occurrences of video events are aligned with moments within the first audio track corresponding to audio event markers. Presentation of an audio mixing option may be effectuated on a graphical user interface of a video application for selection by a user. The audio mixing option may define volume at which the first audio track is played as accompaniment for the video clip.

Claims

1. A system that automatically edits video clips to synchronize accompaniment by different musical tracks, the system comprising:

one or more non-transitory storage media storing video content and initial instructions defining a preliminary version of an edited video clip made up from the stored video content, the initial instructions indicating specific portions of the video content to be included in the preliminary version of the edited video clip and an order in which the specific portions of the video content should be presented; and

one or more physical processors configured by machine readable instructions to;

determine occurrences of video events within the preliminary version of the edited video clip, the individual occurrences of video events corresponding to different moments within the preliminary version of the edited video clip;

access a repository of musical tracks, the repository of musical tracks including a first musical track, a second musical track, a first set of audio event markers associated with the first musical track, and a second set of audio event markers associated with the second musical track, individual sets of audio event markers including one or more audio event markers, the individual audio event markers associated with the first musical track corresponding to different moments within the first musical track and the individual audio event markers associated with the second musical track corresponding to different moments within the second musical track;

effectuate presentation of two or more of the musical tracks on a graphical user interface of a video application for selection by a user to use as accompaniment for the edited video clip;

determine revised instructions defining revised versions of the edited video clip that are synchronized with the two or more of the musical tracks presented on the graphical user interface such that first revised instructions define a first revised version of the edited video clip that is synchronized with the first musical track so that one or more moments within the edited video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the first musical track corresponding to one or more of the audio event markers of the first musical track and second revised instructions define a second revised version of the edited video clip that is synchronized with the second musical track so that one or more moments within the edited video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the second musical track corresponding to one or more of the audio event markers of the second musical track, wherein the revised instructions are determined prior to the user'"'"'s selection of one or more of the musical tracks;

effectuate presentation of an audio mixing option on the graphical user interface of the video application for selection by the user, the audio mixing option defining volume at which the musical tracks are played as accompaniment for the edited video clip;

responsive to the user'"'"'s selection of the first musical track, effectuate playback of the first revised version of the edited video clip along with the first musical track as accompaniment, the first musical track played at the volume defined by the user selection of the audio mixing option; and

responsive to the user'"'"'s selection of the second musical track, effectuate playback of the second revised version of the edited video clip along with the second musical track as accompaniment, the second musical track played at the volume defined by the user selection of the audio mixing option.

2. The system of claim 1, wherein the audio mixing option includes an option to set a percentage of the volume at which the musical tracks are played as accompaniment for the edited video clip.

3. The system of claim 2, wherein the edited video clip includes an audio, and an inverse relationship exists between volume at which the audio of the edited video clip is played and the volume at which the musical tracks are played.

4. The system of claim 3, wherein a hundred percent of the volume for the musical tracks corresponds to a zero percent of the volume for the audio of the edited video clip.

5. The system of claim 3, wherein a zero percent of volume for the musical track corresponds to a hundred percent of volume for the audio of the edited video clip.

6. The system of claim 3, wherein a fifty percent of volume for the musical track corresponds to a fifty percent of volume for the audio of the edited video clip.

7. The system of claim 1, wherein the audio mixing option includes an option to decrease the volume at which the musical tracks are played at a moment in the edited video clip corresponding to a moment of interest.

8. The system of claim 7, wherein the edited video clip includes an audio, and an inverse relationship exists between volume at which the audio of the edited video clip is played and the volume at which the musical tracks are played.

9. The system of claim 7, wherein the moment in the edited video clip corresponding to the moment of interest includes a single point in time corresponding to the moment of interest and a first duration of time preceding the single point in time corresponding to the moment of interest.

10. The system of claim 9, wherein the moment in the edited video clip corresponding to the moment of interest further includes a second duration of time following the single point in time corresponding to the moment of interest.

11. A method for automatically editing video clips to synchronize accompaniment by different musical tracks, the method comprising:

accessing one or more non-transitory storage media storing video content and initial instructions defining a preliminary version of an edited video clip made up from the stored video content, the initial instructions indicating specific portions of the video content to be included in the preliminary version of the edited video clip and an order in which the specific portions of the video content should be presented; and

determining occurrences of video events within the preliminary version of the edited video clip, the individual occurrences of video events corresponding to different moments within the preliminary version of the edited video clip;

accessing a repository of musical tracks, the repository of musical tracks including a first musical track, a second musical track, a first set of audio event markers associated with the first musical track, and a second set of audio event markers associated with the second musical track, individual sets of audio event markers including one or more audio event markers, the individual audio event markers associated with the first musical track corresponding to different moments within the first musical track and the individual audio event markers associated with the second musical track corresponding to different moments within the second musical track;

effectuating presentation of two or more of the musical tracks on a graphical user interface of a video application for selection by a user to use as accompaniment for the edited video clip;

determining revised instructions defining revised versions of the edited video clip that are synchronized with the two or more of the musical tracks presented on the graphical user interface such that first revised instructions define a first revised version of the edited video clip that is synchronized with the first musical track so that one or more moments within the edited video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the first musical track corresponding to one or more of the audio event markers of the first musical track and second revised instructions define a second revised version of the edited video clip that is synchronized with the second musical track so that one or more moments within the edited video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the second musical track corresponding to one or more of the audio event markers of the second musical track, wherein the revised instructions are determined prior to the user'"'"'s selection of one or more of the musical tracks;

effectuating presentation of an audio mixing option on the graphical user interface of the video application for selection by the user, the audio mixing option defining volume at which the musical tracks are played as accompaniment for the edited video clip;

responsive to the user'"'"'s selection of the first musical track, effectuating playback of the first revised version of the edited video clip along with the first musical track as accompaniment, the first musical track played at the volume defined by the user selection of the audio mixing option; and

responsive to the user'"'"'s selection of the second musical track, effectuating playback of the second revised version of the edited video clip along with the second musical track as accompaniment, the second musical track played at the volume defined by the user selection of the audio mixing option.

12. The method of claim 11, wherein the audio mixing option includes an option to set a percentage of the volume at which the musical tracks are played as accompaniment for the edited video clip.

13. The method of claim 12, wherein the edited video clip includes an audio, and an inverse relationship exists between volume at which the audio of the edited video clip is played and the volume at which the musical tracks are played.

14. The method of claim 13, wherein a hundred percent of the volume for the musical tracks corresponds to a zero percent of the volume for the audio of the edited video clip.

15. The method of claim 13, wherein a zero percent of volume for the musical track corresponds to a hundred percent of volume for the audio of the edited video clip.

16. The method of claim 13, wherein a fifty percent of volume for the musical track corresponds to a fifty percent of volume for the audio of the edited video clip.

17. The method of claim 11, wherein the audio mixing option includes an option to decrease the volume at which the musical tracks are played at a moment in the edited video clip corresponding to a moment of interest.

18. The method of claim 17, wherein the edited video clip includes an audio, and an inverse relationship exists between volume at which the audio of the edited video clip is played and the volume at which the musical tracks are played.

19. The method of claim 17, wherein the moment in the edited video clip corresponding to the moment of interest includes a single point in time corresponding to the moment of interest and a first duration of time preceding the single point in time corresponding to the moment of interest.

20. The method of claim 19, wherein the moment in the edited video clip corresponding to the moment of interest further includes a second duration of time following the single point in time corresponding to the moment of interest.