Systems and methods for audio track selection in video editing

• US 9,838,730 B1
• Filed: 04/07/2016
• Issued: 12/05/2017
• Est. Priority Date: 04/07/2016
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Matias, Joven
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Smith, Stephen
• Application Number:
  US15/093,700
• Time in Patent Office: 607 Days
• Field of Search: 386282, 386285, 386207, 386278, 84625, 715769, 715716, 715723
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally

Abstract

Video clips may be automatically edited to be synchronized for accompaniment by audio tracks. A preliminary version of a video clip may be made up from stored video content. Occurrences of video events within the preliminary version may be determined. A first audio track may include audio event markers. A first revised version of the video clip may be synchronized so that moments within the video clip corresponding to occurrences of video events are aligned with moments within the first audio track corresponding to audio event markers. Other audio tracks similar to the first audio track may be identified based on audio characteristic parameters. Other revised versions of the video clip may be synchronized.

Claims

1. A system that automatically edits video clips to

synchronize accompaniment by similar musical tracks, the system comprising:

one or more non-transitory storage media storing video content and initial instructions defining a preliminary version of a video clip made up from the stored video content, the initial instructions indicating specific portions of the video content to be included in the preliminary version of the video clip and an order in which the specific portions of the video content should be presented, the video clip divided into video segments, the video segments including a first video segment and a second video segment; and

one or more physical processors configured by machine readable instructions to;

determine occurrences of video events within the preliminary version of the video clip, the individual occurrences of video events corresponding to different moments within the preliminary version of the video clip;

access a repository of musical tracks, the repository of musical tracks including a first musical track, a second musical track, a third musical track, a first set of audio event markers, a second set of audio event markers, and a third set of audio event markers, the individual musical tracks including one or more audio event markers, wherein the individual audio event markers of the first set of audio event markers correspond to different moments within the first musical track, the individual audio event markers of the second set of audio event markers correspond to different moments within the second musical track, and the individual audio event markers of the third set of audio event markers correspond to different moments within the third musical track;

effectuate presentation of at least some of the musical tracks on a graphical user interface of a video application for selection by a user to use as accompaniment for the first video segment of the video clip, the at least some of the musical tracks including the first musical track;

determine first revised instructions defining a first revised version of the video clip that is synchronized with the first musical track so that one or more moments within the first video segment of the video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the first musical track corresponding to one or more audio event markers;

responsive to the user'"'"'s selection of the first musical, track as accompaniment for the first video segment of the video clip;

effectuate playback of the first revised version of the video clip along with the first musical track as accompaniment; and

identify other musical tracks similar to the first musical track based on an audio characteristic parameter of the first musical track and audio characteristic parameters of the other musical tracks, the other musical tracks including the second musical track and the third musical track, wherein the audio characteristic parameters define one or more characteristics of the first musical track and the other musical tracks; and

effectuate presentation of the other musical tracks similar to the first musical track on the graphical user interface of the video application for selection by the user to use as accompaniment for the second video segment of the video clip;

register the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip; and

responsive to the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip, remove the musical tracks similar to the third musical track from the graphical user interface based on an audio characteristic parameter of the third musical track and audio characteristics parameters of the musical tracks similar to the third musical track, wherein the musical tracks similar to the third musical track include the second musical track.

2. The system of claim 1, wherein the audio characteristic parameter of the first musical track includes one or more of a beat parameter, a tempo parameter, a rhythm parameter, an instrument parameter, a volume parameter, a vocal parameter, a frequency parameter, a style parameter, a genre parameter, a performer parameter, a composer parameter, and/or a lyric parameter.

3. The system of claim 1, wherein the one or more physical processors are further configured by machine readable instructions to determine the audio characteristic parameter of the first musical track based on user input.

4. The system of claim 1, wherein the one or more physical processors are further configured by machine readable instructions to:

determine an audio filter parameter, the audio filter parameter defining one or more criteria for filtering one or more musical tracks;

determine, by comparing the individual musical tracks with the audio filter parameter, whether the individual musical tracks meet the audio filter parameter; and

effectuate presentation of the musical tracks that meets the audio filter parameter on the graphical user interface of the video application for selection by the user to use as accompaniment for the first video segment of the video clip.

5. The system of claim 4, wherein the audio filter parameter includes one or more of a beat parameter, a tempo parameter, a rhythm parameter, an instrument parameter, a volume parameter, a vocal parameter, a frequency parameter, a style parameter, a genre parameter, a performer parameter, a composer parameter, a lyric parameter, and/or a user-specified audio filter parameter.

6. The system of claim 4, wherein the one or more physical processors are further configured by machine readable instructions to determine the audio filter parameter based on user input.

7. The system of claim 1, wherein the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip is registered based on the user'"'"'s interaction with a down-vote button for the third musical track.

8. The system of claim 1, wherein the one or more physical processors are further configured by machine readable instructions to determine the audio characteristic parameter of the third musical track based on user input.

9. The system of claim 1, wherein the audio characteristic parameter of the first musical track and the audio characteristic parameter of the third musical track are of same type.

10. The system of claim 1, wherein the audio characteristic parameter of the first musical track and the audio characteristic parameter of the third musical track are of different types.

11. A method automatically editing video clips to synchronize accompaniment by similar musical tracks, the method performed by a computing device including one or more physical processors, the method comprising:

accessing, by the computing device, one or more non-transitory storage media storing video content and initial instructions defining a preliminary version of a video clip made up from the stored video content, the initial instructions indicating specific portions of the video content to be included in the preliminary version of the video clip and an order in which the specific portions of the video content should be presented, the video clip divided into video segments, the video segments including a first video segment and a second video segment; and

determining, by the computing device, occurrences of video events within the preliminary version of the video clip, the individual occurrences of video events corresponding to different moments within the preliminary version of the video clip;

accessing, by the computing device, a repository of musical tracks, the repository of musical tracks including a first musical track, a second musical track, a third musical track, a first set of audio event markers, a second set of audio event markers, and a third set of audio event markers, the individual musical tracks including one or more audio event markers, wherein the individual audio event markers of the first set of audio event markers correspond to different moments within the first musical track, the individual audio event markers of the second set of audio event markers correspond to different moments within the second musical track, and the individual audio event markers of the third set of audio event markers correspond to different moments within the third musical track;

effectuating, by the computing device, presentation of at least some of the musical tracks on a graphical user interface of a video application for selection by a user to use as accompaniment for the first video segment of the video clip, the at least some of the musical tracks including the first musical track;

determining, by the computing device, first revised instructions defining a first revised version of the video clip that is synchronized with the first musical track so that one or more moments within the first video segment of the video clip corresponding to one or more occurrences of video events are aligned with one or more moments within the first musical track corresponding to one or more audio event markers;

responsive to the user'"'"'s selection of the first musical track as accompaniment for the first video segment of the video clip;

effectuating, by the computing device, playback of the first revised version of the video clip along with the first musical track as accompaniment; and

identifying, by the computing device, other musical tracks similar to the first musical track based on an audio characteristic parameter of the first musical track and audio characteristic parameters of the other musical tracks, the other musical tracks including the second musical track and the third musical track, wherein the audio characteristic parameters define one or more characteristics of the first musical track and the other musical tracks; and

effectuating, by the computing device, presentation of the other musical tracks similar to the first musical track on the graphical user interface of the video application for selection by the user to use as accompaniment for the second video segment of the video clip;

registering, by the computing device, the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip; and

responsive to the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip, removing, by the computing device, the musical tracks similar to the third musical track from the graphical user interface based on an audio characteristic parameter of the third musical track and audio characteristics parameters of the musical tracks similar to the third musical track, wherein the musical tracks similar to the third musical track include the second musical track.

12. The method of claim 11, wherein the audio characteristic parameter of the first musical track includes one or more of a beat parameter, a tempo parameter, a rhythm parameter, an instrument parameter, a volume parameter, a vocal parameter, a frequency parameter, a style parameter, a genre parameter, a performer parameter, a composer parameter, and/or a lyric parameter.

13. The method of claim 11, further comprising determining, by the computing device, the audio characteristic parameter of the first musical track based on user input.

14. The method of claim 11, further comprising:

determining, by the computing device, an audio filter parameter, the audio filter parameter defining one or more criteria for filtering one or more musical tracks;

determining, by the computing device, by comparing the individual musical tracks with the audio filter parameter, whether the individual musical tracks meet the audio filter parameter; and

effectuating, by the computing device, presentation of the musical tracks that meets the audio filter parameter on the graphical user interface of the video application for selection by the user to use as accompaniment for the first video segment of the video clip.

15. The method of claim 14, wherein the audio filter parameter includes one or more of a beat parameter, a tempo parameter, a rhythm parameter, an instrument parameter, a volume parameter, a vocal parameter, a frequency parameter, a style parameter, a genre parameter, a performer parameter, a composer parameter, a lyric parameter, and/or a user-specified audio filter parameter.

16. The method of claim 14, further comprising determining, by the computing device, the audio filter parameter based on user input.

17. The method of claim 11, wherein the user'"'"'s rejection of the third musical track as accompaniment for the second video segment of the video clip is registered based on the user'"'"'s interaction with a down-vote button for the third musical track.

18. The method of claim 11, further comprising determining, by the computing device, the audio characteristic parameter of the third musical track based on user input.

19. The method of claim 11, wherein the audio characteristic parameter of the first musical track and the audio characteristic parameter of the third musical track are of same type.

20. The method of claim 11, wherein the audio characteristic parameter of the first musical track and the audio characteristic parameter of the third musical track are of different types.