Generating video summaries for a video using video summary templates

• US 9,792,502 B2
• Filed: 10/13/2014
• Issued: 10/17/2017
• Est. Priority Date: 07/23/2014
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Newman, David A., Schunck, Brian G.
• Primary Examiner(s): MCINTOSH, ANDREW T
• Application Number:
  US14/513,150
• Publication Number:
  US 20160029105A1
• Time in Patent Office: 1,100 Days
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/179 metadata assisted face recognition
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G11B 27/002 Programmed access in sequence to a plurality of record carriers or indexed parts, e.g. tracks, thereof, e.g. for editing; transfer of record carriers from magazine G11B15/68, G11B17/10; G11B17/22
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/13 the information being derived from movement of the record carrier, e.g. using tachometer
• CPC Class Codes: G11B 27/22 Means responsive to presence or absence of recorded information signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/3081 used signal is a video-frame or a video-field (P.I.P)
• CPC Class Codes: G11B 27/34 : Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 13/172 : image signals comprising non-image signal components, e.g. headers or format information
• CPC Class Codes: H04N 13/178 : Metadata, e.g. disparity information
• CPC Class Codes: H04N 19/513 : Processing of motion vectors
• CPC Class Codes: H04N 21/2353 : specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/8549 : Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 2201/3228 : further additional information (metadata) being comprised in the identification information
• CPC Class Codes: H04N 2201/3229 : further additional information (metadata) being comprised in the file name (including path, e.g. directory or folder names at one or more higher hierarchical levels)
• CPC Class Codes: H04N 2201/3256 : colour related metadata, e.g. colour, ICC profiles
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 5/77 : between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/772 : the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/91 : Television signal processing therefor
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal

Abstract

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. A video summary can be generated including one or more of the identified best scenes. The video summary can be generated using a video summary template with slots corresponding to video clips selected from among sets of candidate video clips. Best scenes can also be identified by receiving an indication of an event of interest within video from a user during the capture of the video. Metadata patterns representing activities identified within video clips can be identified within other videos, which can subsequently be associated with the identified activities.

Claims

1. A computer-implemented method for generating a video summary using a video summary template, the method comprising:

receiving, by one or more processors, a request for the video summary of multiple videos;

segmenting the videos into multiple video clips such that at least two of the video clips are segmented from a common one of the videos;

obtaining the video summary template, the video summary template comprising a plurality of template slots, the plurality of template slots including a first template slot and a second template slot, wherein individual template slots are characterized by one or more video clip criteria and one or more quality video clip criteria specific to the individual template slots such that the first template slot is characterized by a first video clip criteria and a first quality video clip criteria and the second template slot is characterized by a second video clip criteria and a second quality video clip criteria;

identifying, for the individual template slots, different sets of candidate video clips from the video clips that satisfy the one or more video clip criteria of the individual template slots;

selecting, for the individual template slots, individual video clips from the sets of candidate video clips based on the one or more quality video clip criteria of the individual template slots, wherein the one or more quality video clip criteria used to select a video clip for a given template slot includes at least one criterion that is not included in the one or more video clip criteria for the given template slot; and

generating the video summary by placing the selected video clips in the corresponding template slots in the video summary template.

2. The method of claim 1, wherein the video summary template is obtained based on the videos or the video clips.

3. The method of claim 1, wherein at least one of the video clip criteria for the given template slot includes a video clip length, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips having the specified video clip length.

4. The method of claim 1, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips captured by a camera moving in the particular motion.

5. The method of claim 1, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips including a user of a camera moving in the particular motion, the one or more video clips captured by the camera.

6. The method of claim 1, wherein selecting the individual video clips from the sets of candidate video clips comprises:

presenting the sets of candidate video clips to a user; and

receiving, from the user, a selection of at least one of the video clips from the sets of candidate video clips.

19. The method of claim 1, further comprising responsive to none of the video clips satisfying the one or more video clip criteria or the one or more quality video clip criteria of at least one of the individual template slots, removing the at least one of the individual template slots from the video summary template such that responsive to none of the video clips satisfying the one or more video clip criteria and the one or more quality video criteria of the first template slot, the first template slot is removed from the video summary template and no video clip is placed in the first template slot to generate the video summary.

7. A system for generating a video summary using a video summary template, the system comprising:

one or more physical processors configured by computer-readable instructions to;

receive a request for the video summary of multiple videos;

segment the videos into multiple video clips such that at least two of the video clips are segmented from a common one of the videos;

obtain the video summary template, the video summary template comprising a plurality of template slots, the plurality of template slots including a first template slot and a second template slot, wherein individual template slots are characterized by one or more video clip criteria and one or more quality video clip criteria specific to the individual template slots such that the first template slot is characterized by a first video clip criteria and a first quality video clip criteria and the second template slot is characterized by a second video clip criteria and a second quality video clip criteria;

identify, for the individual template slots, different sets of candidate video clips from the video clips that satisfy the one or more video clip criteria of the individual template slots;

select, for the individual template slots, individual video clips from the sets of candidate video clips based on the one or more quality video clip criteria of the individual template slots, wherein the one or more quality video clip criteria used to select a video clip for a given template slot includes at least one criterion that is not included in the one or more video clip criteria for the given template slot; and

generate the video summary by placing the selected video clips in the corresponding template slots in the video summary template.

8. The system of claim 7, wherein the video summary template is obtained based on the videos or the video clips.

9. The system of claim 7, wherein at least one of the video clip criteria for the given template slot includes a video clip length, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips having the specified video clip length.

10. The system of claim 7, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips captured by a camera moving in the particular motion.

11. The system of claim 7, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips including a user of a camera moving in the particular motion, the one or more video clips captured by the camera.

12. The system of claim 7, wherein selecting the individual video clips from the sets of candidate video clips comprises:

presenting the sets of candidate video clips to a user; and

receiving, from the user, a selection of at least one of the video clips from the sets of candidate video clips.

20. The system of claim 7, wherein the one or more physical processors are further configured by computer-readable instructions to:

responsive to none of the video clips satisfying the one or more video clip criteria or the one or more quality video clip criteria of at least one of the individual template slots, remove the at least one of the individual template slots from the video summary template such that responsive to none of the video clips satisfying the one or more video clip criteria and the one or more quality video criteria of the first template slot, the first template slot is removed from the video summary template and no video clip is placed in the first template slot to generate the video summary.

13. A non-transitory computer-readable storage medium storing instructions for generating a video summary using a video summary template, the instructions, when executed by one or more physical processors, configured to:

receive a request for the video summary of multiple videos;

segment the videos into multiple video clips such that at least two of the video clips are segmented from a common one of the videos;

obtain the video summary template, the video summary template comprising a plurality of template slots, the plurality of template slots including a first template slot and a second template slot, wherein individual template slots are characterized by one or more video clip criteria and one or more quality video clip criteria specific to the individual template slots such that the first template slot is characterized by a first video clip criteria and a first quality video clip criteria and the second template slot is characterized by a second video clip criteria and a second quality video clip criteria;

identify, for the individual template slots, different sets of candidate video clips from the video clips that satisfy the one or more video clip criteria of the individual template slots;

select, for the individual template slots, individual video clips from the sets of candidate video clips based on the one or more quality video clip criteria of the individual template slots, wherein the one or more quality video clip criteria used to select a video clip for a given template slot includes at least one criterion that is not included in the one or more video clip criteria for the given template slot; and

generate the video summary by placing the selected video clips in the corresponding template slots in the video summary template.

14. The computer-readable storage medium of claim 13, wherein the video summary template is obtained based on the videos or the video clips.

15. The computer-readable storage medium of claim 13, wherein at least one of the video clip criteria for the given template slot includes a video clip length, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips having the specified video clip length.

16. The computer-readable storage medium of claim 13, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips captured by a camera moving in the particular motion.

17. The computer-readable storage medium of claim 13, wherein at least one of the video clip criteria for the given template slot includes a particular motion, and wherein the set of candidate video clips identified for the given template slot includes one or more video clips including a user of a camera moving in the particular motion, the one or more video clips captured by the camera.

18. The computer-readable storage medium of claim 13, wherein selecting the individual video clips from the sets of candidate video clips comprises:

presenting the sets of candidate video clips to a user; and

receiving, from the user, a selection of at least one of the video clips from the sets of candidate video clips.

21. The computer-readable storage medium of claim 13, wherein the instructions, when executed by the one or more physical processors, are further configured to:

responsive to none of the video clips satisfying the one or more video clip criteria or the one or more quality video clip criteria of at least one of the individual template slots, remove the at least one of the individual template slots from the video summary template such that responsive to none of the video clips satisfying the one or more video clip criteria and the one or more quality video criteria of the first template slot, the first template slot is removed from the video summary template and no video clip is placed in the first template slot to generate the video summary.