Highlight reels

• US 9,652,475 B2
• Filed: 03/31/2014
• Issued: 05/16/2017
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Musatenko, Yuriy, Kasitskyi, Oleksii, Shakib, Babak Robert
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Yang, Nien-Ru
• Application Number:
  US14/231,573
• Publication Number:
  US 20150078732A1
• Time in Patent Office: 1,142 Days
• Field of Search: 386278, 386242, 386267, 386280, 386281, 386283, 386288
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G11B 27/02 Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 1/212 Motion video recording combined with still video recording television signal recording H04N5/76

Abstract

Techniques and apparatuses for generating highlight reels are described. These techniques and apparatuses enable a user to quickly and easily generate a highlight reel to highlight audiovisual media, such as photos and video clips.

Claims

1. A method comprising:

determining, at a computing device, relatedness of video clips within source media, the relatedness being a measure of variety among the video clips;

determining, at the computing device, quality metric values for the video clips by grading the video clips according to quality metrics, the quality metrics including independent metrics and relative metrics, the quality metric values for the independent metrics not based on values given to other video clips and the quality metric values for the relative metrics based at least in part on a quality metric value given to another video clip;

determining, at the computing device, based on the relatedness of the video clips and the quality metric values of the video clips, a subset of the video clips by which to highlight the source media; and

arranging, at the computing device, the subset of the video clips into a highlight reel.

2. The method as recited in claim 1, further comprising determining contexts for the video clips within the source media, and wherein determining the relatedness is based on the determined contexts.

3. The method as recited in claim 2, wherein the contexts determined for the video clips within the source media include the video clips having a scene that is indoors, outdoors, at night, or during daytime.

4. The method as recited in claim 2, wherein the contexts determined for the video clips within the source media include the video clips having been captured at particular times or associated with particular events.

5. The method as recited in claim 2, wherein determining the subset of the video clips assigns a penalty to one of two or more video clips that, based on being captured at particular times, are determined to be captured in chronological proximity, the penalty effective to reduce a likelihood that the one of the two or more video clips will be of the subset.

6. The method as recited in claim 2, wherein determining the subset of the video clips assigns a penalty to one of two or more video clips based on being associated with a same event or a same media album, the penalty effective to reduce a likelihood that the one of the two or more video clips will be of the subset.

7. The method as recited in claim 1, wherein the source media includes a previously created highlight reel, the previously created highlight reel highlighting media at least some of which is not also separately within the source media.

8. The method as recited in claim 1, further comprising determining the source media based on an association between the source media and multiple persons having a same calendar event or an affirmative assent to share media.

9. The method as recited in claim 1, wherein the determining the quality metric values of the independent metrics of the video clips includes, for each of one or more frames within each of the video clips of the source media, determining:

a brightness, a color variance, an overall sharpness, an accumulated-faces area, a head orientation, a face or faces smiling, eye blinks, locations of one or more faces within the frame, eye focal point, an aspect ratio of the frame, a uniqueness, a mistake, or a face blur.

10. The method as recited in claim 1, wherein arranging the subset of the video clips into the highlight reel creates a reel preview, and further comprising enabling review of the reel preview, enabling selection to alter or approve the reel preview and, responsive to receiving approval of the highlight reel, creating a final highlight reel or, responsive to receiving an alteration, altering the reel preview based on the alteration to provide an altered reel preview.

11. The method as recited in claim 10, wherein enabling review of the reel preview presents the reel preview without encoding the reel preview and using sequential subclips of one or more of the video clips, the sequential subclips presented by playing from a start point to a stop point within the respective one or more video clips.

12. The method as recited in claim 1, wherein determining the subset of the video clips by which to highlight the source media is based in part on a selected target duration for the highlight reel and wherein the method continues to determine the subset of the video clips in a greedy manner until the selected target duration is met.

13. The method as recited in claim 1, wherein the quality metric values are calculated based on an average per-frame quality for frames in each of the video clips.

14. The method as recited in claim 1, wherein the quality metric values comprise floating-point values for each image or frame of the video clips.

15. The method as recited in claim 1, wherein the determining the subset of the video clips further comprises ranking the video clips by the quality video metric values and then pulling from the top of the ranked video clips in a greedy manner until a selected target duration is met.

20. The method as recited in claim 1, wherein the determining the relatedness values of the video clips includes, for each of one or more frames within each of the video clips of the source media, determining contextual variety, album source, or time-distance.

16. One or more non-transitory computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, perform operations comprising:

determining, at a computing device, relatedness of video clips within source media, the relatedness being a measure of variety among the video clips;

determining, at the computing device, quality metric values for the video clips by grading the video clips according to quality metrics, the quality metrics including independent metrics and relative metrics, the quality metric values for the independent metrics not based on values given to other video clips and the quality metric values for the relative metrics based at least in part on a quality metric value given to another video clip;

determining, at the computing device, based on the relatedness of the video clips and the quality metric values of the video clips, a subset of the video clips by which to highlight the source media; and

arranging, at the computing device, the subset of the video clips into a highlight reel.

17. The one or more non-transitory computer-readable storage media as recited in claim 16, wherein the operations further comprise determining contexts for the video clips within the source media, and wherein determining the relatedness is based on the determined contexts.

18. The one or more non-transitory computer-readable storage media as recited in claim 16, wherein the determining the quality metric values of the independent metrics of the video clips includes, for each of one or more frames within each of the video clips of the source media, determining:

a brightness, a color variance, an overall sharpness, an accumulated-faces area, a head orientation, a face or faces smiling, eye blinks, locations of one or more faces within the frame, eye focal point, an aspect ratio of the frame, a uniqueness, a mistake, or a face blur.

19. The one or more non-transitory computer-readable storage media as recited in claim 16, wherein arranging the subset of the video clips into the highlight reel creates a reel preview, and further comprising enabling review of the reel preview, enabling selection to alter or approve the reel preview and, responsive to receiving approval of the highlight reel, creating a final highlight reel or, responsive to receiving an alteration, altering the reel preview based on the alteration to provide an altered reel preview.