Highlight Reels

• US 20150078732A1
• Filed: 03/31/2014
• Published: 03/19/2015
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Musatenko, Yuriy, Shakib, Babak Robert, Kasitskyi, Oleksii
• Granted Patent: US 9,652,475 B2
• Time in Patent Office: Days
• US Class Current: 386/278
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G11B 27/02 Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 1/212 Motion video recording combined with still video recording television signal recording H04N5/76

Abstract

Techniques and apparatuses for generating highlight reels are described. These techniques and apparatuses enable a user to quickly and easily generate a highlight reel to highlight audiovisual media, such as photos and video clips.

Claims

1. A method comprising:

determining, at a computing device, contextual variety of video clips within source media;

determining, at the computing device, based on the contextual variety of the video clips and quality metric values of the video clips, a subset of the video clips by which to highlight the source media; and

arranging, at the computing device, the subset of the video clips into a highlight reel.

2. The method as recited in claim 1, further comprising determining contexts for the video clips within the source media, and wherein determining the contextual variety is based on the determined contexts.

3. The method as recited in claim 2, wherein the contexts determined for the video clips within the source media include the video clips having a scene that is indoors, outdoors, at night, or during daytime.

4. The method as recited in claim 2, wherein the contexts determined for the video clips within the source media include the video clips having been captured at particular times or associated with particular events.

5. The method as recited in claim 2, wherein determining the subset of the video clips assigns a penalty to one of two or more video clips that, based on being captured at particular times, are determined to be captured in chronological proximity, the penalty effective to reduce a likelihood that the one of the two or more video clips will be of the subset.

6. The method as recited in claim 2, wherein determining the subset of the video clips assigns a penalty to one of two or more video clips based on being associated with a same event or a same media album, the penalty effective to reduce a likelihood that the one of the two or more video clips will be of these subset.

7. The method as recited in claim 1, wherein the source media includes a previously created highlight reel, the previously created highlight reel highlighting media at least some of which is not also separately within the source media.

8. The method as recited in claim 1, further comprising determining the source media based on an association between the source media and multiple persons having a same calendar event or an affirmative assent to share media.

9. The method as recited in claim 1, further comprising determining the quality metric values of the video clips, the determining including, for each of one or more frames within each of the video clips of the source media:

a brightness, a color variance, an overall sharpness, an accumulated-faces area, a head orientation, a face or faces smiling, eye blinks, locations of one or more faces within the frame, eye focal point, an aspect ratio of the frame, a uniqueness, a mistake, or a face blur.

10. The method as recited in claim 1, wherein arranging the subset of the video clips into the highlight reel creates a reel preview, and further comprising enabling review of the reel preview, enabling selection to alter or approve the reel preview and, responsive to receiving approval of the highlight reel, creating a final highlight reel or, responsive to receiving an alteration, altering the reel preview based on the alteration to provide an altered reel preview.

11. The method as recited in claim 10, wherein creating the final highlight reel encodes the subset of the video clips included within the final highlight reel and saves metadata for the final highlight reel, the metadata usable for creation of a future highlight reel using the final highlight reel as future source media.

12. The method as recited in claim 10, wherein enabling review of the reel preview presents the reel preview without encoding the reel preview and using sequential subclips of one or more of the video clips, the sequential subclips presented by playing from a start point to a stop point within the respective one or more video clips.

13. The method as recited in claim 1, wherein determining the subset of the video clips by which to highlight the source media is based in part on a selected target duration for the highlight reel and wherein the method continues to determine the subset of the video clips in a greedy manner until the selected target duration is met.

14. A method comprising:

determining one or more persons to highlight in a highlight reel;

determining, based on faces of the one or more determined persons appearing in images or video clips and quality metric values of the images or the video clips, determined images or video clips by which to highlight the one or more persons; and

arranging the determined images or video clips into the highlight reel.

15. The method as recited in claim 14, wherein determining the one or more persons to highlight is based on persons tagged or facially recognized within selected source media that includes the images or video clips.

16. The method as recited in claim 14, wherein determining images or video clips is further based on contextual variety of the images or the video clips, the contextual variety sufficient to show the one or more persons in various different contextual environments or different events.

17. The method as recited in claim 14, wherein determining images or video clips by which to highlight the one or more persons is based in part on a selected target duration for the highlight reel and wherein the method continues to determine images or video clips in a greedy manner until the selected target duration is met.

18. The method as recited in claim 14, wherein the one or more persons includes two or more persons and wherein determining images or video clips by which to highlight the two or more persons is based on a weighting assigned to each of the two or more persons.

19. The method as recited in claim 14, further comprising:

receiving or determining, automatically and without user interaction, additional images or video clips;

determining, based on the additional images or video clips having faces of the one or more determined persons and quality metric values of the additional images or video clips, to include one or more of the additional images or video clips within the highlight reel; and

updating the highlight reel to include the one or more of the additional images or video clips.

20. One or more computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, perform operations comprising:

receiving selection of source media having images or video clips;

receiving selection of persons represented in one or more of the images or video clips;

determining context for the images or video clips;

determining quality metric values for the images or video clips;

determining, based on the determined contexts, persons represented, and determined quality metric values, a subset of the images or video clips by which to highlight the source media; and

presenting a highlight reel having the determined subset of the images or video clips.