Systems and methods for creating compilations based on hierarchical clustering

• US 10,614,114 B1
• Filed: 07/10/2017
• Issued: 04/07/2020
• Est. Priority Date: 07/10/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Garcia, Vincent, Caille, Jean
• Primary Examiner(s): Park, Grace
• Assistant Examiner(s): Curran, J Mitchell
• Application Number:
  US15/645,411
• Time in Patent Office: 1,002 Days
• Field of Search: None
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information

Abstract

Media items may be obtained. The media items may be characterized by capture information indicating a capture time and a capture location of individual media items. The media items may be clustered into scenes based on proximity of the capture times of the media items. The scenes may be clustered into collections based on proximity of the capture times and/or the capture locations of the media items within the scenes. The collections may be iteratively clustered into higher collections based on proximity of the capture times and/or the capture locations of the media items within the collections. One or more collections may be identified for inclusion in a media compilation based on a size of the identified collection(s). A media compilation, including one or more of the media items included in the identified collection(s), may be generated.

Claims

1. A system that creates compilations based on hierarchical clustering, the system comprising:

one or more physical processors configured by machine-readable instructions to;

obtain media items, the media items including two or more images, the media items characterized by capture information indicating a capture time and a capture location of individual media items;

cluster the media items into scenes based on proximity of the capture times of the media items;

cluster the scenes into collections based on proximity of the capture times, the capture locations, or the capture times and the capture locations of the media items within the scenes;

iteratively cluster lower collections into higher collections based on proximity of the capture times, the capture locations, or the capture times and the capture locations of the media items within the lower collections, the iterative clustering resulting in a hierarchy of a scene level and two or more collection levels;

identify multiple ones of the collections within the hierarchy for potential inclusion in a media compilation of some or all of the media items included in the identified collections, the media compilation providing a view of the media items organized into the hierarchy, wherein the identified collections are identified based on sizes of the collections, the sizes of the collections characterizing extents of the collections; and

generate the media compilation, the media compilation including some or all of the media items included in the identified collections;

wherein identification of the multiple ones of the collections within the hierarchy for potential inclusion in the media compilation of some or all of the media items included in the identified collections includes;

identification of multiple potential collections;

identification of a subset of the identified potential collections for inclusion in the media compilation of some or all of the media items included in the subset of the multiple potential collections;

wherein;

the identified potential collections include a first combination of collections and a second combination of collections, the first combination of collections including a first collection and a second collection, the second combination of collection including the first collection and a third collection;

the first collection is more proximate to the second collection than the third collection within the hierarchy;

the first combination of collections is identified as the subset of the identified potential collections based on a first preference to include collections with closer proximity within the media compilation; and

the second combination of collections is identified as the subset of the identified potential collections based on a second preference to include collections with farther proximity within the media compilation.

2. The system of claim 1, wherein the identified collections are identified for inclusion in the media compilation further based on a compacity of the media items within the collections, the compacity of the media items within the collections determined based on differences in the capture times of the media items within the collections.

3. The system of claim 2, wherein the identified collections are identified for inclusion in the media compilation further based on a diversity of the media items within the collections, the diversity of the media items within the collections determined based on a standard deviation of the capture times of the media items within the collections.

4. The system of claim 1, wherein the one or more physical processors are further configured by the machine-readable instructions to obtain additional media items, the additional media items clustered without altering the majority of existing collections.

5. The system of claim 1, wherein the media compilation includes all of the media items included in the identified collections.

6. The system of claim 1, wherein the media compilation includes a subset of the media items included in the identified collections, the subset of the media items included in the media compilation based on content of the subset of the media items.

13. The system of claim 1, wherein the sizes of the collections indicate amounts of media items within the collections and the identified collections are identified based on the amounts of media items within the collections such that the media compilation includes a certain number of media items, includes a number of media items within a range of media item, includes less than the certain number of media items, or includes more than the certain number of media items.

14. The system of claim 1, wherein the sizes of the collections indicate amounts of scenes within the collections and the identified collections are identified based on the amounts of scenes within the collections such that the media compilation includes a certain number of scenes, includes a number of scenes within a range of scenes, includes less than the certain number of scenes, or includes more than the certain number of scenes.

15. The system of claim 1, wherein the sizes of the collections indicate time durations covered by the media items or the scenes within the collections such that the media compilation includes a certain time duration, includes time durations within a range of time durations, includes less than the certain time duration, or includes more than the certain time duration.

16. The system of claim 1, wherein the sizes of the collections indicate capture locations covered by the media items or the scenes within the collections such that the media compilation includes a certain capture location, includes capture locations within a range of capture locations, includes less than the certain capture location, or includes more than the certain capture location.

7. A method for creating compilations based on hierarchical clustering, the method comprising:

obtaining media items, the media items including two or more images, the media items characterized by capture information indicating a capture time and a capture location of individual media items;

clustering the media items into scenes based on proximity of the capture times of the media items;

clustering the scenes into collections based on proximity of the capture times, the capture locations, or the capture times and the capture locations of the media items within the scenes;

iteratively clustering lower collections into higher collections based on proximity of the capture times, the capture locations, or the capture times and the capture locations of the media items within the lower collections, the iterative clustering resulting in a hierarchy of a scene level and two or more collection levels;

identifying multiple ones of the collections within the hierarchy for potential inclusion in a media compilation of some or all of the media items included in the identified collections, the media compilation providing a view of the media items organized into the hierarchy, wherein the identified collections are identified based on sizes of the collections, the sizes of the collections characterizing extents of the collections; and

generating the media compilation, the media compilation including some or all of the media items included in the identified collections

wherein identification of the multiple ones of the collections within the hierarchy for potential inclusion in the media compilation of some or all of the media items included in the identified collections includes;

identification of multiple potential collections;

identification of a subset of the identified potential collections for inclusion in the media compilation of some or all of the media items included in the subset of the multiple potential collections;

wherein;

the identified potential collections include a first combination of collections and a second combination of collections, the first combination of collections including a first collection and a second collection, the second combination of collection including the first collection and a third collection;

the first collection is more proximate to the second collection than the third collection within the hierarchy;

the first combination of collections is identified as the subset of the identified potential collections based on a first preference to include collections with closer proximity within the media compilation; and

the second combination of collections is identified as the subset of the identified potential collections based on a second preference to include collections with farther proximity within the media compilation.

8. The method of claim 7, wherein the identified collections are identified for inclusion in the media compilation further based on a compacity of the media items within the collections, the compacity of the media items within the collections determined based on differences in the capture times of the media items within the collections.

9. The method of claim 8, wherein the identified collections are identified for inclusion in the media compilation further based on a diversity of the media items within the collections, the diversity of the media items within the collections determined based on a standard deviation of the capture times of the media items within the collections.

10. The method of claim 7, further comprising obtaining additional media items, the additional media items clustered without altering the majority of existing collections.

11. The method of claim 7, wherein the media compilation includes all of the media items included in the identified collections.

12. The method of claim 7, wherein the media compilation includes a subset of the media items included in the identified collections, the subset of the media items included in the media compilation based on content of the subset of the media items.