METHOD FOR CREATING VIEW-BASED REPRESENTATIONS FROM MULTIMEDIA COLLECTIONS

• US 20150363409A1
• Filed: 06/11/2015
• Published: 12/17/2015
• Est. Priority Date: 06/11/2014
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Inventors: Wood, Mark D., Das, Madirakshi, Stubler, Peter O.
• Granted Patent: US 11,170,037 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

A system that is capable of generating a multiplicity of representations from a set of multimedia objects, each with a potentially different form, is disclosed. In addition, a system that is capable of generating representations that include thematic groupings in addition to the traditional purely sequential groupings is disclosed. Analysis of metadata to generate derived metadata creates a more comprehensive repository of information about media assets. Using the abundance of metadata, embodiments of the methods disclosed teach uniquely grouping and prioritizing media assets into a storyboard, which is in turn mapped onto view-based representations based on a selected output modality.

Claims

1. A method of creating one or more view-based representations from a set of multimedia objects comprising:

using a processor to analyze the multimedia objects and corresponding recorded metadata to generate derived metadata;

applying a selected method to analyze recorded and derived metadata;

ordering and grouping the set of multimedia objects according to said analysis;

prioritizing the set of multimedia objects according to said analysis;

selecting a specific output modality; and

generating a view-based representation of the set of multimedia objects for the selected modality.

2. The method of claim 1 wherein the ordering, grouping, and prioritization is persisted independently of the view modality.

3. The method of claim 1, further comprising saving the ordering, grouping, and prioritization data independently of the elected output modality.

4. The method of claim 1 wherein the view-based representation further comprises computing an emphasis score for members of the set of multimedia objects.

5. The method of claim 1 wherein the view based representation of the set of multimedia objects for the selected modality consists of a subset of the multimedia objects as determined by the priority.

6. The method of claim 1 wherein the grouping consists of one or more of the following techniques:

chronology, frequent itemset mining, face detection, face recognition, location clustering, object detection, object recognition, event detection, and event recognition.

7. The method of claim 1 wherein the ordering consists of a hierarchical arrangement of the multimedia objects from the groups.

8. The method of claim 1 wherein all the elements of select groups of the grouped set of multimedia objects are determined to be semantically equivalent.

9. The method of claim 7 wherein the semantic equivalence is determined by one or more of the following classes of information:

temporal, location, people detection and recognition, and visual similarity.

10. The method of claim 8 wherein visual similarity is determined by comparing block-based color histograms.

11. The method of claim 6 wherein the set of multimedia objects of the hierarchical arrangement are arranged in sequential order or indeterminate order.

12. The method of claim 7 wherein the set of multimedia objects or contains one or more representative members of the set are selected to represent the set.

13. The method of claim 1, wherein the views in the view based representation of the set of multimedia objects for the selected modality are determined by the ordering and grouping.

14. The method of claim 2 wherein a graphic user interface is provided that incorporates a visual representation of the ordering and grouping of the set of multimedia objects according to said analysis.

15. The method of claim 13 wherein the graphic user interface provides a description or indication of size for the groupings.

16. The method of claim 4 wherein the selected modality includes a multimedia presentation, an interactive multimedia presentation, a photobook, a printed collage, a virtual collage, or a digital slide show.

17. The method of claim 1 wherein the grouping method consists of identifying a plurality of thematic sets for each group, wherein each of the thematic sets is determined from the derived metadata and where each thematic set contains a subset of the digital images.

18. The method of claim 16 wherein for each group, a subset of the corresponding thematic sets is selected by evaluating each of the identified thematic sets responsive to one or more of the following:

the probability of occurrence, the size of the thematic set, the user preferences, or other derived metadata.

19. The method of claim 1 wherein the derived metadata includes one or more of the following:

the frequency with which an asset has shared with others, the number of people with which an asset has been shared, or social media feedback such as the number of “

likes,”

comments, star ratings an asset has received.

20. A method of generating one or more view-based representations of an output modality for multimedia assets comprising:

a processor analyzing recorded metadata and user-provided metadata associated with multimedia assets of a multimedia collection;

the processor generating derived metadata based on the recorded metadata and the user-provided metadata;

the processor organizing the media assets in one or more groupings;

the processor prioritizing the media assets in each grouping;

the processor prioritizing the one or more groupings;

the processor generating a storyboard;

mapping the storyboard as one or more view-based representations according to a selected output modality; and

rendering on a display the one or more view-based representations.

21. The method of claim 20, wherein recorded metadata comprises metadata automatically recorded by an image capture device upon capturing a media asset.

22. The method of claim 20, wherein user-provided metadata comprises metadata provided by a user via an interface located on an image capture device or via an image editing application.

23. The method of claim 20, wherein the step of generating derived metadata comprises applying one or more of the techniques chosen from the group consisting of temporal event clustering, geographic naming, scene classification, materials class extraction, low-level feature extraction, content-based image retrieval, face detection, face recognition, facial clustering, semantic event clustering, image value indexing, and video key frame extraction.

24. The method of claim 20, wherein the storyboard is a hierarchical grouped set of the multimedia assets.

25. The method of claim 20, wherein user-provided metadata comprises indication of preference recorded by users of a social networking application or social networking website.

26. The method of claim 20, wherein the processor groups the media assets strictly according to a chronological paradigm.

27. The method of claim 20, wherein the processor groups the media assets according to a thematic paradigm.

28. The method of claim 20, wherein the processor groups the media assets according to a hybrid chronological-thematic paradigm.

29. The method of claim 20, wherein the groupings comprise sequential segments and parallel segments.

30. The method of claim 20 wherein the processor groups events according to a chronological paradigm and groups sub-events according to a thematic paradigm.

31. The method of claim 20, wherein the output modality is selected from the group consisting of hardcopy print media and softcopy digital presentation.

32. The method of claim 20, wherein prioritizing the media assets in each grouping is performed as a function of the recorded metadata, the user-provided metadata, and the derived metadata.

33. A method of grouping and prioritizing media assets in a multimedia collection comprising:

apply a frequent itemset mining method to compute one or more sets of thematic groupings of the media assets, wherein the thematic groupings are constrained by one or more feature descriptors belonging to one or more feature categories; and

computing an asset priority score for every asset in each of the one or more sets of thematic groupings, wherein the asset priority score is a function of the one or more feature descriptors.