ALIGNING AND SUMMARIZING DIFFERENT PHOTO STREAMS

• US 20120113121A1
• Filed: 11/09/2010
• Published: 05/10/2012
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Inventors: Luo, Jiebo, Yang, Jianchao
• Granted Patent: US 8,805,165 B2
• Time in Patent Office: Days
• US Class Current: 345/440
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/70 of video data

Abstract

A method for organizing individual collections of images or videos captured for the same event by different cameras into a master collection, wherein each individual collection forms a media stream in chronological order, includes using a processor to provide the following steps: extracting image features for each image or video of the media stream of each individual collection; analyzing the extracted features to align the media streams to form a master stream in chronological order of the event over a common timeline; producing a master collection of images or videos of the event from the master stream by removing redundant images or videos; and storing, displaying or transmitting the master collection.

Claims

1. A method for organizing individual collections of images or videos captured for the same event by different cameras into a master collection, wherein each individual collection forms a media stream in chronological order, comprising:

(a) extracting image features for each image or video of the media stream of each individual collection;

(b) analyzing the extracted features to align the media streams to form a master stream in chronological order of the event over a common timeline;

(c) producing a master collection of images or videos of the event from the master stream by using a cost function to remove redundant images or videos; and

(d) storing, displaying or transmitting the master collection.

2. A method of claim 1, further including using the master collection to augment one of the individual collections to produce an augmented individual collection, and storing, displaying or transmitting the augmented individual collection.

3. The method of claim 2, further including analyzing the individual collection and the master collection to determine which images or videos from the master collection are relevant to the individual collection, and augmenting the individual collection with the relevant images or videos.

4. A method of claim 1, wherein step (a) further includes extracting geo-location tags or other textual tags associated to each image or video as additional features.

5. A method of claim 1, wherein step (b) includes:

(i) constructing a graph based on the extracted features to link the images or videos from the two media streams;

(ii) finding on the graph at least a pair of images or videos, each from one of the two media streams, that corresponds to correlated captured content; and

(iii) aligning the remaining images or videos in response to the aligned pair so that the all images or videos from the two media streams are aligned over a common timeline.

6. A method of claim 1, wherein the cost function in step (c) incorporates a compactness that the master collection should contain as few redundant photos as possible, and a coverage that the master collection should maintain the integrity of the content information contained in the master stream, the master collection should cover or represent the master stream well in the feature space.