Content aware slideshows

• US 9,881,376 B2
• Filed: 07/28/2014
• Issued: 01/30/2018
• Est. Priority Date: 08/22/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shechtman, Elya, Bagon, Shai, Agarwala, Aseem O.
• Primary Examiner(s): Bekele, Mekonen
• Application Number:
  US14/444,996
• Publication Number:
  US 20140337721A1
• Time in Patent Office: 1,282 Days
• Field of Search: 382284, 382236, 382243, 382268, 382266, 382235, 382173, 382165, 382103, 382190, 382100, 382180, 382195
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49

Abstract

A method, system, and computer-readable storage medium for performing content based transitions between images. Image content within each image of a set of images are analyzed to determine at least one respective characteristic metric for each image. A respective transition score for each pair of at least a subset of the images is determined with respect to each transition effect of a plurality of transition effects based on the at least one respective characteristic metric for each image. Transition effects implementing transitions between successive images for a sequence of the images are determined based on the transition scores. An indication of the determined transition effects is stored. The determined transition effects are useable to present the images in a slideshow or other image sequence presentation.

Claims

1. A method comprising:

measuring at least one image characteristic of multiple images by analyzing content of the multiple images;

determining a sequence of the images in which each pair of consecutive images in the sequence has a similar region of the content according to the measured image characteristic; and

applying a transition effect for the pair of consecutive images to transition from a display of a first image of the pair to a display of a second image of the pair, the transition effect usable to implement a face-based transition effect between the pair of images when the similar region of content includes a face according to the measured image characteristic, the face-based transition effect applied to align the first image and the second image.

2. The method of claim 1, wherein the transition effect includes shifting the display of the first image to align the similar region of the first image and the second image during the transition effect.

3. The method of claim 1, wherein the transition effect includes shifting the display of the second image to align the similar region of the first image and the second image during the transition effect.

4. The method of claim 1, wherein the transition effect includes performing a morphing technique to one of cross fade or transition between the similar region of the first image and the second image.

5. The method of claim 1, wherein the transition effect includes at least one of translating, scaling, or rotating the display of the first image to align the similar region of the first image and the second image during the transition effect.

6. The method of claim 1, wherein the transition effect includes at least one of translating, scaling, or rotating the display of the second image to align the similar region of the first image and the second image during the transition effect.

7. The method of claim 1, further comprising detecting the similar region for the pair of consecutive images based on the measured image characteristic.

8. The method of claim 7, wherein the transition effect is usable to implement a depth-discontinuity transition effect when the similar region detected in the first image and the second image corresponds to depth discontinuities, the depth-discontinuity transition effect determined based on:

computing a depth map for each of the multiple images based on the measuring;

searching the depth maps for the depth discontinuities; and

aligning the similar region of the first image and the second image using the depth discontinuities found in the first image and the second image as a result of the searching.

9. The method of claim 7, wherein the transition effect is further usable to implement:

an object-based transition effect when the similar region detected in the pairs of images corresponds to one or more objects;

a large segment transition effect when the similar region detected in the pairs of images corresponds to one or more regions that are considered large segments of the images;

a panoramic stitch transition effect when the similar region detected in the pairs of images corresponds to one or more regions that can be stitched together to form a panoramic image;

or

a local similar regions transition effect when the similar region detected in the pairs of images corresponds to similar backgrounds, similar objects, or similar logos.

10. One or more computer-readable memory devices storing instructions thereon that are executable by a computing device to perform operations comprising:

measuring at least one image characteristic of at least two images by analyzing image content of the at least two images;

detecting faces that are similar in the image content of the at least two images according to the measured image characteristic; and

applying a face-based transition effect for a transition from a display of a first one of the images to a display of a second one of the images, the face-based transition effect applied effective to align, during the transition, similar faces detected in the at least two images.

11. The one or more computer-readable memory devices of claim 10, wherein the transition effect includes at least one of translating, scaling, or rotating the display of the first image to align the similar faces of the first image and the second image during the transition effect.

12. The one or more computer-readable memory devices of claim 10, wherein the transition effect includes at least one of translating, scaling, or rotating the display of the second image to align the similar faces of the first image and the second image during the transition effect.

13. The one or more computer-readable memory devices of claim 10, wherein said measuring the at least one image characteristic is performed for more than two images to determine a sequence of the images in which each pair of consecutive images in the sequence has a similar region of the image content according to the measured image characteristic.

14. The one or more computer-readable memory devices of claim 13, wherein the operations further comprise applying a transition effect for each of the pairs of consecutive images based on the similar region detected for each of the pairs, the transition effect comprising:

a face-based transition effect when the similar region detected for the pair corresponds to one or more faces;

an object-based transition effect when the similar region detected for the pair corresponds to one or more objects;

a large segment transition effect when the similar region detected for the pair corresponds to one or more regions that are considered large segments of the images;

a depth discontinuity transition effect when the similar region detected for the pair corresponds to one or more depth discontinuities;

a panoramic stitch transition effect when the similar region detected for the pair corresponds to one or more regions that can be stitched together to form a panoramic image;

or

a local similar regions transition effect when the similar region detected for the pair corresponds to similar backgrounds, similar objects, or similar logos.

15. The one or more computer-readable memory devices of claim 10, wherein the face-based transition effect includes shifting the display of the first image to align the similar face of the first image and the second image during the face-based transition effect.

16. The one or more computer-readable memory devices of claim 10, wherein the face-based transition effect includes shifting the display of the second image to align the similar face of the first image and the second image during the face-based transition effect.

17. A system comprising:

a display device configured to display images and transition effects that transition from one of the images to another of the images;

a memory and at least one processor to implement an image analysis module configured to;

measure at least one image characteristic of the images by analyzing content of the images;

determine a sequence of the images in which each pair of consecutive images in the sequence has a similar region of the content according to the measured image characteristic; and

apply a transition effect for each of the pairs of consecutive images to transition from a display of a first image of the pair to a display of a second image of the pair, the transition effect usable to implement a face-based transition between the pair of images when the pair of images have similar faces in the content according to the measured image characteristic, the face-based transition effect applied to align the similar faces of the first and second images.

18. The system of claim 17, wherein to determine the transition effects for each of the pairs of images, the image analysis module is further configured to:

determine a transition score for each of the pairs of images with respect to multiple candidate transition effects that can be applied to the pair to transition from the first image of the pair of images to the second image of the pair of images, wherein the transition score represents suitability of a candidate transition effect to transition between the first image of the pair of images and the second image of the pair of images; and

determine transition effects between the consecutive images for the sequence of the images based on the transition scores.

19. The system of claim 18, wherein to determine the transition effects between consecutive images for the sequence of the images based on the transition scores, the image analysis module is further configured to:

solve an optimization problem to determine a solution that substantially maximizes similarity over the sequence of images.

20. The system of claim 19, wherein the optimization problem is solved subject to one or more constraints, wherein the one or more constraints are specified with respect to one or more of:

priorities or preferences with respect to image attributes;

repetition or frequency of image attributes;

priorities or preferences regarding the candidate transition effects;

repetition or frequency of the transition effects;

or

distribution of image content.