TECHNIQUES TO ENABLE AUTOMATED WORKFLOWS FOR THE CREATION OF USER-CUSTOMIZED PHOTOBOOKS

• US 20120294514A1
• Filed: 05/19/2011
• Published: 11/22/2012
• Est. Priority Date: 05/19/2011
• Status: Abandoned Application

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Saunders, Craig John, Marchesotti, Luca, Lin, Julianna Elizabeth, Rolleston, Robert J., Maloney, Thomas L.
• Application Number:
  US13/111,060
• Publication Number:
  US 20120294514A1
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: H04N 1/00196 Creation of a photo-montage, e.g. photoalbum H04N1/0019 takes precedence

Abstract

A system and method for generating a photobook are provided. The method includes receiving a set of images and automatically selecting a subset of the images as candidates for inclusion in a photobook. At least one design element of a design template for the photobook is automatically selected, based on information extracted from at least one of the images in the subset. Placeholders of the design template are automatically filled with images drawn from the subset to form at least one page of a multipage photobook. The exemplary system and method address some of the problems of photobook creation, thorough combining automatic methods for selecting, cropping, and placing photographs into a photo album template, which the user can then post-edit, if desired. This can greatly reduce the time required to create a photobook and thus encourage users to print photo albums.

Claims

1. A method of generating a photobook comprising:

receiving a set of images;

with a processor;

automatically selecting a subset of the images as candidates for inclusion in a photobook;

automatically selecting at least one design element of a design template for the photobook based on information extracted from at least one of the images in the subset; and

automatically filling placeholders of the design template with images from the subset to form at least one page of a multipage photobook.

2. The method of claim 1, wherein the selection of an element of the design template comprises selection of at least one of the group consisting of font style, border, background images, background color, font color, image layout, combinations thereof.

3. The method of claim 1, wherein the automatic selecting of the subset of the images is based on at least one of image quality assessment criteria, image relevance criteria, and near-duplicate removal criteria.

4. The method of claim 3, wherein the image relevance criterion is based on at least one of:

a user-selected or automatically identified theme or color scheme of the design template;

a user profile; and

other images in the set of images.

5. The method of claim 3, wherein the image quality assessment criteria is based on a measurement, on at least a salient part of the image, of at least one of:

image size;

image blur;

structural noise;

image exposure; and

image contrast.

6. The method of claim 1, wherein the automatic selection of the subset of images is also based on at least one of a user-selected theme and a user-selected style for the photobook.

7. The method of claim 1, wherein the automatic selection includes categorizing images in the set by semantic category based on low level features extracted from the images and grouping images that are categorized in a same one of a finite set of semantic categories for filling the placeholders to form the page.

8. The method of claim 1, wherein the automatic selection of the subset of images comprises removing redundant images comprising identifying images which are near duplicates of each other and removing at least one of the near duplicates from consideration as a candidate image.

9. The method of claim 1, wherein the method further comprises providing for presenting a set of automatically identified similar images to a user as candidates for replacement of an automatically selected image when a user rejects the automatically-selected image.

10. The method of claim 1, wherein the method further comprises providing for presenting at least one selected color palette to a user as a candidate for replacement of at least one automatically selected design element design of a template for the page, the design element being selected from a border color, a border pattern, a background color, a background pattern, and a font color for the page, the color palette in the set being selected based on a computed similarity between the color palette and a color palette extracted from at least one image on the page.

11. The method of claim 1, wherein the automatic filling of placeholders of the design template comprises selecting an anchor image for a first placeholder on the page of the photobook and selection a set of supplementary images which complement the anchor image based on at least one of a similarity of a color palette extracted from a supplementary image to a color palette extracted from the anchor image, a relationship between a time stamp of the supplementary image and a time stamp of anchor image, a similarity of the semantic content of the anchor image and supplementary image based on representations of low level features extracted from patches of respective images.

12. The method of claim 1, wherein the method further includes computing a saliency map of a candidate image in the subset and automatically cropping the candidate image based on the saliency map.

13. The method of claim 12, wherein the computing of the saliency map comprises:

for each image in a dataset of images for which a region of interest has been established respectively, storing a dataset image representation based on features extracted from the training image;

for the candidate image for which a region of interest is to be detected;

generating a candidate image representation for the candidate image based on features extracted from the candidate image;

identifying a subset of similar images from the images in the dataset, the identified subset being based on a measure of similarity between the candidate image representation and respective dataset image representations;

training a classifier with information extracted from the established regions of interest of the subset of similar images;

with the trained classifier, classifying regions of the candidate images with respect to saliency; and

generating a saliency map based on the saliency classifications.

14. The method of claim 12, wherein the cropping is also based on a placeholder shape.

15. The method of claim 1, wherein the method comprises automatically filling multiple pages of the photobook, wherein images of a first page are similar to each other, based on a computed measure of at least one of structural similarity, semantic content similarity and aesthetic similarity, and images of a second page are similar to each other based on a computed measure of at least one of structural similarity, semantic content similarity and aesthetic similarity, and wherein the first and second pages differ in at least one automatically-selected design element, the automatically-selected design element being selected from a border color, a border pattern, a background color, a background pattern, and a font color for the page.

16. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

17. A computer program product comprising a non-transitory recording medium encoding instructions, which when executed by a computer, perform the method of claim 1.

18. A system for generating a photobook comprising:

a selection component for automatically selecting a subset of a set of images as candidates for inclusion in a photobook;

a template component for automatically selecting at least one design element of design template for the photobook based on information extracted from at least one of the images in the subset;

a creation component which automatically fills placeholders of the design template with images from the subset to form a multipage photobook; and

a processor which implements the selection component, template component, and creation component.

19. A workflow process comprising:

automatically selecting a subset of a set of input images based on at least one of a computation of image quality and a computation of near duplicate images;

automatically cropping at least some of images in the subset based on identification of a salient region of the respective image;

grouping similar images in the subset into groups based on a computation of at least one of structural similarity, content similarity, and aesthetic similarity;

automatically selecting at least one design element of design template for a page of a book based on information extracted from at least one of the images in one of the groups, the design element being selected from a border color, a border pattern, a background color, a background pattern, and a font color for the page; and

automatically filling placeholders of the design template with the group of images to form a page, wherein the process is implemented with a computer processor.

20. The method of claim 19, wherein the method further comprises providing for presenting a set of automatically identified similar images to a user as candidates for replacement of an automatically selected image when a user rejects the automatically-selected image.