Methods and systems for automating insertion of content into media-based projects

• US 8,958,662 B1
• Filed: 08/20/2014
• Issued: 02/17/2015
• Est. Priority Date: 08/20/2014
• Status: Active Grant

• Current Assignee: Interactive Memories Incorporated
• Original Assignee: Interactive Memories Incorporated
• Inventors: Grosz, Aryk Erwin, Lai, Anthony Jan Fung, Schultz, Dan, Barushev, Denys, Astashov, Anton Vladimirovich, Wang, Wiley
• Primary Examiner(s): SHAH, UTPAL D
• Application Number:
  US14/464,699
• Time in Patent Office: 181 Days
• Field of Search: 382/100, 382/118, 382/224, 382/305
• US Class Current: 382/305
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

Methods and systems for automating insertion of content into media-based projects are disclosed. In one embodiment, images are uploaded from a user device to a server. The server performs image analysis on the uploaded images and obtains automatic image information. After presentation of the initial project, the user manually interacts with the uploaded images to provide user inputs to the project. The system observes the user interactions to generate manual image information based on the user interactions with the project. Based on the automatic image information and the manual image information, the present invention automatically updates a printable photobook or other media-based project after automatically inserting relevant content into the project. This produces a faster and more seamless user experience, and increases the number of customers purchasing the ultimate product by minimizing friction points to purchase.

Claims

1. A non-transitory, computer-readable storage medium storing executable instructions, which when executed by a processor of a server, causes the processor to perform a process for creating a photo-project over a communications network, the instructions causing the processor to perform the steps of:

establishing a current active client-server connection between the server and a computing device over said communications network;

receiving a plurality of images from said computing device through an upload to said server, and storing said plurality of images in a data repository connected to said server;

performing an automatic image analysis on said plurality of images to derive automatic image information from said plurality of images, the automatic image information comprising at least an automatically selected area of interest for at least one image;

grouping said plurality of images based on said automatic image information;

inserting content into said photo-project based on the grouping of said plurality of images to create an initial recommended design of the photo-project;

presenting said initial recommended design of the photo-project to a user of said computing device;

monitoring user interactions on said computing device while the user is manipulating said plurality of images to derive observed image information from said user interactions, wherein the user interactions comprise at least selecting an area of interest, wherein the user selects a user selected area of interest that is different from the automatically selected area of interest selected by the automatically image analysis; and

automatically modifying an arrangement of content in the photo-project based on said observed image information to produce a modified recommended design of the photo-project, wherein said observed image information, comprising at least said user selected area of interest, is utilized to optimize a next iteration of said automatic image analysis to update said automatic image information based on said user selected area of interest revealed through said user interactions, wherein said next iteration of said automatic image analysis occurs at least during the current active client-server connection between the server and the corn muting device, and

wherein when the user over-rides the automatically selected area of interest in at least one image of the photo-project with the user selected area of interest, the server automatically updates at least one of the plurality of images with the user selected area of interest in the current active client-server connection.

2. The storage medium of claim 1, wherein said automatic image information is derived though one or more algorithms for:

saliency detection, face detection, and photo similarity detection.

3. The storage medium of claim 1, wherein the user selected area of interest is selected by panning, zooming, and/or moving of said images by said user.

4. The storage medium of claim 1, wherein said grouping of said plurality of images is achieved through at least one of:

timestamp analysis, image similarity analysis, and photo importance scoring.

5. The storage medium of claim 1, wherein the step of inserting content is derived through one or more algorithms for:

a ranking of images, a buffer calculation for a best fitting content into a photo slot, an area of interest selection on a photo, a score for inserting a photo into a photo slot of a layout, a determination of a best design of a design page, a dynamic text insertion algorithm, and a positioning of a photo into a photo slot having a focal box.

6. The storage medium of claim 5, wherein said ranking of images comprises a plurality of image heuristics for calculating image importance.

7. A method for automating a creation of a media project in a client-server environment via a graphical user interface (GUI) of a user device, after a user of said user device uploads content comprising media files from said user device to a server, the method comprising:

selecting a first set of content for insertion into a design of said media project, wherein said first set of content is based on automatic image information derived though an automatic image analysis of said media files performed by said server, wherein said first set of content is automatically inserted into said design without said user'"'"'s interaction utilizing automated rules for content insertion, the automatic image information comprising at least an automatically selected area of interest for at least media file;

grouping of said media files into a plurality of groups by using automated rules of content grouping, and inserting said groups of media files into said design;

presenting to said user at said GUI an automated design of said media project by automatically inserting said first set of content, grouped into the plurality of groups, into the media project, wherein said automated design is generated using said automated rules of content insertion and said automated rules of content grouping;

monitoring user interactions to said automated design on said GUI of said user device, wherein the user interactions comprise at least selecting an area of interest, wherein the user selects a user selected area of interest that is different from the automatically selected area of interest selected by the automatic image analysis;

inserting a second set of content into said automated design based on observed user interactions with said first set of content, either wherein said second set of content is inserted based on information derived from observing said user interactions with said media project, and wherein said insertion of said second set of content either alters said first set of content or adds additional content into said automated design;

automatically re-arranging said first set and said second set of content based on said observed user interactions, wherein said observed image information, comprising at least said user selected area of interest, is utilized to optimize a next iteration of said automatic image analysis to update said automatic image information based on said user selected area of interest, wherein said next iteration of said automatic image analysis occurs at least during a current active client-server connection between the server and the user device; and

presenting to said user a final recommended design of said media project, wherein said final recommended design comprises said first set of content, said second set of content, and said automatic re-arrangement of said first set and said second set of content, and wherein said final recommended design may be further customized by said user prior to ordering.

8. The method of claim 7, wherein the first set of content and the second set of content are inserted automatically by said server at a background computer process that is invisible to said user utilizing said GUI.

9. The method of claim 7, wherein the user selected area of interest is selected by panning, zooming, and/or moving of said first set of content by said user.

10. The method of claim 7, further comprising:

sending an order to print said automated design to a printer, wherein said printer prints said automated design on paper, and wherein said paper is selected from a plurality of configurable print sizes.

11. The method of claim 7, further comprising:

sending an order to print said final recommended design to a printer, wherein said printer prints said final design on paper, and wherein said paper is selected from a plurality of configurable print sizes.

12. The method of claim 7, wherein the method is practiced over the Internet network.

13. A system for automating a design of a media-project, the system comprising:

a user device;

a server comprising a memory and a processor;

a communications network connecting said user device and said server; and

a plurality of computer codes embodied on said memory of said server, said plurality of computer codes which when executed causes said processor to;

establish a current active client-server connection between the server and the user device over said communications network;

receive a plurality of images from said user device through an upload to said server;

perform an automatic image analysis on said plurality of images to derive automatic image information from said plurality of images, the automatic image information comprising at least an automatically selected area of interest for at least one image;

group said plurality of images based on said automatic image information;

insert content into said media-project based on the grouping of said plurality of images to create an initial recommended design of the media-project;

present said initial recommended design of the media-project to a user of said user device;

monitor user interactions on said user device while the user is manipulating said plurality of images to derive observed image information from said user interactions, wherein the user interactions comprise at least selecting an area of interest, wherein the user selects a user selected area of interest that is different from the automatically selected of interest selected by the automatically image analysis; and

automatically modify an arrangement of content in the media-project based on said observed image information to produce a modified recommended design of the media-project, wherein said observed image information, comprising at least said user selected area of interest, is utilized to optimize a next iteration of said automatic image analysis to update said automatic image information based on said user selected area of interest revealed through said user interactions, wherein said next iteration of said automatic image analysis occurs at least during the current active client-server connection between the server and the user device.

14. The system of claim 13, wherein said automatic image analysis comprises an execution of computer algorithms for:

saliency detection, face detection, photo similarity detection, and object detection.

15. The system of claim 13, wherein said grouping of said plurality of images comprises an execution of computer algorithms for:

image similarity detection and photo importance scoring.

16. The system of claim 13, wherein said server, after performing said automatic image analysis, is adapted to auto-populate a layout from a layoutset.

17. The system of claim 16, wherein said server, after performing said automatic image analysis, is adapted to auto-populate a design from a designset, and wherein said designset is inter-related to said layoutset.

18. The system of claim 13, wherein said sever, after performing said automatic image analysis, is adapted to auto-populate a layout and a design of the media-project.

19. The system of claim 13, wherein said media-project is sent to a printer, and wherein said printer prints said media-project on paper, thereby turning said media-project into a shippable product for delivery to a plurality of users.

20. The system of claim 13, wherein the user selected area of interest is selected by panning and/or zooming of said images by said user.