Generating theme-based videos

• US 10,642,893 B2
• Filed: 09/01/2017
• Issued: 05/05/2020
• Est. Priority Date: 09/05/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Novikoff, Timothy, Cui, Jingyu
• Primary Examiner(s): Vu, Bai D
• Application Number:
  US15/694,728
• Publication Number:
  US 20180068019A1
• Time in Patent Office: 977 Days
• Field of Search: None
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings

Abstract

Implementations relate to generating theme-based videos. In some implementations, a computer-implemented method to automatically generate a theme-based video includes obtaining image criteria for a theme from a theme definition, querying an image database to obtain a plurality of images that meet the image criteria, determining that a count of the plurality of images satisfies a threshold based on the theme definition, and generating the theme-based video that includes one or more of the plurality of images.

Claims

1. A computer-implemented method to automatically generate a theme-based video, the computer-implemented method comprising:

selecting a theme from one or more available themes;

obtaining a theme definition associated with the theme;

obtaining stored image criteria associated with the theme from the theme definition;

querying an image database to obtain one or more candidate images from multiple images stored in the image database that meet the stored image criteria associated with the theme definition;

causing display of the one or more candidate images on a user device;

receiving a selection of at least one particular candidate image of the one or more candidate images;

determining selected image criteria based on the at least one particular candidate image;

querying the image database to obtain a plurality of images that meet the stored image criteria and meet the one or more selected image criteria;

generating the theme-based video that includes one or more of the plurality of images; and

causing the theme-based video to be displayed.

2. The computer-implemented method of claim 1, wherein the stored image criteria include at least one of:

one or more image features visually depicted by images, one or more types of image features visually depicted by the images, one or more geographic locations of image capture, or one or more times of image capture.

3. The computer-implemented method of claim 1, further comprising:

determining a respective engagement metric for the one or more available themes; and

selecting the theme from the one or more available themes based at least in part on the respective engagement metric.

4. The computer-implemented method of claim 3, further comprising:

after at least a portion of the theme-based video has been displayed, receiving feedback related to the theme-based video from a user; and

adjusting the engagement metric for the theme based on the feedback for the theme-based video,

wherein receiving the feedback comprises obtaining one or more of;

a share indicator that indicates the user shared the theme-based video, a view indicator that indicates a count of views of the theme-based video, a deletion indicator that indicates that the theme-based video was deleted, and a notification response indicator that indicates that the user accessed the theme-based video from a notification.

5. The computer-implemented method of claim 1, further comprising selecting the theme based on one or more of:

user profile information of a user for the theme-based video, a number of recent images that the user has uploaded to the image database, or a current date.

6. The computer-implemented method of claim 1, further comprising:

determining user profile information for a user, wherein the user profile information includes one or more of;

a user location or a user preference; and

selecting a soundtrack for the theme-based video based at least in part on the user profile information, wherein the selecting comprises one or more of;

determining that the user location matches a location parameter of the soundtrack;

or

determining that the user preference matches an audio parameter of the soundtrack,

wherein generating the theme-based video includes generating the theme-based video to include the soundtrack.

7. The computer-implemented method of claim 6, wherein the selected image criteria include one or more content features depicted in the at least one particular candidate image or one or more types of content features depicted in the at least one particular candidate image.

8. The computer-implemented method of claim 1, further comprising:

calculating a confidence score based on the plurality of images and the theme definition, wherein the confidence score indicates a strength of match between the plurality of images and the image criteria; and

selecting a soundtrack for the theme-based video based at least in part on the confidence score.

9. The computer-implemented method of claim 1, further comprising:

determining a threshold count based on the theme definition;

determining whether a count of the plurality of images is less than the threshold count;

in response to determining that the count of the plurality of images is less than the threshold count, checking the image database until one or more additional images are detected in the image database; and

in response to the one or more additional images being detected in the image database, repeating the determining whether the count of the plurality of images is greater than the threshold count.

10. The computer-implemented method of claim 1, wherein the theme definition includes one or more parameters for generating the theme-based video, wherein the one or more parameters include at least one of:

a minimum or maximum duration for the theme-based video, a frame rate for playback of the theme-based video, an indication of audio data to be played with the theme-based video, a transition effect displayed for images of the theme-based video, one or more visual filters to be applied to the theme-based video, or an indication of a title for the theme-based video.

11. A system comprising:

a memory; and

at least one processor configured to access instructions stored on the memory that cause the at least one processor to perform operations comprising;

obtaining a plurality of themes from a storage, each of the plurality of themes including a respective theme definition, wherein the operation of obtaining the plurality of themes includes;

determining available themes from a themes database;

determining a respective engagement metric for each of the available themes; and

selecting the plurality of themes from the available themes based at least in part on the respective engagement metric;

determining respective image criteria from the respective theme definition for each of the plurality of themes;

obtaining a plurality of images associated with a user from an image database;

determining a respective subset of the plurality of images that meet the respective image criteria for each of the plurality of themes;

selecting a particular theme of the plurality of themes for a video; and

generating the video based on the selected particular theme of the plurality of themes, wherein the video includes one or more images from the respective subset of the plurality of images for the particular theme.

12. The system of claim 11, wherein the at least one processor is configured to perform the operation of selecting the particular theme to include:

ranking the plurality of themes based on one or more of;

the respective engagement metric for the plurality of themes,

a respective diversity metric for the plurality of themes, or

a user profile for the user for the video; and

determining the particular theme based on the ranking.

13. The system of claim 11, wherein the respective theme definition includes one or more parameters for generating the video, wherein the one or more parameters include at least one of:

a minimum or maximum duration for the video, a frame rate for playback of the video, an indication of audio data to be played with the video, a transition effect displayed for images of the video, one or more visual filters to be applied to the video, or an indication of a title for the video.

14. The system of claim 11, wherein the at least one processor is configured to perform the operation of obtaining the plurality of themes to further comprise:

determining a respective diversity metric for each of the available themes; and

selecting the plurality of themes from the available themes based at least in part on the respective diversity metric, wherein the respective diversity metric are based on one or more of;

a number of generated videos for the available themes, or historical data for the available themes.

15. The system of claim 11, wherein the at least one processor is configured to perform further operations comprising:

determining a threshold time difference based on the respective theme definition; and

determining whether a time difference between a first timestamp associated with a first image of the plurality of images and a second timestamp associated with a second image of the plurality of images is greater than the threshold time difference,

wherein generating the video is in response to determining that the time difference is greater than the threshold time difference.

16. A system comprising:

a memory; and

at least one processor configured to access instructions stored on the memory that cause the at least one processor to perform operations comprising;

obtaining a plurality of themes from a storage, each of the plurality of themes including a respective theme definition, wherein the operation of obtaining the plurality of themes includes;

determining available themes from a themes database;

determining a respective diversity metric for each of the available themes, wherein the respective diversity metric is based on one or more of;

a number of generated videos for the available themes, or historical data for the available themes; and

selecting the plurality of themes from the available themes based at least in part on the respective diversity metric;

determining respective image criteria from the respective theme definition for each of the plurality of themes;

obtaining a plurality of images associated with a user from an image database;

determining a respective subset of the plurality of images that meet the respective image criteria for each of the plurality of themes;

selecting a particular theme of the plurality of themes for a video; and

generating the video based on the selected particular theme of the plurality of themes, wherein the video includes one or more images from the respective subset of the plurality of images for the particular theme.

17. The system of claim 16, wherein the operation of selecting the particular theme includes:

ranking the plurality of themes based on one or more of;

the respective diversity metric for the plurality of themes,

a respective engagement metric for the plurality of themes, or

a user profile for the user for the video; and

determining the particular theme based on the ranking.

18. The system of claim 16, wherein the operation of obtaining the plurality of themes further comprises:

determining a respective engagement metric for each of the available themes; and

selecting the plurality of themes from the available themes based at least in part on the respective engagement metric.

19. The system of claim 16, wherein the respective theme definition includes one or more parameters for generating the video, wherein the one or more parameters include at least one of:

a minimum or maximum duration for the video, a frame rate for playback of the video, an indication of audio data to be played with the video, a transition effect displayed between display of images of the video, one or more visual filters to be applied to the video, or an indication of a title for the video.

20. The system of claim 16, wherein the at least one processor is configured to perform further operations comprising:

determining a threshold time difference based on the respective theme definition;

determining that a time difference between a first timestamp associated with a first image of the plurality of images and a second timestamp associated with a second image of the plurality of images is greater than or less than the threshold time difference based on the respective theme definition; and

based on determining that the time difference meets the threshold time difference, generating a theme-based video that includes one or more of the plurality of images.