Intelligent selection of images to create image narratives

• US 10,242,034 B1
• Filed: 12/28/2015
• Issued: 03/26/2019
• Est. Priority Date: 12/28/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Li, Yi, Wu, Yuxin, Chen, Yushan, Wang, Meng
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US14/980,775
• Time in Patent Office: 1,184 Days
• Field of Search: 707696, 382170, 382305, 282170, 282305
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Images are intelligently selected to create image narratives. Instead of a user having to manually search and locate images to view, the images to associate with a particular image narrative are programmatically determined. Many different types of image narratives may be created. For example, one image narrative may show images that include both a first user and a second user over some period of time. Another image narrative may show images that relate to an activity that a first user enjoys or an event that included the user (e.g., a graduation). The tags and metadata associated with the images of the user are analyzed to determine the tags that are important to the user. For example, the importance might be determined based on the frequency of the tags within the images. After creation, the user may select one of the image narratives to view the associated images.

Claims

1. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to:

access images associated with a user;

identify tags that are associated with the images, wherein individual ones of the tags identify at least one or more objects within the images, one or more individuals within the images, or one or more scenes within the images;

determine frequencies of the tags, wherein the frequencies include a first frequency of a first tag, a second frequency of a second tag, and a third frequency of a third tag;

identify a portion of the tags that occur above a threshold based, at least in part, on the frequencies;

identify candidate images associated with the portion of the tags that occur above the threshold;

determining whether the candidate images are associated with a particular time period, an event, an individual, or an activity;

identify, based at least in part on the determining, narrative images for an image narrative from the candidate images;

select at least a first portion of the narrative images based, at least in part, on a first one of the tags that occur above the threshold;

associate the at least first portion of the images with an image narrative;

provide for display, to a client computing device, an indication of the image narrative;

receive an indication of a selection of the image narrative; and

provide, to the client computing device, an instruction to display a portion of the narrative images associated with the image narrative.

2. The non-transitory computer-readable storage medium of claim 1, wherein the computer-executable instructions further cause the computer to sort] the tags according to a number of times the tags appear in the images, and selecting the first one of the tags based, at least in part, on the sorting of the tags.

3. The non-transitory computer-readable storage medium of claim 1, wherein select at least the first portion of the images comprises selecting the first portion of the narrative images based, at least in part, on a confidence level associated with the tags.

4. The non-transitory computer-readable storage medium of claim 1, wherein the computer-executable instructions further cause the computer to store, within a data store of the client computing device, an indication of narrative images identified.

5. The non-transitory computer-readable storage medium of claim 1, wherein the computer-executable instructions further cause the computer to identify tags modified by the user, and identify the narrative images, based, at least in part, on the tags modified by the user.

6. A system, comprising:

a data store operative to store images associated with a first user; and

one or more computing devices performing operations to;

access, from the data store, at least a portion of the images associated with a first user;

obtain tags associated with the images, the tags describing individuals identified, scenes identified, and locations identified;

determine frequencies of the tags for the images;

identify, from the tags, first images from the images associated with the first user that depict one or more of;

an activity that the user participates in;

or an event the user attended;

select a first portion of the first images based, at least in part, on the frequencies of the tags for the images and a determination that a confidence score of the first portion of the first images is above a confidence threshold, the confidence score indicating a level of confidence that tags within the first portion of the first images are associated;

associate the first portion of the first images with an image narrative; and

provide, to a client computing device, data identifying the first portion of the first images of the image narrative.

7. The system of claim 6, wherein the one or more computing devices perform further operations to:

receive an indication of a selection of a user interface element associated with the image narrative; and

provide, to the client computing device, an instruction to provide a display of the image narrative.

8. The system of claim 6, wherein the one or more computing devices perform further operations to determine a user and a second user depicted within at least a portion of the images.

9. The system of claim 8, wherein select the first portion of the first images, comprises selecting the first portion of the first images based at least in part on a confidence score that a first tag exceeds a first confidence level and a second tag exceeds a second confidence level.

10. The system of claim 6, wherein the one or more computing devices perform further operations to store, within the data store, an indication of the first portion of the first images associated with the image narrative.

11. The system of claim 6, wherein the one or more computing devices perform further operations to identify tags modified by the user, and select the first portion of the first images based, at least in part, on the tags modified by the user.

12. The system of claim 6, wherein the one or more computing devices perform further operations to access, from one or more other image sources, second images associated with the user, identify second tags from image data of the second images, and aggregate the second tags with the first tags.

13. The system of claim 6, wherein the image narrative includes images that relate to one or more of an event at a particular location, an activity over a period of time, or a connection between the first user and a second user over a period of time.

14. The system of claim 13, wherein the images for the image narrative are based at least in part on the period of time.

15. The system of claim 6, wherein the one or more computing devices perform further operations to provide data for display on the client computing device within a graphical user interface that includes a first user interface element for the image narrative and a second user interface element for a second image narrative.

16. A computer-implemented method, comprising:

accessing, from one or more data stores, images associated with a first user;

obtaining tags associated with the images, the tags identifying faces within the images, scenes within the images, and locations of the images;

identifying a first portion of the images to associate with a first image narrative based, at least in part, on a number of occurrences of individual ones of the tags and one or more of a specified time period, an activity, an event, or a friend identified by the tags; and

providing, to a computing device, data identifying the images associated with the image narrative.

17. The computer-implemented method of claim 16, wherein accessing the images associated with the first user comprises accessing a local storage of a client computing device and accessing a social networking site.

18. The computer-implemented method of claim 16, further comprising:

providing for presentation by a user interface, a first user interface element associated with the first image narrative and a second user interface element associated with a second image narrative;

receiving, from the client computing device, an indication of a selected one of the first image narrative or the second image narrative; and

providing, to the client computing device, an instruction to provide for display the selected one of the first image narrative or the second image narrative.

19. The computer-implemented method of claim 16, wherein identifying the first portion of the images to associate with the first image narrative comprise identifying one or more of:

a frequently occurring scene;

a frequently occurring face other than a face of the first user;

or a frequently occurring activity and identifying the first portion of the images based, at least in part, on a determination that a confidence score of the first portion of the images is above a confidence threshold, the confidence score indicating a level of confidence that tags within the first portion of the images are associated.

20. The computer-implemented method of claim 16, further comprising:

identifying tags modified by the first user; and

identifying the at least the first portion of the images based, at least in part, on the tags modified by the first user.