Context-based extraction and information overlay for photographic images

• US 10,217,000 B2
• Filed: 01/17/2017
• Issued: 02/26/2019
• Est. Priority Date: 01/17/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hardee, Christopher J., Rakshit, Sarbajit K.
• Primary Examiner(s): Potts, Ryan P
• Application Number:
  US15/407,484
• Publication Number:
  US 20180204063A1
• Time in Patent Office: 770 Days
• Field of Search: None
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

An example method for extracting information from images published to a network site includes generating a knowledge base of stored activity related image objects from the published images, wherein each activity related image object is indicative of an activity, and wherein each activity related image object is associated with an image location of the published images; performing a contextual analysis of a new image published to the network site to identify a new activity, to determine an activity position of the activity, and to determine a new image location; identifying a stored activity related image object from the knowledge base as relating to the new image location based on the image location of the published images; and overlaying an indicium of the stored activity related image object identified as relating to the image location of the new image on the new image.

Claims

1. A computer-implemented method for organizing information of a plurality of images published to a network site, the method comprising:

generating, by a processor, a knowledge base of stored activity related image objects from the plurality of published images, wherein each activity related image object is indicative of an activity, and wherein each activity related image object is associated with an image location of the plurality of published images;

performing a contextual analysis of a new image published to the network site to identify new activities, to determine an activity position of the activities in the new image, and to determine a new image location of the new image;

identifying at least two stored activity related image objects from the knowledge base as relating to the new image location based on the image location of the plurality of published images; and

overlaying, by the processor, an indicium of the at least two stored activity related image objects identified as relating to the image location of the new image on the new image at the activity position of the activities in the new image, the indicium being an animation of the at least two stored activity related image objects displayed sequentially to depict a sequence of activities.

2. The computer-implemented method of claim 1, wherein the indicium comprises text describing at least one of the new activities.

3. The computer-implemented method of claim 1, wherein the indicium comprises an icon relating to at least one of the new activities.

4. The computer-implemented method of claim 1, wherein the indicium comprises a hyperlink to a website that provides information about at least one of the new activities.

5. The computer-implemented method of claim 1, wherein generating the knowledge base further comprises identifying, by the processor, the plurality of published images from a plurality of user profiles.

6. The computer-implemented method of claim 1, wherein generating the knowledge base further comprises extracting, by the processor, activity related image objects from the plurality of published images.

7. The computer-implemented method of claim 1, wherein generating the knowledge base further comprises generating, by the processor, a correlation between the activity related image objects and the image location of the plurality of published images.

8. The computer-implemented method of claim 1, wherein generating the knowledge base further comprises performing, by the processor, a contextual analysis of data relating to the plurality of published images to identify the activity and to identify the image location.

9. The computer-implemented method of claim 1, wherein generating the knowledge base further comprises storing, by the processor, the activity related image objects as stored activity related image objects in the knowledge base.

10. A system for organizing information of a plurality of images published to a network site, the system comprising:

a memory comprising computer readable instructions; and

a processing device for executing the computer readable instructions for performing a method, the method comprising;

generating a knowledge base of stored activity related image objects from the plurality of published images, wherein each activity related image object is indicative of an activity, and wherein each activity related image object is associated with an image location of the plurality of published images;

performing a contextual analysis of a new image published to the network site to identify new activities, to determine an activity position of the activity in the new image, and to determine a new image location of the new image;

identifying at least two stored activity related image objects from the knowledge base as relating to the new image location based on the image location of the plurality of published images; and

overlaying an indicium of the at least two stored activity related image objects identified as relating to the image location of the new image on the new image at the activity position of the activities in the new image, the indicium being an animation of the at least two stored activity related image objects displayed sequentially to depict a sequence of activities.

11. The system of claim 10, wherein the indicium comprises text describing at least one of the new activities.

12. The system of claim 10, wherein the indicium comprises an icon relating to at least one of the new activities.

13. The system of claim 10, wherein the indicium comprises a hyperlink to a website that provides information about at least one of the new activities.

14. The system of claim 10, wherein generating the knowledge base further comprises identifying, by the processing device, the plurality of published images from a plurality of user profiles.

15. The system of claim 10, wherein generating the knowledge base further comprises extracting, by the processing device, activity related image objects from the plurality of published images.

16. The system of claim 10, wherein generating the knowledge base further comprises generating, by the processing device, a correlation between the activity related image objects and the image location of the plurality of published images.

17. The system of claim 10, wherein generating the knowledge base further comprises performing, by the processing device, a contextual analysis of data relating to the plurality of published images to identify the activity and to identify the image location.

18. A computer program product for organizing information of a plurality of images published to a network site, the computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processing device to cause the processing device to perform a method comprising;

generating a knowledge base of stored activity related image objects from the plurality of published images, wherein each activity related image object is indicative of an activity, and wherein each activity related image object is associated with an image location of the plurality of published images;

performing a contextual analysis of a new image published to the network site to identify new activities, to determine an activity position of the activities in the new image, and to determine a new image location of the new image;

identifying at least two stored activity related image objects from the knowledge base as relating to the new image location based on the image location of the plurality of published images; and

overlaying an indicium of the at least two stored activity related image objects identified as relating to the image location of the new image on the new image at the activity position of the activities in the new image, wherein the activity position is indicative of a location where the activities occurred in the new image, the indicium being an animation of the at least two stored activity related image objects displayed sequentially to depict a sequence of the activities.