Analyzing and exploring images posted on social media

• US 9,830,533 B2
• Filed: 12/30/2015
• Issued: 11/28/2017
• Est. Priority Date: 12/30/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cavalin, Paulo Rodrigo, Gatti, Maira Athanazio de Cerqueira, Mendes, Carlos, Moyano, Luis Gregorio, Pinhanez, Claudio Santos
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/984,542
• Publication Number:
  US 20170193333A1
• Time in Patent Office: 699 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Embodiments focus on the real-time processing and analysis of event-related images posts on social media. By computing the most posted images and grouping them in accordance to their similarity, these images can be organized in a visual interface to provide the users an image-driven social media topic detection system. The groups of images with the highest frequency represent the most prominent topics. By keeping track of the temporal occurrence of the images, it is possible to visualize the temporal evolution of the topics.

Claims

1. A computer implemented method of analyzing image data, the method comprising:

receiving, using a processor system, image data of one or more images that have been posted to internet websites;

analyzing, using the processor system, the image data to extract image feature data for each one of the one or more images;

analyzing, using the processor system, the image data to extract metadata of the one or more images;

creating multiple image files;

wherein each one of the multiple image files comprises individual ones of the one or more images linked together based on overlaps between the image feature data of the individual ones of the one or more images;

wherein each one of the multiple image files further comprises individual ones of the one or more images linked together based on overlaps between the metadata of the individual ones of the one or more images;

indexing the multiple image files to form multiple indexed image files;

wherein the indexing is based on the image feature data of the individual ones of the one or more images in an individual indexed image file;

wherein the indexing is further based on the metadata of the individual ones of the one or more images in the individual indexed image file; and

storing the multiple indexed image files in a memory of the processor system having a searchable indexed data storage structure.

2. The computer implemented method of claim 1 further comprising;

applying the multiple indexed image files to a clustering algorithm to generate clustered results; and

displaying the clustered results on a display of the processor system.

3. The computer implemented method of claim 1, wherein:

the metadata comprises multiple discrete time frames; and

the computer implemented method further comprises;

searching the searchable indexed data storage structure of the memory to access the multiple indexed image files based on a predetermined subject matter and the multiple discrete time frames; and

displaying results of the searching on a display of the processor system.

4. The computer implemented method of claim 3 further comprising:

accessing sentiment data based on textual sentiment posted on internet web sites;

wherein the textual sentiment corresponds to the predetermined subject matter and the multiple discrete time frames; and

displaying the sentiment data with the results of the searching.

5. The computer implemented method of claim 1, wherein:

the processor system includes a machine learning processor;

the analyzing of the image data to extract image features data for each one of the one or more images is performed using the machine learning processor; and

the analyzing of the image data to extract metadata of the one or more images is performed using the machine learning processor.

6. The computer implemented method of claim 5 wherein the image features data comprises image feature vectors.

7. The computer implemented method of claim 5 wherein the metadata comprises metadata vectors.

8. A computer system for analyzing image data, the system comprising:

a memory; and

a processor system communicatively coupled to the memory;

the processor system configured to perform a method comprising;

receiving image data of one or more images that have been posted to internet websites;

analyzing the image data to extract image feature data for each one of the one or more images;

analyzing the image data to extract metadata of the one or more images;

creating multiple image files;

wherein each one of the multiple image files comprises individual ones of the one or more images linked together based on overlaps between the image feature data of the individual ones of the one or more images;

wherein each one of the multiple image files further comprises individual ones of the one or more images linked together based on overlaps between the metadata of the individual ones of the one or more images;

indexing the multiple image files to form multiple indexed image files;

wherein the indexing is based on the image feature data of the individual ones of the one or more images in an individual indexed image file;

wherein the indexing is further based on the metadata of the individual ones of the one or more images in the individual indexed image file; and

storing the multiple indexed image files in the memory, wherein the memory includes a searchable indexed data storage structure.

9. The computer system of claim 8 further comprising;

applying the multiple indexed image files to a clustering algorithm to generate clustered results; and

displaying the clustered results on a display of the processor system.

10. The computer system of claim 8, wherein:

the metadata comprises multiple discrete time frames; and

the method performed by the processor system further comprises;

searching the searchable indexed data storage structure of the memory to access the multiple indexed image files based on a predetermined subject matter and the multiple discrete time frames; and

displaying results of the searching on a display of the processor system.

11. The computer system of claim 10 further comprising:

accessing sentiment data based on textual sentiment posted on internet web sites;

wherein the textual sentiment corresponds to the predetermined subject matter and the multiple discrete time frames; and

displaying the sentiment data with the results of the searching.

12. The computer system of claim 8, wherein:

the processor system includes a machine learning processor;

the analyzing of the image data to extract image features data for each one of the one or more images is performed using the machine learning processor;

the analyzing of the image data to extract metadata of the one or more images is performed using the machine learning processor.

13. The computer system of claim 12 wherein the image features data comprises image feature vectors.

14. The computer system of claim 12 wherein the metadata comprises metadata vectors.

15. A computer program product for analyzing image data, the computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions readable by a processor system to cause the processor system to perform a method comprising;

receiving, using the processor system, image data of one or more images that have been posted to internet websites;

analyzing, using the processor system, the image data to extract image features data for each one of the one or more images;

analyzing, using the processor system, the image data to extract metadata of the one or more images;

creating multiple image files;

wherein each one of the multiple image files comprises individual ones of the one or more images linked together based on overlaps between the image feature data of the individual ones of the one or more images;

wherein each one of the multiple image files further comprises individual ones of the one or more images linked together based on overlaps between the metadata of the individual ones of the one or more images;

indexing the multiple image files to form multiple indexed image files;

wherein the indexing is based on the image feature data of the individual ones of the one or more images in an individual indexed image file;

wherein the indexing is further based on the metadata of the individual ones of the one or more images in the individual indexed image file; and

storing the multiple indexed image files in a memory of the processor system having a searchable indexed data storage structure.

16. The computer program product of claim 15 further comprising;

applying the multiple indexed image files to a clustering algorithm to generate clustered results; and

displaying the clustered results on a display of the processor system.

17. The computer program product of claim 15, wherein:

the metadata comprises multiple discrete time frames; and

the method performed by the processor system further comprises;

searching the searchable indexed data storage structure of the memory to access the multiple indexed image files based on a predetermined subject matter and the multiple discrete time frames; and

displaying results of the searching on a display of the processor system.

18. The computer program product of claim 17 further comprising:

accessing sentiment data based on textual sentiment posted on internet web sites;

wherein the textual sentiment corresponds to the predetermined subject matter and the multiple discrete time frames; and

displaying the sentiment data with the results of the searching.

19. The computer program product of claim 15, wherein:

the processor system includes a machine learning processor;

the analyzing of the image data to extract image features data for each one of the one or more images is performed using the machine learning processor;

the analyzing of the image data to extract metadata of the one or more images is performed using the machine learning processor.

20. The computer program product of claim 15 wherein:

the image features data comprises image feature vectors; and

the metadata comprises metadata vectors.