Social Media Event Detection and Content-Based Retrieval

• US 20140085329A1
• Filed: 09/24/2012
• Published: 03/27/2014
• Est. Priority Date: 09/24/2012
• Status: Active Grant

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Codella, Noel C., Smith, John R., Natsev, Apostol
• Granted Patent: US 9,002,069 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and article of manufacture for social media event detection and content-based retrieval include obtaining multiple images from at least one social media source, extracting at least one visual semantic concept from the multiple images, differentiating an event semantic concept signal from a background semantic concept signal to detect an event in the multiple images, and retrieving one or more images associated with the event semantic concept signal for presentation as a visual description of the detected event.

Claims

1. An article of manufacture comprising a computer readable storage medium having computer readable instructions tangibly embodied thereon which, when implemented, cause a computer to carry out a plurality of method steps comprising:

obtaining multiple images from at least one social media source;

extracting at least one visual semantic concept from the multiple images;

differentiating an event semantic concept signal from a background semantic concept signal to detect an event in the multiple images; and

retrieving one or more images associated with the event semantic concept signal for presentation as a visual description of the detected event.

2. The article of manufacture of claim 1, wherein the multiple images from at least one social media source comprise images derived from photo and/or video input.

3. The article of manufacture of claim 2, wherein the photo and/or video input comprises user-posted images and/or videos on at least one social media source.

4. The article of manufacture of claim 1, wherein said obtaining comprises streaming multiple images from at least one social media source in real-time.

5. The article of manufacture of claim 1, wherein the at least one visual semantic concept characterizes an event.

6. The article of manufacture of claim 1, wherein said differentiating is based on comparing observed signal strength of each of the at least one extracted visual semantic concept to a predetermined outlier threshold.

7. The article of manufacture of claim 1, wherein said differentiating comprises measuring a mean and standard deviation of each semantic concept over a user-specified background time period.

8. The article of manufacture of claim 1, wherein said differentiating comprises using an interactive machine learning algorithm.

9. The article of manufacture of claim 1, wherein said differentiating comprises performing a statistical analysis to compute a probability that an event has occurred.

10. The article of manufacture of claim 1, wherein the method steps comprise:

measuring the signal of each semantic concept over time.

11. The article of manufacture of claim 1, wherein the method steps comprise:

identifying one or more semantic concept signal peaks that signify a real-time event.

12. The article of manufacture of claim 1, wherein the method steps comprise:

overlaying images annotated with a global position system (GPS) function onto a map to provide a geo-spatial representation of the event.

13. The article of manufacture of claim 1, wherein the method steps comprise:

retrieving one or more additional images that are relevant to a detected event.

14. The article of manufacture of claim 1, wherein the method steps comprise:

monitoring semantics of images uploaded to one or more social media websites over a specified time period; and

streaming images to a parallel system which applies a set of generated semantic models to the images in real-time.

15. The article of manufacture of claim 1, wherein the method steps comprise:

specifying a background time period and an event time period.

16. A system for detecting an event via social media content, comprising:

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory; and

at least one processor coupled to the memory and operative for;

obtaining multiple images from at least one social media source;

extracting at least one visual semantic concept from the multiple images;

differentiating an event semantic concept signal from a background semantic concept signal to detect an event in the multiple images; and

retrieving one or more images associated with the event semantic concept signal for presentation as a visual description of the detected event.

17. An article of manufacture comprising a computer readable storage medium having computer readable instructions tangibly embodied thereon which, when implemented, cause a computer to carry out a plurality of method steps comprising:

obtaining multiple images from at least one social media source;

extracting at least one visual semantic concept from the multiple images;

differentiating an event semantic concept signal from a background semantic concept signal to detect an event in the multiple images;

retrieving one or more images associated with the event semantic concept signal;

grouping the one or more images associated with the event semantic concept signal;

annotating the group of one or more images with user feedback; and

displaying the annotated group of one or more images as a visual description of the detected event.

18. The article of manufacture of claim 17, wherein said differentiating is based on comparing signal strength of each of the at least one extracted visual semantic concept to a predetermined outlier threshold.

19. The article of manufacture of claim 17, wherein the method steps comprise:

overlaying images annotated with a global position system (GPS) function onto a map to provide a geo-spatial representation of the event.

20. A system for detecting an event via social media content, comprising:

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory; and

at least one processor coupled to the memory and operative for;

obtaining multiple images from at least one social media source;

extracting at least one visual semantic concept from the multiple images;

differentiating an event semantic concept signal from a background semantic concept signal to detect an event in the multiple images;

retrieving one or more images associated with the event semantic concept signal;

grouping the one or more images associated with the event semantic concept signal;

annotating the group of one or more images with user feedback; and

displaying the annotated group of one or more images as a visual description of the detected event.