Methods and systems for analysis of real-time user-generated text messages

• US 8,688,791 B2
• Filed: 02/16/2011
• Issued: 04/01/2014
• Est. Priority Date: 02/17/2010
• Status: Active Grant

• Current Assignee: Wright State University
• Original Assignee: Wright State University
• Inventors: Sheth, Amit, Gomadam, Karthik, Nagarajan, Meenakshi
• Primary Examiner(s): Chriss, Andrew
• Assistant Examiner(s): MAK, PETER K
• Application Number:
  US13/028,397
• Publication Number:
  US 20120042022A1
• Time in Patent Office: 1,140 Days
• Field of Search: 709/206, 709/224
• US Class Current: 709/206
• CPC Class Codes: H04L 51/222 using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

The present invention generally relates to methods and systems for analysis of real-time user-generated text messages. The methods and systems allow analysis to be performed using term associations and geographical and temporal constraints.

Claims

1. A system for analyzing electronic messages related to an event in context of a given geographical location and time period, comprising a processing circuit that stores one or more electronic messages related to the event in a memory, wherein the one or more electronic messages comprise information text and metadata providing temporal and geographical information;

the processing circuit further determines sets of electronic messages using the temporal and geographical information provided by the one or more electronic messages to create a plurality of spatio-temporal sets of electronic messages, wherein the plurality of spatio-temporal sets of electronic messages comprise a spatio-temporal set of electronic messages for the event in a given geographical location and time period, and wherein the spatio-temporal set of electronic messages is related to the event;

the processing circuit also parses the spatio-temporal set of electronic messages for the event to determine event descriptor terms from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

the processing circuit further generates term frequency values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period and association strength values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

wherein the processing circuit also uses the term frequency values and the association strength values to generate term weight values that indicate relative thematic importance of the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period; and

wherein the processing circuit discounts the term weight values that indicate the relative thematic importance of the event descriptor terms based upon a fraction of the spatio-temporal sets of electronic messages in which the event descriptor terms occur;

the processing circuit further uses the term weight values to select one or more event descriptor terms for the event in the given geographical location and time period, wherein the selected one or more event descriptor terms collectively reflect a general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set; and

wherein the processing circuit also provides the selected event descriptor terms which collectively reflect the general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set to an electronic display.

2. The system of claim 1, wherein the processing circuit further generates the term frequency values using term frequency-inverse document frequency scores for the event descriptor terms.

3. The system of claim 2, wherein the processing circuit further generates the association strength values using the word co-occurrence frequencies between the event descriptor terms.

4. The system of claim 3, wherein the word co-occurrence frequencies are calculated by the processing circuit in part by determining a context for each event descriptor term;

wherein the context for each event descriptor term comprises a set of associated words that co-occur with the event descriptor term in an electronic message; and

wherein the set of associated words is limited to the most frequently co-occurring words in the message.

5. The system of claim 4, wherein the word co-occurrence frequencies are further calculated in part by determining the point-wise mutual information between each event descriptor term and the context of each word in the set of associated words.

6. The system of claim 1, wherein the given geographical location and time period spans a duration of time;

and

the processing circuit discounts the term weight values that indicate the relative thematic importance of the event descriptor terms based upon a relative time of occurrence of the event descriptor terms within the duration of time.

7. The system of claim 1, wherein the processing circuit also determines visual indicia characteristics for the selected event descriptor terms using the term weight values and provides the visual indicia characteristics to the display.

8. A method for analyzing electronic messages related to an event in context of a given geographical location and time period, comprising:

storing one or more electronic messages related to the event in a memory, wherein the one or more electronic messages comprise information text and metadata providing temporal and geographical information;

determining, via a processor coupled to the memory, a set of electronic messages using the temporal and geographical information provided by the one or more electronic messages that is related to the event to create a spatio-temporal set of electronic messages for the event in a given geographical location and time period;

parsing, via the processor, the spatio-temporal set of electronic messages for the event in the given geographical location and time period to determine event descriptor terms from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

generating, via the processor, term frequency values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

generating, via the processor, association strength values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

using, via the processor, the term frequency values and the association strength values to generate term weight values that indicate relative thematic importance of the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period, wherein the given geographical location and time period spans a duration of time;

discounting, via the processor, the term weight values that indicate the relative thematic importance of the event descriptor terms based upon a relative time of occurrence of the event descriptor terms within the duration of time;

using, via the processor, the term weight values to select one or more event descriptor terms for the event in the given geographical location and time period, wherein the selected one or more event descriptor terms collectively reflect a general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set; and

providing, via the processor, the selected event descriptor terms which collectively reflect the general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set to an electronic display coupled to the processor.

9. The method of claim 8, wherein the term frequency values are generated using term frequency-inverse document frequency scores of the event descriptor terms.

10. The method of claim 9, wherein the association strength values are generated using the word co-occurrence frequencies between the event descriptor terms.

11. The method of claim 10, wherein the word co-occurrence frequencies are calculated in part by determining a context for each event descriptor term;

wherein the context for each event descriptor term comprises a set of associated words that co-occur with the event descriptor term in an electronic message; and

wherein the set of associated words is limited to the most frequently co-occurring words in the message.

12. The method of claim 11, wherein the word co-occurrence frequencies are further calculated in part by determining the point-wise mutual information between each event descriptor term and the context of each word in the set of associated words.

13. The method of claim 8, further comprising:

determining, via the processor, an additional set of electronic messages using the temporal and geographical information provided by the one or more electronic messages; and

discounting, via the processor, the term weight values that indicate the relative thematic importance of the event descriptor terms based upon a fraction of a group of sets of electronic messages in which the event descriptor terms occur, wherein the group of sets of electronic messages comprise the spatio-temporal set of electronic messages and the additional set of electronic messages.

14. A system for analyzing electronic messages related to an event in context of a given geographical location and time period, comprising:

a memory containing program instructions and one or more electronic messages related to the event, wherein the one or more electronic messages comprise information text and metadata providing temporal and geographical information;

a display; and

a processor coupled to the memory and the display, said program instructions when read from the memory and executed by the processor cause the processor to;

determine sets of electronic messages using the temporal and geographical information provided by the one or more electronic messages to create a plurality of spatio-temporal sets of electronic messages, wherein the plurality of spatio-temporal sets of electronic messages comprise a spatio-temporal set of electronic messages for the event in a given geographical location and time period, and wherein the spatio-temporal set of electronic messages is related to the event;

parse spatio-temporal set of electronic messages for the event to determine event descriptor terms from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

generate term frequency values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

generate association strength values for the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period;

use the term frequency values and the association strength values to generate term weight values that indicate relative thematic importance of the event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period, wherein the given geographical location and time period spans a duration of time;

discount the term weight values that indicate the relative thematic importance of the event descriptor terms based upon a fraction of the spatio-temporal sets of electronic messages in which the event descriptor terms occur and a relative time of occurrence of the event descriptor terms within the duration of time;

use the term weight values to select one or more event descriptor terms determined from the information text provided by the electronic messages in the spatio-temporal set for the event in the given geographical location and time period, wherein the selected one or more event descriptor terms collectively reflect a general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set; and

provide the selected event descriptor terms which collectively reflect the general viewpoint or opinion of the event in the given geographical location and time period based on the electronic messages from the spatio-temporal set to the electronic display.

15. The system of claim 14, wherein the term frequency values are generated using term frequency-inverse document frequency scores of the event descriptor terms.

16. The system of claim 15, wherein the association strength values are generated using the word co-occurrence frequencies between the event descriptor terms.

17. The system of claim 16, wherein the word co-occurrence frequencies are calculated in part by determining a context for each event descriptor term;

wherein the context for each event descriptor term comprises a set of associated words that co-occur with the event descriptor term in an electronic message; and

wherein the set of associated words is limited to the most frequently co-occurring words in the message.

18. The system of claim 17, wherein the word co-occurrence frequencies are further calculated in part by determining the point-wise mutual information between each event descriptor term and the context of each word in the set of associated words.

19. The system of claim 14, wherein the program instructions further cause the processor to use the term weight values to determine visual indicia characteristics for the selected event descriptor terms and to provide the visual indicia characteristics to the display.