NATURAL LANGUAGE PROCESSING OPTIMIZED FOR MICRO CONTENT

• US 20130290317A1
• Filed: 02/19/2013
• Published: 10/31/2013
• Est. Priority Date: 02/17/2012
• Status: Active Grant

• Current Assignee: Blazent Incorporated (ServiceNow Incorporated)
• Original Assignee: Bottlenose, Inc. (ServiceNow Incorporated)
• Inventors: ter Heide, Dominiek, Spivack, Nova T.
• Granted Patent: US 8,832,092 B2
• Time in Patent Office: Days
• US Class Current: 707/723
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A system and a method for microcontent natural language processing are presented. The method comprising steps of receiving a microcontent message from a social networking server, tokenizing the microcontent message into one or more text tokens, detecting the language of the microcontent message and selecting the property dictionary for part-of-speech tag, part-of-speech tagging the microcontent message to identify related pronouns and nouns based on the selected dictionary, and extracting topics form the microcontent messages and assigning confidence values to the topics.

Claims

1. A method for microcontent natural language processing comprising:

receiving a microcontent message from a social networking server;

breaking up the microcontent message into one or more text tokens;

identifying type metadata of the microcontent message from an ontology of predetermined microcontent types by applying a database of annotation rules to the text tokens of the microcontent message; and

associating the identified type metadata to the microcontent message.

2. The method of claim 1, wherein the ontology of predetermined microcontent types are organized as a relational database including trees of the predetermined microcontent types.

3. The method of claim 2, wherein the annotation rules includes list of names and patterns that help assigning a tree of the trees of the predetermined microcontent types to the microcontent message.

4. The method of claim 1, further comprising:

providing an interface for users to edit the ontology of predetermined microcontent types or the database of annotation rules.

5. The method of claim 1, wherein the step of identifying type metadata of the microcontent message includes:

compiling a matching index including a set of expressions based on the ontology of predetermined microcontent types or the database of annotation rules; and

matching the expressions of the matching index with the microcontent message.

6. The method of claim 1, further comprising:

performing a language detection on the microcontent message to detect language metadata; and

associating the detected language metadata with the microcontent message.

7. The method of claim 6, wherein the language detection is based on a dictionary-based language detector or an NGram language detector.

8. The method of claim 1, further comprising:

performing a speech tagging on the microcontent message to identify nouns in the microcontent message.

9. The method of claim 8, wherein the speech tagging is based on a Brill tagger method.

10. The method of claim 1, further comprising:

performing a topic extraction on the microcontent message to extract topic metadata;

assigning a confidence score for each piece of the extracted topic metadata; and

associating the topic metadata and confidence scores with the microcontent message.

11. The method of claim 10, wherein the topic extraction is based on factors including capitalization.

12. The method of claim 1, further comprising:

identifying sentiment metadata by performing a sentiment analysis on the microcontent message; and

associating the sentiment metadata with the microcontent message.

13. The method of claim 12, wherein the sentiment analysis is based on a Naï

ve Bayesian classifier that is trained offline to an annotated set of positive, negative and neutral microcontent messages.

14. A method for message stream rank comprising:

receiving a message stream including a plurality of messages, wherein each of the messages is associated with metadata attributes;

counting the occurrence of the metadata attributes associated with the messages in the message stream; and

generating a list of trending topics based on the occurrences of the metadata attributes.

15. The method of claim 14, further comprising:

producing a trend database, wherein each record of the trend database includes a count for a specific condition based on the metadata attributes in which a message occurred in a timeframe.

16. The method of claim 15, further comprising:

presenting list of trending metadata attributes based on the counts of the records of the trend database.

17. The method of claim 14, wherein the metadata attributes include topics, types, hashtags, mentioned people, author, links, geographic segments, ethnicity, or genders.

18. The method of claim 15, wherein a record of the trend database further include a relative counter, an acceleration counter or a ratio counter.

19. The method of claim 14, further comprising:

identifying an audience segment correlated with a trending topic based on the messages from users of the audience segment.

20. The method of claim 14, further comprising:

presenting the trending topics as scatters in quadrant scatter plot, wherein a volume of each scatter indicates a metadata attribute of the topics.

21. The method of claim 20, further comprising:

producing an animation of the quadrant scatter plot over a time period to reflect how users'"'"' attentions changes on the trending topics.

22. A system comprising:

a network component configured for receiving a microcontent message from a social networking server;

a processor; and

a memory storing instructions which, when executed by the processor, cause the system to perform a process including;

breaking up the microcontent message into one or more text tokens;

identifying type metadata of the microcontent message from an ontology of predetermined microcontent types by applying a database of annotation rules to the text tokens of the microcontent message; and

associating the identified type metadata to the microcontent message.

23. The system of claim 22, wherein the step of identifying type metadata of the microcontent message includes:

compiling a matching index including a set of expressions based on the ontology of predetermined microcontent types or the database of annotation rules; and

matching the expressions of the matching index with the microcontent message.

24. The system of claim 22, wherein the process further includes:

performing a language detection on the microcontent message to detect language metadata;

performing a speech tagging on the microcontent message to identify nouns in the microcontent message;

performing a topic extraction on the microcontent message to extract topic metadata; and

identifying sentiment metadata by performing a sentiment analysis on the microcontent message.

25. The system of claim 22, wherein the process is performed as a functionality of a browser plug-in application.

26. A method for microcontent natural language processing comprising:

receiving a microcontent from a social networking site;

tokenizing the microcontent;

selecting a dictionary based on a detected language of the microcontent; and

identifying related topics of the microcontent using part-of-speech tagging based on the dictionary.

27. The method of claim 26, wherein the topics are indicated by pronouns and nouns form the microcontent.

28. The method of claim 26, wherein the microcontent is tokenized into a plurality of token texts.

29. The method of claim 26, further comprising:

receiving a message stream including the microcontent, wherein the microcontent and other messages in the message stream are associated with metadata attributes;

counting the occurrence of the metadata attributes; and

generating a list of trending topics based on the occurrences of the metadata attributes.