Method for updating a knowledge base of a sentiment analysis system

• US 10,565,311 B2
• Filed: 02/15/2017
• Issued: 02/18/2020
• Est. Priority Date: 02/15/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Crudele, Michele, Perrone, Antonio
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Chavez, Rodrigo A
• Application Number:
  US15/433,717
• Publication Number:
  US 20180232359A1
• Time in Patent Office: 1,098 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A mechanism is provided updating a knowledge base of a sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term. Messages comprising natural language are received. Using content of the knowledge base, a decision is made as to whether at least one message of the received messages has a positive sentiment or a negative sentiment. A term is extracted from the message that is not present in the knowledge base. Based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, a score value of the term is calculated, and the term and the calculated score value are stored into the knowledge base.

Claims

1. A method, in a data processing apparatus comprising at least one processor and at least one memory, the at least one memory comprising instructions that are executed by the at least one processor to cause the at least one processor to be configured to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, the method comprising:

receiving, by the sentiment analysis system, messages comprising natural language from one or more public social media systems;

deciding, by the sentiment analysis system, using content of the knowledge base, whether at least one message of the received messages has a positive sentiment or a negative sentiment;

extracting, by the sentiment analysis system, a term rom the message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, calculating, by the sentiment analysis system, a score value of the term wherein the score value of the term is defined using the following formula;

2. The method of claim 1, further comprising:

classifying, by the sentiment analysis system, the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

3. The method of claim 2, further comprising:

for each set of the positive set of messages and the negative set of set messages;

identifying, by the sentiment analysis system, terms in each message of the set; and

creating, by the sentiment analysis system, a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.

4. The method of claim 1, wherein the term is stored if the frequency is higher than a predefined threshold.

5. The method of claim 2, further comprising:

for each set of the positive setoff messages and the negative set of messages;

identifying, by the sentiment analysis system, in each message of the set new terms that are not stored in the knowledge base; and

creating, by the sentiment analysis system, a set of candidates comprising the identified new terms having a frequency of occurrence higher than a predefined threshold, wherein extracting the term comprises selecting the term from the set of candidates.

6. The method of claim 5, further comprising:

repeating, by the sentiment analysis system, the extracting, the calculating, and the storing for each term of the set of candidates.

7. The method of claim 5, wherein the term is present only in the set of candidates of the positive set or in the set of candidates of the negative set.

8. The method of claim 1, further comprising:

storing, by the sentiment analysis system, the term in a first set of candidate terms having a negative sentiment if the frequency of occurrence of the term is greater than or equal to a first threshold and the term occurs in a message having a negative sentiment.

9. The method of claim 1, further comprising:

storing, by the sentiment analysis system, the term in a second set of candidate terms having a positive sentiment if the frequency of occurrence of the term is greater than or equal to a second threshold and the term occurs in a message having a positive sentiment.

10. The method of claim 9, further comprising:

calculating, by the sentiment analysis system, a difference set of a first set of candidate terms and a second set of candidate terms in order to obtain a final set of terms to be added to the knowledge base.

11. The method of claim 1, further comprising:

monitoring, by the sentiment analysis system, the frequency of occurrence of at least one term present in the knowledge base; and

updating, by the sentiment analysis system, the score value of the term or removing the term from the knowledge base based on the monitoring.

12. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable storage medium is not a transitory signal per se, and wherein the computer readable program, when executed on a computing device, causes the computing device to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, and further causes the computing device to:

receive, by the sentiment analysis system, messages comprising natural language from one or more public social media systems;

decide, by the sentiment analysis system, using content of a knowledge base, whether at least one message of the received messages has a positive sentiment or a negative sentiment;

extract, by the sentiment analysis system, a term from the message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, calculate, by the sentiment analysis system, a score value of the term wherein the score value of the term is defined using the following formula;

13. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

classify, by the sentiment analysis system, the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

14. The computer program product of claim 13, wherein computer readable program further causes the computing device to:

for each set of the positive set of message and the negative messages;

identify, the sentiment analysis system, term in each message of the set and

create, by the sentiment analysis system, a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.

15. An apparatus comprising;

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, and further causes the processor to;

receive, by the sentiment analysis system, multiple messages comprising natural language from one or more public social media systems;

decide, by the sentiment analysis system, using the content of a knowledge base, whether at least one message of the received messages has a positive or a negative sentiment;

extract, by the sentiment analysis system, a term from the message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the messages and the sentiment of the messages in that the term occurs, calculate, by the sentiment analysis system, a score value of the term wherein the score value of the term is defined using the following formula;

16. The apparatus of claim 15, wherein tine instructions further cause the processor to:

classify, by the sentiment analysis system the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

17. The apparatus of claim 16, wherein the instructions further cause the processor to:

for each set of the positive set of messages and the negative set of set messages;

identify, by the sentiment analysis system, terms in each message of the set; and

create, by the sentiment analysis system, a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the, knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.