Method for updating a knowledge base of a sentiment analysis system

• US 11,755,841 B2
• Filed: 11/19/2019
• Issued: 09/12/2023
• Est. Priority Date: 02/15/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Crudele, Michele, Perrone, Antonio
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Chavez, Rodrigo A
• Application Number:
  US16/687,915
• Publication Number:
  US 20200134264A1
• Time in Patent Office: 1,393 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A mechanism is provided updating a knowledge base of a sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term. Messages comprising natural language are received. Using content of the knowledge base, a decision is made as to whether at least one message of the received messages has a positive sentiment or a negative sentiment. A term is extracted from the message that is not present in the knowledge base. Based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, a score value of the term is calculated, and the term and the calculated score value are stored into the knowledge base.

Claims

1. A method, in a data processing apparatus comprising at least one processor and at least one memory, the at least one memory comprising instructions that are executed by the at least one processor to cause the at least one processor to be configured to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, the method comprising:

receiving messages comprising natural language from one or more public social media systems;

deciding using content of the knowledge base, whether at least one message of the received messages has a positive sentiment or a negative sentiment;

classifying the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment;

extracting a term from the at least one message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, calculating a score value of the term; and

storing the term and the calculated score value into the knowledge base, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages, and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

2. The method of claim 1, further comprising:

for each set of the positive set of messages and the negative set of set messages;

identifying terms in each message of the set; and

creating a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.

3. The method of claim 2, wherein the score value of the term is defined using the following formula:

4. The method of claim 1, wherein the term is stored if the frequency is higher than a predefined threshold.

5. The method of claim 1, further comprising:

for each set of the positive set of messages and the negative set of messages;

identifying in each message of the set new terms that are not stored in the knowledge base; and

creating a set of candidates comprising the identified new terms having a frequency of occurrence higher than a predefined threshold, wherein extracting the term comprises selecting the term from the set of candidates.

6. The method of claim 5, further comprising:

repeating the extracting, the calculating, and the storing for each term of the set of candidates.

7. The method of claim 5, wherein the term is present only in the set of candidates of the positive set or in the set of candidates of the negative set.

8. The method of claim 1, further comprising:

storing the term in a first set of candidate terms having a negative sentiment if the frequency of occurrence of the term is greater than or equal to a first threshold and the term occurs in a message having a negative sentiment.

9. The method of claim 1, further comprising:

storing the term in a second set of candidate terms having a positive sentiment if the frequency of occurrence of the term is greater than or equal to a second threshold and the term occurs in a message having a positive sentiment.

10. The method of claim 9, further comprising:

calculating a difference set of a first set of candidate terms and a second set of candidate terms in order to obtain a final set of terms to be added to the knowledge base.

11. The method of claim 1, further comprising:

monitoring the frequency of occurrence of at least one term present in the knowledge base; and

updating the score value of the term or removing the term from the knowledge base based on the monitoring.

12. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable storage medium is not a transitory signal per se, and wherein the computer readable program, when executed on a computing device, causes the computing device to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, and further causes the computing device to:

receive messages comprising natural language from one or more public social media systems;

decide using content of a knowledge base, whether at least one message of the received messages has a positive sentiment or a negative sentiment;

classify the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment;

extract a term from the at least one message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, calculate a score value of the term; and

store the term and the calculated score value into the knowledge base, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages, and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

13. The computer program product of claim 12, wherein the computer readable program further causes the computing device to:

for each set of the positive set of messages and the negative set of set messages;

identify terms in each message of the set; and

create a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.

14. The computer program product of claim 13, wherein the score value of the term is defined using the following formula:

15. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to implement a sentiment analysis system for updating a knowledge base of the sentiment analysis system, the knowledge base being operable for storing natural language terms and a score value related to each natural language term, the score value characterizing the sentiment of the natural language term, and further causes the processor to;

receive multiple messages comprising natural language from one or more public social media systems;

decide using the content of a knowledge base, whether at least one message of the received messages has a positive or a negative sentiment;

classify the received messages into a positive set of messages having a positive sentiment and a negative set of messages having a negative sentiment;

extract a term from the at least one message that is not present in the knowledge base;

based on a frequency of occurrence of the term in the received messages and the sentiment of the messages in which the term occurs, calculate a score value of the term; and

store the term and the calculated score value into the knowledge base, wherein the term is extracted from the message of one of the positive set of messages or the negative set of messages, and wherein the frequency of occurrence is the fraction of messages of the one of the positive set of messages and negative set of messages that contain the term.

16. The apparatus of claim 15, wherein the instructions further cause the processor to:

for each set of the positive set of messages and the negative set of set messages;

identify terms in each message of the set; and

create a vocabulary comprising the identified terms, wherein each identified term is tagged as a new term or as an existing term in the knowledge base and wherein extracting the term comprises selecting a term of the vocabulary being tagged as a new term.

17. The apparatus of claim 16, wherein the score value of the term is defined using the following formula: