Method and system for extracting opinions from text documents

• US 8,200,477 B2
• Filed: 10/22/2003
• Issued: 06/12/2012
• Est. Priority Date: 10/22/2003
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yi, Jeonghee, Nasukawa, Tetsuya, Bunescu, Razvan Constantin
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US10/692,025
• Publication Number:
  US 20050091038A1
• Time in Patent Office: 3,156 Days
• Field of Search: 704/1, 704/256.5, 704/9, 704/209, 704/260, 704/4, 706/45, 707/100, 707/5, 707/3, 709/224, 709/203, 715/531, 715/202, 715/256, 715/532
• US Class Current: 704/9
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: Y10S 707/939 Idea solicitation

Abstract

A method and system for extracting opinions about a subject of interest from a text document in which each sentence is analyzed individually to identify the opinions. The most relevant feature terms related to the subject are extracted from the document based on their relevancy scores. Candidate feature terms are definite noun phrases at the beginning of the sentences. For each sentence that refers to the subject or a feature term, the invention determines whether the sentence includes an opinion polarity about the subject or the feature term. The opinion polarity is detected by identifying opinion terms in the sentence using an opinion dictionary or an opinion rule base, parsing the sentence with an English parser to identify grammatical components in the sentence and their relationships, and finding a matching entry in the dictionary or the rule base.

Claims

1. A method performed on a computer for extracting opinions about a subject of interest from a text document having a plurality of sentences, the subject associated with a plurality of features, the method comprising:

determining from the document feature terms related to the features most relevant to the subject, comprising;

determining a definite article is located at the beginning of a sentence in the plurality of sentences; and

determining the definite article precedes the feature term;

extracting from the document feature terms related to the features most relevant to the subject;

for each sentence referring to a feature term, determining whether the sentence includes an opinion polarity about the feature term;

for each sentence referring to the subject, determining whether the sentence includes an opinion polarity about the subject,

wherein the determining comprises;

identifying opinion terms in the sentence using an opinion dictionary, each entry in the dictionary having an opinion term, a part-of-speech tag, and an associated opinion polarity,

for each sentence having a feature term and an opinion term, parsing the sentence with an English parser to identify grammatical components in the sentence and relationships between said components, and

identifying an opinion polarity associated with said feature term using the opinion dictionary,

wherein;

the grammatical components include verb phrases, subject phrases, object phrases, complements, and prepositional phrases, and

each feature term may have a modifier describing the feature term, and the identifying includes for each sentence having a verb phrase where the verb phrase has no matching entry in the opinion dictionary, assigning an opinion polarity of the modifier of the feature term to the feature term, the opinion polarity of the modifier being defined in the opinion dictionary; and

deriving an opinion skeleton for each of the extracted opinion, thereby providing supporting details for this opinion.

2. The method as recited in claim 1, wherein the opinion polarity associated with the feature term is identified based on an opinion rule.

3. The method as recited in claim 2, wherein the rule base comprises a plurality of rules each having a relationship term, a target of the opinion, and a polarity of the opinion.

4. The method as recited in claim 2, wherein the rule base comprises a plurality of rules each having a relationship term, a source of the opinion, and a target of the opinion.

5. The method as recited in claim 4, wherein the target of the opinion is a component of the sentence to which the opinion is to be assigned.

6. The method as recited in claim 4, wherein the source of the opinion is a component of the sentence of which opinion polarity is to be assigned to the target.