Extracting quotes from customer reviews

• US 9,672,555 B1
• Filed: 03/18/2011
• Issued: 06/06/2017
• Est. Priority Date: 03/18/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Fox, Eric B., Dicker, Russell A., Dillard, Logan L.
• Primary Examiner(s): LEVINE, ADAM L
• Application Number:
  US13/051,278
• Time in Patent Office: 2,272 Days
• Field of Search: 705 26- 272, 705347
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06Q 30/0282 Rating or review of business operators or products
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0631 Item recommendations

Abstract

Technologies are described herein for extracting quotes for an item from customer reviews. Individual sentences or phrases contained in customer reviews regarding an item or category of items are parsed into a collection of sentences. A list of topics is generated from the collection of sentences, and each sentence or phrases is assigned a topic from the list of topics. The most relevant topics from the list of topics are identified for a particular item, and one or more sentences or phrases from customer reviews for the particular item are selected for each of the most relevant topics that best represent the sentences associated with the item and assigned to that topic, according to content and sentiment.

Claims

1. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon that, when executed by a computer, cause the computer to:

receive customer reviews for a category of items from a plurality of customers, wherein the category of items encompasses a particular item and items of a similar type;

aggregate the customer reviews for the category of items;

parse a plurality of sentences from the customer reviews, wherein the parsing comprises breaking compound sentences in the customer reviews based on one or more of commas, coordinating conjunctions, or transition terms;

classify the plurality of sentences by sentiment by utilizing a logistic regression classifier trained using sentiment scores of terms generated based on sentences contained in training data and manually labeled as to sentiment, wherein positive, negative and mixed sentiment scores of the terms are used as positive coefficients in the logistic regression classifier and neutral sentiment scores of the terms are used as negative coefficients in the logistic regression classifier;

remove those sentences having a neutral sentiment from the plurality of sentences;

generate a list of topics from the plurality of sentences utilizing latent Dirichlet allocation;

assign individual sentences in the plurality of sentences to a topic in the list of topics;

select a relevant topic from the list of topics for the particular item, wherein the relevant topic comprises the topic from the list of topics assigned to the most sentences parsed from customer reviews regarding the particular item;

select a representative sentence from the plurality of sentences assigned to the relevant topic and parsed from the customer reviews regarding the particular item, the representative sentence having a highest relevance to the relevant topic and expressing a majority sentiment from among the plurality of sentences assigned to the relevant topic and parsed from the customer reviews regarding the particular item; and

send the selected representative sentence to a customer computing device, wherein the selected representative sentence is presented to another customer on the customer computing device.

2. The non-transitory computer-readable storage medium of claim 1, wherein selecting the representative sentence further comprises utilizing cosine similarity with term frequency-inverse document frequency weighting to determine the relevance to the relevant topic of individual sentences from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item.

3. The non-transitory computer-readable storage medium of claim 1, wherein selecting the representative sentence further comprises selecting a sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item having one or more of a minimum level of specificity or a minimum level of readability.

4. A system for summarizing customer reviews regarding a particular item, the system comprising:

one or more application servers configured to receive, from a plurality of customers, customer reviews for a category of items comprising the particular item and items of a similar type; and

a quote extraction module executing on the one or more application servers and configured to

aggregate the customer reviews for the category of items,

parse a plurality of sentences from the aggregated customer reviews, wherein the parsing comprises breaking compound sentences in the customer reviews based on one or more of commas, coordinating conjunctions, or transition terms,

classify the plurality of sentences by sentiment by utilizing a logistic regression classifier trained using sentiment scores of terms generated based on sentences contained in training data and manually labeled as to sentiment, wherein positive, negative and mixed sentiment scores of the terms are used as positive coefficients in the logistic regression classifier and neutral sentiment scores of the terms are used as negative coefficients in the logistic regression classifier,

remove those sentences having a neutral sentiment from the plurality of sentences,

generate a list of topics from the plurality of sentences utilizing latent Dirichlet allocation,

assign each sentence in the plurality of sentences to a topic in the list of topics,

select a relevant topic from the list of topics for a particular item, wherein the relevant topic comprises the topic from the list of topics assigned to the most sentences parsed from the customer reviews regarding the particular item,

select a representative sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item, the representative sentence having a highest relevance to the relevant topic and expressing a majority sentiment from among the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item, and

cause the selected representative sentence to be sent to a customer computing device, wherein the selected representative sentence is presented to another customer on the customer computing device.

5. The system of claim 4, wherein selecting the representative sentence for the relevant topic further comprises utilizing cosine similarity with term frequency-inverse document frequency weighting to determine the relevance to the relevant topic of individual sentences from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item.

6. The system of claim 4, wherein selecting the representative sentence for the relevant topic further comprises utilizing a graph-based centrality algorithm with term frequency-inverse document frequency weighting to determine the relevance to the relevant topic of individual sentences from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item.

7. The system of claim 6, wherein the graph-based centrality algorithm comprises the LexRank algorithm.

8. The system of claim 4, wherein the sentences contained in the training data comprise sentences parsed from customer reviews for the category of items.

9. The system of claim 4, wherein selecting the representative sentence further comprises selecting a sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item having one or more of a minimum level of specificity or a minimum readability level.

10. The system of claim 4, further comprising an online shopping module executing on the one or more application servers and configured to present the representative sentence selected for the relevant topic to the another customer in conjunction with other information regarding the particular item.

11. A computer-implemented method of extracting a quote regarding a relevant topic from customer reviews for a particular item, the method comprising executing instructions in a computer system to perform the operations of:

receiving customer reviews for a category of items from a plurality of customers, wherein the category of items encompasses a particular item and items of a similar type;

aggregating, by the computer system, the customer reviews for the category of items;

parsing, by the computer system, a plurality of sentences from the customer reviews, wherein the parsing comprises breaking compound sentences in the customer reviews based on one or more of commas, coordinating conjunctions, or transition terms;

classifying, by the computer system, the plurality of sentences by sentiment by utilizing a logistic regression classifier trained using sentiment scores of terms generated based on sentences contained in training data and manually labeled as to sentiment, wherein positive, negative and mixed sentiment scores of the terms are used as positive coefficients in the logistic regression classifier and neutral sentiment scores of the terms are used as negative coefficients in the logistic regression classifier;

removing, by the computer system, those sentences having a neutral sentiment from the plurality of sentences;

generating, by the computer system, a list of topics from the plurality of sentences utilizing latent Dirichlet allocation;

assigning individual sentences in the plurality of sentences to a topic in the list of topics;

selecting, by the computer system, the relevant topic from the list of topics for the particular item, wherein the relevant topic comprises the topic from the list of topics assigned to the most sentences parsed from the customer reviews regarding the particular item;

selecting, by the computer system, a representative sentence from the plurality of sentences assigned to the relevant topic and parsed from the customer reviews regarding the particular item, the representative sentence having a highest relevance to the relevant topic and expressing a majority sentiment from among the plurality of sentences assigned to the relevant topic and parsed from the customer reviews regarding the particular item; and

sending, by the computer system, the selected representative sentence to a customer computing device, wherein the selected representative sentence is presented to another customer on the customer computing device.

12. The computer-implemented method of claim 11, further comprising:

selecting a plurality of most relevant topics from the list of topics for the particular item; and

selecting one or more representative sentences for individual ones of the plurality of most relevant topics from sentences assigned to the plurality of most relevant topics and parsed from the customer reviews regarding the particular item.

13. The computer-implemented method of claim 11, wherein selecting the representative sentence comprises utilizing cosine similarity with term frequency-inverse document frequency weighting to select the sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item having a highest relevance to the relevant topic.

14. The computer-implemented method of claim 11, wherein selecting the representative sentence comprises utilizing the LexRank algorithm with term frequency-inverse document frequency weighting to select the sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item having a highest relevance to the relevant topic.

15. The computer-implemented method of claim 11, wherein the sentences contained in the training data comprise sentences parsed from customer reviews for the category of items.

16. The computer-implemented method of claim 11, wherein selecting the representative sentence further comprises selecting a sentence from the plurality of sentences assigned to the relevant topic and parsed from the customer reviews regarding the particular item having a minimum level of specificity.

17. The computer-implemented method of claim 11, wherein selecting the representative sentence further comprises selecting a sentence from the plurality of sentences assigned to the relevant topic and parsed from customer reviews regarding the particular item having a minimum readability level.

18. The computer-implemented method of claim 11, wherein the representative sentence selected for the relevant topic is presented to the another customer in conjunction with other information regarding the particular item.

19. The computer-implemented method of claim 11, wherein aggregating the customer reviews for the category of items further comprises aggregating feedback regarding the customer reviews for the category of items.