Calculating document quality

• US 9,760,607 B1
• Filed: 09/25/2013
• Issued: 09/12/2017
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Tweedy, Carolyn Grace, Varma, Sanjay, Ringhiser, Raymond Robert, Kelly, Nicole Bethany-Abbott, Burrow, William Sean, Goglia, Todd
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US14/036,202
• Time in Patent Office: 1,448 Days
• Field of Search: 707722, 707728, 707748
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique

Abstract

Disclosed are various implementations for a content evaluation application. Various component scores are calculated to reflect a variety of attributes of a document. The component scores may reflect a readability score, a word count, user interactions, content timeliness, or other data. An overall quality score is calculated for a document by applying selected weights to the component scores.

Claims

1. A method comprising:

receiving, by a server computing device from a client computing device over a network, a network request for a network page;

associating, by the server computing device, one of a plurality of user accounts with the network request;

retrieving, by the server computing device over the network, the network page from a network page server;

calculating, by the server computing device, a plurality of component scores for the network page including a readability score, a word count, and a timeliness score;

determining, by the server computing device, a plurality of weights specified by configuration data associated with the user account, each of the plurality of user accounts associated with configuration data;

generating, by the server computing device, a quality score for the network page by multiplying the plurality of weights specified by the configuration data associated with the user account to the plurality of component scores;

calculating, by the server computing device, a dynamic threshold value as a function of aggregate values of quality scores of a plurality of other network pages;

responsive to the quality score falling below the dynamic threshold value, determining, by the server computing device, at least one suggested modification to the network page to improve the quality score based on the plurality of weights specified by the configuration data associated with the user account;

encoding, by the server computing device, for rendering by the client computing device a report embodying at least the quality score or the at least one suggested modification; and

generating, by the server computing device, search results comprising a list of network pages, wherein the list of network pages is sorted according to respective quality scores associated with the network pages in the list.

2. The method of claim 1, wherein the configuration data associated with the user account specifies a weight to be applied for a particular language of the network page.

3. The method of claim 1, wherein the plurality of component scores further includes a user rating score, the method further comprising aggregating feedback data to generate the user rating score, wherein the feedback data comprises a user feedback of the network page.

4. A system, comprising:

at least one computing device;

a content evaluation application executed in the at least one computing device, the content evaluation application configured to at least;

receive from a client computing device over a network, a network request for a network page;

associate one of a plurality of user accounts with the network request;

retrieve the network page from a network page server over the network;

calculate a plurality of component scores for the network page including a user interaction score, wherein the user interaction score is a function of user interaction data, the user interaction data comprising at least data representing previous interactions with the network page by at least one viewer of the network page;

determine a plurality of weights to be multiplied to the plurality of component scores from configuration data associated with the user account, each of the plurality of user accounts associated with configuration data;

generate a quality score for the network page by applying the plurality of weights specified by the configuration data associated with the user account to the plurality of component scores;

encode for rendering by the client computing device a report embodying at least the quality score within a header portion of the network page;

determine a dynamic threshold value as a function of aggregate values of quality scores of a plurality of other network pages; and

responsive to the quality score falling below the dynamic threshold value, output a one of the plurality of component scores to be improved; and

a network page server application executed in the at least one computing device, the network page server application configured to generate search results comprising a list of network pages, wherein the list of network pages is sorted according to respective quality scores associated with the network pages in the list.

5. The system of claim 4, wherein the user interaction score is calculated as a function of a quantity of views of the network page.

6. The system of claim 5, wherein the network page is a help document, and wherein the user interaction score is further calculated as a function of a quantity of support contacts performed after viewing the help document.

7. The system of claim 4, wherein the network page is one of a plurality of network pages, and the component scores are calculated as a function of another plurality of component scores corresponding to another of the plurality of network pages.

8. The system of claim 4, wherein the content evaluation application is further configured to obtain a definition of at least a portion of the plurality of weights from the client computing device.

9. The system of claim 4, wherein the plurality of component scores further includes a user rating score, wherein the content evaluation application is further configured to aggregate feedback data to generate the user rating score, wherein the feedback data comprises a user feedback of the network page.

10. The system of claim 4, wherein the network page is one of a plurality of network pages, and the content evaluation application is further configured to perform a statistical correlation on the plurality of network pages and a corresponding plurality of quality scores to identify a subset of the plurality of component scores that are more statistically significant with respect to a remainder of the component scores.

11. The system of claim 4, wherein the user interaction data further comprises a browsing history for the at least one viewer subsequent to viewing the network page.

12. A method, comprising:

receiving, by a server computing device from a client computing device over a network, a network request for a network page;

associating, by the server computing device, one of a plurality of user accounts with the network request;

retrieving, by the server computing device, the network page from a network page server over the network;

calculating, by the server computing device, for a document, a plurality of component scores including a readability score, a word count, or a timeliness score, wherein the readability score is calculated by applying a readability test to the network page, wherein the readability score is based at least in part on a word complexity, a word diversity, or a sentence length;

determining, by the server computing device, a plurality of weights to be multiplied to the plurality of component scores based at least in part on configuration data associated with the user account, each of the plurality of user accounts associated with configuration data;

calculating, by the server computing device, a dynamic threshold value as a function of a plurality of quality scores of other network pages;

generating, by the server computing device, a quality score for the network page by applying the plurality of weights to the plurality of component scores; and

encoding, by the server computing device, for rendering by the client computing device a report embodying at least the quality score within a user interface component distinct from the network page, wherein responsive to the quality score of the network page falling below the dynamic threshold value, the report provides at least one suggested modification to the network page to improve the quality score.

13. The method of claim 12, wherein the plurality of component scores is a first plurality of component scores, and further comprising:

identifying at least one reference document from the configuration data associated with the user account;

performing, by the server computing device, a regression analysis on the at least one reference document to determine a second plurality of component scores of the at least one reference document contributing to a quality score of the at least one reference document; and

wherein the plurality of weights are determined as a function of the second plurality of component scores.

14. The method of claim 13, wherein the at least one reference document is further identified based on a language of the network page.

15. The method of claim 13, further comprising including the network page as a reference document in the configuration data when the quality score for the network page is above the dynamic threshold value.

16. The method of claim 12, further comprising determining, by the server computing device, the modification to the network page to increase the quality score of the network page.

17. The method of claim 12, wherein the plurality of component scores comprises the readability score, a word count, and timeliness score.

18. The method of claim 12, wherein user interface component comprises a graphical window.