Document quality measurement

• US 9,286,379 B2
• Filed: 11/26/2012
• Issued: 03/15/2016
• Est. Priority Date: 11/26/2012
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal-Mart Stores Texas LLC (WalMart Inc.)
• Inventors: Yang, Fan, Lamba, Digvijay Singh
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US13/685,570
• Publication Number:
  US 20140147048A1
• Time in Patent Office: 1,205 Days
• Field of Search: 382/103, 382/154, 382/159, 382/209, 382/204, 707/737, 707/741, 707/709, 707/102
• US Class Current: 1/1
• CPC Class Codes: G06F 16/353 into predefined classes

Abstract

Systems and methods are disclosed herein for ranking the quality of documents, such as documents shared or referenced in postings by users. For a first set of documents quality attributes that are indicative of quality or lack of quality are identified. Ratings of the quality of the first set of documents are received. Classifiers are associated with each document and the ratings and quality attributes for each attribute used to train class-specific models corresponding to the classifiers. Subsequently received documents are then classified and corresponding quality attributes are evaluated using the corresponding class-specific model in order to rank the quality of the document.

Claims

1. A method for evaluating document quality, the method comprising:

for a first plurality of documents, performing for each document of the first plurality of documents;

identifying, by a computer system, quality attributes associated with the each document of the first plurality of documents, the quality attributes including characterizations of usage of the each document of the first plurality of documents, relevance of text in the each document of the first plurality of documents to one or more topics, and quantity of text and media;

associating, by the computer system, a classifier from a plurality of classifiers with the each document of the first plurality of documents in accordance with relevance of content of the each document of the first plurality of documents to the classifier, by selecting the classifier from a taxonomy of concepts as being representative of the each document of the first plurality of documents by evaluating meanings of textual representations within the each document of the first plurality of documents; and

receiving, by the computer system, a ranking of the each document of the first plurality of documents;

training a plurality of class-specific models each corresponding to a classifier of the plurality of classifiers by, for each classifier of the plurality of classifiers, training, by the computer system, a class-specific model of the plurality of class-specific models corresponding to the each classifier of the plurality of classifiers according to the ranking of the first plurality of documents associated with the each classifier of the plurality of classifiers and both of the quality attributes associated with documents of the first plurality of documents associated with the each classifier of the plurality of classifiers and the content of the documents of the first plurality of documents associated with the each classifier of the plurality of classifiers; and

for a second plurality of documents, performing, by the computer system, for each document of the second plurality of documents;

identifying quality attributes associated with the each document of the second plurality of documents;

associating a classifier with the each document of the second plurality of documents in accordance with relevance of content of the each document of the second plurality of documents to the classifier;

inputting, by the computer system, the each document of the second plurality of documents to a selected class-specific model of the plurality of class-specific models corresponding to the classifier associated with the each document of the second plurality of documents;

ranking, by the computer system, each document of the second plurality of documents according to the selected class-specific model using as inputs both of the quality attributes and the content of the each document of the second plurality of documents.

2. The method of claim 1, further comprising filtering the second plurality of documents in accordance with rankings of the documents of the second plurality of documents.

3. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises identifying popularity of a domain that is at least one of referenced in the document and a host of the document.

4. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises retrieving a quality ranking of a domain that is at least one of referenced in the document and a source of the document.

5. The method of claim 1, wherein ranking a document of the second plurality of documents according to a class-specific model corresponding to the classifier associated further comprises updating a quality ranking of a domain associated with the document of the second plurality of documents in accordance with the ranking of the document of the second plurality of documents.

6. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises identifying popularity of a user that posted the document.

7. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises retrieving a quality ranking of a user that is at least one of referenced in the document and a source of the document.

8. The method of claim 1, wherein ranking a document of the second plurality of documents according to the class-specific model corresponding to the classifier associated further comprises updating a quality ranking of a user associated with the document of the second plurality of documents in accordance with the ranking of the document of the second plurality of documents.

9. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises identifying a richness of content of the document.

10. The method of claim 1, wherein identifying the richness of content of the document comprises identifying at least one of an amount of content and a variety of types of content.

11. The method of claim 1, wherein identifying the richness of content of the document comprises identifying similarity between the content of the document and a canonical article, wherein the document is deemed to be less rich the more similar the document is to the canonical article.

12. The method of claim 11, wherein the canonical article is a product description authored by a manufacturer of a product.

13. The method of claim 11, wherein the canonical article is a product description authored by a retailer of a product.

14. The method of claim 1, wherein identifying quality attributes of a document of one or both of the first and second plurality of documents comprises identifying occurrences in keywords in one of the document and a document referenced by the document.

15. The method of claim 1, wherein training the class-specific model corresponding to the classifier associated comprises training a decision tree model.

16. A system for rule generation, the system comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or more memory devices storing executable and operational data effective to cause the one or more processors to:

for a first plurality of documents, perform for each document of the first plurality of documents;

identifying quality attributes associated with the each document of the first plurality of documents, the quality attributes including characterizations of usage of the each document, relevance of text in the each document to one or more topics, and quantity of text and media;

associating a classifier from a plurality of classifiers with the each document of the first plurality of documents in accordance with relevance of content of the each document of the first plurality of documents to the classifier, by selecting the classifier from a taxonomy of concepts as being representative of the each document by evaluating meanings of textual representations within the each document; and

receiving a ranking of the each document of the first plurality of documents;

training a plurality of class-specific models each corresponding to a classifier of the plurality of classifiers by, for each classifier of the plurality of classifiers, training each class-specific model of the plurality of class-specific models corresponding to the each classifier according to both of the quality attributes associated with the each document of the first plurality of documents associated with the each classifier and the content of the each documents of the first plurality of documents associated with the each classifier; and

for a second plurality of documents, perform for each document in the second plurality of documents;

identifying quality attributes associated with the each document of the second plurality of documents;

associating a classifier with the each document of the second plurality of documents in accordance with relevance of content of the each document of the second plurality of documents to the classifier;

inputting the each document of the second plurality of documents to a selected class-specific model of the plurality of class-specific models corresponding to the classifier associated with the each document;

ranking the each document of the second plurality of documents according to the selected class-specific model using as inputs one or both of the quality attributes and the content of the each document of the second plurality of documents.

17. The system of claim 16, wherein the executable and operational data are further effective to cause the one or more processors to filter the second plurality of documents in accordance with rankings of the each document of the second plurality of documents.

18. The system of claim 16, wherein the executable and operational data are further effective to cause the one or more processors to rank a document of the second plurality of documents according to the class-specific model corresponding to the classifier associated by updating a quality ranking of a domain associated with the each document of the second plurality of documents in accordance with the ranking of the each document of the second plurality of documents.

19. The system of claim 16, wherein the executable and operational data are further effective to cause the one or more processors to rank a document of the second plurality of documents according to the class-specific model corresponding to the classifier associated by updating a quality ranking of a user associated with the each document of the second plurality of documents in accordance with the ranking of the each document of the second plurality of documents.

20. The system of claim 16, wherein the executable and operational data are further effective to cause the one or more processors to train the class-specific model corresponding to the classifier associated comprises training a decision tree model.