USING SPECIFICITY MEASURES TO RANK DOCUMENTS

• US 20100205184A1
• Filed: 02/10/2009
• Published: 08/12/2010
• Est. Priority Date: 02/10/2009
• Status: Abandoned Application

• Current Assignee: Oath Inc. (Verizon Communications Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: MARCINIAK, Tomasz, Marson, Yoel David
• Application Number:
  US12/368,932
• Publication Number:
  US 20100205184A1
• Time in Patent Office: Days
• US Class Current: 707/750
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method of ranking documents by specificity values includes specifying a reference set of documents, each document including one or more terms, and specifying a first document that includes one or more terms that are included in the reference set of documents. The method includes determining, from the reference set of documents, one or more term-specificity values for the one or more terms of the first document by calculating frequencies of terms within the reference set of documents, wherein a larger term-specificity value corresponds to a lower likelihood relative to the reference set of documents, and determining a document-specificity value for the first document by combining the one or more term-specificity values for the first document, wherein larger term-specificity values correspond to a larger document-specificity value.

Claims

1. A method of ranking documents by specificity values, comprising:

specifying a reference set of documents, each document including one or more terms;

specifying a first document that includes one or more terms that are included in the reference set of documents;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of the first document by calculating frequencies of terms within the reference set of documents, wherein a larger term-specificity value corresponds to a lower likelihood relative to the reference set of documents;

determining a document-specificity value for the first document by combining the one or more term-specificity values for the first document, wherein larger term-specificity values correspond to a larger document-specificity value; and

saving one or more values for the document-specificity value of the first document in a computer-readable medium.

2. A method according to claim 1, further comprising:

calculating term specificity values for terms in the reference set of documents as inverse document frequency values relative to the reference set of documents by comparing a number of documents including each term to a total number of documents.

3. A method according to claim 1, further comprising:

calculating the document-specificity value for the first document as a non-negative arithmetic combination of the corresponding term specificity values.

4. A method according to claim 1, wherein determining the document-specificity value for the first document includes calculating a norm of a vector that includes the corresponding term-specificity values.

5. A method according to claim 1, wherein the reference set of documents includes the first document.

6. A method according to claim 1, further comprising:

specifying a plurality of input documents that include one or more terms that are included in the reference set of documents, wherein the input documents include the first document;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of each input document;

determining, from the one or more term-specificity values for each input document, a document-specificity value for each input document;

determining a rank ordering of the input documents corresponding to an ordering of the document-specificity values of the documents; and

saving one or more values for the rank ordering in the computer-readable medium.

7. A computer-readable medium that stores a computer program for ranking documents by specificity values, wherein the computer program includes instructions for:

specifying a reference set of documents, each document including one or more terms;

specifying a first document that includes one or more terms that are included in the reference set of documents;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of the first document by calculating frequencies of terms within the reference set of documents, wherein a larger term-specificity value corresponds to a lower likelihood relative to the reference set of documents;

determining a document-specificity value for the first document by combining the one or more term-specificity values for the first document, wherein larger term-specificity values correspond to a larger document-specificity value; and

saving one or more values for the document-specificity value of the first document.

8. A computer-readable medium according to claim 7, wherein the computer program further includes instructions for:

calculating term specificity values for terms in the reference set of documents as inverse document frequency values relative to the reference set of documents by comparing a number of documents including each term to a total number of documents.

9. A computer-readable medium according to claim 7, wherein the computer program further includes instructions for:

calculating the document-specificity value for the first document as a non-negative arithmetic combination of the corresponding term specificity values.

10. A computer-readable medium according to claim 7, wherein determining the document-specificity value for the first document includes calculating a norm of a vector that includes the corresponding term-specificity values.

11. A computer-readable medium according to claim 7, wherein the reference set of documents includes the first document.

12. A computer-readable medium according to claim 7, wherein the computer program further includes instructions for:

specifying a plurality of input documents that include one or more terms that are included in the reference set of documents, wherein the input documents include the first document;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of each input document;

determining, from the one or more term-specificity values for each input document, a document-specificity value for each input document;

determining a rank ordering of the input documents corresponding to an ordering of the document-specificity values of the documents; and

saving one or more values for the rank ordering.

13. An apparatus for ranking documents by specificity values, the apparatus comprising a computer for executing computer instructions, wherein the computer includes computer instructions for:

specifying a reference set of documents, each document including one or more terms;

specifying a first document that includes one or more terms that are included in the reference set of documents;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of the first document by calculating frequencies of terms within the reference set of documents, wherein a larger term-specificity value corresponds to a lower likelihood relative to the reference set of documents;

determining a document-specificity value for the first document by combining the one or more term-specificity values for the first document, wherein larger term-specificity values correspond to a larger document-specificity value; and

saving one or more values for the document-specificity value of the first document.

14. An apparatus according to claim 13, wherein the computer further includes computer instructions for:

calculating term specificity values for terms in the reference set of documents as inverse document frequency values relative to the reference set of documents by comparing a number of documents including each term to a total number of documents.

15. An apparatus according to claim 13, wherein the computer further includes computer instructions for:

calculating the document-specificity value for the first document as a non-negative arithmetic combination of the corresponding term specificity values.

16. An apparatus according to claim 13, wherein determining the document-specificity value for the first document includes calculating a norm of a vector that includes the corresponding term-specificity values.

17. An apparatus according to claim 13, wherein the reference set of documents includes the first document.

18. An apparatus according to claim 13, wherein the computer further includes computer instructions for:

specifying a plurality of input documents that include one or more terms that are included in the reference set of documents, wherein the input documents include the first document;

determining, from the reference set of documents, one or more term-specificity values for the one or more terms of each input document;

determining, from the one or more term-specificity values for each input document, a document-specificity value for each input document;

determining a rank ordering of the input documents corresponding to an ordering of the document-specificity values of the documents; and

saving one or more values for the rank ordering.

19. An apparatus according to claim 13, wherein the computer includes a processor with memory for executing at least some of the computer instructions.

20. An apparatus according to claim 13, wherein the computer includes circuitry for executing at least some of the computer instructions.