Ranking similar passages

• US 20090055389A1
• Filed: 06/05/2008
• Published: 02/26/2009
• Est. Priority Date: 08/20/2007
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent-Foglesong, Justin John Paul, Kolak, Okan, Schilit, William Noah
• Application Number:
  US12/134,145
• Publication Number:
  US 20090055389A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars

Abstract

Passages in a digital corpus are scored and ranked based at least in part on characteristics of instances of the passages occurring in the corpus. Such characteristics include the popularity of the author, the characteristics of the words introducing and following the similar passage, frequency of appearance of the passage in the digital corpus, the length of the similar passage, the words of the similar passage, the usage of punctuation with the similar passage, and the diffusion of the similar passage within the digital corpus. The characteristics are scored and weighted to produce ranking scores for the associated passages. The ranking scores are used for purposes including selecting passages to display in association with a document and ranking passages displayed in response to a search.

Claims

1. A computer-implemented method for calculating a score for a passage having a plurality of instances occurring in a digital corpus, comprising:

calculating at least one score based at least in part on characteristics of instances of the passage occurring in the digital corpus;

generating a ranking score associated with the passage based at least in part on the calculated at least one score; and

storing the ranking score in association with the passage in a computer-readable medium.

2. The method of claim 1, wherein a plurality of scores are calculated based on a plurality of characteristics of the instances of the passage occurring in the digital corpus, and wherein generating the ranking score comprises combining the plurality of scores to form the ranking score.

3. The method of claim 1, wherein calculating the at least one score comprises:

accessing a database identifying authors and having associated author scores;

determining whether an author of a document in the digital corpus in which a passage instance occurs is found in the database; and

responsive to the author being found in the database, calculating the score based at least in part on the author score associated with the author in the database.

4. The method of claim 1, wherein calculating the at least one score comprises:

accessing a database identifying documents and having associated document scores;

determining whether a document in the digital corpus in which a passage instance occurs is found in the database; and

responsive to the document being found in the database, calculating the score based at least in part on the document score associated with the document in the database.

5. The method of claim 1, wherein calculating the at least one score comprises:

identifying a frequency that the passage instances appear in the digital corpus; and

calculating the score based at least in part on the frequency.

6. The method of claim 1, wherein calculating the at least one score comprises:

determining a length of the passage; and

calculating the score based at least in part on the length.

7. The method of claim 1, wherein calculating the at least one score comprises:

determining an amount of variation of words of the passage; and

calculating the score based at least in part on the amount of variation of words of the passage.

8. The method of claim 1, wherein calculating the at least one score comprises:

applying one or more language models to analyze words within the passage; and

calculating the score based at least in part on the application of the one or more language models.

9. The method of claim 1, wherein calculating the at least one score comprises:

determining a usage of punctuation associated with the passage; and

calculating the score based at least in part on the usage of punctuation associated with the passage.

10. The method of claim 1, wherein calculating the at least one score comprises:

identifying words introducing the passage and/or following the passage in a document in the digital corpus containing an instance of the passage;

ascertaining whether the words introducing and/or following the passage denote a speech act; and

calculating the score based at least in part on whether the words introducing and/or following the similar passage denote a speech act.

11. The method of claim 1, wherein calculating the at least one score comprises:

identifying a characteristic of the plurality of passage instances occurring in the digital corpus;

examining the plurality of passage instances to determine an amount of variation in the identified characteristic over the plurality of passage instances; and

calculating the at least one score based at least in part on the amount of variation in the characteristic.

12. The method of claim 11, wherein an identified characteristic is an author of a document in which a passage instance appears.

13. The method of claim 11, wherein an identified characteristic is a publisher of a document in which a passage instance appears.

14. The method of claim 11, wherein an identified characteristic is a library containing a document in which a passage instance appears.

15. The method of claim 11, wherein an identified characteristic is a part of a document in which a passage instance appears.

16. The method of claim 1, wherein a plurality of ranking scores are calculated for a plurality of different passages occurring in the digital corpus and further comprising:

ranking the plurality of different passages in an order responsive to the ranking scores calculated for the passages.

17. A computer-readable storage medium containing executable program code for calculating a score for a passage having multiple occurrences in a digital corpus, the program code comprising code for:

calculating at least one score based at least in part on characteristics of instances of the passage occurring in the digital corpus;

generating a ranking score associated with the passage based at least in part on the calculated at least one score; and

storing the ranking score in association with the passage in a computer-readable medium.

18. The computer-readable storage medium of claim 17, wherein a plurality of scores are calculated based on a plurality of characteristics of the instances of the passage occurring in the digital corpus, and wherein generating the ranking score comprises combining the plurality of scores to form the ranking score.

19. The computer-readable storage medium of claim 17, wherein calculating the at least one score comprises:

identifying a characteristic of the plurality of passage instances occurring in the digital corpus;

examining the plurality of passage instances to determine an amount of variation in the identified characteristic over the plurality of passage instances; and

calculating the at least one score based at least in part on the amount of variation in the characteristic.

20. The computer-readable storage medium of claim 17, wherein a plurality of ranking scores are calculated for a plurality of different passages occurring in the digital corpus and further comprising:

ranking the plurality of different passages in an order responsive to the ranking scores calculated for the passages.

21. A computer system for calculating a score for a passage having multiple occurrences in a digital corpus, the system comprising:

a computer-readable storage medium containing executable program code for calculating a score for a passage having multiple occurrences in a digital corpus, the program code comprising code for;

calculating at least one score based at least in part on characteristics of instances of the passage occurring in the digital corpus;

generating a ranking score associated with the passage based at least in part on the calculated at least one score; and

storing the ranking score in association with the passage in a computer-readable medium.

22. The computer system of claim 21, wherein a plurality of scores are calculated based on a plurality of characteristics of the instances of the passage occurring in the digital corpus, and wherein generating the ranking score comprises combining the plurality of scores to form the ranking score.

23. The computer system of claim 21, wherein calculating the at least one score comprises:

identifying a characteristic of the plurality of passage instances occurring in the digital corpus;

examining the plurality of passage instances to determine an amount of variation in the identified characteristic over the plurality of passage instances; and

calculating the at least one score based at least in part on the amount of variation in the characteristic.

24. The computer system of claim 21, wherein a plurality of ranking scores are calculated for a plurality of different passages occurring in the digital corpus and further comprising:

ranking the plurality of different passages in an order responsive to the ranking scores calculated for the passages.