Method for re-ranking documents retrieved from a document database

• US 20060089926A1
• Filed: 10/27/2004
• Published: 04/27/2006
• Est. Priority Date: 10/27/2004
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Frieder, Ophir, Knepper, Margaret M., Fox, Kevin Lee
• Granted Patent: US 7,801,887 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing

Abstract

A computer-implemented method for processing documents in a document database includes generating an initial ranking of retrieved documents using an information retrieval system and based upon a user search query, and processing vocabulary words based upon occurrences thereof in at least some of the retrieved documents. Respective relevancies of the vocabulary words based on the occurrences thereof and the user search query are generated. A re-ranking of the retrieved documents is generated based on the relevancies of the vocabulary words.

Claims

1. A computer-implemented method for processing documents in a document database comprising:

generating an initial ranking of retrieved documents using an information retrieval system and based upon a user search query;

processing a plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents;

generating respective relevancies of the vocabulary words based on the occurrences thereof and the user search query; and

generating a re-ranking of the retrieved documents based on the relevancies of the vocabulary words.

2. A computer-implemented method according to claim 1 further comprising generating the plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents before the processing.

3. A computer-implemented method according to claim 1 further comprising selecting a vocabulary comprising the plurality of vocabulary words before the processing.

4. A computer-implemented method according to claim 3 wherein the plurality of vocabulary words are based upon words in at least one predetermined document.

5. A computer-implemented method according to claim 4 wherein the at least one predetermined document is not part of the retrieved documents.

6. A computer-implemented method according to claim 3 further comprising adding vocabulary words to the vocabulary based upon occurrences of words in at least some of the retrieved documents.

7. A computer-implemented method according to claim 6 further comprising determining a quality of the vocabulary based upon how many vocabulary words are added thereto.

8. A computer-implemented method according to claim 1 further comprising selecting N top ranked documents from the retrieved documents before processing the plurality of vocabulary words, with N being an integer greater than 1;

and wherein generating the respective relevancies and generating the re-ranking are with respect to the N top-ranked documents.

9. A computer-implemented method according to claim 8 wherein generating the respective relevancies of the vocabulary words comprises:

counting how many times a respective vocabulary word is used in the N top ranked documents; and

counting how many of the N top ranked documents uses the respective vocabulary word.

10. A computer-implemented method according to claim 9 further comprising generating a word/document ratio for each respective vocabulary word based upon the counting, and if the word/document ratio is less than a threshold, then the relevancy of the word is not used when generating the re-ranking of the N top ranked documents.

11. A computer-implemented method according to claim 1 further comprising determining which documents from at least some of the retrieved documents are irrelevant to the user search query;

and wherein generating the re-ranking of the retrieved documents is also based on the irrelevant documents.

12. A computer-implemented method according to claim 1 further comprising determining which documents from at least some of the retrieved documents are relevant to the user search query;

and wherein generating the re-ranking of the retrieved documents is also based on the relevant documents.

13. A computer-implemented method according to claim 12 further comprising determining if the respective vocabulary words are relevant to the user search query;

and wherein determining the relevant documents is based upon the relevant vocabulary words.

14. A computer-implemented method according to claim 1 further comprising determining a respective source of at least some of the retrieved documents, and assigning priority to documents provided by preferred sources;

and wherein generating the re-ranking of the retrieved documents is also based on documents with preferred sources.

15. A computer-implemented method according to claim 1 further comprising generating a second re-ranking of the retrieved documents based upon a combination of the initial ranking and the re-ranking of the retrieved documents.

16. A computer-implemented method according to claim 1 further comprising displaying the re-ranked documents.

17. A computer-implemented method for processing documents in a document database, the documents having an initial ranking based upon a user search query, the method comprising;

selecting N top ranked documents from the retrieved documents, with N being an integer greater than 1;

processing a plurality of vocabulary words based upon occurrences thereof in the N top ranked documents;

generating respective relevancies of the vocabulary words based on the occurrences thereof in the N top ranked documents and the user search query;

generating a re-ranking of the N top ranked documents based on the relevancies of the vocabulary words; and

displaying the re-ranked documents.

18. A computer-implemented method according to claim 17 further comprising generating the plurality of vocabulary words based upon occurrences thereof in the N top ranked documents.

19. A computer-implemented method according to claim 17 further comprising selecting a vocabulary comprising the plurality of vocabulary words before the processing.

20. A computer-implemented method according to claim 19 wherein the plurality of vocabulary words are based upon words in at least one predetermined document.

21. A computer-implemented method according to claim 20 wherein the at least one predetermined document is not part of the retrieved documents.

22. A computer-implemented method according to claim 20 further comprising adding vocabulary words to the vocabulary based upon occurrences of words in at least some of the retrieved documents.

23. A computer-implemented method according to claim 22 further comprising determining a quality of the vocabulary based upon how many vocabulary words are added thereto.

24. A computer-implemented method according to claim 17 wherein generating the respective relevancies of the vocabulary words comprises:

counting how many times a respective vocabulary word is used in the N top ranked documents; and

counting how many of the N top ranked documents uses the respective vocabulary word.

25. A computer-implemented method according to claim 24 further comprising generating a word/document ratio for each respective vocabulary word based upon the counting, and if the word/document ratio is less than a threshold, then the relevancy of the word is not used when generating the re-ranking of the N top ranked documents.

26. A computer-implemented method according to claim 17 further comprising determining which documents from at least some of the retrieved documents are irrelevant to the user search query;

and wherein generating the re-ranking of the retrieved documents is also based on the irrelevant documents.

27. A computer-implemented method according to claim 17 further comprising determining which documents from the N top ranked documents are relevant to the user search query;

and wherein generating the re-ranking of the retrieved documents is also based on the relevant documents.

28. A computer-implemented method according to claim 27 further comprising determining if the respective vocabulary words are relevant to the user search query;

and wherein determining the relevant documents is based upon the relevant vocabulary words.

29. A computer-implemented method according to claim 17 further comprising determining a respective source of the N top ranked documents, and assigning priority to documents provided by preferred sources;

and wherein generating the re-ranking of the N top ranked documents is also based on documents with preferred sources.

30. A computer-implemented method according to claim 17 further comprising generating a second re-ranking of the N top ranked documents based upon a combination of the initial ranking and the re-ranking of the N top ranked documents.

31. A computer-readable medium having stored thereon a data structure for processing documents in a document database, the computer-readable medium comprising:

a first data field for generating an initial ranking of retrieved documents using an information retrieval system and based upon a user search query;

a second data field for processing a plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents;

a third data field for generating respective relevancies of the vocabulary words based on the occurrences thereof and the user search query; and

a fourth data field for generating a re-ranking of the retrieved documents based on the relevancies of the vocabulary words.

32. A computer-readable medium according to claim 31 further comprising a fifth data field for generating the plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents before the processing.

33. A computer-readable medium according to claim 31 further comprising a sixth data field for selecting a vocabulary comprising the plurality of vocabulary words before the processing, the plurality of vocabulary words corresponding to the user search topic.

34. A computer-readable medium according to claim 33 wherein the plurality of vocabulary words are based upon words in at least one predetermined document.

35. A computer-readable medium according to claim 34 wherein the at least one predetermined document is not part of the retrieved documents.

36. A computer-readable medium according to claim 34 further comprising a seventh data field for adding vocabulary words to the vocabulary based upon occurrences of words in at least some of the retrieved documents.

37. A computer-readable medium according to claim 36 further comprising an eighth data field for determining a quality of the vocabulary based upon how many vocabulary words are added thereto.

38. A computer-readable medium according to claim 31 further comprising a ninth data field for selecting N top ranked documents from the retrieved documents before processing the plurality of vocabulary words, with N being an integer greater than 1;

and wherein generating the respective relevancies in said third data field and generating the re-ranking in said fourth data field are with respect to the N top-ranked documents.

39. A computer-readable medium according to claim 38 wherein generating the respective relevancies of the vocabulary words comprises:

a tenth data field for counting how many times a respective vocabulary word is used in the N top ranked documents; and

an eleventh data field for counting how many of the N top ranked documents uses the respective vocabulary word.

40. A computer-readable medium according to claim 39 further comprising a twelfth data field for generating a word/document ratio for each respective vocabulary word based upon the counting, and if the word/document ratio is less than a threshold, then the relevancy of the word is not used when generating the re-ranking of the N top ranked documents.

41. A computer-readable medium according to claim 31 further comprising a thirteenth data field for determining which documents from at least some of the retrieved documents are relevant to the user search query;

and wherein generating the re-ranking of the retrieved documents in said fourth data field is also based on the relevant documents.

42. A computer-readable medium according to claim 41 further comprising a fourteenth data field for determining if the respective vocabulary words are relevant to the user search query;

and wherein determining the relevant documents in said third data field is based upon the relevant vocabulary words.

43. A computer-readable medium according to claim 31 further comprising a fifteenth data field for determining a respective source of at least some of the retrieved documents, and assigning priority to documents provided by preferred sources;

and wherein generating the re-ranking of the retrieved documents in said fourth data field is also based on documents with preferred sources.

44. A computer-readable medium according to claim 31 further comprising a sixteenth data field for generating a second re-ranking of the retrieved documents based upon a combination of the initial ranking and the re-ranking of the retrieved documents.

45. A computer-readable medium according to claim 31 further comprising a seventeenth data field for displaying the re-ranked documents.

46. A computer implemented system for processing documents in a document database comprising:

a first module for generating an initial ranking of retrieved documents using an information retrieval system and based upon a user search query;

a second module for processing a plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents;

a third module for generating respective relevancies of the vocabulary words based on the occurrences thereof and the user search query; and

a fourth module for generating a re-ranking of the retrieved documents based on the relevancies of the vocabulary words.

47. A computer implemented system according to claim 46 further comprising a fifth module for generating the plurality of vocabulary words based upon occurrences thereof in at least some of the retrieved documents before the processing.

48. A computer implemented system according to claim 46 further comprising a sixth module for selecting a vocabulary comprising the plurality of vocabulary words before the processing.

49. A computer implemented system according to claim 48 wherein the plurality of vocabulary words are based upon words in at least one predetermined document.

50. A computer implemented system according to claim 49 wherein the at least one predetermined document is not part of the retrieved documents.

51. A computer implemented system according to claim 49 further comprising a seventh module for adding vocabulary words to the vocabulary based upon occurrences of words in at least some of the retrieved documents.

52. A computer implemented system according to claim 51 further comprising an eighth module for determining a quality of the vocabulary based upon how many vocabulary words are added thereto.

53. A computer implemented system according to claim 46 further comprising a ninth module for selecting N top ranked documents from the retrieved documents before processing the plurality of vocabulary words, with N being an integer greater than 1;

and wherein generating the respective relevancies in said third module and generating the re-ranking in said fourth module are with respect to the N top-ranked documents.

54. A computer implemented system according to claim 53 wherein generating the respective relevancies of the vocabulary words comprises:

a tenth module for counting how many times a respective vocabulary word is used in the N top ranked documents; and

an eleventh module for counting how many of the N top ranked documents uses the respective vocabulary word.

55. A computer implemented system according to claim 54 further comprising a twelfth module for generating a word/document ratio for each respective vocabulary word based upon the counting, and if the word/document ratio is less than a threshold, then the relevancy of the word is not used when generating the re-ranking of the N top ranked documents.

56. A computer implemented system according to claim 46 further comprising a thirteenth module for determining which documents from at least some of the retrieved documents are irrelevant to the user search query;

and wherein generating the re-ranking of the retrieved documents in said fourth module is also based on the irrelevant documents.

57. A computer implemented system according to claim 46 further comprising a fourteenth module for determining which documents from at least some of the retrieved documents are relevant to the user search query;

and wherein generating the re-ranking of the retrieved documents in said fourth module is also based on the relevant documents.

58. A computer implemented system according to claim 57 further comprising a fifteenth module for determining if the respective vocabulary words are relevant to the user search query;

and wherein determining the relevant documents in said third module is based upon the relevant vocabulary words.

59. A computer implemented system according to claim 46 further comprising a sixteenth module for determining a respective source of at least some of the retrieved documents, and assigning priority to documents provided by preferred sources;

and wherein generating the re-ranking of the retrieved documents in said fourth module is also based on documents with preferred sources.

60. A computer implemented system according to claim 46 further comprising a seventeenth module for generating a second re-ranking of the retrieved documents based upon a combination of the initial ranking and the re-ranking of the retrieved documents.

61. A computer implemented system according to claim 46 further comprising a display for displaying the re-ranked documents.