Method of searching text to find relevant content

• US 20070239707A1
• Filed: 03/30/2007
• Published: 10/11/2007
• Est. Priority Date: 04/03/2006
• Status: Active Grant

• Current Assignee: Needlebot, Inc.
• Original Assignee: Needlebot, Inc.
• Inventors: Verlin, Jerome, Donnelly, Stuart, Collins, John, Akyuz, Can
• Granted Patent: US 8,019,754 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

A method of locating relevant documents wherein documents are given a fingerprint comprising weights associated with particular topic categories of a classification system, each weight representing a degree to which the document relates to the particular topic category, a first piece of text is identified and given a fingerprint comprising a list of other weights associated with similar topic categories, the other weights representing a degree to which the first piece of text relates to the particular topic category. All or a portion of the universe of documents is searched by comparing the fingerprint for the first piece of text with the fingerprint for each document. You select those documents whose fingerprints have a predetermined degree of mathematical overlap with the fingerprint of the first piece of text. A user fingerprint of the user'"'"'s recently accessed texts can be used in place of the first piece of text.

Claims

1. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified so that each document in the universe has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in the all or a portion of the universe of documents,

ranking the all or a portion of the universe of documents based on a degree to which a document has a mathematical overlap with the fingerprint of the first piece of text.

2. The method of claim 1, wherein either the weights are scaled or else the mathematical overlaps are scaled.

3. The method of claim 2, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

4. The method of claim 1, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

5. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified so that each document in the universe has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in that all or a portion of the universe of documents, and

selecting those documents whose fingerprints have a predetermined degree of mathematical overlap with the fingerprint of the first piece of text.

6. The method of claim 5, wherein either the weights are scaled or else the mathematical overlaps are scaled.

7. The method of claim 6, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

8. The method of claim 5, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

9. A method of locating relevant documents within a universe of documents, comprising:

creating a fingerprint for each document in the universe of documents, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in that all or a portion of the universe of documents, and

ranking the all or a portion of the universe of documents based on a degree to which a document has a mathematical overlap with the fingerprint of the first piece of text.

10. The method of claim 9, wherein either the weights are scaled or else the mathematical overlaps are scaled.

11. The method of claim 10, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

12. The method of claim 9, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

70. A method according to claims 2 or 6 or 10 or 14 or 18 or 22 or 26 or 31 or 35 or 39 or 44 or 49 or 54 or 58 or 62 or 66, wherein a numerical magnitude of topic categories can be adjusted.

71. A method according to claims 2 or 6 or 10 or 14 or 18 or 22 or 26 or 31 or 35 or 39 or 44 or 49 or 54 or 58 or 62 or 66, wherein some weights of the fingerprint are disregarded because they are not relevant.

13. A method of locating relevant documents within a universe of documents, comprising:

creating a fingerprint for each document in the universe of documents, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in that all or a portion of the universe of documents, and

selecting those documents whose fingerprints have a predetermined degree of mathematical overlap with the fingerprint of the first piece of text.

14. The method of claim 13, wherein either the weights are scaled or else the mathematical overlaps are scaled.

15. The method of claim 14, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

16. The method of claim 13, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

17. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, the method comprising:

classifying the list of documents so that each document in the list has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching the list of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in the list of documents,

re-ranking the list of documents based on a degree to which a document in the list has a mathematical overlap with the fingerprint of the first piece of text.

18. The method of claim 17, wherein either the weights are scaled or else the mathematical overlaps are scaled.

19. The method of claim 18, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

20. The method of claim 17, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

21. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, and wherein each document in the list has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for the first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching the list of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in the list of documents,

re-ranking the list of documents based on a degree to which a fingerprint of a document in the list has a mathematical overlap with the fingerprint of the first piece of text.

22. The method of claim 21, wherein either the weights are scaled or else the mathematical overlaps are scaled.

23. The method of claim 22, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

24. The method of claim 21, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

25. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, wherein the list of documents has been classified and appears in an inverted list, said inverted list comprising for each topic category of a classification system a weight associated with a particular document of the list of documents, the weight representing a degree to which the particular document relates to said each topic category, the weights obtained automatically from a computer program, the method comprising:

identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching the list of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in the list of documents, and

re-ranking the list of documents based on a degree to which a fingerprint of a document in the list has a mathematical overlap with the fingerprint of the first piece of text.

26. The method of claim 25, wherein either the weights are scaled or else the mathematical overlaps are scaled.

27. The method of claim 26, wherein an upper bound to the mathematical overlap is calculated dynamically, said upper bound used to reduce a magnitude of documents for which the mathematical overlap is calculated.

28. The method of claim 26, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

29. The method of claim 25, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

30. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, the method comprising:

classifying the list of documents so that each document in the list has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching the list of documents by comparing the user fingerprint with the fingerprint for each document in the list of documents, and

re-ranking the list of documents based on a degree to which a fingerprint of a document in the list has a mathematical overlap with the user fingerprint.

31. The method of claim 30, wherein either the weights are scaled or else the mathematical overlaps are scaled.

32. The method of claim 31, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

33. The method of claim 30, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

69. A method according to claims 30 or 34 or 38 or 53 or 57 or 61 or 65, wherein a second user fingerprint is maintained, the second user fingerprint constructed identically to the user fingerprint except that for the second user fingerprint the text or texts recently accessed by the user is more recently accessed by the user than the text or texts recently accessed by the user for the user fingerprint, calculating a mathematical overlap between the user fingerprint and the second user fingerprint, the second user fingerprint superseding the user fingerprint when said mathematical overlap exceeds a set value.

34. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, and wherein each document in the list has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching the list of documents by comparing the user fingerprint with the fingerprint for each document in the list of documents, and

re-ranking the list of documents based on a degree to which a fingerprint of a document in the list has a mathematical overlap with the user fingerprint.

35. The method of claim 34, wherein either the weights are scaled or else the mathematical overlaps are scaled.

36. The method of claim 35, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

37. The method of claim 34, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

38. A method of re-ranking a list of documents obtained from a search wherein a ranking of a document in the list is determined by a relevance of the document to a search text, wherein the list of documents has been classified and appears in an inverted list, said inverted list comprising for each topic category of a classification system a weight associated with a particular document of the list of documents, the weight representing a degree to which the particular document relates to said each topic category, the weights obtained automatically from a computer program, the method comprising:

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching the inverted list by comparing the user fingerprint with the fingerprint for each document in the inverted list, and

re-ranking the list of documents by making use of a degree to which a fingerprint of a document in the inverted list has a mathematical overlap with the user fingerprint.

39. The method of claim 38, wherein either the weights are scaled or else the mathematical overlaps are scaled.

40. The method of claim 39, wherein an upper bound to the mathematical overlap is calculated dynamically, said upper bound used to reduce a magnitude of documents for which the mathematical overlap is calculated.

41. The method of claim 39, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

42. The method of claim 38, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

43. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified and appears in an inverted list, said inverted list comprising for each topic category of a classification system a weight associated with a particular document of the list of documents, each of the weights representing a degree to which the particular document relates to said each topic category, the weights obtained automatically from a computer program, identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in that all or a portion of the universe of documents, and

ranking the all or a portion of the universe of documents by making use of a degree to which a fingerprint of a document in the inverted list has a mathematical overlap with the fingerprint of the first piece of text.

44. The method of claim 43, wherein either the weights are scaled or else the mathematical overlaps are scaled.

45. The method of claim 44, wherein an upper bound to the mathematical overlap is calculated dynamically, said upper bound used to reduce a magnitude of documents for which the mathematical overlap is calculated.

46. The method of claim 44, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

47. The method of claim 43, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

48. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified and appears in an inverted list, said inverted list comprising for each topic category of a classification system a weight associated with a particular document of the list of documents, each of the weights representing a degree to which the particular document relates to said each topic category, the weights obtained automatically from a computer program, identifying a first piece of text,

creating a fingerprint for said first piece of text, the fingerprint comprising a list of weights associated with particular topic categories in the classification system, each of the weights in the fingerprint for said first piece of text representing a degree to which the first piece of text relates to the particular topic category that the weight in the fingerprint for said first piece of text is associated with, the weights in the fingerprint for said first piece of text obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the fingerprint for the first piece of text with the fingerprint for each document in that all or a portion of the universe of documents, and

selecting those documents whose fingerprints have a predetermined degree of mathematical overlap with the fingerprint of the first piece of text.

49. The method of claim 48, wherein either the weights are scaled or else the mathematical overlaps are scaled.

50. The method of claim 49, wherein an upper bound to the mathematical overlap is calculated dynamically, said upper bound used to reduce a magnitude of documents for which the mathematical overlap is calculated.

51. The method of claim 49, wherein a fingerprint of a document and/or a fingerprint of the first piece of text only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

52. The method of claim 48, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

53. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified so that each document in the universe has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the user fingerprint with the fingerprint for each document in that all or a portion of the universe of documents, and

ranking the all or a portion of the universe of documents based on a degree to which a document has a mathematical overlap with the user fingerprint.

54. The method of claim 53, wherein either the weights are scaled or else the mathematical overlaps are scaled.

55. The method of claim 54, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

56. The method of claim 53, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

57. A method of locating relevant documents within a universe of documents, the documents of said universe having been classified so that each document in the universe has a fingerprint, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program, the method comprising:

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the user fingerprint with the fingerprint for each document in that all or a portion of the universe of documents, and

selecting those documents whose fingerprints have a predetermined degree of mathematical overlap with the user fingerprint.

58. The method of claim 57, wherein either the weights are scaled or else the mathematical overlaps are scaled.

59. The method of claim 58, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

60. The method of claim 57, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

61. A method of locating relevant documents within a universe of documents, comprising:

creating a fingerprint for each document in the universe of documents, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the user fingerprint with the fingerprint for each document in that all or a portion of the universe of documents, and

ranking the all or a portion of the universe of documents based on a degree to which a document has a mathematical overlap with the user fingerprint.

62. The method of claim 61, wherein either the weights are scaled or else the mathematical overlaps are scaled.

63. The method of claim 62, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

64. The method of claim 61, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.

65. A method of locating relevant documents within a universe of documents, comprising:

creating a fingerprint for each document in the universe of documents, said fingerprint comprising a list of weights associated with particular topic categories in a classification system, each of the weights representing a degree to which the document relates to the particular topic category that the weight is associated with, the weights obtained automatically from a computer program,

providing a user fingerprint, the user fingerprint comprising a list of cumulative weights associated with particular topic categories in the classification system, each of the cumulative weights representing a degree to which text or texts recently accessed by a user relates to the particular topic category that the cumulative weight is associated with, the cumulative weights obtained from weights that in turn were obtained automatically from a computer program,

searching all or a portion of the universe of documents by comparing the user fingerprint with the fingerprint for each document in that all or a portion of the universe of documents, and

selecting those documents whose fingerprints have a predetermined degree of mathematical overlap with the user fingerprint.

66. The method of claim 65, wherein either the weights are scaled or else the mathematical overlaps are scaled.

67. The method of claim 66, wherein a fingerprint of a document only includes those topic categories whose associated weights are among a selected number of highest associated weights for that document.

68. The method of claim 65, wherein a selected number of weights is approximately 75 and the classification system is the Dewey Decimal System.