Method and apparatus for constructing a compact similarity structure and for using the same in analyzing document relevance

• US 20080275870A1
• Filed: 05/15/2008
• Published: 11/06/2008
• Est. Priority Date: 12/12/2005
• Status: Active Grant

• Current Assignee: Justsystems Evans Research Incorporated
• Original Assignee: Justsystems Evans Research Incorporated
• Inventors: Evans, David A., Roma, Norbert, Shanahan, James G.
• Granted Patent: US 7,949,644 B2
• Time in Patent Office: Days
• US Class Current: 707/5
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A computer-readable medium comprises data structure for providing information about levels of similarity between pairs of N documents. The data structure comprises a plurality of entries of similarity values representing levels of similarity for a plurality of pairs of the documents. Each of the similarity values represents a level of similarity of one document of a given pair relative to the other document of the given pair. The similarity value of each entry is greater than a threshold similarity value that is greater than zero. The plurality of similarity-value entries are fewer than N²−N in number if the similarity values are asymmetric with regard to document pairing, and the plurality of similarity-value entries are fewer than

$\frac{N^2-N}{2}$

in number if the similarity values are symmetric with regard to document pairing. A method and apparatus for generating the data structure are described.

Claims

1. A method for retrieving similarity values from a data structure for a set of documents, comprising:

accessing the data structure to determine whether the data structure contains an explicit entry for a similarity value for a given pair of documents;

if the data structure contains an explicit entry of the similarity value for the given pair of documents, retrieving the similarity value; and

if the data structure does not contain an explicit entry of the similarity value for the given pair of documents, retrieving a default similarity value from the data structure or from another memory location,

wherein the data structure comprises a plurality of entries of the similarity values, and wherein the plurality of entries of the similarity values are fewer than N²−

N in number if the similarity values are asymmetric with regard to document pairing and wherein the plurality of entries of the similarity values are fewer than

$\frac{N^2-N}{2}$

in number if the similarity values are symmetric with regard to document pairing, where N equals the number of documents.

2. The method of claim 1, wherein the default similarity value is equal to the threshold similarity value.

3. The method of claim 1, wherein the default similarity value is less than the threshold similarity value.

4. A computer readable medium storing processing instructions adapted to cause a processing unit to execute a method, comprising:

accessing the data structure to determine whether the data structure contains an explicit entry for a similarity value for a given pair of documents;

if the data structure contains an explicit entry of the similarity value for the given pair of documents, retrieving the similarity value; and

if the data structure does not contain an explicit entry of the similarity value for the given pair of documents, retrieving a default similarity value from the data structure or from another memory location,

wherein the data structure comprises a plurality of entries of the similarity values, and wherein the plurality of entries of the similarity values are fewer than N²−

N in number if the similarity values are asymmetric with regard to document pairing and wherein the plurality of entries of the similarity values are fewer than

$\frac{N^2-N}{2}$

in number if the similarity values are symmetric with regard to document pairing, where N equals the number of documents.

5. An apparatus for retrieving similarity values from a data structure for a set of documents, comprising:

a memory; and

a processing unit coupled to the memory, wherein the processing unit is configured to execute the steps of;

accessing the data structure to determine whether the data structure contains an explicit entry for a similarity value for a given pair of documents;

if the data structure contains an explicit entry of the similarity value for the given pair of documents, retrieving the similarity value; and

if the data structure does not contain an explicit entry of the similarity value for the given pair of documents, retrieving a default similarity value from the data structure or from another memory location,

wherein the data structure comprises a plurality of entries of the similarity values, and wherein the plurality of entries of the similarity values are fewer than N²−

N in number if the similarity values are asymmetric with regard to document pairing and wherein the plurality of entries of the similarity values are fewer than

$\frac{N^2-N}{2}$

in number if the similarity values are symmetric with regard to document pairing, and N equals the number of documents.

6. A computer-readable medium having stored thereon a data structure for providing information about levels of similarity between pairs of documents of a set of documents, the documents being N in number, the data structure comprising:

a plurality of entries of similarity values representing levels of similarity for a plurality of pairs of said documents, each of said similarity values representing a level of similarity of one document of a given pair relative to the other document of the given pair,

wherein the similarity value of each entry is greater than a threshold similarity value that is greater than zero, and

wherein the plurality of entries of similarity values are fewer than N²−

N in number if the similarity values are asymmetric with regard to document pairing and wherein the plurality of entries of similarity values are fewer than

$\frac{N^2-N}{2}$

in number if the similarity values are symmetric with regard to document pairing, where N equals the number of documents.

7. The computer readable medium of claim 6, further comprising:

an entry comprising the threshold similarity value; and

an entry comprising a default similarity value.

8. The computer readable medium of claim 6, wherein the default similarity value is equal to the threshold similarity value.

9. The computer readable medium of claim 6, wherein the default similarity value is less than the threshold similarity value.