Document analysis and association system and method

• US 8,666,994 B2
• Filed: 09/24/2010
• Issued: 03/04/2014
• Est. Priority Date: 09/26/2009
• Status: Active Grant

• Current Assignee: Sajari Pty Ltd. (Algolia, Inc.)
• Original Assignee: Sajari Pty Ltd. (Algolia, Inc.)
• Inventors: Prime, Owen James, Burns, Phillip Anthony, Ogilvy, Hamish
• Primary Examiner(s): Bhatia, Ajay
• Assistant Examiner(s): Le, Jessica N
• Application Number:
  US13/498,244
• Publication Number:
  US 20120278341A1
• Time in Patent Office: 1,257 Days
• Field of Search: None
• US Class Current: 707/749
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06Q 10/00 Administration; Management

Abstract

Methods and systems for indexing a plurality of documents, each document comprising a text portion, the method parses the text portion of each of the plurality of documents to form a plurality of respective local document indexes each associated with a respective document, and stores the local document index in a database, Each local document index has a plurality of local text terms and a local weighting associated with each text term From the plurality of local document indexes, forming a global document index associated with each global text term. The global weighting is determined with respect to a parameter associated with a reference global text term. Also, methods and systems for analyzing a text portion, retrieving documents from a database relevant to the text portion and for refining the results of a search are disclosed.

Claims

1. A method for analysing a text portion and retrieving documents relevant to the text portion, the method comprising:

a) receiving an input comprising an input text portion;

b) using a processor to identify at least one text term in the input text portion;

c) using the processor, assigning at least one weight associated with the at least one text term;

d) using the processor, to form an input local index of the at least one text term in the input text portion and at least one associated local term weight, wherein the at least one associated local term weight is determined with reference to a global term index stored in a database, the global term index comprising a plurality of global text terms and associated global text term weights, and being formed from a plurality of reference documents, wherein a representation of each of the reference documents is stored in the database;

e) using the processor, querying the database to identify one or more of the reference documents of relevance with respect to the input text portion;

f) outputting a representation of the identified relevant reference documents;

g) displaying the relevant reference documents on a user interface, the user interface comprising input means for receiving user input with respect to each of the displayed reference documents;

h) accepting user-input with respect to one or more of the displayed documents;

i) using the processor, re-forming the input local term index on the basis of the user input;

j) on the basis of the re-formed input local term index, using the processor to query the database to identify one or more relevant reference documents of enhanced relevance to the input text portion; and

k) using the processor, outputting a representation of the further identified reference documents of enhanced relevance to the user interface for presentation to a user;

wherein, the processor comprises a programmable data processing apparatus comprising instructions which execute via the processor to implement the steps (a) to (k).

2. The method as claimed in claim 1 wherein in step i) of claim 1, re-forming the input local term index comprises:

i.1) re-assigning the input local text term weights of the input text terms which also appear in each of the reference documents for which user-determined input is received; and

wherein step j) of claim 1 comprises;

j.1) on the basis of the re-assigned input local text term weights, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion.

3. The method as claimed in claim 1 further comprising repeating steps h) to k) of claim 1 thereby identifying and outputting one or more relevant further documents of increased enhanced relevance in respect of the relevant text terms in the text portion.

4. The method as claimed in claim 1 wherein the user input comprises additional relevance information comprising either a positive indication of relevance of a particular document or a negative indication of relevance of a particular document.

5. The method as claimed in claim 4 wherein for each reference document for which a positive indication of relevance is received, the associated weighting of each of the input text terms in the input local index which also appear in the local text term index of the positively-identified reference document is increased by a predetermined amount.

6. The method as claimed in claim 4 wherein for each reference document for which a negative indication of relevance is received, the associated weighting of each of the input text terms in the input local index which also appear in the local text term index of the negatively-identified reference document is decreased by a predetermined amount.

7. The method as claimed in claim 5 wherein the predetermined amount is a multiplier applied to the index term weight.

8. The method as claimed in claim 7 wherein the multiplier is zero such that a selected term has no relevance to subsequent interactions.

9. The method as claimed in claim 4 wherein where a selected text term appears in one or more documents which receive a positive indication of relevance, and the selected text term also appears in one or more documents which receive a negative indication of relevance, the associated weighting of the selected text term in the input local index is updated based on a combination of the positive and negative indications.

10. The method as claimed in claim 5 wherein in step step i) of claim 1, re-forming the input local term index comprises:

i.2) forming an augmented input local term index on the basis of text terms in the local term index of documents receiving a positive indication of relevance; and

wherein step j) of claim 1 comprises;

j.2) on the basis of the input local text term weights in the augmented input local text term index, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion.

11. The method as claimed in claim 10 wherein for each reference document for which a positive indication of relevance is received, terms in the positively identified reference document which do not appear in the input local term index are added thereto to form the augmented local text term index together with associated local index text term weights determined.

12. The method as claimed in claim 1 wherein the text portion is a text string comprising a plurality of text words or a text document.

13. The method as claimed in claim 1 wherein each of the reference documents is assigned a document relevance score representative of the relevance to the input text portion.

14. The method as claimed in claim 1 wherein the reference documents are text documents representative of a document selected from one or more of the group of:

a book;

an article;

a text record;

a certificate;

an agreement;

a contract;

a manuscript;

a paper;

a scientific paper;

a resume;

a patent specification;

an employment advertisement;

a legal transcript;

a legal document;

an advertisement;

or a news report.

15. The method for refining the results of a search, the search results comprising a representation of a selected plurality of reference documents, such reference documents displayed being of relevance to an input text portion comprising one or more search terms, the selected plurality of reference documents comprising a subset of a plurality of documents in a database, the method comprising the steps of:

a) using a processor to form a local term index from the search terms in the input text portion, the local term index comprising one or more text terms, each local text term associated with a local text term weight;

b) using the processor, receiving and displaying the search results on a user interface, the user interface comprising input means for receiving user input with respect to one or more of the plurality of the displayed reference documents;

c) accepting user input on one or more of the displayed reference documents;

d) using the processor, re-forming the local term index on the basis of the user input;

e) on the basis of the re-formed input local term index, using the processor to query the database to identify one or more documents of enhanced relevance to the input text portion; and

f) outputting a representation of the further identified reference documents of enhanced relevance to the user interface for presentation to a user;

wherein, the processor comprises a programmable data processing apparatus comprising instructions which execute via the processor to implement the steps (a) to (f);

wherein in step a) the local text term weights for each of the local text terms are derived from a global text term index, the global text term index comprising a plurality of text terms associated with global text term weights, wherein the global text term weights are derived from text term analysis of a plurality of documents.

16. A method as claimed in claim 15 wherein in step d), re-forming the input local term index comprises:

d.1) re-assigning the input local text term weights of the input text terms which also appear in each of the reference documents for which user-determined input is received; and

wherein step e) of claim 15 comprises;

e.1) on the basis of the re-assigned input local text term weights, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion.

17. The method as claimed in claim 15 wherein in step a) the local text term weights for each of the local text terms are equal.

18. The method as claimed in claim 15 wherein in step d), re-forming the input local term index comprises:

d.2) forming an augmented input local term index on the basis of text terms in the local term index of documents receiving a positive indication of relevance; and

wherein step e) comprises;

e.2) on the basis of the input local text term weights in the augmented input local text term index, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion.

19. The method as claimed in claim 18 wherein for each reference document for which a positive indication of relevance is received, new terms in the positively identified reference document which do not appear in the local term index are added thereto to form the augmented local text term index and associated local index text term weights for the new terms are determined.

20. The system for refining the results of a search, the search results comprising a representation of a selected plurality of documents of relevance to one or more search terms, the selected plurality of documents comprising a subset of a plurality of documents in a database, the system comprising:

processing means for forming a local term index from the search terms, the local term index comprising one or more text terms, each local text term associated with a local text term weight;

processing means for receiving and displaying the search results on a user interface, the user interface comprising input means for receiving user input with respect to each of the displayed reference documents, weights for each of the local text terms are derived from a global text term index, the global text term index comprising a plurality of text terms associated with global text term weights, wherein the global text term weights are derived from text term analysis of a plurality of documents;

user input means for accepting user input on one or more of the displayed documents;

processing means for analysing the user input and re-forming the input local term index on the basis of the user input;

query means for querying the database on the basis of the re-formed input local term index to identify one or more documents of enhanced relevance to the input text portion; and

output means for outputting a representation of the further identified reference documents of enhanced relevance.

21. A system for analysing an input text portion and retrieving documents relevant to the text portion, the system comprising:

input means for receiving an input comprising an input text portion;

identification means to identify at least one text term in the text portion;

assignment means for assigning at least one weight associated with the at least one text term;

indexing means for forming an input local term index of the at least one text term in the input text portion and at least one associated local term weight, wherein the at least one associated local text term weights is determined with reference to a global term index stored in a database, the global term index comprising a plurality of global text terms and associated global text term weights, and being formed from a plurality of reference documents, wherein a representation of each of the reference documents is stored in the database;

query means for querying the database to identify one or more relevant reference documents with respect to the input text portion;

output means for outputting a representation of the identified relevant reference documents;

display means for displaying the relevant reference documents on a user interface, the user interface comprising input means for receiving user input with respect to each of the displayed reference documents;

user input means accepting user-input on one or more of the displayed documents;

processing means for analysing the user input and re-forming the input local text term index;

query means for querying the database on the basis of the re-formed input local text term index to identify one or more relevant reference documents of enhanced relevance to the input text portion; and

output means for outputting a representation of the further identified reference documents of enhanced relevance.

22. The system as claimed in claim 21 wherein the re-forming of the input local term index comprises:

with the processing means, re-assigning the input local text term weights of the input text terms which also appear in each of the reference documents for which user-determined input is received; and

the querying of the database on the basis of the re-formed input local text term index comprises;

with the query means, querying the database on the basis of the re-assigned input local text term weights to identify one or more relevant reference documents of enhanced relevance to the input text portion.

23. The system as claimed in claim 21 wherein the re-forming of the input local term index comprises:

with the processing means, forming an augmented input local term index on the basis of text terms in the local term indexes of documents receiving a positive indication of relevance; and

the querying of the database on the basis of the re-formed input local text term index comprises;

with the query means, querying the database on the basis of the input local text term weights in the augmented input local text term index to identify one or more relevant reference documents of enhanced relevance to the input text portion.

24. A non-transitory computer readable medium comprising a program for analysing a text portion and retrieving documents relevant to the text portion, said program controlling the operation of a data processing apparatus on which the program executes on a processor to perform the steps of:

a) receiving an input comprising an input text portion;

b) identify at least one text term in the text portion;

c) assigning at least one weight associated with the at least one text term;

d) forming an input local index of the at least one text term and at least one associated local term weight, wherein the at least one associated local term weight is determined with reference to a global term index stored in a database, the global term index comprising a plurality of global text terms and associated global text term weights, and being formed from a plurality of reference documents, wherein a representation of each of the reference documents is stored in the database

e) querying the database to identify one or more of the reference documents of relevance with respect to the input text portion;

f) outputting a representation of the identified relevant reference documents;

g) displaying the relevant reference documents on a user interface, the user interface comprising input means for receiving user input with respect to each of the displayed reference documents;

h) accepting user-input with respect to one or more of the displayed documents;

i) re-forming the input local term index on the basis of the user input;

j) on the basis of the re-formed input local term index, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion; and

k) outputting a representation of the further identified reference documents of enhanced relevance.

25. A non-transitory computer readable medium comprising a program for refining the results of a search, the search results comprising a representation of a selected plurality of reference documents, such reference documents displayed being of relevance to an input text portion comprising one or more search terms, the selected plurality of documents comprising a subset of a plurality of documents in a database, said program controlling the operation of a data processing apparatus on which the program executes on a processor to perform the steps of:

a) forming a local term index from the search terms, the local term index comprising one or more text terms, each local text term associated with a local text term weight, wherein the local text term weights for each of the local text terms are derived from a global text term index, the global text term index comprising a plurality of text terms associated with global text term weights, wherein the global text term weights are derived from text term analysis of a plurality of documents;

b) receiving and displaying the search results on a user interface, the user interface comprising input means for receiving user input with respect to one or more of the plurality of the displayed reference documents;

c) accepting user input on one or more of the displayed documents;

d) re-forming the input local term index on the basis of the user input;

e) on the basis of the re-formed input local term index, querying the database to identify one or more documents of enhanced relevance to the input text portion; and

f) outputting a representation of the further identified reference documents of enhanced relevance.