Perturbing latent semantic indexing spaces

• US 20060235661A1
• Filed: 03/31/2006
• Published: 10/19/2006
• Est. Priority Date: 04/06/2005
• Status: Active Grant

• Current Assignee: Relativity ODA LLC
• Original Assignee: Content Analyst Company, LLC (Relativity ODA LLC)
• Inventors: Price, Robert Jenson
• Granted Patent: US 7,580,910 B2
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A text processing method is provided that includes the following steps. First, an abstract mathematical vector space is generated based on a collection of documents. Respective documents in the collection of documents have a representation in the abstract mathematical vector space and respective terms contained in the collection of documents have a representation in the abstract mathematical vector space. Then, the abstract mathematical vector space is perturbed to produce a perturbed abstract mathematical vector space that is stored in an electronic format accessible to a user. Perturbing the abstract mathematical vector space may include modifying the representation of a document with a newly computed representation for that document, or modifying the representation of a term with a newly computed representation for that term.

Claims

1. A computer-based text processing method, comprising:

(a) generating an abstract mathematical vector space based on a collection of documents, wherein respective documents in the collection of documents have a representation in the abstract mathematical vector space and respective terms contained in the collection of documents have a representation in the abstract mathematical vector space; and

(b) perturbing the abstract mathematical vector space to produce a perturbed abstract mathematical vector space that is stored in an electronic format accessible to a user, wherein perturbing the abstract mathematical vector space comprises at least one of (i) modifying the representation of a document with a newly computed representation for that document, or (ii) modifying the representation of a term with a newly computed representation for that term.

2. The computer-based method of claim 1, wherein generating an abstract mathematical vector space comprises generating a Latent Semantic Indexing (LSI) space.

3. The computer-based method of claim 1, wherein prior to the perturbing step the method further comprises representing a document not included in the collection of documents in the abstract mathematical vector space.

4. The computer-based method of claim 1, wherein prior to the perturbing step the method further comprises representing a term not contained in the collection of documents in the abstract mathematical vector space.

5. The computer-based method of claim 1, wherein perturbing the abstract mathematical vector space further comprises representing an additional document in the perturbed abstract mathematical vector space.

6. The computer-based method of claim 1, wherein perturbing the abstract mathematical vector space further comprises representing an additional term in the perturbed abstract mathematical vector space.

7. The computer-based method of claim 1, wherein perturbing the abstract mathematical vector space further comprises deleting from the perturbed abstract mathematical vector space document data relating to a document.

8. The computer-based method of claim 1, wherein perturbing the abstract mathematical vector space further comprises deleting from the perturbed abstract mathematical vector space term data relating to a term.

9. A computer-based text processing method, comprising:

(a) providing a first abstract mathematical vector space and a second abstract mathematical vector space, wherein the first abstract mathematical vector space is based on a first collection of documents and the second abstract mathematical vector space is based on a second collection of documents; and

(b) merging the first abstract mathematical vector space with the second abstract mathematical vector space to produce a merged abstract mathematical vector space that is stored in an electronic format accessible to a. user, wherein merging is based on a vector averaging of vectors in the first abstract mathematical vector space with vectors in the second abstract mathematical vector space.

10. The computer-based method of claim 9, wherein step (a) comprises:

generating a first abstract mathematical vector space based on a first collection of documents; and

generating a second abstract mathematical vector space based on a second collection of documents.

11. The computer-based method of claim 9, wherein step (a) comprises:

generating a first abstract mathematical vector space based on a first collection of documents; and

receiving a second abstract mathematical vector space that is based on a second collection of documents, wherein the second abstract mathematical vector space comprises a perturbed abstract mathematical vector space.

12. The computer-based method of claim 9, wherein step (a) comprises:

receiving a first abstract mathematical vector space that is based on a first collection of documents, wherein the first abstract mathematical vector space comprises a perturbed abstract mathematical vector space; and

receiving a second abstract mathematical vector space that is based on a second collection of documents, wherein the second abstract mathematical vector space comprises a perturbed abstract mathematical vector space.

13. The computer-based method of claim 9, wherein step (b) comprises:

merging the first abstract mathematical vector space with the second abstract mathematical vector space to produce a merged abstract mathematical vector space that is stored in an electronic format accessible to a user, wherein merging is based on at least one of (i) a vector averaging of document vectors in the first abstract mathematical vector space with document vectors in the second abstract mathematical vector space, or (ii) a vector averaging of term vectors in the first abstract mathematical vector space with term vectors in the second abstract mathematical vector space.

14. A computer program product comprising a computer usable medium having control logic stored therein for processing text, the control logic comprising:

computer readable first program code that causes the computer to generate an abstract mathematical vector space based on a collection of documents, wherein respective documents in the collection of documents have a representation in the abstract mathematical vector space and respective terms contained in the collection of documents have a representation in the abstract mathematical vector space; and

computer readable second program code that causes the computer to perturb the abstract mathematical vector space to produce a perturbed abstract mathematical vector space that is stored in an electronic format accessible to a user, wherein perturbing the abstract mathematical vector space comprises at least one of (i) modifying the representation of a document with a newly computed representation for that document, or (ii) modifying the representation of a term with a newly computed representation for that term.

15. The computer program product of claim 14, wherein the abstract mathematical vector space comprises a Latent Semantic Indexing (LSI) space.

16. The computer program product of claim 14, wherein the computer readable second program code comprises code to cause the computer to represent a document not included in the collection of documents in the abstract mathematical vector space prior to perturbing the abstract mathematical vector space to produce the perturbed abstract mathematical vector space.

17. The computer program product of claim 14, wherein the computer readable second program code comprises code to cause the computer to represent a term not contained in the collection of documents in the abstract mathematical vector space prior to perturbing the abstract mathematical vector space to produce the perturbed abstract mathematical vector space.

18. The computer program product of claim 14, wherein the computer readable second program code further comprises code that causes the computer to represent an additional document in the perturbed abstract mathematical vector space.

19. The computer program product of claim 14, wherein the computer readable second program code further comprises code that causes the computer to represent an additional term in the perturbed abstract mathematical vector space.

20. The computer program product of claim 14, wherein the computer readable second program code further comprises code that causes the computer to delete from the perturbed abstract mathematical vector space document data relating to a document.

21. The computer program product of claim 14, wherein the computer readable second program code further comprises code that causes the computer to delete from the perturbed abstract mathematical vector space term data relating to a term.

22. A computer program product comprising a computer usable medium having control logic stored therein for processing text, the control logic comprising:

computer readable first program code that causes the computer to provide a first abstract mathematical vector space and a second abstract mathematical vector space, wherein the first abstract mathematical vector space is based on a first collection of documents and the second abstract mathematical vector space is based on a second collection of documents; and

computer readable second program code that causes the computer to merge the first abstract mathematical vector space with the second abstract mathematical vector space to produce a merged abstract mathematical vector space that is stored in an electronic format accessible to a user, wherein merging is based on a vector averaging of vectors in the first abstract mathematical vector space with vectors in the second abstract mathematical vector space.

23. The computer program product of claim 22, wherein the computer readable first program code comprises:

code that causes the computer to generate a first abstract mathematical vector space based on a first collection of documents; and

code that causes the computer to generate a second abstract mathematical vector space based on a second collection of documents.

24. The computer program product of claim 22, wherein the computer readable first program code comprises:

code that causes the computer to generate a first abstract mathematical vector space based on a first collection of documents; and

code that causes the computer to receive a second abstract mathematical vector space that is based on a second collection of documents, wherein the second abstract mathematical vector space comprises a perturbed abstract mathematical vector space.

25. The computer program product of claim 22, wherein the computer readable first program code comprises:

code that causes the computer to receive a first abstract mathematical vector space that is based on a first collection of documents, wherein the first abstract mathematical vector space comprises a perturbed abstract mathematical vector space; and

code that causes the computer to receive a second abstract mathematical vector space that is based on a second collection of documents, wherein the second abstract mathematical vector space comprises a perturbed abstract mathematical vector space.

26. The computer program product of claim 22, wherein the computer readable second program code comprises:

code that causes the computer to merge the first abstract mathematical vector space with the second abstract mathematical vector space to produce a merged abstract mathematical vector space that is stored in an electronic format accessible to a user, wherein merging is based on at least one of (i) a vector averaging of document vectors in the first abstract mathematical vector space with document vectors in the second abstract mathematical vector space, or (ii) a vector averaging of term vectors in the first abstract mathematical vector space with term vectors in the second abstract mathematical vector space.