Method and mechanism for the creation, maintenance, and comparison of semantic abstracts

• US 7,653,530 B2
• Filed: 11/27/2006
• Issued: 01/26/2010
• Est. Priority Date: 07/13/2000
• Status: Expired due to Fees

• Current Assignee: RPX Corporation
• Original Assignee: Novell Incorporated (Open Text Corporation)
• Inventors: Millett, Ronald P., Jensen, Delos C., Carter, Stephen R.
• Primary Examiner(s): Hudspeth, David R
• Assistant Examiner(s): Spooner, Lamont M
• Application Number:
  US11/563,659
• Publication Number:
  US 20070106491A1
• Time in Patent Office: 1,156 Days
• Field of Search: 704/1, 704/9, 704/10, 715/259, 715/254, 707 2- 6, 707/101
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Codifying the “most prominent measurement points” of a document can be used to measure semantic distances given an area of study (e.g., white papers on some subject area). A semantic abstract is created for each document. The semantic abstract is a semantic measure of the subject or theme of the document providing a new and unique mechanism for characterizing content. The semantic abstract includes state vectors in the topological vector space, each state vector representing one lexeme or lexeme phrase about the document. The state vectors can be dominant phrase vectors in the topological vector space mapped from dominant phrases extracted from the document. The state vectors can also correspond to words in the document that are most significant to the document'"'"'s meaning (the state vectors are called dominant vectors in this case). One semantic abstract can be directly compared with another semantic abstract, resulting in a numeric semantic distance between the semantic abstracts being compared.

Claims

1. A method implemented in a computer system including one or more computers communicating with each other, each of the one or more computers including a memory, for determining a semantic abstract in a topological vector space for a semantic content of a document using a dictionary and a basis, where the document, dictionary, and basis are each stored on at least one of the one or more computers, comprising:

accessing the dictionary including a directed set of concepts, the directed set including at least one chain from a maximal element to each other concept in the dictionary;

accessing the basis, the basis including a subset of chains from the dictionary;

identifying dominant phrases in the document;

measuring how concretely each identified dominant phrase is represented in each chain in the basis and the dictionary;

constructing in the memory of one of the one or more computers dominant phrase vectors for the document using the measures of how concretely each identified dominant phrase is represented in each chain in the basis and the dictionary; and

determining the semantic abstract using the dominant phrase vectors.

2. A method according to claim 1, wherein measuring how concretely each identified dominant phrase is represented includes:

mapping each dominant phrase to at least one concept in the dictionary; and

measuring how concretely each mapped concept is represented in each chain in the basis and the dictionary.

3. A method according to claim 1, wherein identifying dominant phrases for a document includes identifying dominant phrases for a portion of the document.

4. A method according to claim 1, further comprising:

determining a second semantic abstract for a second document;

measuring a distance between the first semantic abstract and the second semantic abstract; and

determining if the first semantic abstract is within a threshold distance of the second semantic abstract.

5. A method according to claim 4, further comprising identifying the second document as including similar content as the document if the first semantic abstract is within a threshold distance of the second semantic abstract.

6. A method according to claim 4, wherein measuring a distance includes measuring a Hausdorff distance between the first semantic abstract and the second semantic abstract.

7. A method according to claim 4, wherein measuring a distance includes determining a centroid vector in the topological vector space for each semantic abstract.

8. A method according to claim 7, wherein measuring a distance further includes measuring an angle between each centroid vector.

9. A method according to claim 7, wherein measuring a distance further includes measuring a Euclidean distance between each centroid vector.

10. A computer-readable medium, said computer-readable medium having stored thereon a program, that, when executed by a computer, result in:

accessing a dictionary including a directed set of concepts, the directed set including at least one chain from a maximal element to each other concept in the dictionary;

accessing a basis, the basis including a subset of chains from the dictionary;

identifying dominant phrases in the document;

measuring how concretely each identified dominant phrase is represented in each chain in the basis and the dictionary;

constructing dominant phrase vectors for the document using the measures of how concretely each identified dominant phrase is represented in each chain in the basis and the dictionary; and

determining a semantic abstract using the dominant phrase vectors.

11. A computer-readable medium according to claim 10, wherein measuring how concretely each identified dominant phrase is represented includes:

mapping each dominant phrase to at least one concept in the dictionary; and

measuring how concretely each mapped concept is represented in each chain in the basis and the dictionary.

12. A computer-readable medium according to claim 10, wherein identifying dominant phrases for a document includes identifying dominant phrases for a portion of the document.

13. A computer-readable medium according to claim 10, further comprising:

determining a second semantic abstract for a second document;

measuring a distance between the first semantic abstract and the second semantic abstract; and

determining if the first semantic abstract is within a threshold distance of the second semantic abstract.

14. A computer-readable medium according to claim 13, further comprising identifying the second document as including similar content as the document if the first semantic abstract is within a threshold distance of the second semantic abstract.

15. A computer-readable medium according to claim 13, wherein measuring a distance includes measuring a Hausdorff distance between the first semantic abstract and the second semantic abstract.

16. A computer-readable medium according to claim 13, wherein measuring a distance includes determining a centroid vector in the topological vector space for each semantic abstract.

17. A computer-readable medium according to claim 16, wherein measuring a distance further includes measuring an angle between each centroid vector.

18. A computer-readable medium according to claim 16, wherein measuring a distance further includes measuring a Euclidean distance between each centroid vector.