GRAPHICAL MODELS FOR REPRESENTING TEXT DOCUMENTS FOR COMPUTER ANALYSIS

• US 20110302168A1
• Filed: 06/08/2010
• Published: 12/08/2011
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aggarwal, Charu
• Granted Patent: US 8,375,061 B2
• Time in Patent Office: Days
• US Class Current: 707/739
• CPC Class Codes: G06F 16/316 Indexing structures

Abstract

In a method for representing a text document with a graphical model, a document including a plurality of ordered words is received and a graph data structure for the document is created. The graph data structure includes a plurality of nodes and edges, with each node representing a distinct word in the document and each edge identifying a number of times two nodes occur within a predetermined distance from each other. The graph data structure is stored in an information repository.

Claims

1. A method, comprising:

receiving a document including a plurality of ordered words;

creating a graph data structure for the document, wherein the graph data structure includes a plurality of nodes and edges, each node representing a distinct word in the document and each edge identifying a number of times two nodes occur within a predetermined distance from each other;

storing the graph data structure in an information repository;

receiving a request to perform text analysis on the document; and

performing text analysis on the graph data structure and providing a result that is responsive to the request.

2. The method of claim 1, wherein the predetermined distance is based on an order value of the graph data structure.

3. The method of claim 1, wherein an edge is a directed edge or an undirected edge.

4. The method of claim 1, wherein after receiving the document and before creating the graph data structure, the method further comprises pruning stop words from the document, wherein the graph data structure is created from the pruned document.

5. The method of claim 1, wherein the text analysis is a text search.

6. A method, comprising:

receiving a graph data structure of a document that includes a plurality of words, wherein the graph data structure includes a plurality of nodes and edges, each node representing a distinct word in the document, each edge identifying a number of times two nodes occur within a predetermined distance from each other;

constructing a vector-space representation of the document by assigning a unique token to each edge in the graph data structure, wherein for each edge, a frequency of the token is equivalent to the number of times the two nodes occur with the predetermined distance from each other, wherein the vector-space representation contains the tokens; and

outputting the vector-space representation.

7. The method of claim 6, wherein a token includes a pseudo-word.

8. The method of claim 6, further comprising:

receiving a request to perform text analysis on the document; and

performing text analysis on the vector-space representation to obtain a result that is responsive to the request.

9. The method of claim 6, wherein prior to receiving the graph data structure of the document the method further comprises:

receiving a document corpus; and

creating a unique graph data structure for each document in the corpus.

10. The method of claim 9, wherein the nodes of the unique graph data structures are referenced to each other if the nodes share the same word labels.

11. A computer program product, comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to receive a document including a plurality of ordered words; and

computer readable program code configured to create a graph data structure for the document, wherein the graph data structure includes a plurality of nodes and edges, each node representing a distinct word in the document and each edge identifying a number of times two nodes occur within a predetermined distance from each other.

12. The computer program product of claim 11, wherein the predetermined distance is based on an order value of the graph data structure.

13. The computer program product of claim 11, wherein an edge is a directed edge or an undirected edge.

14. The computer program product of claim 11, wherein after receiving the document and before creating the graph data structure, the computer readable program code of the computer readable storage medium further comprises computer readable program code configured to prune stop words from the document, wherein the graph data structure is created from the pruned document.

15. The computer program product of claim 11, wherein the computer readable program code of the computer readable storage medium further comprises:

computer readable program code configured to receive a request to perform text analysis on the document; and

computer readable program code configured to perform text analysis on the graph data structure to obtain a result that is responsive to the request.

16. A computer program product, comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to receive a graph data structure of a document that includes a plurality of words, wherein the graph data structure includes a plurality of nodes and edges, each node representing a distinct word in the document, each edge identifying a number of times two nodes occur within a predetermined distance from each other; and

computer readable program code configured to construct a vector-space representation of the document by assigning a unique token to each edge in the graph data structure, wherein for each edge, a frequency of the token is equivalent to the number of times the two nodes occur with the predetermined distance from each other, wherein the vector-space representation contains the tokens.

17. The computer program product of claim 16, wherein a token includes a pseudo-word.

18. The computer program product of claim 16, wherein the computer readable program code of the computer readable storage medium further comprises:

computer readable program code configured to receive a request to perform text analysis on the document; and

computer readable program code configured to perform text analysis on the vector-space representation to obtain a result that is responsive to the request.

19. The computer program product of claim 16, wherein prior to receiving the graph data structure of the document the computer readable program code of the computer readable storage medium further comprises:

computer readable program code configured to receive a document corpus; and

computer readable program code configured to create a unique graph data structure for each document in the corpus.

20. The computer program product of claim 19, wherein the nodes of the unique graph data structures are linked to each other if the nodes share the same word labels.