Techniques for knowledge discovery by constructing knowledge correlations using concepts or terms

• US 8,108,389 B2
• Filed: 11/14/2005
• Issued: 01/31/2012
• Est. Priority Date: 11/12/2004
• Status: Active Grant

• Current Assignee: Make Sence Incorporated
• Original Assignee: Make Sence Incorporated
• Inventors: Bobick, Mark, Wimmer, Carl
• Primary Examiner(s): Yen, Syling
• Application Number:
  US11/273,568
• Publication Number:
  US 20060253431A1
• Time in Patent Office: 2,269 Days
• Field of Search: 707/104.1, 707/722, 707/736, 707/737, 707/741, 707/748, 707/758, 707/781, 707999003-999007, 709/202, 704/10, 704/9, 706/46, 715/256
• US Class Current: 707/736
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Techniques for identifying knowledge use an graphical user interface for inputting one or more terms to be explored for additional knowledge. Then a search is conducted across one or more sources of information to identify resources containing information about or information associated with said terms. The resources are decomposed into elemental units of information and stored in a data structures called nodes. A group of nodes are stored in a node pool and, from the node pool, correlations of nodes are constructed that represent knowledge.

Claims

1. A method for identifying knowledge comprising:

using an input device coupled to a processor and associated memory for permitting inputting at least one term to be explored for additional knowledge;

using the processor and the associated memory coupled to the input device for searching one or more sources of information to identify resources comprising information associated with the at least one term;

using the processor and the associated memory for decomposing the resources into a plurality of nodes using natural language parsing to perform word classification based upon parts of speech and so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween;

using the processor and the memory for storing the plurality of nodes in a node pool; and

using the processor and the associated memory for constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

2. The method of claim 1 wherein inputting comprises at least one of typing at least one term into a command line, inputting at least one term into a graphical user interface, and inputting a natural language description of a concept.

3. The method of claim 2 wherein inputting the natural language description of the concept comprises parsing the description into tokens to be explored for additional knowledge.

4. The method of claim 1 wherein the searching comprises at least one of searching files on a personal computer, searching files on at least one computer on a network, and searching files on a network server.

5. The method of claim 4 wherein the files are translated into files comprising text.

6. The method of claim 4 wherein searching comprises using at least one spider to explore files.

7. The method of claim 6 wherein the at least one spider clones other spiders for searching.

8. The method of claim 6 wherein the at least one spider captures information about relevancy of a resource.

9. The method of claim 6 wherein the at least one spider comprises a meta-crawler.

10. The method of claim 1 wherein the one or more sources of information comprises at least one of a file system, a world wide web, an email repository, attachments to email in an email repository, a relational data base management system, a data warehouse, a taxonomy, an ontology, a semantic net, a neural net, a search engine, and an index engine.

11. The method of claim 1 wherein searching the one or more sources of information comprises searching at least one of a file name, a title of a document, an abstract of a document, a full text of a document, and a pool of nodes.

12. The method of claim 1 wherein the resources comprise at least one of a portable document format (PDF) file, a rich text format (RTF) file, a word processing file, a Microsoft PowerPoint (PPT) file, a Hyper Text Markup Language (HTML) page, a file comprising an email, and a file comprising an email attachment.

13. The method of claim 11 wherein identifying resources comprises at least one of finding a match between content of the one or more sources of information and the at least one term to be explored, finding a version of the at least one term to be explored in the content of the one or more sources, finding a synonym of the at least one term to be explored in the content of the one or more sources, finding terms in a dictionary definition of the at least one term to be explored in the content of the one or more sources, finding terms in an authoritative source of the at least one term to be explored in the content of the one or more sources, finding terms in an entry in an encyclopedia including the at least one term to be explored in the content of the one or more sources, finding terms from a taxonomy, from an ontology, or from a semantic net adjacent to the at least one term to be explored in the content of the one or more sources.

14. The method of claim 1 wherein each node further comprises at least one of a source entry and a sequence entry.

15. The method of claim 1 wherein the natural language parsing discards any characters that are not permissible to be used in constructing tokens in the language.

16. The method of claim 15 wherein the tokens are placed into fields of at least one node based on their category.

17. The method of claim 16 wherein the node pool permits selection and retrieval of individual nodes based on the contents of at least one part of a node.

18. The method of claim 16 wherein the node pool permits selection and retrieval using a hash map.

19. The method of claim 16 wherein the node pool is a database.

20. The method of claim 1 wherein the chain of nodes begins with an origin node comprising a term of interest.

21. The method of claim 20 wherein the nodes are added to the chain of nodes until at least one of the following occurs:

a user halts correlation;

a set time expires;

said chain of nodes comprises a number of nodes greater than a specified number;

no further nodes in the node pool can be associated with the chain of nodes;

a pre-selected term from a target node is added to the correlation; and

a pre-selected target node is added to the correlation.

22. A physical computer-readable medium having instructions stored thereon which, when executed by a computer, cause the computer to perform an method for identifying knowledge comprising:

inputting at least one term to be explored for additional knowledge;

searching one or more sources of information to identify resources comprising information associated with the at least one term;

decomposing the resources into a plurality of nodes using natural language parsing to perform word classification based upon parts of speech and so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween;

storing the plurality of nodes in a node pool; and

constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

23. The physical computer-readable medium of claim 22 wherein inputting comprises at least one of typing at least one term into a command line, inputting at least one term into a graphical user interface, and inputting a natural language description of a concept.

24. The physical computer-readable medium of claim 23 wherein inputting the natural language description of the concept comprises parsing the description into tokens to be explored for additional knowledge.

25. The physical computer-readable medium of claim 22 wherein the searching comprises at least one of searching files on a personal computer, searching files on at least one computer on a network, and searching files on a network server.

26. The physical computer-readable medium of claim 25 wherein the files are translated into files comprising text.

27. The physical computer-readable medium of claim 25 wherein searching comprises using at least one spider to explore files.

28. The physical computer-readable medium of claim 27 wherein the at least one spider clones other spiders for searching.

29. The physical computer-readable medium of claim 27 wherein the at least one spider captures information about relevancy of a resource.

30. An apparatus for identifying knowledge comprising:

an input device for permitting inputting at least one term to be explored for additional knowledge; and

a processor and memory associated therewith and coupled to said input device for

inputting at least one term to be explored for additional knowledge,

searching one or more sources of information to identify resources comprising information associated with the at least one term,

decomposing the resources into a plurality of nodes using natural language parsing to perform word classification based upon parts of speech and so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween,

storing the plurality of nodes in a node pool, and

constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

31. A system comprising:

a network; and

at least one computer connected to said network comprising

an input device for permitting inputting of at least one term to be explored for additional knowledge, and

a processor and memory associated therewith and coupled to said input device for

inputting at least one term to be explored for additional knowledge,

searching one or more sources of information to identify resources comprising information associated with the at least one term,

decomposing the resources into a plurality of nodes using natural language parsing to perform word classification based upon parts of speech and so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween,

storing the plurality of nodes in a node pool, and

constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

32. A method for identifying knowledge comprising:

using an input device coupled to a processor and associated memory for permitting inputting at least one term to be explored for additional knowledge;

using the processor and the associated memory coupled to the input device for searching one or more sources of information to identify resources comprising information associated with the at least one term;

using the processor and the associated memory for decomposing the resources into a plurality of nodes by using an intermediate format wherein a first term is paired with a second term based upon a relation defined by a context so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween, the context comprising at least one of a schema, a tree graph and a directed graph;

using the processor and the memory for storing the plurality of nodes in a node pool; and

using the processor and the associated memory for constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

33. The method of claim 32 wherein inputting comprises at least one of typing at least one term into a command line, inputting at least one term into a graphical user interface, and inputting a natural language description of a concept.

34. The method of claim 33 wherein inputting the natural language description of the concept comprises parsing the description into tokens to be explored for additional knowledge.

35. The method of claim 32 wherein the searching comprises at least one of searching files on a personal computer, searching files on at least one computer on a network, and searching files on a network server.

36. A physical computer-readable medium having instructions stored thereon which, when executed by a computer, cause the computer to perform an method for identifying knowledge comprising:

inputting at least one term to be explored for additional knowledge;

searching one or more sources of information to identify resources comprising information associated with the at least one term;

decomposing the resources into a plurality of nodes by using an intermediate format wherein a first term is paired with a second term based upon a relation defined by a context so that each node comprises a data structure sufficient to independently convey meaning and including a subject, an attribute, and a word based relational bond therebetween, the context comprising at least one of a schema, a tree graph and a directed graph;

storing the plurality of nodes in a node pool; and

constructing and storing correlations of linked together nodes of the node pool representing knowledge associated with the at least one term to be explored by at least iteratively adding nodes from the node pool onto an end of a chain of nodes by searching the node pool for a match between an attribute of a chained node and a subject of another unchained node in the node pool.

37. The physical computer-readable medium of claim 36 wherein inputting comprises at least one of typing at least one term into a command line, inputting at least one term into a graphical user interface, and inputting a natural language description of a concept.

38. The physical computer-readable medium of claim 37 wherein inputting the natural language description of the concept comprises parsing the description into tokens to be explored for additional knowledge.

39. The physical computer-readable medium of claim 36 wherein the searching comprises at least one of searching files on a personal computer, searching files on at least one computer on a network, and searching files on a network server.