Knowledge system method and apparatus

• US 8,874,431 B2
• Filed: 12/23/2010
• Issued: 10/28/2014
• Est. Priority Date: 03/16/2001
• Status: Expired due to Fees

• Current Assignee: JGC Corporation (JGC Holdings Corporation)
• Original Assignee: Meaningful Machines LLC
• Inventors: Abir, Eli
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US12/977,887
• Publication Number:
  US 20110202334A1
• Time in Patent Office: 1,405 Days
• Field of Search: 704 1- 10
• US Class Current: 704/9
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 40/42 Data-driven translation
• CPC Class Codes: G06F 40/55 Rule-based translation

Abstract

A method and apparatus for automating the acquisition, reconstruction, and generation of knowledgebases of associated ideas and using such knowledgebases in many application including machine translation of human languages, search and retrieval of unstructured text, or other data, based on concept search, voice recognition, data compression, and artificial intelligence systems.

Claims

1. A computer device including a processor, a memory coupled to the processor, and a program stored in the memory, wherein the computer is configured to execute the program and perform the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined amount of words or word strings or both to the left of said query to be analyzed in said returned documents based on their frequency and creating a Left Signature List comprising said words or word strings or both to the left of said query to be analyzed in said returned documents;

e. searching said collection of documents for each word and word string on said Left Signature List and returning documents containing said words and word strings on said Left Signature List;

f. determining a user-defined amount of words or word strings or both to the right of said words or word strings or both on said Left Signature List in said returned documents and creating Left Anchor Lists comprising said words or word strings or both to the right of said words or word strings or both on said Left Signature List based on their frequency in said returned documents;

g. determining a user-defined number of words or word strings or both to the right of said query to be analyzed in said returned documents and creating a Right Signature List comprising said words or word strings or both to the right of said query to be analyzed in said returned documents based on their frequency;

h. searching said collection of documents for words or word strings or both on said Right Signature List and returning documents containing said words and word strings on said Right Signature List;

i. determining a user-defined number of words or word strings or both to the left of said words or word strings or both on said Right Signature List in said returned documents and creating Right Anchor Lists comprising said words or word strings or both to the left of said words or words strings or both on said Right Signature List based on their frequency in said returned documents; and

j. ranking results based on the frequency of each word or word string occurring on said Left Anchor Lists and the frequency of said word or word string occurring on said Right Anchor Lists.

2. The computer device of claim 1, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

3. The computer device of claim 1, wherein a result is modified by designating each of said results as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said result of said query based on the number of lists of the new queries where both the query and the result appear together.

4. The computer device of claim 1, wherein ranking a result is modified by designating said result as a new query, repeating steps a. through h. and modifying said ranking of said result of said query based on the words and word strings on the Left Signature List and/or words and word strings on the Right Signature List of the new query that do not appear on the Left Signature List and/or the Right Signature List of the query.

5. A computer device including a processor, a memory coupled to the processor, and a program stored in the memory, wherein the computer is configured to execute the program and perform the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined amount of words or word strings or both to the left of said query to be analyzed in said returned documents based on their frequency and creating a Left Signature List comprising said words or word strings or both to the left of said query to be analyzed in said returned documents;

e. searching said collection of documents for each word and word string on said Left Signature List and returning documents containing said words and word strings on said Left Signature List;

f. determining a user-defined amount of words or word strings or both to the right of said words or word strings or both on said Left Signature List in said returned documents and creating Left Anchor Lists comprising said words or word strings or both to the right of said words or word strings or both on said Left Signature List based on their frequency in said returned documents;

g. determining a user-defined number of words or word strings or both to the right of said query to be analyzed in said returned documents and creating a Right Signature List comprising said words or word strings or both to the right of said query to be analyzed in said returned documents based on their frequency;

h. searching said collection of documents for words or word strings or both on said Right Signature List and returning documents containing said words and word strings on said Right Signature List;

i. determining a user-defined number of words or word strings or both to the left of said words or word strings or both on said Right Signature List in said returned documents and creating Right Anchor Lists comprising said words or word strings or both to the left of said words or words strings or both on said Right Signature List based on their frequency in said returned documents; and

j. ranking the results by multiplying the total frequency of each word or word string occurring on said Left Anchor Lists by the total frequency of said word or word string occurring on said Right Anchor Lists.

6. The computer device of claim 5, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

7. The computer device of claim 5, wherein a result is modified by designating each of said results as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said result of said query based on the number of lists of the new queries where both the query and the result appear together.

8. The computer device of claim 5, wherein ranking a result is modified by designating said result as a new query, repeating steps a. through h. and modifying said ranking of said result of said query based on the words and word strings on the Left Signature List and/or words and word strings on the Right Signature List of the new query that do not appear on the Left Signature List and/or the Right Signature List of the query.

9. A non-transistory computer readable storage medium having stored thereon a program executable by a computer processor for performing the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined amount of words or word strings or both to the left of said query to be analyzed in said returned documents based on their frequency and creating a Left Signature List comprising said words or word strings or both to the left of said query to be analyzed in said returned documents;

e. searching said collection of documents for each word and word string on said Left Signature List and returning documents containing said words and word strings on said Left Signature List;

f. determining a user-defined amount of words or word strings or both to the right of said words or word strings or both on said Left Signature List in said returned documents and creating Left Anchor Lists comprising said words or word strings or both to the right of said words or word strings or both on said Left Signature List based on their frequency in said returned documents;

g. determining a user-defined number of words or word strings or both to the right of said query to be analyzed in said returned documents and creating a Right Signature List comprising said words or word strings or both to the right of said query to be analyzed in said returned documents based on their frequency;

h. searching said collection of documents for words or word strings or both on said Right Signature List and returning documents containing said words and word strings on said Right Signature List;

i. determining a user-defined number of words or word strings or both to the left of said words or word strings or both on said Right Signature List in said returned documents and creating Right Anchor Lists comprising said words or word strings or both to the left of said words or words strings or both on said Right Signature List based on their frequency in said returned documents; and

j. ranking results based on the frequency of each word or word string occurring on said Left Anchor Lists and the frequency of said word or word string occurring on said Right Anchor Lists.

10. The medium of claim 9, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

11. The medium of claim 9, wherein a result is modified by designating each of said results as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said result of said query based on the number of lists of the new queries where both the query and the result appear together.

12. The medium of claim 9, wherein ranking a result is modified by designating said result as a new query, repeating steps a. through h. and modifying said ranking of said result of said query based on the words and word strings on the Left Signature List and/or words and word strings on the Right Signature List of the new query that do not appear on the Left Signature List and/or the Right Signature List of the query.

13. A non-transitory computer readable storage medium having stored thereon a program executable by a computer processor for performing the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined amount of words or word strings or both to the left of said query to be analyzed in said returned documents based on their frequency and creating a Left Signature List comprising said words or word strings or both to the left of said query to be analyzed in said returned documents;

e. searching said collection of documents for each word and word string on said Left Signature List and returning documents containing said words and word strings on said Left Signature List;

f. determining a user-defined amount of words or word strings or both to the right of said words or word strings or both on said Left Signature List in said returned documents and creating Left Anchor Lists comprising said words or word strings or both to the right of said words or word strings or both on said Left Signature List based on their frequency in said returned documents;

g. determining a user-defined number of words or word strings or both to the right of said query to be analyzed in said returned documents and creating a Right Signature List comprising said words or word strings or both to the right of said query to be analyzed in said returned documents based on their frequency;

h. searching said collection of documents for each word and word string on said Right Signature List and returning documents containing said words and word strings on said Right Signature List;

i. determining a user-defined number of words or word strings or both to the left of said words or word strings or both on said Right Signature List in said returned documents and creating Right Anchor Lists comprising said words or word strings or both to the left of said words or word strings or both on said Right Signature List based on their frequency in said returned documents; and

j. ranking the results by multiplying the total frequency of each word or word string occurring on said Left Anchor Lists by the total frequency of said word or word string occurring on said Right Anchor Lists.

14. The medium of claim 13, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

15. The medium of claim 13, wherein a result is modified by designating each of said results as a new query, and repeating steps a. through j. to determine and return the results of each of the new queries, and modifying said result of said query based on the number of lists of the new queries where both the query and the result appear together.

16. The medium of claim 13, wherein ranking a result is modified by designating said result as a new query, repeating steps a. through h. and modifying said ranking of said result of said query based on the words and word strings on the Left Signature List and/or words and word strings on the Right Signature List of the new query that do not appear on the Left Signature List and/or the Right Signature List of the query.

17. A computer device including a processor, a memory coupled to the processor, and a program stored in the memory, wherein the computer is configured to execute the program and perform the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined number of words or word strings of user-defined size or both to the left and right of the query in said returned documents containing the query to be analyzed;

e. returning a list with an entry or plurality of entries, wherein said entry or said plurality of entries contain said determined words or word strings or both to the left and right of the query in said returned documents;

f. searching said collection of documents for said entry or plurality of entries in said returned list of entry or plurality of entries and returning second documents containing said entry or plurality of entries;

g. returning a list of words or word strings of user defined size or both that occur between said determined words or word strings or both in said entry or plurality of entries in said returned second documents and ranking said returned list of words or word strings by frequency of occurrence.

18. The device of claim 17, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through g. to determine and return the results of the new query, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

19. The device of claim 17, wherein ranking a result is modified by designating each of said results as a new query, and repeating steps a. through g. to determine and return the results of the new queries, and modifying said ranking of the result of said query based on the number of lists of the new queries where both the query and the result appear together.

20. The device of claim 17, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the query and/or words or word strings or both to the right of the query that do not appear to the left and/or to the right of the new query.

21. The device of claim 17, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the new query and/or words or word strings or both to the right of the new query that do not appear to the left and/or to the right of the query.

22. A computer device including a processor, a memory coupled to the processor, and a program stored in the memory, wherein the computer is configured to execute the program and perform the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined number of words or word strings of user-defined size or both to the left and right of the query in said returned documents containing the query to be analyzed;

e. returning a list with an entry or plurality of entries, wherein said entry or said plurality of entries contain said determined words or word strings or both to the left and right of the query in said returned documents;

f. searching said collection of documents for said entry or plurality of entries in said returned list of entry or plurality of entries and returning second documents containing said entry or plurality of entries;

g. returning a list of words or word strings of user defined size or both that occur between said determined words or word strings or both in said entry or plurality of entries in said returned second documents and ranking said returned list of words or word strings based on the number of unique entries on the returned list that appear to the left and right of the result in the collection of documents.

23. The device of claim 22, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through g. to determine and return the results of the new query, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

24. The device of claim 22, wherein ranking a result is modified by designating each of said results as a new query, and repeating steps a. through g. to determine and return the results of the new queries, and modifying said ranking of the result of said query based on the number of lists of the new queries where both the query and the result appear together.

25. The device of claim 22, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the query and/or words or word strings or both to the right of the query that do not appear to the left and/or to the right of the new query.

26. The device of claim 22, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the new query and/or words or word strings or both to the right of the new query that do not appear to the left and/or to the right of the query.

27. A non-transitory computer readable storage medium having stored thereon a program executable by a computer processor for performing the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined number of words or word strings of user-defined size or both to the left and right of the query in said returned documents containing the query to be analyzed;

e. returning a list with an entry or plurality of entries, wherein said entry or said plurality of entries contain said determined words or word strings or both to the left and right of the query in said returned documents;

f. searching said collection of documents for said entry or plurality of entries in said returned list of entry or plurality of entries and returning second documents containing said entry or plurality of entries;

g. returning a list of words or word strings of user defined size or both that occur between said determined words or word strings or both in said entry or plurality of entries in said returned second documents and ranking said returned list of words or word strings by frequency of occurrence.

28. The medium of claim 27, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through g. to determine and return the results of the new query, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

29. The medium of claim 27, wherein ranking a result is modified by designating each of said results as a new query, and repeating steps a. through g. to determine and return the results of the new queries, and modifying said ranking of the result of said query based on the number of lists of the new queries where both the query and the result appear together.

30. The medium of claim 27, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the query and/or words or word strings or both to the right of the query that do not appear to the left and/or to the right of the new query.

31. The medium of claim 27, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the new query and/or words or word strings or both to the right of the new query that do not appear to the left and/or to the right of the query.

32. A non-transitory computer readable storage medium having stored thereon a program executable by a computer processor for performing the steps of:

a. providing a collection of documents, wherein said collection includes at least one document;

b. receiving from a user a word or word string query to be analyzed;

c. searching said collection of documents for the query to be analyzed and returning documents containing the query to be analyzed;

d. determining a user-defined number of words or word strings of user-defined size or both to the left and right of the query in said returned documents containing the query to be analyzed;

e. returning a list with an entry or plurality of entries, wherein said entry or said plurality of entries contain said determined words or word strings or both to the left and right of the query in said returned documents;

f. searching said collection of documents for said entry or plurality of entries in said returned list of entry or plurality of entries and returning second documents containing said entry or plurality of entries;

g. returning a list of words or word strings of user defined size or both that occur between said determined words or word strings or both in said entry or plurality of entries in said returned second documents and ranking said returned list of words or word strings based on the number of unique entries on the returned list that appear to the left and right of the result in the collection of documents.

33. The medium of claim 32, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through g. to determine and return the results of the new query, and modifying said ranking of the result of said query based on the rank of the query on the results of the new query.

34. The medium of claim 32, wherein ranking a result is modified by designating each of said results as a new query, and repeating steps a. through g. to determine and return the results of the new queries, and modifying said ranking of the result of said query based on the number of lists of the new queries where both the query and the result appear together.

35. The medium of claim 32, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the query and/or words or word strings or both to the right of the query that do not appear to the left and/or to the right of the new query.

36. The medium of claim 32, wherein ranking a result is modified by designating said result as a new query, and repeating steps a. through e. to determine and return words or word strings or both to the left of the new query and to the right of the new query, and modifying said ranking of the result of said query based on the words or word strings or both to the left of the new query and/or words or word strings or both to the right of the new query that do not appear to the left and/or to the right of the query.