Method for mining, mapping and managing organizational knowledge from text and conversation

• US 7,165,023 B2
• Filed: 12/17/2001
• Issued: 01/16/2007
• Est. Priority Date: 12/15/2000
• Status: Expired due to Fees

• Current Assignee: Arizona Board of Regents (University of Arizona)
• Original Assignee: Arizona Board of Regents (University of Arizona)
• Inventors: Corman, Steven R., Dooley, Kevin J.
• Primary Examiner(s): Hudspeth, David
• Assistant Examiner(s): Wozniak, James S.
• Application Number:
  US10/023,556
• Publication Number:
  US 20020116170A1
• Time in Patent Office: 1,856 Days
• Field of Search: 704/1, 704/9, 704/10, 707/102
• US Class Current: 704/9
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars

Abstract

A network text analysis called center resonance analysis (CRA) is presented which represents texts (or transcribed conversations) as networks of centering words. The networks of centering words are components of utterances or (specifically noun phrases) that authors and/or speakers deploy in a manner that makes their utterances coherent. A CRA network can be derived for any text, and abstractly represents its main concepts, their influence, and their interrelationships.

Claims

1. A method for categorizing text, comprising the steps of:

dividing the text into sentences;

parsing the sentences into one or more noun phrases;

converting the noun phrases into networks of word relationships by linking sequentially occurring noun phrases within each sentence; and

analyzing the networks of word relationships to determine the influence of each word by utilizing betweenness centrality, wherein each centering noun phrase in the networks of word relationships is centrally related with respect to peripheral words to the centering noun phrase while the peripheral words have no relationship between one another so that any association between the peripheral words must pass through the centering noun phrase.

2. The method of claim 1 wherein the step of parsing the sentences into one or more noun phrases comprises the step of substituting disambiguated nouns in place of pronouns which are related to text analysis.

3. The method of claim 1 wherein the step of parsing the sentences into one or more noun phrases further comprises the step of converting plural words to their singular form.

4. The method of claim 1 wherein the step of converting the words into networks of word relationships further comprises the step of linking all possible pairs of words in the noun phrases having three or more words.

5. The method of claim 1 where the step of applying the analyzed network to perform a specific analysis task comprises the step of applying the analyzed network to perform at least one of visualization of the network to understand text, spatial modeling of resonance scores, information retrieval, and thematic analysis of collections.

6. A method for categorizing text comprising the steps of:

dividing the text into sentences;

parsing the sentences into one or more noun phrases;

converting words in the noun phrases into networks of word relationships; and

analyzing the word relationship networks to determine the influence of each word by determining influence by utilizing the following formula;

$I=\frac{g_{\mathrm{jk}}<br><br>\left(i\right)/g_{\mathrm{jk}}}{\left[\left(N-1\right)<br><br>\left(N-2\right)/2\right]}$

where I is the influence of a word (i) in the text (T) where g_jk is the number of shortest paths connecting the j^th and k^th words, g_jk (i) is the number of those paths containing word (i), and N is the number of words in the network.

7. A method for analyzing text comprising the steps of:

dividing the text into sentences;

parsing the sentences into one or more noun phrases;

converting one or more words within each of the noun phrases into networks of relationships between words;

analyzing the networks to determine the influence for each word by utilizing betweenness centrality, wherein each centering noun phrase in the networks of word relationships is centrally related with respect to peripheral words to the centering noun phrase while the peripheral words have no relationship between one another so that any association between the peripheral words must pass through the centering noun phrase; and

applying the analyzed networks to perform a specific analysis task.

8. The method of claim 7 wherein the step of parsing the sentences into one or more noun phrases further comprises the step of substituting disambiguated nouns in place of pronouns which are relevant to text analysis.

9. The method of claim 7 wherein the step of parsing the sentences into one or more noun phrases further comprises the step of converting plural words to their singular form.

10. The method of claim 7 wherein the step of converting the word or words into networks of relationships comprises the step of linking all sequentially occurring noun phrases within a sentence.

11. The method of claim 10 wherein the step of converting the words into networks of relationships between words further comprises the step of linking all possible pairs of words in the noun phrases having three or more words.

12. The method of claim 7 wherein the step of analyzing the networks to determine the influence of each word comprises the step of determining influence by utilizing the following formula:

$I=\frac{g_{\mathrm{jk}}<br><br>\left(i\right)/g_{\mathrm{jk}}}{\left[\left(N-1\right)<br><br>\left(N-2\right)/2\right]}$

where I is the influence of a word (i) in the text (T) where g_jk is the number of shortest paths connecting the j^th and k^th words, g_jk (i) is the number of those paths containing word (i), and N is the number of words in the network.

13. The method of claim 7 where the step of applying the analyzed network to perform a specific analysis task comprises the step of applying the analyzed network to perform at least one of visualization of the network to understand text, spatial modeling of resonance scores, information retrieval, and thematic analysis of collections.

14. A method for determining resonance based on common words in two sets of text comprising the step of utilizing the following formula:

${\mathrm{WR}}_{\mathrm{AB}}=\underset{i=1}{\overset{N<br><br>\left(A\right)}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{N<br><br>\left(B\right)}{\sum <br><br>}}<br><br>I_i^A·<br><br>I_j^B·<br><br>{\mathrm{\alpha <br><br>}}_{\mathrm{ij}}^{\mathrm{AB}}$

where WR_AB is the word resonance between texts A and B, {w₁^A, w₂^A, . . . w_N(A)^A) are unique words for text A after parsing into phrases where N(A) is the number of unique words in text A, {I₁^A, I₂^A, . . . I_N(A)^A} are influence scores calculated for the unique words in text A, {w₁^B, w₂^B, . . . w_N(B)^B) are unique words for text B after parsing into phrases where N(B) is the number of unique words in text B, {I₁^B, I₂^B, . . . I_N(B)^B} are influence scores calculated for the unique words in text B, and indicator function α

^AB_ij is equal to 1 if w_i^A and w_j^B are the same words, and the indicator function is equal to zero if w_i^A and w_i^B are not the same words.

15. The method of claim 14 further comprising the step of determining standardized resonance based on common words in texts A and B comprising the step of utilizing the following formula:

${\mathrm{WR}}_{\mathrm{AB}}^{\mathrm{\prime <br><br>}}={\mathrm{WR}}_{\mathrm{AB}}/\sqrt{\underset{i=1}{\overset{N<br><br>\left(A\right)}{\sum <br><br>}}<br><br>{\left(I_i^A\right)}^2·<br><br>\underset{j=1}{\overset{N<br><br>\left(B\right)}{\sum <br><br>}}<br><br>{\left(I_j^B\right)}^2}$

where WR_AB′

is the standardized word resonance between texts A and B, WR_AB is the actual word resonance between texts A and B,

$\underset{i=1}{\overset{N<br><br>\left(A\right)}{\sum <br><br>}}<br><br><br><br>{\left(I_i^A\right)}^2$

is the sum of all influence scores for the unique words in text A squared, and

$\underset{j=1}{\overset{N<br><br>\left(B\right)}{\sum <br><br>}}<br><br><br><br>{\left(I_j^B\right)}^2$

is the sum of all influence scores for the unique words in text B squared.

16. A method for searching two or more texts utilizing resonance scores obtained in accordance with claim 14.

17. A method for searching two or more texts utilizing resonance scores obtained in accordance with claim 15.

18. A method for modeling two or more texts utilizing resonance scores obtained in accordance with claim 14.

19. A method for modeling two or more texts utilizing resonance scores obtained in accordance with claim 15.

20. A method for determining pair resonance based on common word pairs in two sets of text comprising the step of utilizing the following formula:

${\mathrm{PR}}_{\mathrm{AB}}=\underset{i=1}{\overset{N<br><br>\left(A\right)-1}{\sum <br><br>}}<br><br>\left(\underset{j=i+1}{\overset{N<br><br>\left(A\right)}{\sum <br><br>}}<br><br>\left(\underset{k=1}{\overset{N<br><br>\left(B\right)-1}{\sum <br><br>}}<br><br>\left[\underset{l=k+1}{\overset{N<br><br>\left(B\right)}{\sum <br><br>}}<br><br>P_{\mathrm{ij}}^A·<br><br>P_{\mathrm{kl}}^B·<br><br>{\mathrm{\beta <br><br>}}_{\mathrm{ijkl}}^{\mathrm{AB}}\right]\right)\right)$

where PR_AB is the word pair resonance between texts A and B, P_ij^A is the frequency weighted pair influence of words i and j in text A and is equal to I_i^A·

I_j^A·

f_ij^A where F_ij^A is the number of times that w_i^A and w_j^A co-occur in text A, P_ij^B is the frequency weighted pair influence of words k and 1 in text B and is equal to I_k^B·

I_l^B·

F_kl^B where F_kl^B is the number of times that w_k^B and w_l^B co-occur in text B, and indicator function β

_ijkl^AB is equal to 1 if the two word sets (w_i^A, w_j^A) and (w_k^B, w_l^B) are equivalent and if F_ij^A and F_kl^B both are equal to one, otherwise the indicator is zero.

21. The method of claim 20 further comprising the step of determining standardized resonance based on common word pairs in texts A and B comprising the step of utilizing the following formula:

${\mathrm{PR}}_{\mathrm{AB}}^{\mathrm{\prime <br><br>}}={\mathrm{PR}}_{\mathrm{AB}}/\sqrt{\left(\underset{i=1}{\overset{N<br><br>\left(A\right)-1}{\sum <br><br>}}<br><br>\underset{j=i+1}{\overset{N<br><br>\left(A\right)}{\sum <br><br>}}<br><br>{\left(P_{\mathrm{ij}}^A\right)}^2\right)}·<br><br>\sqrt{\left(\underset{k=1}{\overset{N<br><br>\left(B\right)-1}{\sum <br><br>}}<br><br>\underset{l=k+1}{\overset{N<br><br>\left(B\right)}{\sum <br><br>}}<br><br>{\left(P_{\mathrm{kl}}^B\right)}^2\right)}$

where PR′

_AB is the standardized word pair resonance between texts A and B and PR_AB is the actual word pair resonance between texts A and B.

22. A method for searching two or more texts utilizing resonance scores obtained in accordance with claim 20.

23. A method for searching two or more texts utilizing resonance scores obtained in accordance with claim 21.

24. A method for modeling two or more texts utilizing resonance scores obtained in accordance with claim 20.

25. A method for modeling two or more texts utilizing resonance scores obtained in accordance with claim 21.

26. A method for analyzing text comprising the steps of:

a) compartmentalizing the text into defined units;

b) categorizing the defined units by;

parsing the units into one or more noun phrases each comprising one or more words,

converting the word or words into networks of relationships between words by linking sequentially occurring noun phrases within a defined unit, and

analyzing the networks of word associations to determine the structural influence of each word by utilizing betweenness centrality, wherein each centering noun phrase in the networks of word relationships is centrally related with respect to peripheral words to the centering noun phrase while the peripheral words have no relationship between one another so that any association between the peripheral words must pass through the centering noun phrase; and

c) applying the analyzed network to perform a specific analysis task.

27. The method of claim 26 wherein the step of compartmentalizing the text into defined units comprises the step of breaking down the text into sentences.

28. The method of claim 26 wherein the step of parsing the units into one or more noun phrases further comprises the step of substituting disambiguated nouns in place of pronouns which are relevant to text analysis.

29. The method of claim 26 wherein the step of parsing the units into one or more noun phrases further comprises the step of converting plural words to their singular form.

30. The method of claim 26 wherein the step of converting the word or words into networks of relationships further comprises the step of linking all possible pairs of words in those noun phrases having three or more words.

31. The method of claim 26 wherein the step of analyzing the network to determine the structural influence of each word comprises the step of determining structural influence by utilizing the following formula:

$I=\frac{g_{\mathrm{jk}}<br><br>\left(i\right)/g_{\mathrm{jk}}}{\left[\left(N-1\right)<br><br>\left(N-2\right)/2\right]}$

where I is the influence of a word (i) in the text (T) where g_jk is the number of shortest paths connecting the j^th and k^th words, g_jk (i) is the number of those paths containing word (i), and N is the number of words in the network.