System for Compiling Word Usage Frequencies

• US 20090063483A1
• Filed: 11/13/2008
• Published: 03/05/2009
• Est. Priority Date: 01/13/2005
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Yen-Fu, Dunsmoir, John W.
• Granted Patent: US 8,346,533 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A system for assisting a user who is learning a language to prioritize words to be learned in order of usage frequency is disclosed. A frequency determination program running on a computer determines the frequency of usage of each word at a list of locations provided by the user. Different algorithms to identify what constitutes a word are employed depending upon the language of the source data. The total number of words at each location and their usage frequency found during the user session, along with a total number of words and their usage frequency for all user sessions performed regardless of location, are calculated and made available to the user. The user can view usage frequencies for words from a single location, a group of locations, or all user sessions performed.

Claims

1. A method for compiling word usage frequencies comprising the steps of:

(a) obtaining a list of locations having words to be counted;

(b) counting a number of times each of said words is present at each of said locations in said list;

(c) calculating a total number of words present at each of said locations in said list; and

(d) displaying a total number of times a word is present at each of said locations in said list and a total number of words present at each of said locations in said list.

2. The method of claim 1, further comprising the steps of:

calculating a location word frequency ratio by dividing the total number of words present at each of said locations in said list by the number of times each word is present at each of said locations in said list; and

displaying said location word frequency ratio for each of said words present at each of said locations in said list.

3. The method of claim 1, further comprising the steps of:

calculating a location word frequency percentage by dividing said number of times each word is present at each of said locations in said list by said total number of words present at each of said locations in said list and multiplying by 100; and

displaying said location word frequency percentage for each of said words present at each of said locations in said list.

4. The method of claim 1, further comprising the steps of:

calculating an overall word frequency ratio by adding said total number of words present at all of said locations in said list, adding said total number of times each word is present at any of said locations in said list, and dividing said total number of words present at all of said locations in said list by said total number of times each word is present at any of said locations in said list; and

displaying said overall word frequency ratio for each of said words present at any of said locations in said list.

5. The method of claim 1, further comprising the steps of:

calculating an overall word frequency percentage by adding said total number of words present at all of said locations in said list, adding said total number of times each word is present at any of said locations in said list, dividing said total number of times each word is present at any of said locations in said list by said total number of words present at all of said locations in said list, and multiplying by 100; and

displaying said overall word frequency percentage for each of said words present at any of said locations in said list.

6. The method of claim 1, wherein said locations are selected from the group consisting of documents, electronic mail, and websites having hyperlinks.

7. The method of claim 6, wherein the depth to which said method traverses said hyperlinks to count said number of times each of said words is present at said websites is user configurable.

8. The method of claim 1, wherein said total number of words present at said location in said list must equal or exceed a user-configurable quantity for step (d) to be performed for said location.

9. The method of claim 1, wherein prior to performing step (b), said method determines if said words at said locations are graphic based or alphabet based.

10. The method of claim 9, wherein said determination if said words at said locations are graphic based or alphabet based uses Unicode encoding of said words at said locations.

11. The method of claim 9, wherein step (b) comprises the steps of:

(a) adding a first character to a character list;

(b) searching a list of punctuation marks and spaces for a match to said character list;

(c) responsive to finding a match in step (b), clearing the character list;

(d) responsive to finding a match in step (b), determining if there is a next character;

(e) responsive to finding a next character in step (d), adding said next character to said character list;

(f) responsive to finding a next character in step (d), repeating steps (b)-(f);

(g) responsive to determining said words at said location are graphic based and responsive to not finding a match in step (b), searching a graphic dictionary for a match to said character list;

(h) responsive to determining said words at said location are graphic based and responsive to finding a match in steps (g) or (i), determining if there is a next character;

(i) responsive to determining said words at said location are graphic based and responsive to finding a next character in step (h), searching said graphic dictionary for a match to said character list and said next character;

(j) responsive to determining said words at said location are graphic based and responsive to finding a match in step (i), adding said next character to said character list;

(k) responsive to determining said words at said location are graphic based and responsive to finding a next character is present in step (h), repeating steps (h)-(k);

(l) responsive to determining said words at said location are graphic based and responsive to not finding a match in steps (g) or (i) or a next character in step (h), adding said character list to a location word list;

(m) responsive to determining said words at said location are alphabet based, determining if there is a next character;

(n) responsive to determining said words at said location are alphabet based and responsive to finding a next character in step (m), searching said list of punctuation marks and spaces for said next character;

(o) responsive to determining said words at said location are alphabet based and responsive to not finding a match in step (n), repeating steps (m)-(o);

(p) responsive to determining said words at said location are alphabet based and responsive to not finding a next character in step (m) or finding a match in step (n), adding said character list to said location word list;

(q) incrementing a counter in said location word list;

(r) clearing said character list;

(s) determining if there is a next character;

(t) responsive to finding a next character in step (s), adding said next character to said character list;

(u) responsive to finding a next character in step (s), searching said list of punctuation marks and spaces for a match to said character list;

(v) responsive to finding a match in step (u), repeating steps (r)-(v);

(w) responsive to determining said words at said location are graphic based and responsive to not finding a match in step (u), repeating steps (g)-(w);

(x) responsive to determining said words at said location are alphabet based and responsive to not finding a match in step (u), repeating steps (m)-(x); and

(y) adding said counters associated with said location word list together to calculate said total number of times a word is present at said location in said list.

12. The method of claim 1, wherein said method is operable in an apparatus comprising a processor and memory.

13. The method of claim 12, wherein said apparatus further comprises a network connection to a web server containing a website.

14. The method of claim 12 wherein results are displayed on a user-interface.

15. The method of claim 12 wherein locations to be analyzed are selected on a user-interface.

16-25. -25. (canceled)

26. A method for compiling word usage frequencies comprising the steps of:

(a) obtaining a list of locations having words to be counted;

(b) obtaining a minimum number of words that must be present at each location;

(c) obtaining a depth of hyperlinks to be traversed for each location;

(d) calculating a total number of words present at each of said locations in said list;

(e) responsive to finding a location has at least said minimum number of words, counting a number of times each of said words is present at that location; and

(f) displaying a total number of times a word is present at each of said locations in said list having said minimum number of words and a total number of words present at each of said locations having said minimum number of words in said list.

27. The method of claim 26, further comprising the steps of:

calculating a location word frequency ratio by dividing the total number of words present at each of said locations in said list by the number of times each word is present at each of said locations in said list;

displaying said location word frequency ratio for each of said words present at each of said locations in said list;

calculating a location word frequency percentage by dividing said number of times each word is present at each of said locations in said list by said total number of words present at each of said locations in said list and multiplying by 100;

displaying said location word frequency percentage for each of said words present at each of said locations in said list;

calculating an overall word frequency ratio by adding said total number of words present at all of said locations in said list, adding said total number of times each word is present at any of said locations in said list, and dividing said total number of words present at all of said locations in said list by said total number of times each word is present at any of said locations in said list;

displaying said overall word frequency ratio for each of said words present at any of said locations in said list;

calculating an overall word frequency percentage by adding said total number of words present at all of said locations in said list, adding said total number of times each word is present at any of said locations in said list, dividing said total number of times each word is present any of said locations in said list by said total number of words present at all of said locations in said list, and multiplying by 100; and

displaying said overall word frequency percentage for each of said words present at any of said locations in said list.

28. The method of claim 26, wherein prior to performing step (d), said method determines if said words at said locations are graphic based or alphabet based.

29. The method of claim 26, wherein said determination if said words at said locations are graphic based or alphabet based uses Unicode encoding of said words at said locations.

30. The method of claim 26, wherein step (d) comprises the steps of:

(a) adding a first character to a character list;

(b) searching a list of punctuation marks and spaces for a match to said character list;

(c) responsive to finding a match in step (b), clearing the character list;

(d) responsive to finding a match in step (b), determining if there is a next character;

(e) responsive to finding a next character in step (d), adding said next character to said character list;

(f) responsive to finding a next character in step (d), repeating steps (b)-(f);

(g) responsive to determining said words at said location are graphic based and responsive to not finding a match in step (b), searching a graphic dictionary for a match to said character list;

(h) responsive to determining said words at said location are graphic based and responsive to finding a match in steps (g) or (i), determining if there is a next character;

(i) responsive to determining said words at said location are graphic based and responsive to finding a next character in step (h), searching said graphic dictionary for a match to said character list and said next character;

(j) responsive to determining said words at said location are graphic based and responsive to finding a match in step (i), adding said next character to said character list;

(k) responsive to determining said words at said location are graphic based and responsive to finding a next character is present in step (h), repeating steps (h)-(k);

(l) responsive to determining said words at said location are graphic based and responsive to not finding a match in steps (g) or (i) or a next character in step (h), adding said character list to a location word list;

(m) responsive to determining said words at said location are alphabet based, determining if there is a next character;

(n) responsive to determining said words at said location are alphabet based and responsive to finding a next character in step (m), searching said list of punctuation marks and spaces for said next character;

(o) responsive to determining said words at said location are alphabet based and responsive to not finding a match in step (n), repeating steps (m)-(o);

(p) responsive to determining said words at said location are alphabet based and responsive to not finding a next character in step (m) or finding a match in step (n), adding said character list to said location word list;

(q) incrementing a counter in said location word list;

(r) clearing said character list;

(s) determining if there is a next character;

(t) responsive to finding a next character in step (s), adding said next character to said character list;

(u) responsive to finding a next character in step (s), searching said list of punctuation marks and spaces for a match to said character list;

(v) responsive to finding a match in step (u), repeating steps (r)-(v);

(w) responsive to determining said words at said location are graphic based and responsive to not finding a match in step (u), repeating steps (g)-(w);

(x) responsive to determining said words at said location are alphabet based and responsive to not finding a match in step (u), repeating steps (m)-(x); and

(y) adding said counters associated with said location word list together to calculate said total number of words present at said location in said list.

31. The method of claim 26, wherein said method is operable in an apparatus comprising a processor, memory, and a network connection to a web server hosting a website.

32. The method of claim 31 wherein a location and a depth are entered into an interface.