Lexicon with sectionalized data and method of using the same

• US 20030204392A1
• Filed: 04/30/2002
• Published: 10/30/2003
• Est. Priority Date: 04/30/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Powell, Kevin R., Finnigan, James P., Huttenhower, Curtis E., Potter, Douglas W.
• Granted Patent: US 7,490,034 B2
• Time in Patent Office: Days
• US Class Current: 704/10
• CPC Class Codes: G06F 40/242 Dictionaries

Abstract

A data structure for a word lexicon includes a plurality of separate data sections for storing information related to word entries. An indices section includes pointers indicating the location of the information. The location of the associated pointers for each word entry is obtained as a function of a list of the word entries.

Claims

1. A computer readable medium having a lexicon for storing word information and adapted for use in a language processing system, the lexicon comprising:

a word list section for storing a plurality of words;

a plurality of data sections, each data section for storing substantially different selected information about the words in the word list; and

an indices section for storing pointers, each of the pointers pointing to data in the plurality of data sections, wherein an identification of the which pointer to use is a function of the corresponding word in the word list section.

2. The computer readable medium of claim 1 wherein the word list section comprises a Trie data structure.

3. The computer readable medium of claim 2 wherein the identification is a function of nodes in the Trie data structure.

4. The computer readable medium of claim 2 wherein the identification is an offset value stored with the Trie data structure.

5. The computer readable medium of claim 1 wherein the identification is an offset value stored with the word list section.

6. The computer readable medium of claim 1 and further comprising an index table section for storing each of the identifications correlated with words of the word list section, each word in the word list section having a corresponding entry in the index table section.

7. The computer readable medium of claim 1 wherein each of the pointers includes a first indication of which data section to access and a second indication of an offset value related to the information stored therein.

8. The computer readable medium of claim 7 wherein each of the pointers includes a third indication of a classification of the word.

9. The computer readable medium of claim 8 wherein the classification comprises part of speech (POS).

10. The computer readable medium of claim 8 wherein the classification comprises one of tonal and inflection features.

11. The computer readable medium of claim 8 wherein the pointers of each of classification of each of the words are sequentially provided as a set in the indices section, and wherein each of the pointers includes a fourth indication comprising a flag indicating the last pointer of the corresponding set.

12. The computer readable medium of claim 11 and further comprising a string heap section for storing a plurality of strings, wherein a plurality of data locations in the plurality of data sections reference the same string in the string heap section.

13. The computer readable medium of claim 11 and further comprising a header section providing information as to the size and location of the word list section, the indices sections and each of the plurality of data sections.

14. The computer readable medium of claim 1 wherein one of the plurality of data sections stores information related to spell checking.

15. The computer readable medium of claim 1 wherein one of the plurality of data sections stores information related to morphology.

16. The computer readable medium of claim 1 wherein one of the plurality of data sections stores information related to linguistics.

17. The computer readable medium of claim 1 wherein one of the plurality of data sections stores information indicating a word belongs to a multiple word expression.

18. The computer readable medium of claim 1 wherein two data sections of the plurality of data sections separately store information selected from the group consisting of spell checking information, morphology information, linguistic information and multiple word expression information.

19. A computer-implemented method for obtaining word information about a word by accessing a lexicon storing word information pertaining to a plurality of words, the lexicon having a word list section storing a plurality of words, a plurality of data sections, each data section storing different selected information about the words in the word list, and an indices section storing pointers, each of the pointers pointing to data in the plurality of data sections, the method comprising:

accessing the word list section as a function of said word to ascertain a pointer identification for the indices section;

using the pointer identification to obtain a pointer of the indices section;

using the pointer to ascertain which data section of the plurality of data section has information about said word and where the information is located in the data section.

20. The computer-implemented method of claim 19 wherein the identification is an offset value stored with the word list section.

21. The computer-implemented method of claim 19 wherein the lexicon includes an index table section storing each of the identifications correlated with words of the word list section, and wherein accessing the word list section as a function of said word to ascertain a pointer identification for the indices section further includes using the word list section to find a corresponding entry in the index table section having the corresponding identification.

22. The computer-implemented method of claim 21 wherein the pointers pertaining to each word are grouped together as a function of a classification, and wherein the method further comprises identifying the classification and using the pointers of the identified classification.

23. The computer-implemented method of claim 22 wherein the classification is a function of part of speech (POS).

24. A method for storing word information in a lexicon pertaining to a plurality of words in a word list, the method comprising:

storing word information in a plurality of data sections, each data section storing substantially different selected information about the words in the word list;

storing pointer information in an indices section separate from the plurality of data sections, each of the pointers pointing to selected data in the plurality of data sections; and

storing the word list in a word list section separate from the plurality of data sections and the indices section, the word list having information for identifying the corresponding pointers related to a selected word.

25. The computer-implemented method of claim 24 wherein the information for identifying is an offset value stored with the word list section.

26. The computer-implemented method of claim 24 and further comprising:

storing identification values in an index table section separate from the plurality of data sections, each identification value corresponding to a word in the word list section and being associated with a pointer in the indices section.

27. The computer-implemented method of claim 26 wherein the pointers pertaining to each word are grouped together as a function of a classification, and wherein storing pointer information includes assigning an indication of the classification in each of the pointers.

28. The computer-implemented method of claim 27 wherein the classification is a function of part of speech (POS).

29. A computer-implemented method for obtaining word information from a plurality of lexicons, each lexicon having a word list section storing a plurality of words, a plurality of data sections, each data section storing different selected information about the words in the word list, and an indices section storing pointers, each of the pointers pointing to data in the plurality of data sections, the method comprising:

selectively accessing data sections of each of the lexicons having similar information and obtaining information therefrom.

30. The computer-implemented method of claim 29 wherein selectively accessing includes combining the word information from at least two data sections of each of the lexicons having similar information.

31. The computer-implemented method of claim 29 wherein selectively accessing includes obtaining the word information from a data section of a first lexicon, obtaining the word information from a data section of a second lexicon and then using only the information from the data section of the second lexicon.

32. The computer-implemented method of claim 29 wherein selectively accessing includes obtaining the word information from similar data sections of each lexicon until a stop indicator is located.

33. The computer-implemented method of claim 29 selectively accessing the data sections includes sequentially accessing the lexicons in a selected order.

34. The computer-implemented method of claim 29 wherein selectively accessing the data sections comprises:

for each lexicon that is accessed,

accessing the corresponding word list section as a function of a given word to ascertain a pointer identification for the corresponding indices section;

using the pointer identification to obtain a pointer of the corresponding indices section; and

using the pointer to ascertain which corresponding data section of the plurality of data sections has information about said word and where the information is located in the corresponding data section.

35. The computer-implemented method of claim 34 selectively accessing the data sections includes sequentially accessing the lexicons in a selected order.