Method and apparatus to provide a hierarchical index for a language model data structure

• US 20050055199A1
• Filed: 10/19/2001
• Published: 03/10/2005
• Est. Priority Date: 10/19/2001
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ryzchachkin, Ivan, Kibkalo, Alexander
• Application Number:
  US10/492,857
• Publication Number:
  US 20050055199A1
• Time in Patent Office: Days
• US Class Current: 704/4
• CPC Class Codes: G06F 16/322 Trees
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

A method for storing bigram word indexes of a language model for a consecutive speech recognition system (200) is described. The bigram word indexes (321) are stored as a common two-byte base with a specific one-byte offset to significantly reduce storage requirements of the language model data file. In one embodiment the storage space required for storing the bigram word indexes (321) sequentially is compared to the storage space required to store the bigram word indexes as a common base with specific offset. The bigram word indexes (321) are then stored so as to minimize the size of the language model data file.

Claims

1. A method for storing a plurality of bigram word indexes corresponding to a specified unigram as a common base with a specific offset characterized in that the bigram word indexes are part of a trigram language model of a consecutive speech recognition system wherein language model models the Wall Street Journal task.

2. The method of claim 1 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

3. A method for storing a plurality of bigram word indexes, each bigram word index corresponding to a specified unigram as a common base with a specific offset, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Wall Street Journal task, the method comprising:

determining storage space required for sequential storage of the plurality of bigram word indexes corresponding to a specified unigram;

determining storage space required for hierarchical data structure storage of the plurality of bigram word indexes; and

implementing hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

4. The method of claim 3 wherein the hierarchical data structure storage of the plurality of bigram word indexes includes storing each bigram word index as a common base with a specific offset.

5. The method of claim 4 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

6. A machine-readable medium that provides executable instructions which, when executed by a processor, cause the processor to perform a method for storing a plurality of bigram word indexes, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Wall Street Journal task, the method comprising:

determining storage space required for sequential storage of the plurality of bigram word indexes corresponding to a specified unigram;

determining storage space required for hierarchical data structure storage of the plurality of bigram word indexes; and

implementing hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

7. The machine-readable medium of claim 6 wherein the hierarchical data structure storage of the bigram word indexes includes storing each bigram word index as a common base with a specific offset.

8. The method of claim 7 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

9. An apparatus comprising a processor with a memory coupled thereto, characterized in that

the memory has stored therein instructions which, when executed by the processor, cause the processor to (a) determine storage space required for sequential storage of a plurality of bigram word indexes, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Wall Street Journal task (b) determine storage space required for hierarchical data structure storage of the plurality of bigram word indexes, and (c) implement hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

10. The apparatus of claim 9 wherein the hierarchical data structure storage of the bigram word indexes includes storing the bigram word indexes corresponding to a specified unigram as a common base with a specific offset.

11. The apparatus of claim 10 wherein the bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

12. A method for storing a plurality of bigram word indexes corresponding to a specified unigram as a common base with a specific offset characterized in that the bigram word indexes are part of a trigram language model of a consecutive speech recognition system wherein language model models the Chinese Task 863.

13. The method of claim 12 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

14. A method for storing a plurality of bigram word indexes, each bigram word index corresponding to a specified unigram as a common base with a specific offset, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Chinese Task 863, the method comprising:

determining storage space required for sequential storage of the plurality of bigram word indexes corresponding to a specified unigram;

determining storage space required for hierarchical data structure storage of the plurality of bigram word indexes; and

implementing hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

15. The method of claim 14 wherein the hierarchical data structure storage of the plurality of bigram word indexes includes storing each bigram word index as a common base with a specific offset.

16. The method of claim 15 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

17. A machine-readable medium that provides executable instructions which, when executed by a processor, cause the processor to perform a method for storing a plurality of bigram word indexes, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Chinese Task 863, the method comprising:

determining storage space required for sequential storage of the plurality of bigram word indexes corresponding to a specified unigram;

determining storage space required for hierarchical data structure storage of the plurality of bigram word indexes; and

implementing hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

18. The machine-readable medium of claim 17 wherein the hierarchical data structure storage of the bigram word indexes includes storing each bigram word index as a common base with a specific offset.

19. The method of claim 18 wherein each bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.

20. An apparatus comprising a processor with a memory coupled thereto, characterized in that

the memory has stored therein instructions which, when executed by the processor, cause the processor to (a) determine storage space required for sequential storage of a plurality of bigram word indexes, the bigram word indexes part of a trigram language model of a consecutive speech recognition system wherein language model models the Chinese Task 863 (b) determine storage space required for hierarchical data structure storage of the plurality of bigram word indexes, and (c) implement hierarchical data structure storage of the plurality of bigram word indexes if the storage space required for hierarchical data structure storage of the plurality of bigram word indexes is less than the storage space required for sequential storage of the plurality of bigram word indexes.

21. The apparatus of claim 20 wherein the hierarchical data structure storage of the bigram word indexes includes storing the bigram word indexes corresponding to a specified unigram as a common base with a specific offset.

22. The apparatus of claim 21 wherein the bigram word index has a length of three bytes, the common base has a length of two bytes, and the specific offset has a length of one byte.