Language context dependent data labeling

• US 7,295,979 B2
• Filed: 02/22/2001
• Issued: 11/13/2007
• Est. Priority Date: 09/29/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Rajput, Nitendra, Verma, Ashish, Neti, Chalapathy Venkata, Subramaniam, L. Venkata
• Primary Examiner(s): Edouard; Patrick N.
• Assistant Examiner(s): Wozniak; James S.
• Application Number:
  US09/790,296
• Publication Number:
  US 20020152068A1
• Time in Patent Office: 2,455 Days
• Field of Search: 704/1, 704/2, 704/8, 704/9, 704256-257, 704/245, 704243-244, 704254-255, 704/277
• US Class Current: 704/243
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

Bootstrapping of a system from one language to another often works well when the two languages share the similar acoustic space. However, when the new language has sounds that do not occur in the language from which the bootstrapping is to be done, bootstrapping does not produce good initial models and the new language data is not properly aligned to these models. The present invention provides techniques to generate context dependent labeling of the new language data using the recognition system of another language. Then, this labeled data is used to generate models for the new language phones.

Claims

1. A method of aligning continuous speech data of a new language to a phone set associated with the new language using a speech recognition system trained in accordance with a base language, the method comprising the steps of:

applying a mapping to a new language phonetic vocabulary to generate a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language;

aligning continuous speech data, input in the new language, to word lexemes in the generated base language phonetic vocabulary using the speech recognition system trained in accordance with the base language; and

realigning the aligned speech data to the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base language vocabularies.

2. The method of claim 1, wherein the mapping applied to the new language phonetic vocabulary is a many-to-one mapping.

3. The method of claim 1, wherein the aligning step comprises labeling feature vectors, generated from the input speech data, by phones that the feature vectors represent in the phonetic space of the base language phone set.

4. The method of claim 3, wherein the realigning step comprises relabeling the feature vectors by clustering the feature vectors according to phones of the new language phone set.

5. The method of claim 1, wherein the speech recognition system trained in accordance with the base language is a large vocabulary continuous speech recognition system.

6. A method of labeling continuous speech data of a new language with a phone set associated with the new language using a speech recognition system trained in accordance with a base language, the method comprising the steps of:

using the speech recognition system trained in accordance with the base language to label the continuous speech data uttered in the new language using word lexemes in a base language phonetic vocabulary, wherein a new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language; and

relabeling the labeled speech data using the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base languages.

7. A method of generating a speech recognition system for a new language using a speech recognition system previously generated for a base language, the method comprising the steps of:

applying a mapping to a new language phonetic vocabulary to generate a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using a phone set associated with the new language and wherein the base language phonetic vocabulary represents a new language phonetic vocabulary mapped to a phone set associated with the base language;

aligning continuous training speech data, input in the new language, to word lexemes in the generated base language phonetic vocabulary using the speech recognition system previously generated for the base language;

realigning the aligned continuous training speech data to the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base language vocabularies;

constructing acoustic models using the realigned training speech data; and

associating the constructed acoustic models with a speech recognition engine for subsequent use in recognizing real-time data input speech data uttered in the new language.

8. Apparatus for aligning continuous speech data of a new language to a phone set associated with the new language using a speech recognizer trained in accordance with a base language, the apparatus comprising:

at least one processor operative to;

(i) apply a mapping to a new language phonetic vocabulary to generate a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language;

(ii) align continuous speech data, input in the new language, to word lexemes in the generated base language phonetic vocabulary using the speech recognizer trained in accordance with the base language; and

(iii) realign the aligned continuous speech data to the new language phone set by sequentially comparing phonetic spelling of word lexemes in the new and base language vocabularies ; and

memory, coupled to the at least one processor, operative to store at least one of results associated with the mapping, aligning and realigning operations.

9. The apparatus of claim 8, wherein the mapping applied to the new language phonetic vocabulary is a many-to-one mapping.

10. The apparatus of claim 8, wherein the aligning operation comprises labeling feature vectors, generated from the input speech data, by phones that the feature vectors represent in the phonetic space of the base language phone set.

11. The apparatus of claim 10, wherein the realigning operation comprises relabeling the feature vectors by clustering the feature vectors according to phones of the new language phone set.

12. The apparatus of claim 8, wherein the speech recognizer trained in accordance with the base language is a large vocabulary continuous speech recognizer.

13. Apparatus for labeling continuous speech data of a new language with a phone set associated with the new language using a speech recognizer trained in accordance with a base language, the apparatus comprising:

at least one processor operative to;

(i) use the speech recognizer trained in accordance with the base language to label the continuous speech data uttered in the new language using word lexemes in a base language phonetic vocabulary, wherein a new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language; and

(iii) relabel the labeled continuous speech using the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base languages; and

memory, coupled to the at least one processor, operative to store at least one of results associated with the obtaining, labeling and relabeling operations.

14. Apparatus for generating a speech recognizer for a new language using a speech recognizer previously generated for a base language, the apparatus comprising:

at least one processor operative to;

(i) apply a mapping to a new language phonetic vocabulary to generate a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using a phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language;

(ii) align continuous training speech data, input in the new language to word lexemes in the generated base language phonetic vocabulary using the speech recognizer previously generated for the base language;

(iii) realign the aligned continuous training speech data to the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base language vocabularies;

(iv) construct acoustic models using the realigned continuous training speech data; and

(v) associate the constructed acoustic models with a speech recognition engine for subsequent use in recognizing real-time speech data uttered in the new language; and

memory, coupled to the at least one processor, operative to store at least one of results associated with the applying, aligning, realigning, constructing and associating operations.

15. A continuous speech data alignment system, comprising:

a mapping module which applies a new language-to-a base language mapping to a new language phonetic vocabulary to generate a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using a phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language;

a speech recognizer trained in accordance with the base language, coupled to the mapping module, which aligns continuous speech data, input in the new language to word lexemes in the generated base language phonetic vocabulary; and

a lexeme context comparator, coupled to the speech recognizer, which realigns the aligned continuous speech data to the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base languages.

16. The system of claim 15, wherein the speech recognizer trained in accordance with the base language is a large vocabulary continuous speech recognizer.

17. An article of manufacture for aligning continuous speech data of a new language to a phone set associated with the new language using a speech recognition system trained in accordance with a base language, comprising a computer readable medium containing one or more programs which when executed by a computer implement the steps of:

applying a mapping to a new language phonetic vocabulary to generate a new base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language;

aligning continuous speech data, input in the new language to word lexemes in the generated base language phonetic vocabulary using the speech recognition system trained in accordance with the base language; and

realigning the aligned continuous speech data to the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base language vocabularies.

18. An article of manufacture for labeling continuous speech data of a new language with a phone set associated with the new language using a speech recognition system trained in accordance with a base language, comprising a computer readable medium containing one or more programs which when executed by a computer implement the steps of:

using the speech recognition system trained in accordance with the base language to label the continuous speech data uttered in the new language using word lexemes in a base language phonetic vocabulary, wherein the new language phonetic vocabulary comprises new language words built using the phone set associated with the new language and wherein the base language phonetic vocabulary represents the new language phonetic vocabulary mapped to a phone set associated with the base language; and

relabeling the labeled continuous speech data using the new language phone set by sequentially comparing phonetic spellings of word lexemes in the new and base languages.