Symbol insertion apparatus and method

• US 6,778,958 B1
• Filed: 08/30/2000
• Issued: 08/17/2004
• Est. Priority Date: 08/30/1999
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mori, Shinsuke, Nishimura, Masafumi, Itoh, Nobuyasu
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Lerner, Martin
• Application Number:
  US09/651,679
• Time in Patent Office: 1,448 Days
• Field of Search: 704/235, 704/240, 704/251, 704/243, 704/244, 704/255, 704/256, 704/257
• US Class Current: 704/235
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

An apparatus and method are provided for the insertion of punctuation marks into appropriate positions in a sentence. An acoustic processor processes input utterances to extract voice data, and transforms the data into a feature vector. When the automatic insertion of punctuation marks is not performed, a language decoder processes the feature vector using only a general-purpose language model, and inserts a comma at a location marked in the voice data by the entry “ten,” for example, which is clearly a location at which a comma should be inserted. When automatic punctuation insertion is performed, the language decoder employs the general-purpose language model and the punctuation mark language model to identify an unvoiced, pause location for the insertion of a punctuation mark, such as a comma.

Claims

1. A speech recognition apparatus comprising:

a transformer for transforming sequences of phonemes extracted from an utterance into one or more word sequences, and for assigning to said word sequences appearance probabilities, in accordance with which said word sequences are originally represented by said phoneme sequences;

a renewer for renewing said appearance probability assigned to each of said word sequences by employing a renewal value represented by a language model corresponding to each of said word sequences; and

a speech recognizer for selecting the word sequence having the highest appearance probability, by screening all said word sequences assigned appearance probabilities, according to which said word sequences are originally represented by said phoneme sequences,

wherein said renewer calculates said renewal value using a first language model which is employed when each of said word sequences always includes a specific symbol as a word and a second language model which is employed in other situations to renew said appearance probabilities based on said renewal value.

2. The speech recognition apparatus according to claim 1, wherein said first language model is demonstratively reliable, in that when each of said word sequences includes a specific symbol, such as a word, each of said word sequences was originally represented by one of said phoneme sequences;

wherein said second language model is also demonstratively reliable, in that when other situations are encountered, each of said word sequences was originally represented by one of said phoneme sequences;

wherein said renewer calculates said renewal value based on said first and said second language models, and employs said renewal value to renew said appearance probability assigned to each of said sequences, so as to reflect said reliability demonstrated by said word sequences having been represented originally by said phoneme sequences.

3. The speech recognition apparatus according to claim 2, wherein said first and said second language models are N-gram models;

and wherein said renewer employs, as said renewal value, a weighted average calculated for said first and said second language models.

4. The speech recognition apparatus according to claim 2, wherein said symbol is a comma or a period.

5. The speech recognition apparatus according to claim 2, wherein, in order to include said symbol in the speech recognition results that are obtained, said phoneme sequence/word sequence transformer transforms said phoneme sequences into one or more word sequences, or in the other situation, transforms said phoneme sequences into one or more word sequences each of which includes words other than said symbol, and provides said appearance probability;

and wherein, in order to include said symbol in the results obtained by the speech recognition process, said renewer employs said first and said second language models to renew said appearance probability assigned to each of said word sequences, or in the other situation, employs only said second language model to renew said appearance probability.

6. The speech recognition apparatus according to claim 5, wherein said first and said second language models are N-gram models;

and wherein said renewer employs, as said renewal value, a weighted average calculated for said first and said second language models.

7. The speech recognition apparatus according to claim 5, wherein said symbol is a coma or a period.

8. The speech recognition apparatus according to claim 1, wherein said first language model is demonstratively reliable in that when each of said word sequences includes a specific symbol, analogous to a word, each of said word sequences includes one or more words in the order for each of said word sequences;

wherein said second language model is demonstratively reliable in that in other situations the pertinent word sequence includes one or more words arranged in the same order as that of each of said word sequences; and

wherein said renewer calculates said renewal value based on said first and said second language models, and employs said renewal value to renew said appearance probability assigned to each of said sequences, so as to demonstrate said reliability for each of said word sequences that was originally represented by said phoneme/sequences.

9. The speech recognition apparatus according to claim 8, wherein said first and said second language models are N-gram models;

and wherein said renewer employs, as said renewal value, a weighted average calculated for said first and said second language models.

10. The speech recognition apparatus according to claim 8, wherein said symbol is a comma or a period.

11. The speech recognition apparatus according to claim 8, wherein, in order to include said symbol in the speech recognition results that are obtained, said phoneme sequence/word sequence transformer transforms said phoneme sequences into one or more word sequences, or in the other situation, transforms said phoneme sequences into one or more word sequences each of which includes words other than said symbol, and provides said appearance probability;

and wherein, in order to include said symbol in the results obtained by the speech recognition process, said renewer employs said first and said second language models to renew said appearance probability assigned to each of said word sequences, or in the other situation, employs only said second language model to renew said appearance probability.

12. The speech recognition apparatus according to claim 11, wherein said symbol is a comma or a period.

13. The speech recognition apparatus according to claim 1, wherein said first and said second language models are N-gram models;

and wherein said renewer employs, as said renewal value, a weighted average calculated for said first and said second language models.

14. The speech recognition apparatus according to claim 1, wherein said symbol is a comma or a period.

15. A method of speech recognition comprising the steps of:

transforming sequences of phonemes extracted from an utterance into one or more word sequences, and of assigning to said word sequences appearance probabilities, in accordance with which said word sequences are originally represented by said phoneme sequences;

renewing said appearance probability assigned to each of said word sequences by employing a renewal value represented by a language model corresponding to each of said word sequences; and

selecting the word sequence having the highest appearance probability, by screening all said word sequences assigned appearance probabilities, according to which said word sequences are originally represented by said phoneme sequences,

wherein, at during said renewing, said renewal value is calculated using a first language model which is employed when each of said word sequences always includes a specific symbol as a word and a second language model which is employed in other situations to renew said appearance probabilities based on said renewal value.

16. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for providing symbol insertion, said method comprising the steps of:

transforming sequences of phonemes extracted from an utterance into one or more word sequences, and of assigning to said word sequences appearance probabilities, in accordance with which aid word sequences are originally represented by said phoneme sequences;

renewing said appearance probability assigned to each of said word sequences by employing a renewal value represented by a language model corresponding to each of said word sequences; and

selecting the word sequence having the highest appearance probability, by screening all said word sequences assigned appearance probabilities, according to which said word sequences arm originally represented by said phoneme sequences,

wherein, during said renewing, said renewal value is calculated using a first language model which is employed when each of said word sequences always includes a specific symbol as a word and a second language model which is employed to renew said appearance probabilities based on said renewal value.