Modelling and processing filled pauses and noises in speech recognition

• US 20040181410A1
• Filed: 03/13/2003
• Published: 09/16/2004
• Est. Priority Date: 03/13/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hwang, Mei-Yuh
• Granted Patent: US 7,076,422 B2
• Time in Patent Office: Days
• US Class Current: 704/256
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 2021/02168 the estimation exclusively taking place during speech pauses

Abstract

A speech recognition system recognizes filled pause utterances made by a speaker. In one embodiment, an ergodic model is used to acoustically model filled pauses that provides flexibility allowing varying utterances of the filled pauses to be made. The ergodic HMM model can also be used for other types of noise such as but limited to breathing, keyboard operation, microphone noise, laughter, door openings and/or closings, or any other noise occurring in the environment of the user or made by the user. Similarly, silence can be modeled using an ergodic HMM model. Recognition can be used with N-gram, context-free grammar or hybrid language models.

Claims

1. A computer readable media including instructions readable by a computing device which, when implemented, cause the computing device to handle information by performing steps comprising:

receiving input speech;

accessing information indicative of a lexicon including entries transcribed by phones, an acoustic model modeling phones for entries in the lexicon, the acoustic model including at least one ergodic Hidden Markov Model (HMM) model for a filled pause, and a language model modeling word order with filled pauses; and

processing the input speech using the information to identify filled pauses in the input speech.

2. The computer readable media of claim 1 wherein the language model comprises an N-gram language model, and wherein processing includes processing the input speech where a filled pause is a function of a unigram probability.

3. The computer readable media of claim 1 wherein the language model comprises a context-free grammar, and wherein processing includes processing the input speech allowing identification for filled pauses to occur at a position corresponding to the beginning nodes of the context-free grammar rules being processed.

4. The computer readable media of claim 1 wherein the acoustic model includes an ergodic model for silence, and wherein processing includes processing the input speech to identify silence.

5. The computer readable media of claim 1 wherein the acoustic model includes an ergodic model for noise, and wherein processing includes processing the input speech to identify noise.

6. A computer-implemented method of forming a context-free grammar language model for use in speech recognition, the method comprising:

accessing a set of context-free grammar rules; and

creating a context-free grammar language model as a function of the context-free grammar rules, wherein portions of the model for at least some of the rules include allowing identification for filled pauses to occur at a position corresponding to beginning nodes of said at least some the context-free grammar rules.

7. The computer-implemented method of claim 6 wherein creating the context-free grammar language model includes calculating a transition probability to a filled pause for each of the beginning nodes of said at least some the context-free grammar rules.

8. The computer-implemented method of claim 6 wherein calculating the transition probability includes ascertaining a fan-out of the corresponding context-free grammar rule.

9. A speech recognition system, comprising:

an input receiving a signal indicative of speech;

a lexicon including entries transcribed by phones;

an acoustic model modeling phones for entries in the lexicon, the acoustic model including at least one ergodic Hidden Markov Model (HMM) model for a filled pause;

a language model modeling word order with filled pauses; and

a decoder coupled to the input, the acoustic model and the language model, recognizing speech including filled pauses represented by the signal.

10. The speech recognition system of claim 9 wherein said at least one HMM ergodic model of the filled pause comprises an HMM ergodic model where every state includes a transition probability to every other state.

11. The speech recognition system of claim 9 wherein the language model comprises a statistically based language model.

12. The speech recognition system of claim 11 wherein the language model comprises an N-gram language model.

13. The speech recognition system of claim 12 wherein the language model comprises a probability for said at least one filled pause as a function of a unigram probability for said at least one filled pause.

14. The speech recognition system of claim 9 wherein the language model comprises a context-free grammar.

15. The speech recognition system of claim 14 wherein the language model includes at least some context-free grammar rules having allowable transitions to a filled pause located at beginning nodes of the context-free grammar rules.

16. The speech recognition system of claim 15 wherein a transition probability to a filled pause is a function of a fan-out of the corresponding context-free grammar rule.

17. The speech recognition system of claim 9 wherein the acoustic model includes an ergodic model for silence.

18. The speech recognition system of claim 9 wherein the acoustic model includes an ergodic model for noise.

19. The speech recognition system of claim 18 wherein the acoustic model for noise is a single model for a plurality of different noises.

20. The speech recognition system of claim 19 wherein the plurality of different noises are selected from a group comprising environmental noise and speaker made noise.

21. The speech recognition system of claim 19 wherein the environmental noise includes microphone noise, keyboard operation, and door open and closings.

22. The speech recognition system of claim 21 wherein the speaker made noise includes breathing noise, laughter and background talking.