A SYSTEM AND APPARATUS FOR RECOGNIZING SPEECH

• US 20010011218A1
• Filed: 03/13/2001
• Published: 08/02/2001
• Est. Priority Date: 09/30/1997
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Anne Rogers, Steven Phillips
• Inventors: Phillips, Steven, Rogers, Anne
• Granted Patent: US 6,374,212 B2
• Time in Patent Office: Days
• US Class Current: 704/256
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/34 Adaptation of a single recogniser for parallel processing, e.g. by use of multiple processors or cloud computing

Abstract

A continuous, speaker independent, speech recognition method and system for recognizing a variety of vocabulary input signals. A language model which is an implicit description of a graph consisting of a plurality of states and arcs is inputted into the system. An input speech signal, corresponding to a plurality of speech frames is received and processed using a shared memory multipurpose machine having a plurality of microprocessors working in parallel to produce a textual representation of the speech signal.

Claims

1. A speech recognition method for recognizing a variety of speech inputs comprising the steps of:

receiving a signal corresponding to a plurality of speech frames;

inputting a language model in a general purpose shared memory machine having a plurality of processors wherein said model is an implicit description of a graph consisting of a plurality of states and arcs;

assigning each state to at least one of said processors; and

processing said states using said plurality of processors.

2. The method of

claim 1

further including the step of determining which of said states are active.

3. The method of

claim 1

further including the step of determining which of said arcs, associated with each of said states, are active.

4. The method of

claim 3

further including the step of assigning each of said active states to a processing thread associated with each of said processors.

5. The method of

claim 4

wherein said thread also processes said active arcs associated with said active state.

6. The method of

claim 3

wherein each of said arcs having a label corresponding to a speech sound.

7. The method of

claim 5

further including the step of determining the transitions from said active states using an on-demand composition of finite state transducers.

8. The method of

claim 7

wherein said step of determining is performed in parallel using said threads associated with said processors.

9. The method of

claim 6

wherein said language model includes models of said speech sounds, said method further including the step of evaluating said input speech frames with said models of speech sounds.

10. The method of

claim 9

further including the step of modeling each of said speech sounds by a multiple state Hidden Markov model, each of said states corresponding to a portion of said speech sound and having a probability distribution associated therewith.

11. The method of

claim 9

further including the step of calculating, for each of said frames, the likelihood cost of observing each of said frames in a set of Hidden Markov models.

12. The method of

claim 9

wherein each of said states having at least one incoming or outgoing arc within said graph, said method further including the steps of:

calculating the likelihood costs of said active arcs;

defining a source state cost by calculating the cost of arriving at a particular state within said graph;

determining a cost of a particular arc by adding said likelihood cost of said arc and said source state cost for the state associated with said arc;

calculating a state cost by determining the minimum of said costs of said incoming arcs associated with said particular state;

determining which of said active arcs has the lowest cost;

calculating a particular value range by adding a predetermined threshold value to the cost of said arc having the lowest cost;

determining which of said active arcs fall within said value range;

pruning said active arcs such that only said arcs having costs that fall within said range remain active.

13. The method of

claim 11

further including the step of processing said likelihood calculations in parallel in said machine.

14. The method of

claim 11

further including the step of updating said active arc list with said active arcs that remain after said pruning.

15. The method of

claim 13

further including the step of activating additional active states corresponding to said updated active arc list.

16. The method of

claim 14

wherein said step of activating additional active states is performed in parallel in said shared memory machine.

17. A general purpose speech recognition system for receiving a speech signal and producing a textual representation of said speech signal, said system capable of recognizing a variety of speech input, said recognition system comprising:

a data processing machine having at least two microprocessors and a memory means;

a speech signal inputted to said machine;

means for receiving said speech signal and digitally sampling said signal at a predetermined rate resulting in a representation of said input signal as a series of speech frames;

an inputted language model stored in said memory means, said model including models relating to probability distributions corresponding to a plurality of speech sounds;

means responsive to said speech frame for producing a set of signals representative of said input speech frame, said means for producing including processing a Viterbi speech algorithm in parallel on said microprocessors, said algorithm mapping said speech frames to Hidden Markov models and producing a textual representation of said received speech signal based on likelihood calculations; and

means for providing a representation of said speech signal.

18. The speech recognition system of

claim 17

wherein said algorithm maintains a set of active states and a set of active arcs having one of said models associated with it.

19. The speech recognition system of

claim 18

wherein said active state set is partitioned into subsets where each subset is assigned to a particular thread, said active state subsets being processed in parallel within said machine.

20. The speech recognition system of

claim 19

wherein each of said active states within said active state subsets having corresponding active arcs, each of said active arcs assigned to said processor that processes said active states.

21. The speech recognition system of

claim 20

wherein said algorithm calculates the likelihood of observing said speech frame in said model in each of said active arcs, said likelihood calculations occurring in parallel such that said processor receiving said active arc also performs said likelihood calculations.

22. A method of speech recognition having a plurality of active states, said method comprising the steps of:

partitioning said plurality of active states to create one or more active state subsets, each of said subsets including a number of active states;

assigning each of said active state subsets to one or a plurality of microprocessors included in a multiprocessor shared memory machine;

determining active arcs associated with said active states;

assigning said active arcs to a particular processor based on said assignment of said active states;

performing a likelihood calculation for each of said active arcs; and

pruning said active arcs based on said likelihood calculation such that said arcs having a likelihood calculation within a computed range are included in an active arc sublist.

23. The method of

claim 22

wherein the step of performing likelihood calculations for each of said active arcs is performed in parallel using at least two or more of said processors.

24. The method of

claim 22

further comprising the step of storing said likelihood calculation associated with an active arc.

25. The method of

claim 24

further comprising the steps of:

determining whether a likelihood calculation associated with an active arc has previously been performed; and

storing said likelihood calculation in a memory in parallel.

26. The method of

claim 23

further including the step of updating said active arc subset with active arcs that remain after said pruning.

27. The method of

claim 26

further including the step of creating a new active state subset based on said updated active arc subset.

28. The method of

claim 26

wherein said step of creating a new active state subset is performed in parallel using at least two or more of said processors.

29. The method of

claim 25

further including the step of determining the transitions from said active states using an on-demand composition of finite state transducers.

30. The method of

claim 29

further including the step up multi-threading said finite state transducers.