Method and apparatus for probabilistic recognition using small number of state clusters

• US 6,725,195 B2
• Filed: 10/22/2001
• Issued: 04/20/2004
• Est. Priority Date: 08/25/1998
• Status: Expired due to Term

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Sankar, Ananth, Gadde, Venkata Ramana Rao
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): Azad, Abul K.
• Application Number:
  US10/029,420
• Publication Number:
  US 20030040906A1
• Time in Patent Office: 911 Days
• Field of Search: 704/231, 704/240, 704/243, 704/244, 704/245, 704/251, 704/256
• US Class Current: 704/240
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/144 Training of HMMs
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

Probabilistic recognition using clusters and simple probability functions provides improved performance by employing a limited number of clusters each using a relatively large number of simple probability functions. The simple probability functions for each of the limited number of state clusters are greater in number than the limited number of state clusters.

Claims

1. In a speech recognition system using a method for recognizing human speech, the method comprising the steps of:

selecting a model to represent a selected subunit of speech, the model having associated with it a plurality of states;

determining states that may be represented by a set of simple probability functions; and

clustering said states that may be represented by a set of simple probability functions into a limited number of clusters, wherein said simple probability functions for each of said limited number of state clusters is greater in number than said limited number of state clusters.

2. The method according to claim 1 wherein the number of clusters is kept low to improve processing speed while the number of simple probability functions per cluster is increased for greater recognition accuracy.

3. The method according to claim 1 wherein the number of clusters is between approximately 40 and 200 and wherein at least one cluster has assigned to it 500 to 2000 simple probability functions.

4. The method according to claim 1 wherein the number of clusters is more than 10 and wherein the ratio of the number of clusters to the total number of simple probability functions in the system is less than 0.002.

5. The method according to claim 1 wherein the number of clusters is more than 9 and at least one cluster has more than approximately 1,000 simple probability functions.

6. The method according to claim 1 wherein the simple probability functions are Gaussians.

7. The method according to claim 1 wherein different numbers of simple probability functions are used in different clusters.

8. The method according to claim 7 wherein the number of simple probability functions used for a particular cluster is determined by a training algorithm.

9. The method according to claim 7 wherein the number of simple probability functions used for a particular cluster is indicated by a human system designer.

10. The method according to claim 1 wherein the number of said clusters is equal to the number of phones in the system.

11. The method according to claim 1 wherein the model is a three-state Hidden Markov Model.

12. The method according to claim 1 wherein states are clustered according to an agglomerative hierarchical clustering scheme.

13. The method according to claim 1 wherein states are clustered so as to nearly eliminate overlap between clusters.

14. The method according to claim 1 further comprising:

caching log-likelihoods for the simple probability functions in a mixture as soon as they are computed for a frame so that if the same mixture needs to be evaluated at that frame for another triphone state, the cache is used.

15. The method according to claim 1 wherein redundant simple probability functions in the state cluster overlap region are more effectively used to cover the acoustic space of the clusters, resulting in smaller variances and a reducing the number of distance components to be computed.

16. The method according to claim 1 further comprising:

reducing the size of a simple probability function shortlists by decreasing the number of state clusters with a corresponding reduction in simple probability function computations.

17. A computer readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

selecting a model to represent a selected subunit of speech, the model having associated with it a plurality of states;

determining states that may be represented by a set of simple probability functions; and

clustering said states that may be represented by a set of simple probability functions into a limited number of clusters, wherein said simple probability functions for each of said limited number of state clusters is greater in number than said limited number of state dusters.

18. A speech recognizer comprising:

a logic processing device;

storage means;

a set of probabilistic models stored in the storage means;

said models including a limited number of state clusters, wherein at least one of said limited number of state clusters is represented by a number of simple probability functions, wherein said simple probability functions for each of said limited number of state clusters is greater in number than said limited number of state clusters;

a feature extractor in a computer for extracting feature data capable of being processed by said computer from a speech signal; and

recognizing means for matching features from unidentified speech data to the models to produce a most likely path through the models where the path defines the most likely subunits and words in the speech data.