System and method for speech verification using out-of-vocabulary models

• US 6,778,959 B1
• Filed: 10/18/2000
• Issued: 08/17/2004
• Est. Priority Date: 10/21/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Wu, Duanpei, Chen, Ruxin, Olorenshaw, Lex, Menendez-Pidal, Xavier
• Primary Examiner(s): Chawan, Vijay
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/691,877
• Time in Patent Office: 1,399 Days
• Field of Search: 704/256, 704/255, 704/250, 704/251
• US Class Current: 704/256
• CPC Class Codes: G10L 15/08 Speech classification or search

Abstract

A system and method for speech verification using out-of-vocabulary models includes a speech recognizer that has a model bank with system vocabulary word models, a garbage model, and one or more noise models. The model bank may reject an utterance or other sound as an invalid vocabulary word when the model bank identifies the utterance or other sound as corresponding to the garbage model or the noise models. Initial noise models may be selectively combined into a pre-determined number of final noise model clusters to effectively reduce the number of noise models that are utilized by the model bank of the speech recognizer to verify system vocabulary words.

Claims

1. A system for performing a speech verification of a sound, comprising:

a recognizer configured to verify that said sound is valid by referencing a model bank that includes system vocabulary word models and out-of-vocabulary models, said out-of-vocabulary models including at least one of a garbage model and one or more noise models; and

a processor configured to control said recognizer to thereby perform said speech verification, said speech verification including selecting noise model types for said noise models based upon an intended operating environment, and creating respective noise databases for training initial noise models by recording one or more samples of said noise model types.

2. The system of claim 1 wherein said recognizer includes at least one of a search engine, a speech verifier, and said model bank.

3. The system of claim 1 wherein at least one of said system vocabulary word models and said out-of-vocabulary models is implemented as a Hidden Markov Model.

4. The system of claim 1 wherein said model bank rejects said sound as an invalid vocabulary word when said model bank identifies said sound as corresponding to said garbage model.

5. The system of claim 1 wherein said garbage model is trained with system vocabulary words and out-of-vocabulary words.

6. The system of claim 5 wherein said garbage model is trained before training said system vocabulary word models.

8. The system of claim 1 wherein said initial noise models are trained using said respective noise databases so that each of said initial noise models corresponds to one of said noise model types.

9. The system of claim 8 wherein a set of test noises is input to each of said initial noise models, said initial noise models responsively generating recognition scores for each of said test noises.

10. The system of claim 9 wherein a design program normalizes said recognition scores to produce normalized recognition scores by dividing said recognition scores by a corresponding test noise duration time.

11. The system of claim 10 wherein said design program calculates individual differential scores between each of said initial noise models by calculating a difference of said recognition scores for each of said test noises.

12. The system of claim 11 wherein said design program calculates an average differential score between each of said initial noise models based on all of said individual differential scores corresponding to said test noises.

13. The system of claim 12 wherein said design program creates a distance matrix that includes every possible pairing of said initial noise models, said distance matrix including distance entries that each corresponds to said average differential score for a corresponding pairing of said initial noise models.

14. The system of claim 13 wherein said design program regroups said most-similar pair of said initial noise models to generate a noise model cluster.

15. The system of claim 13 wherein said design program calculates a minimum non-zero distance entry to identify a most-similar pair of said initial noise models.

16. The system of claim 15 wherein said design program generates a reconfigured distance matrix that substitutes said noise model cluster for said most-similar pair of said initial noise models and generates recalculated distance entries, said reconfigured distance matrix including every possible pairing of said initial noise models and said noise model cluster, said reconfigured distance matrix including said recalculated distance entries that each corresponds to said average differential score for one of said corresponding pairings of said initial noise models and said noise model cluster.

17. The system of claim 16 wherein said design program repeatedly recalculates new minimum non-zero distances, repeatedly generates new noise model clusters, and repeatedly generates new reconfigured distance matrices until a pre-determined number of final noise model clusters is reached.

18. The system of claim 17 wherein said model bank includes final noise models that are each trained using a respective one of said final noise model clusters.

7. A system for performing a speech verification of a sound, comprising:

a recognizer configured to verify that said sound is valid by referencing a model bank that includes system vocabulary word models and out-of-vocabulary models, said out-of-vocabulary models including at least one of a garbage model and one or more noise models, each of said one or more noise models being trained to correspond to one of a specific noise and a noise cluster, said model bank rejecting said sound as an invalid vocabulary word when said model bank identifies said sound as corresponding to one of said specific noise and said noise cluster; and

a processor configured to control said recognizer to thereby perform said speech verification.

19. A method for performing a speech verification of a sound, comprising:

verifying that said sound is valid with a recognizer by referencing a model bank that includes system vocabulary word models, and out-of-vocabulary models, said out-of-vocabulary models including at least one of a garbage model and one or more noise models, each of said one or more noise models being trained to correspond to one of a specific noise and a noise cluster, said model bank rejecting said sound as an invalid vocabulary word when said model bank identifies said sound as corresponding to one of said specific noise and said noise cluster;

and

controlling said recognizer with a processor to thereby perform said speech verification.

20. The method of claim 19 wherein said recognizer includes at least one of a search engine, a speech verifier, and said model bank.

21. The method of claim 19 wherein at least one of said system vocabulary word models and said out-of-vocabulary models is implemented as a Hidden Markov Model.

22. The method of claim 19 wherein said model bank rejects said sound as an invalid vocabulary word when said model bank identifies said sound as corresponding to said garbage model.

23. The method of claim 19 wherein said garbage model is trained with system vocabulary words and out-of-vocabulary words.

24. The method of claim 23 wherein said garbage model is trained before training said system vocabulary word models.

25. A method for performing a speech verification of a sound, comprising:

verifying that said sound is valid with a recognizer by referencing a model bank that includes system vocabulary word models, and out-of-vocabulary models, said out-of-vocabulary models including at least one of a garbage model and one or more noise models; and

controlling said recognizer with a processor to thereby perform said speech verification, said speech verification including selecting noise model types for said noise models based upon an intended operating environment, and creating respective noise databases for training initial noise models by recording one or more samples of said noise model types.

26. The method of claim 25 wherein said initial noise models are trained using said respective noise databases so that each of said initial noise models corresponds to one of said noise model types.

27. The method of claim 26 wherein a set of test noises is input to each of said initial noise models, said initial noise models responsively generating recognition scores for each of said test noises.

28. The method of claim 27 wherein a design program normalizes said recognition scores to produce normalized recognition scores by dividing said recognition scores by a corresponding test noise duration time.

29. The method of claim 28 wherein said design program calculates individual differential scores between each of said initial noise models by calculating a difference of said recognition scores for each of said test noises.

30. The method of claim 29 wherein said design program calculates an average differential score between each of said initial noise models based on all of said individual differential scores corresponding to said test noises.

31. The method of claim 30 wherein said design program creates a distance matrix that includes every possible pairing of said initial noise models, said distance matrix including distance entries that each corresponds to said average differential score for a corresponding pairing of said initial noise models.

32. The method of claim 31 wherein said design program regroups said most-similar pair of said initial noise models to generate a noise model cluster.

33. The method of claim 31 wherein said design program calculates a minimum non-zero distance entry to identify a most-similar pair of said initial noise models.

34. The method of claim 33 wherein said design program generates a reconfigured distance matrix that substitutes said noise model cluster for said most-similar pair of said initial noise models and generates recalculated distance entries, said reconfigured distance matrix including every possible pairing of said initial noise models and said noise model cluster, said reconfigured distance matrix including said recalculated distance entries that each corresponds to said average differential score for one of said corresponding pairings of said initial noise models and said noise model cluster.

35. The method of claim 34 wherein said design program repeatedly recalculates new minimum non-zero distances, repeatedly generates new noise model clusters, and repeatedly generates new reconfigured distance matrices until a pre-determined number of final noise model clusters is reached.

36. The method of claim 35 wherein said model bank includes final noise models that are each trained using a respective one of said final noise model clusters.