Speaker verification system using acoustic data and non-acoustic data

• US 7,016,833 B2
• Filed: 06/12/2001
• Issued: 03/21/2006
• Est. Priority Date: 11/21/2000
• Status: Expired due to Fees

• Current Assignee: Lawrence Livermore National Security LLC (Government of the United States of America)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Ng, Lawrence C., Gable, Todd J., Burnett, Greg C., Holzrichter, John F.
• Primary Examiner(s): Young, W. R.
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/879,469
• Publication Number:
  US 20050060153A1
• Time in Patent Office: 1,743 Days
• Field of Search: 704/200, 704/201, 704/209, 704/250
• US Class Current: 704/209
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction

Abstract

A method and system for speech characterization. One embodiment includes a method for speaker verification which includes collecting data from a speaker, wherein the data comprises acoustic data and non-acoustic data. The data is used to generate a template that includes a first set of “template” parameters. The method further includes receiving a real-time identity claim from a claimant, and using acoustic data and non-acoustic data from the identity claim to generate a second set of parameters. The method further includes comparing the first set of parameters to the set of parameters to determine whether the claimant is the speaker. The first set of parameters and the second set of parameters include at least one purely non-acoustic parameter, including a non-acoustic glottal shape parameter derived from averaging multiple glottal cycle waveforms.

Claims

1. A method for speaker verification, comprising:

collecting a plurality of data from a speaker, wherein the plurality of data comprises acoustic data and non-acoustic data;

using the plurality of data to generate a template comprising a first plurality of parameters;

receiving a real-time identity claim from a claimant;

using a plurality of acoustic data and non-acoustic data from the identity claim to generate a second plurality of parameters; and

comparing the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker, wherein the first plurality of parameters and the second plurality of parameters include at least one purely non-acoustic parameter, including a non-acoustic glottal shape parameter parameter derived from averaging multiple glottal cycle waveforms.

2. The method of claim 1, wherein the first plurality of parameters and the second plurality of parameters each comprise:

a pitch parameter extracted using non-acoustic data;

at least one pitch synchronous spectral coefficient extracted using non-acoustic data;

pitch synchronous auto-regressive and moving average (ARMA) coefficients extracted using non-acoustic data.

3. The method of claim 2, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system, including computing the fast Fourier transform of the acoustic data and the non-acoustic data and solving for a transfer function, wherein the non-acoustic data comprises input of the modeled speech system, and the acoustic data comprises output of the modeled speech system.

4. The method of claim 2, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system using a parametric linear model.

5. The method of claim 1, wherein the non-acoustic data comprises an electromagnetic (EM) signal that characterizes a motion of the speaker'"'"'s tracheal and glottal tissues.

6. The method of claim 5, wherein the EM signal is sampled during the middle of phonation.

7. The method of claim 6, wherein a glottal shape parameter (GSP) is based on averaged multiple glottal cycle waveforms generated when the speaker utters a test sentence.

8. The method of claim 7, wherein non-consecutive two-glottal cycle waveforms are averaged to produce the GSP.

9. The method of claim 1, wherein generating the template comprises:

extracting a parameter from each of multiple repetitions of each of a set of test sentences by the speaker;

producing multiple feature vectors, each corresponding to a parameter;

selecting one of the multiple feature vectors as a guide vector for dynamic time warping; and

averaging the multiple feature vectors to produce a resultant feature vector that is part of the template.

10. The method of claim 9, wherein collecting the first plurality of data comprises the speaker uttering each of a set of test sentences, and wherein subsequent utterances of the test sentences by the speaker cause the template to be updated.

11. The method of claim 9, wherein comparing the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker comprises:

using a dynamic warping algorithm to calculate a warping distance between a feature vector in the template and a corresponding feature vector generated from the second plurality of parameters; and

determining whether the calculated distance is above or below a predetermined threshold.

12. The method of claim 11, wherein using a dynamic warping algorithm, comprises applying constraints comprising:

a monotonicity constraint;

at least one endpoint constraint;

at least one global path constraint; and

at least one local path constraint.

13. The method of claim 11, wherein the predetermined threshold is chosen such that a false acceptance error rate and a false rejection error rate are substantially equal.

14. The method of claim 13, wherein each feature vector generated from the second plurality of parameters has its own equal error rate (EER) based upon a corresponding warping distance from a feature vector that is part of the template.

15. The method of claim 13, wherein EERs of each feature vector generated from the second plurality of parameters are combined to generate an overall EER used to evaluate the speaker verification method.

16. A system for speaker verification, comprising:

at least one microphone for collecting acoustic data from a speaker'"'"'s voice;

at least one sensor for collecting non-acoustic data from movements of the speaker'"'"'s body;

at least one processor;

a memory device coupled to the processor, wherein the memory device stores instructions that when executed cause the processor to generate a template using the acoustic data and non-acoustic data, wherein the template comprises a first plurality of parameters, wherein when a claimant speaks an identity claim into the at least one microphone, the instruction further cause the processor to generate a second plurality of parameters, and to compare the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker, wherein the first plurality of parameters and the second plurality of parameters include at least one purely non-acoustic parameter, including a non-acoustic glottal shape parameter parameter derived from averaging multiple glottal cycle waveforms.

17. The system of claim 16, wherein the first plurality of parameters and the second plurality of parameters each comprise:

a pitch parameter extracted using non-acoustic data;

at least one pitch synchronous spectral coefficient extracted using non-acoustic data;

pitch synchronous auto-regressive and moving average (ARMA) coefficients extracted using non-acoustic data.

18. The system of claim 17, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system, including computing the fast Fourier transform of the acoustic data and the non-acoustic data and solving for a transfer function, wherein the non-acoustic data comprises input of the modeled speech system, and the acoustic data comprises output of the modeled speech system.

19. The system of claim 17, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system using a parametric linear model.

20. The system of claim 16, wherein the non-acoustic data comprises an electromagnetic (EM) signal that characterizes a motion of the speaker'"'"'s tracheal and glottal tissues.

21. The system of claim 20, wherein the EM signal is sampled during the middle of phonation.

22. The system of claim 21, wherein a glottal shape parameter (GSP) is based on averaged multiple glottal cycle waveforms generated when the speaker utters a test sentence.

23. The system of claim 22, wherein non-consecutive two-glottal cycle waveforms are averaged to produce the GSP.

24. The system of claim 16, wherein generating the template comprises:

extracting a parameter from each of multiple repetitions of each of a set of test sentences by the speaker;

producing multiple feature vectors, each corresponding to a parameter;

selecting one of the multiple feature vectors as a guide vector for dynamic time warping; and

averaging the multiple feature vectors to produce a resultant feature vector that is part of the template.

25. The system of claim 24, wherein collecting the first plurality of data comprises the speaker uttering each of a set of test sentences, and wherein subsequent utterances of the test sentences by the speaker cause the template to be updated.

26. The system of claim 24, wherein comparing the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker comprises:

using a dynamic warping algorithm to calculate a warping distance between a feature vector in the template and a corresponding feature vector generated from the second plurality of parameters; and

determining whether the calculated distance is above or below a predetermined threshold.

27. The system of claim 26, wherein using a dynamic warping algorithm, comprises applying constraints comprising:

a monotonicity constraint;

at least one endpoint constraint;

at least one global path constraint; and

at least one local path constraint.

28. The system of claim 26, wherein the predetermined threshold is chosen such that a false acceptance error rate and a false rejection error rate are substantially equal.

29. The system of claim 28, wherein each feature vector generated from the second plurality of parameters has its own equal error rate (EER) based upon a corresponding warping distance from a feature vector that is part of the template.

30. The system of claim 28, wherein EERs of each feature vector generated from the second plurality of parameters are combined to generate an overall EER used to evaluate the speaker verification system.

31. An electromagnetic medium, having stored thereon instructions that when executed, cause a processor to:

collect a plurality of data from a speaker, wherein the plurality of data comprises acoustic data and non-acoustic data;

use the plurality of data to generate a template comprising a first plurality of parameters;

receive a real-time identity claim from a claimant;

use a plurality of acoustic data and non-acoustic data from the identity claim to generate a second plurality of parameters; and

compare the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker, wherein the first plurality of parameters and the second plurality of parameters include at least one purely non-acoustic parameter, including a non-acoustic glottal shape parameter derived from averaging multiple glottal cycle waveforms.

32. The electromagnetic medium of claim 31, wherein the first plurality of parameters and the second plurality of parameters each comprise:

a pitch parameter extracted using non-acoustic data;

at least one pitch synchronous spectral coefficient extracted using non-acoustic data;

pitch synchronous auto-regressive and moving average (ARMA) coefficients extracted using non-acoustic data.

33. The electromagnetic medium of claim 32, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system, including computing the fast Fourier transform of the acoustic data and the non-acoustic data and solving for a transfer function, wherein the non-acoustic data comprises input of the modeled speech system, and the acoustic data comprises output of the modeled speech system.

34. The electromagnetic medium of claim 32, wherein extracting the ARMA coefficients comprises ARMA pole-zero modeling of a speech system using a parametric linear model.

35. The electromagnetic medium of claim 31, wherein the non-acoustic data comprises an electromagnetic (EM) signal that characterizes a motion of the speaker'"'"'s tracheal and glottal tissues.

36. The electromagnetic medium of claim 35, wherein the EM signal is sampled during the middle of phonation.

37. The electromagnetic medium of claim 36, wherein a glottal shape parameter (GSP) is based on averaged two-glottal cycle waveforms generated when the speaker utters a test sentence.

38. The electromagnetic medium of claim 37, wherein non-consecutive two-glottal cycle waveforms are averaged to produce the GSP.

39. The electromagnetic medium of claim 31, wherein generating the template comprises:

extracting a parameter from each of multiple repetitions of each of a set of test sentences by the speaker;

producing multiple feature vectors, each corresponding to a parameter;

selecting one of the multiple feature vectors as a guide vector for dynamic time warping; and

averaging the multiple feature vectors to produce a resultant feature vector that is part of the template.

40. The electromagnetic medium of claim 39, wherein collecting the first plurality of data comprises the speaker uttering each of a set of test sentences, and wherein subsequent utterances of the test sentences by the speaker cause the template to be updated.

41. The electromagnetic medium of claim 39, wherein comparing the first plurality of parameters to the second plurality of parameters to determine whether the claimant is the speaker comprises:

using a dynamic warping algorithm to calculate a warping distance between a feature vector in the template and a corresponding feature vector generated from the second plurality of parameters; and

determining whether the calculated distance is above or below a predetermined threshold.

42. The electromagnetic medium of claim 41, wherein using a dynamic warping algorithm, comprises applying constraints comprising:

a monotonicity constraint;

at least one endpoint constraint;

at least one global path constraint; and

at least one local path constraint.

43. The electromagnetic medium of claim 41, wherein the predetermined threshold is chosen such that a false acceptance error rate and a false rejection error rate are substantially equal.

44. The electromagnetic medium of claim 43, wherein each feature vector generated from the second plurality of parameters has its own equal error rate (EER) based upon a corresponding warping distance from a feature vector that is part of the template.

45. The method of claim 43, wherein EERs of each feature vector generated from the second plurality of parameters are combined to generate an overall EER used to evaluate the speaker verification method.