Methods of identification using voice sound analysis

• US 8,036,891 B2
• Filed: 06/26/2008
• Issued: 10/11/2011
• Est. Priority Date: 06/26/2008
• Status: Active Grant

• Current Assignee: California State University Chico (California State University System)
• Original Assignee: California State University Chico (California State University System)
• Inventors: Fulop, Sean
• Primary Examiner(s): Vo; Huyen X.
• Application Number:
  US12/146,971
• Publication Number:
  US 20090326942A1
• Time in Patent Office: 1,202 Days
• Field of Search: 704/200, 704/203, 704/204, 704/205, 704/206, 704/246, 704/278
• US Class Current: 704/246
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band

Abstract

Methods of using individually distinctive patterns of voice characteristics to identify a speaker include computing the reassigned spectrogram of each of at least two voice samples, pruning each reassigned spectrogram to remove noise and other computational artifacts, and comparing (either visually or with the aid of a processor) the strongest points to determine whether the voice samples belong to the same speaker.

Claims

1. A method of comparing voice signals, comprising:

a. receiving a digital representation of each of a plurality of voice signals;

b. generating at least one reassigned spectrogram corresponding to each of said plurality of digital representations;

c. pruning each of said plurality of reassigned spectrograms to remove noise and computational artifacts, wherein said pruning comprises computing a mixed partial derivative of the phase of said spectrogram and removing data points from said spectrogram when said derivative is outside of a range;

d. comparing a first of said plurality of reassigned spectrograms to at least one other of said plurality of reassigned spectrograms, wherein said first reassigned spectrogram corresponds to a voice signal to be validated; and

e. identifying any similarities between said first spectrogram and any of said remaining spectrograms.

2. The method of claim 1, wherein the step of generating a reassigned spectrogram comprises:

a. identifying a target location within said digital representation;

b. selecting a portion of said digital representation within said target location;

c. sectioning said selected portion into a plurality of partially overlapping analysis time frames;

d. obtaining a spectrogram by calculating a short-time Fourier Transform calculation over the plurality of said analysis time frames; and

e. reassigning the spectrogram by calculating a time derivative and a frequency derivative of a phase argument of the spectrogram.

3. The method of claim 2, wherein said target location corresponds to a vocal vibration and has a sonority greater than a sonority threshold.

4. The method of claim 2, wherein said target location corresponds to a vowel sound.

5. The method of claim 2, wherein said target location corresponds to a letter selected from the group consisting of a, e, i, l, m, n, o, r, and u.

6. The method of claim 2, wherein said target location corresponds to any sound represented by an IPA character.

7. The method of claim 2, wherein said target location is identified by a human operator.

8. The method of claim 2, wherein said target location is identified by a processor.

9. The method of claim 8, wherein said step of identifying a target location comprises:

a. performing an autocorrelation analysis of said digital representation;

b. limiting potential target locations to those having a value greater than a threshold.

10. The method of claim 8, wherein said step of identifying a target location comprises limiting potential target locations to those having a harmonics-to-noise ratio of greater than about 10 decibels.

11. The method of claim 8, wherein said step of identifying a target location comprises:

a. performing a cepstral analysis of said digital representation;

b. determining a harmonicity; and

c. limiting potential target locations to those having a value greater than a cepstral threshold.

12. The method of claim 2, wherein said selected portion includes at least four phonation cycles.

13. The method of claim 12, wherein each analysis time frame has a length that is between about 5 percent and about 30 percent of the length of said selected portion.

14. The method of claim 12 wherein the length of each analysis time frame is at least about 75% of a phonation cycle.

15. The method of claim 12 wherein each analysis time frame overlaps an adjacent analysis time frame by at least about 50%.

16. The method of claim 2, wherein said selected portion includes at least one phonation cycle.

17. The method of claim 2, wherein the length of said selected portion is between about 25 and about 50 milliseconds.

18. The method of claim 17, wherein the length of each analysis time frame is between about 4 and about 7 milliseconds.

19. The method of claim 2, wherein the length of said selected portion is approximately 40 milliseconds.

20. The method of claim 2 wherein each analysis time frame is customized using pitch-synchronous analysis to determine phonation cycle time.

21. The method of claim 1, wherein the step of generating a reassigned spectrogram comprises:

a. identifying a target location within said digital representation;

b. selecting a portion of said digital representation within said target location;

c. sectioning said selected portion into a plurality of analysis time frames;

d. obtaining a spectrogram by generating a magnitude plot transform; and

e. reassigning the spectrogram by calculating a time derivative and a frequency derivative of the phase argument of the spectrogram.

22. The method of claim 1 wherein said cepstral threshold has a high amplitude peak.

23. The method of claim 22 wherein said cepstral threshold peak is at least 1decibel.

24. The method of claim 1, wherein said range is between about −

0.25 and about 0.25.

25. The method of claim 1, wherein said range is between about 0.5 and about 1.5.

26. The method of claim 1, wherein said a first range is between about −

0.25 and about 0.25, and a second range is between about 0.75 and about 1.25.

27. The method of claim 1, wherein said range is between about −

0.25 and about 1.5.

28. The method of claim 1, wherein said range is between about −

0.25 and about 1.25.

29. The method of claim 1, wherein said range is between about 0.75 and about 1.25.

30. The method of claim 1, wherein the frequency range of said reassigned spectrogram is between about 50 and 3000 Hz.

31. The method of claim 1, wherein the frequency range of said reassigned spectrogram is between about 100 and 1500 Hz.

32. The method of claim 1, wherein said step of comparing said reassigned spectrograms comprises:

a. generating a colormap corresponding to each of said reassigned spectrograms;

b. displaying said plurality of colormaps;

c. visually comparing the most pronounced and intense areas of said first reassigned spectrogram to the most pronounced and intense areas of said other reassigned spectrograms; and

d. selecting at least one of said other reassigned spectrograms that most closely correlates to said first reassigned spectrogram.

33. The method of claim 1, wherein said step of comparing said reassigned spectrograms comprises:

a. generating outputs corresponding to each of said reassigned spectrograms;

b. comparing the most pronounced and intense areas of the output for said first reassigned spectrogram to the most pronounced and intense areas of the outputs for said other reassigned spectrograms; and

c. selecting at least one of said outputs that most closely correlates to said first output.

34. The method of claim 1, wherein said step of comparing said reassigned spectrograms comprises:

a. selecting data points in each of said reassigned spectrograms exceeding a threshold value;

b. calculating the Euclidean squared distances between said selected data points in said first reassigned spectrogram and said selected data points in said other reassigned spectrograms, and

c. selecting at least one of said other reassigned spectrograms having the least total distance to said first reassigned spectrogram.

35. The method of claim 1, wherein said step of comparing said reassigned spectrograms comprises:

a. processing each of said spectrograms through a statistical pattern-identification device;

b. selecting one of said other reassigned spectrograms that is best matched to the first spectrogram according to the device.

36. The method of claim 35, wherein said device is a support vector machine.

37. The method of claim 1, wherein said step of comparing said reassigned spectrograms utilizes a Gaussian mixture model comparison procedure.

38. The method of claim 1 wherein the step of comparing said reassigned spectrograms comprises:

a. generating images for each of said reassigned spectrograms;

b. comparing sets of points and lines in the image for said first reassigned spectrogram for similarity to sets of points and lines in the images for said other reassigned spectrograms; and

c. selecting one of said other reassigned spectrograms that most closely correlates to said first reassigned spectrogram.

39. A method of comparing two voice sounds to determine whether they came from the same source, comprising:

a. recording a first voice signal;

b. selecting a first vocal vibration from said first voice signal;

c. isolating at least four cycles of phonation of said first vocal vibration;

d. computing a first reassigned spectrogram of said first vocal vibration for said isolated phonation cycles;

e. pruning said first reassigned spectrogram to remove unwanted elements by computing a mixed partial derivative of the phase of said first reassigned spectrogram and removing data points from said first reassigned spectrogram when said derivative is outside of a range;

f. recording a second voice signal;

g. selecting a second vocal vibration from said second voice signal;

h. isolating at least four cycles of phonation of said second vocal vibration;

i. computing a second reassigned spectrogram of said second vocal vibration for said isolated phonation cycles;

j. pruning said second reassigned spectrogram to remove unwanted signal elements by computing a mixed partial derivative of the phase of said second reassigned spectrogram and removing data points from said second reassigned spectrogram when said derivative is outside of a range; and

k. comparing said first and said second reassigned spectrograms.

40. The method of claim 39, wherein said first and said second reassigned spectrograms are computed for a range of between about 50 and 3000 Hz.

41. The method of claim 39, further comprising sectioning said at least four cycles of phonation into between about 3 and about 20 partially overlapping time frames.

42. The method of claim 39, wherein said step of selecting said first vocal vibration comprises performing an autocorrelation analysis of said first voice signal and selecting a time during which the autocorrelation exceeds a threshold indicative of a periodic signal.

43. The method of claim 39, wherein said step of selecting said first vocal vibration comprises performing a cepstral analysis of said first voice signal and selecting a time during which the first peak amplitude of said cepstral analysis exceeds a threshold.

44. The method of claim 43, wherein said step of selecting said second vocal vibration comprises comparing the mel-frequency cepstral coefficients of said first vocal vibration to said second voice signal, wherein said second vocal vibration corresponds to the time at which said mel-frequency cepstral coefficients match the most.

45. The method of claim 39, wherein said range is between about −

0.25 and about 0.25.

46. The method of claim 39, wherein said range is between about 0.5 and about 1.5

47. The method of claim 39, wherein a first range is between about −

0.25 and about 0.25, and a second range is between about 0.5 and about 1.5.

48. The method of claim 39, wherein a first range is between about −

0.25 and about 0.25, and a second range is between about 0.75 and about 1.25.

49. The method of claim 39, wherein said range is between about −

0.25 and about 1.5.

50. The method of claim 39, wherein said range is between about −

0.25 and about 1.25.

51. A method of verifying the identity of a person, comprising:

a. generating a first reassigned spectrogram of a selected portion of a first voice sample;

b. generating a second reassigned spectrogram of a selected portion of a second voice sample;

c. pruning unwanted elements from both of said spectrograms by computing mixed partial derivatives of the phases of each of said spectrograms and removing data points from said spectrogram when said derivatives are outside of a range; and

d. comparing said first and said second pruned reassigned spectrograms.

52. The method of claim 51, wherein a first range is between about −

0.25 and about 0.25, and a second range is between about 0.75 and about 1.25.

53. The method of claim 51, wherein said step of comparing said voiceprints comprises displaying said voiceprints on a colormap.