Fast frequency-domain pitch estimation

• US 6,587,816 B1
• Filed: 07/14/2000
• Issued: 07/01/2003
• Est. Priority Date: 07/14/2000
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Hoory, Ron, Chazan, Dan, Zibulski, Meir
• Primary Examiner(s): Banks-Harold, Marsha D.
• Assistant Examiner(s): Storm, Donald L.
• Application Number:
  US09/617,582
• Time in Patent Office: 1,082 Days
• Field of Search: 704/207, 704/203, 704/204
• US Class Current: 704/207
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A method for estimating a pitch frequency of an audio signal includes computing a first transform of the signal to a frequency domain over a first time interval, and computing a second transform of the signal to the frequency domain over a second time interval, which contains the first time interval. A line spectrum of the signal is found, based on the first and second transforms, the spectrum including spectral lines having respective line amplitudes and line frequencies. A utility function that is periodic in the frequencies of the lines in the spectrum is then computed. This function is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency. The pitch frequency of the speech signal is estimated responsive to the utility function.

Claims

1. A method for estimating a pitch frequency of a speech signal, comprising:

computing a first transform of the speech signal to a frequency domain over a first time interval;

computing a second transform of the speech signal to the frequency domain over a second time interval, which contains the first time interval; and

estimating the pitch frequency of the speech signal responsive to the first and second transforms,

wherein the first and second transforms comprise Short Time Fourier Transforms.

2. A method according to claim 1, wherein the first time interval comprises a current frame of the speech signal, and the second time interval comprises the current frame and a preceding frame, and wherein computing the second transform comprises combining the first transform with a transform computed over the preceding frame.

3. A method according to claim 2, wherein the transforms generate respective spectral coefficients, and wherein combining the first transform with the transform computed over the preceding frame comprises applying a phase shift to the coefficients generated by the transform computed over the preceding frame and adding the phase-shifted coefficients to the coefficients generated by the first transform.

4. A method according to claim 3, wherein for a given frequency, the phase shift applied to the corresponding coefficient is proportional to the frequency and to a duration of the frame.

5. A method according to claim 1, wherein estimating the pitch frequency comprises deriving first and second line spectra of the signal from the first and second transforms, respectively, and determining the pitch frequency based on the line spectra.

6. A method according to claim 5, wherein determining the pitch frequency comprises deriving first and second candidate pitch frequencies from the first and second line spectra, respectively, and choosing one of the first and second candidates as the pitch frequency.

7. A method according to claim 6, wherein deriving the first and second candidates comprises defining high and low ranges of possible pitch frequencies, and finding the first candidate in the high range and the second candidate in the low range.

8. A method according to claim 5, wherein the line spectra comprise spectral lines having respective line frequencies, and wherein determining the pitch frequency comprises computing a function that is periodic in the line frequencies, which function is indicative of the pitch frequency.

9. A method according to claim 1, and comprising encoding the speech signal responsive to the estimated pitch frequency.

10. A method for estimating a pitch frequency of a speech signal, comprising:

finding a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies;

computing a utility function, which is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency, the utility function comprising at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency; and

estimating the pitch frequency of the speech signal responsive to the utility function.

11. A method according to claim 10, wherein computing the at least one influence function comprises computing a function of the ratio having maxima at integer values of the ratio and minima therebetween.

12. A method according to claim 10, wherein computing the at least one influence function comprises computing respective influence functions for multiple lines in the spectrum, and wherein computing the utility function comprises computing a superposition of the influence functions.

13. A method according to claim 10, wherein estimating the pitch frequency comprises choosing a candidate pitch frequency at which the utility function has a local maximum.

14. A method according to claim 13, wherein the candidate pitch frequency is one of a plurality of frequencies at which the utility function has local maxima, and wherein choosing the candidate pitch frequency comprises preferentially selecting one of the maxima because it has a higher frequency than another one of the maxima.

15. A method according to claim 13, wherein the candidate pitch frequency is one of a plurality of frequencies at which the utility function has local maxima, and wherein choosing the candidate pitch frequency comprises preferentially selecting one of the maxima because it is near in frequency to a previously estimated pitch frequency of a preceding frame of the speech signal.

16. A method according to claim 13, and comprising determining whether the speech signal is voiced or unvoiced by comparing a value of the local maximum to a predetermined threshold.

17. A method according to claim 10, and comprising encoding the speech signal responsive to the estimated pitch frequency.

18. A method for estimating a pitch frequency of a speech signal, comprising:

finding a line spectrum of the signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies;

computing a utility function that is periodic in the frequencies of the lines in the spectrum, which function is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency; and

estimating the pitch frequency of the speech signal responsive to the utility function,

wherein computing the utility function comprises computing at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and

wherein computing the at least one influence function comprises computing a function of the ratio having maxima at integer values of the ratio and minima therebetween, and

wherein computing the function of the ratio comprises computing values of a piecewise linear function c(f), having a maximum value in a first interval surrounding f=0, a minimum value in a second interval surrounding f=1/2, and a value that varies linearly in a transition interval between the first and second intervals.

19. A method for estimating a pitch frequency of a speech signal, comprising:

finding a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies;

computing a utility function that is periodic in the frequencies of the lines in the spectrum, which function is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency; and

estimating the pitch frequency of the speech signal responsive to the utility function,

wherein computing the utility function comprises computing at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and

wherein computing the at least one influence function comprises computing respective influence functions for multiple lines in the spectrum, and wherein computing the utility function comprises computing a superposition of the influence functions, and

wherein the respective influence functions comprise piecewise linear functions having break points, and wherein computing the superposition comprises calculating values of the influence functions at the break points, such that the utility function is determined by interpolation between the break points.

20. A method according to claim 19, wherein computing the respective influence functions comprises computing at least first and second influence functions for first and second lines in the spectrum in succession, and wherein computing the utility function comprises computing a partial utility function including the first influence function and then adding the second influence function to the partial utility function by calculating the values of the second influence function at the break points of the partial utility function and calculating the values of the partial utility function at the break points of the second influence function.

21. A method for estimating a pitch frequency of a speech signal, comprising:

finding a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies;

computing a utility function that is periodic in the frequencies of the lines in the spectrum, which function is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency; and

estimating the pitch frequency of the speech signal responsive to the utility function,

wherein computing the utility function comprises computing at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and

wherein computing the at least one influence function comprises computing respective influence functions for multiple lines in the spectrum, and wherein computing the utility function comprises computing a superposition of the influence functions, and

wherein computing the respective influence functions comprises performing the following steps iteratively over the lines in the spectrum;

computing a first influence function for a first line in the spectrum;

responsive to the first influence function, identifying one or more intervals in the pitch frequency range that are incompatible with the spectrum;

defining a reduced pitch frequency range from which the one or more intervals have been eliminated; and

computing a second influence function for a second line in the spectrum, while substantially restricting computation of the second influence function to pitch frequencies within the reduced range.

22. A method according to claim 21, wherein computing the superposition comprises calculating a partial utility function including the first influence function but not including the second influence function, and wherein identifying the one or more intervals comprises eliminating the intervals in which the partial utility function is below a specified level.

23. A method according to claim 22, wherein the specified level is determined responsive to the line amplitudes of the lines in the spectrum that are not included in the partial utility function.

24. A method according to claim 21, wherein performing the steps iteratively comprises iterating over the lines in the spectrum in order of decreasing amplitude.

25. Apparatus for estimating a pitch frequency of a speech signal, comprising an audio processor, which is adapted to compute a first transform of the speech signal to a frequency domain over a first time interval and a second transform of the speech signal to a frequency domain over a second time interval, which contains the first time interval, and to estimate the pitch frequency of the speech signal responsive to the first and second frequency transforms,

wherein the first and second transforms comprise Short Time Fourier Transforms.

26. Apparatus according to claim 25, wherein the first time interval comprises a current frame of the speech signal, and the second time interval comprises the current frame and a preceding frame, and wherein the processor is adapted to compute the second transform by combining the first transform with a transform computed over the preceding frame.

27. Apparatus according to claim 25, wherein the processor is further adapted to encode the speech signal responsive to the estimated pitch frequency.

28. Apparatus for estimating a pitch frequency of a speech signal, comprising an audio processor, which is adapted to compute a first transform of the speech signal to a frequency domain over a first time interval and a second transform of the speech signal to a frequency domain over a second time interval, which contains the first time interval, and to estimate the pitch frequency of the speech signal responsive to the first and second frequency transforms,

wherein the first time interval comprises a current frame of the speech signal, and the second time interval comprises the current frame and a preceding frame, and wherein the processor is adapted to compute the second transform by combining the first transform with a transform computed over the preceding frame, and

wherein the transforms generate respective spectral coefficients, and wherein the processor is adapted to apply a phase shift to the coefficients generated by the transform computed over the preceding frame and to add the phase-shifted coefficients to the coefficients generated by the transform computed over the first time interval.

29. Apparatus according to claim 28, wherein for a given frequency, the phase shift applied to the corresponding coefficient is proportional to the frequency and to a duration of the frame.

30. Apparatus for estimating a pitch frequency of a speech signal, comprising an audio processor, which is adapted to compute a first transform of the speech signal to a frequency domain over a first time interval and a second transform of the speech signal to a frequency domain over a second time interval, which contains the first time interval, and to estimate the pitch frequency of the speech signal responsive to the first and second frequency transforms,

wherein the processor is adapted to derive first and second line spectra of the signal from the first and second transforms, respectively, and to determine the pitch frequency based on the line spectra.

31. Apparatus according to claim 30, wherein the processor is adapted to derive first and second candidate pitch frequencies from the first and second line spectra, respectively, and to choose one of the first and second candidates as the pitch frequency.

32. Apparatus according to claim 31, wherein high and low ranges of possible pitch frequencies are defined, and the processor is adapted to derive the first candidate in the high range and the second candidate in the low range.

33. Apparatus according to claim 30, wherein the line spectra comprise spectral lines having respective line frequencies, and wherein the processor is adapted to generate a function that is periodic in the line frequencies, which function is indicative of the pitch frequency.

34. Apparatus for estimating a pitch frequency of a speech signal, comprising an audio processor, which is adapted to find a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies, to compute a utility function, which is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency, the utility function comprising at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and to estimate the pitch frequency of the speech signal responsive to the periodic function.

35. Apparatus according to claim 34, wherein the at least one influence function comprises a function of the ratio having maxima at integer values of the ratio and minima therebetween.

36. Apparatus according to claim 34, wherein the processor is adapted to compute respective influence functions for multiple lines in the spectrum, and to compute the utility function by finding a superposition of the influence functions for use in estimating the pitch frequency.

37. Apparatus according to claim 36, wherein the influence functions comprise piecewise linear functions having break points, and wherein the processor is adapted to calculate values of the influence functions at the break points, such that the utility function is determined by interpolation between the break points.

38. Apparatus according to claim 37, wherein the influence functions comprise at least first and second influence functions, computed for first and second lines in the spectrum in succession, and wherein the processor is adapted to compute a partial utility function including the first influence function and then to add the second influence function to the partial utility function by calculating the values of the second influence function at the break points of the partial utility function and calculating the values of the partial utility function at the break points of the second influence function.

39. Apparatus according to claim 36, wherein the processor is adapted to perform the following steps iteratively over the lines in the spectrum:

computing a first influence function for a first line in the spectrum;

responsive to the first influence function, identifying one or more intervals in the pitch frequency range that are incompatible with the spectrum;

defining a reduced pitch frequency range from which the one or more intervals are eliminated; and

computing a second influence function for a second line in the spectrum, while substantially restricting computation of the second influence function to pitch frequencies within the reduced range.

40. Apparatus according to claim 39, wherein the processor is adapted to calculate a partial utility function including the first influence function but not including the second influence function, and to eliminate the intervals in which the partial utility function is below a specified level from consideration in computing the second influence function.

41. Apparatus according to claim 40, wherein the specified level is determined responsive to the line amplitudes of the lines in the spectrum that are not included in the partial utility function.

42. Apparatus according to claim 39, wherein the processor is adapted to iterate over the lines in the spectrum in order of decreasing amplitude.

43. Apparatus according to claim 34, wherein the estimated pitch frequency comprises a pitch frequency at which the utility function has a local maximum.

44. Apparatus according to claim 43, wherein the candidate pitch frequency is one of a plurality of frequencies at which the utility function has local maxima, and wherein the processor is adapted to preferentially select as the candidate pitch frequency one of the maxima because it has a higher frequency than another one of the maxima.

45. Apparatus according to claim 43, wherein the candidate pitch frequency is one of a plurality of frequencies at which the periodic function has local maxima, and wherein the processor is adapted to preferentially select as the candidate pitch frequency one of the maxima because it is near in frequency to a previously-estimated pitch frequency of a preceding frame of the speech signal.

46. Apparatus according to claim 43, wherein the processor is adapted to determine whether the speech signal is voiced or unvoiced by comparing a value of the local maximum to a predetermined threshold.

47. Apparatus according to claim 34, wherein the processor is further adapted to encode the speech signal responsive to the estimated pitch frequency.

48. Apparatus for estimating a pitch frequency of a speech signal, comprising an audio processor, which is adapted to find a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies, to compute a utility function that is periodic in the frequencies of the lines in the spectrum, which function is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency, and to estimate the pitch frequency of the speech signal responsive to the periodic function,

wherein the utility function comprises at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and

wherein the at least one influence function comprises a function of the ratio having maxima at integer values of the ratio and minima therebetween, and

wherein the at least one influence function comprises a piecewise linear function c(f), having a maximum value in a first interval surrounding f=0, a minimum value in a second interval surrounding f=1/2, and a value that varies linearly in a transition interval between the first and second intervals.

49. A computer software product, comprising a computer-readable storage medium in which program instructions are stored, which instructions, when read by a computer receiving a speech signal, cause the computer to compute a first transform of the speech signal to a frequency domain over a first time interval and a second transform of the speech signal over a second time interval to the frequency domain, which contains the first time interval, and to estimate the pitch frequency of the speech signal responsive to the first and second transforms,

wherein the first and second transforms comprise Short Time Fourier Transforms.

50. A product according to claim 49, wherein the instructions further cause the computer to encode the speech signal responsive to the estimated pitch frequency.

51. A computer software product, comprising a computer-readable storage medium in which program instructions are stored, which instructions, when read by a computer receiving a speech signal, cause the computer to find a line spectrum of the speech signal, the spectrum comprising spectral lines having respective line amplitudes and line frequencies, to compute a utility function, which is indicative, for each candidate pitch frequency in a given pitch frequency range, of a compatibility of the spectrum with the candidate pitch frequency, the utility function comprising at least one influence function that is periodic in a ratio of the frequency of one of the spectral lines to the candidate pitch frequency, and to estimate the pitch frequency of the speech signal responsive to the periodic function.

52. A product according to claim 51, wherein the instructions further cause the computer to encode the speech signal responsive to the estimated pitch frequency.