Pitch detection of speech signals

• US 7,660,718 B2
• Filed: 09/23/2004
• Issued: 02/09/2010
• Est. Priority Date: 09/26/2003
• Status: Active Grant

• Current Assignee: STMicroelectronics Asia Pacific Pte Limited (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Asia Pacific Pte Limited (STMicroelectronics NV)
• Inventors: Padhi, Kabi Prakash, George, Sapna
• Primary Examiner(s): Hudspeth; David R
• Assistant Examiner(s): Jackson; Jakieda R
• Application Number:
  US10/948,950
• Publication Number:
  US 20050149321A1
• Time in Patent Office: 1,965 Days
• Field of Search: 704/268, 704/205, 704/207
• US Class Current: 704/268
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

Pitch detection of speech signals finds numerous applications in karaoke, voice recognition and scoring applications. While most of the existing techniques rely on time domain methods, the invention utilizes frequency domain methods. There is provided a method and system for determining the pitch of speech from a speech signal. The method includes the steps of: producing or obtaining the speech signal; distinguishing the speech signal into voiced, unvoiced or silence sections using speech signal energy levels; applying a Fourier Transform to the speech signal and obtaining speech signal parameters; determining peaks of the Fourier transformed speech signal; tracking the speech signal parameters of the determined peaks to select partials; and determining the pitch from the selected partials using a two-way mismatch error calculation.

Claims

1. A system for determining a pitch of speech from a speech signal, the system including:

(1) an input device to receive the speech and generate the speech signal; and

(2) a processor structured to;

(a) distinguish the speech signal into voiced, unvoiced or silenced sections using speech signal energy levels;

(b) apply a Fourier Transform to the voiced speech signal section and obtain speech signal parameters;

(c) determine peaks of the Fourier transformed voiced speech signal section;

(d) select partials by tracking the speech signal parameters of the determined peaks over a plurality of frames of the speech signal to determine trajectories; and

(e) determine the pitch from the selected partials using a two-way mismatch error calculation, the two-way mismatch error calculation including;

setting a trial fundamental frequency (ƒ

_fund);

determining a plurality of predicted harmonics corresponding to the trial fundamental frequency;

for one of the plurality of predicted harmonics, determining if any of the selected partials is within (ƒ

_fund/2) of the predicted harmonic;

setting a harmonic frequency error equal to a frequency value of the predicted harmonic in response to determining that none of the selected partials is within (ƒ

_fund/2) of the predicted harmonic; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the harmonic frequency error.

2. The system according to claim 1, wherein the speech signal is a coded, compressed or real-time audio or data signal.

3. The system according to claim 1, adapted to perform real-time processing of live speech signals.

4. The system according to claim 1, wherein the speech signal is a Pulse Code Modulated signal.

5. The system according to claim 1, wherein the system is incorporated into a karaoke system, computer system or voice recognition system.

6. The system according to claim 1, wherein the input device is a microphone or audio receiver.

7. A method of determining a pitch of speech from a speech signal, the method including the steps of:

obtaining the speech signal;

that has been received at a microphone

distinguishing the speech signal into voiced, unvoiced or silenced sections using speech signal energy levels;

applying a Fourier Transform to the voiced speech signal section and obtaining speech signal parameters;

determining peaks of the Fourier transformed voiced speech signal section;

selecting partials by tracking the speech signal parameters of the determined peaks over a plurality of frames of the speech signal to determine trajectories; and

determining the pitch from the selected partials using a two-way mismatch error calculation, the two-way mismatch error calculation including;

setting a trial fundamental frequency (ƒ

_fund);

determining a plurality of predicted harmonics corresponding to the trial fundamental frequency;

for one of the plurality of predicted harmonics, determining if any of the selected partials is within (ƒ

_fund/2) of the predicted harmonic;

setting a harmonic frequency error equal to a frequency value of the predicted harmonic in response to determining that none of the selected partials is within (ƒ

_fund/2) of the predicted harmonic; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the harmonic frequency error.

8. The method according to claim 7, wherein prior to applying the Fourier Transform a windowing procedure is applied to the voiced speech signal section.

9. The method according to claim 8, wherein the windowing procedure utilizes a Blackman window, a Kaiser window, a Raised Cosine window or other sinusoidal models.

10. The method according to claim 7, wherein applying the Fourier Transform comprises applying the Fourier Transform to a frame of the voiced speech signal section.

11. The method according to claim 10, wherein the frame is one of a plurality of overlapping frames.

12. The method according to claim 10, wherein the signal parameters are tracked over the plurality of frames of the voiced speech signal section.

13. The method according to claim 12, wherein the trajectories persisting over more than one frame of the plurality of frames are utilized.

14. The method according to claim 7, wherein the Fourier Transform is a Fast Fourier Transform.

15. The method according to claim 7, wherein the speech signal parameters are frequency, phase and amplitude.

16. The method according to claim 7, wherein a zero padding procedure is used in determining the peaks of the Fourier transformed voiced speech signal section.

17. The method according to claim 7, wherein a frequency of a determined peak falling within a specified frequency range of a frequency of a harmonic of the pitch is set equal to the frequency of the harmonic.

18. The method according to claim 7, wherein the peaks are determined in an amplitude spectrum.

19. The method according to claim 18, wherein the peaks are determined in the amplitude spectrum using a logarithmic scale.

20. The method according to claim 7, wherein the partials are selected from the determined peaks based on a greatest common divisor of a maximum number of partials in a voiced speech signal section spectrum.

21. The method according to claim 7, wherein the two-way mismatch error calculation further includes, if a nearest of the selected partials is within (ƒ

_fund/2) of the predicted harmonic, setting the harmonic frequency error equal to an absolute value of a frequency value of the nearest selected partial subtracted from the frequency value of the predicted harmonic.

22. The method according to claim 21, wherein the two-way mismatch error calculation further includes:

for one of the selected partials, determining if any of the plurality of the predicted harmonics is within (ƒ

_fund/2) of the selected partial;

setting a partial frequency error equal to a frequency value of the selected partial in response to determining that none of the predicted harmonics is within (ƒ

_fund/2) of the selected partial; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the partial frequency error.

23. The method according to claim 22, wherein the two-way mismatch error calculation further includes, if a nearest of the plurality of predicted harmonics is within (ƒ

_fund/2) of the selected partial, setting the partial frequency error equal to an absolute value of a frequency value of the nearest predicted harmonic subtracted from the frequency value of the selected partial.

24. The method according to claim 7, wherein the speech signal energy levels are short-term signal energy levels.

25. The method according to claim 7, wherein distinguishing the speech signal further comprises utilizing an energy estimation calculation.

26. The method according to claim 7, wherein the speech signal corresponds to a sum of sinusoids of varying amplitudes in frequency domain that extends from a minimum speech signal frequency (ƒ

_min) to a maximum speech signal frequency (ƒ

_max), and further comprising limiting a pitch search for domain space speech signal parameters to a maximum speech search frequency (ƒ

_search-max) that is less than the maximum speech signal frequency (ƒ

_max).

27. The method according to claim 26, wherein limiting a pitch search for domain space speech signal parameters to a maximum speech search frequency (ƒ

_search-max) that is less than the maximum speech signal frequency (ƒ

_max) includes limiting the pitch search to a frequency range of about 50-500 Hz.

28. A system for determining a pitch of speech from a speech signal, the system comprising:

(1) a processor structured to;

(a) distinguish the speech signal into voiced, unvoiced or silenced speech signal sections using speech signal energy levels;

(b) apply a windowing procedure to the voiced speech signal section to generate a frame;

(c) apply a Fourier Transform to the frame and obtain speech signal parameters;

(d) determine peaks of the Fourier transformed frame;

(e) select partials by tracking the speech signal parameters of the determined peaks over a plurality of frames of the speech signal to determine trajectories; and

(f) determine the pitch from the selected partials using a two-way mismatch error calculation, the two-way mismatch error calculation including;

setting a trial fundamental frequency (ƒ

_fund);

determining a plurality of predicted harmonics corresponding to the trial fundamental frequency;

for one of the plurality of predicted harmonics, determining if any of the selected partials is within (ƒ

_fund/2) of the predicted harmonic;

setting a harmonic frequency error equal to a frequency value of the predicted harmonic in response to determining that none of the selected partials is within (ƒ

_fund/2) of the predicted harmonic; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the harmonic frequency error.

29. The system of claim 28, wherein the windowing procedure utilizes a Blackman window, a Kaiser window, a Raised Cosine window or other sinusoidal models.

30. The system of claim 28, wherein the frame is one of a plurality of overlapping frames.

31. The system of claim 28, wherein the signal parameters are tracked over the plurality of frames of the voiced speech signal section.

32. The system of claim 31, wherein the trajectories persisting over more than one frame of the plurality of frames are utilized.

33. The system of claim 28, wherein the Fourier Transform is a Fast Fourier Transform.

34. The system of claim 28, wherein the processor is further adapted to determine peaks of the Fourier transformed frame using a zero padding procedure.

35. The system of claim 28, wherein the processor is further adapted to set a frequency of a determined peak falling within a specified frequency range of a frequency of a harmonic of the pitch equal to the frequency of the harmonic.

36. The system of claim 28, wherein the processor is further configured to select partials from the determined peaks based on a greatest common divisor of a maximum number of partials in the Fourier transformed frame.

37. The system of claim 28, wherein the two-way mismatch error calculation further includes, if a nearest of the selected partials is within (ƒ

_fund/2) of the predicted harmonic, setting the harmonic frequency error equal to an absolute value of a frequency value of the nearest selected partial subtracted from the frequency value of the predicted harmonic.

38. The system of claim 37, the two-way mismatch error calculation further includes:

for one of the selected partials, determining if any of the plurality of predicted harmonics is within (ƒ

_fund/2) of the selected partial;

setting a partial frequency error equal to a frequency value of the selected partial in response to determining that none of the predicted harmonics is within (ƒ

_fund/2) of the selected partial; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the partial frequency error.

39. A system for estimating a pitch of speech from a speech signal, the system including:

(1) a memory unit adapted to communicate required data to a processing unit; and

(2) the processing unit operating on the speech signal and structured to;

(a) section the speech signal into voiced, unvoiced or silenced sections using speech signal energy levels;

(b) apply a Fast Fourier Transform to the voiced speech signal section and generate speech signal parameters;

(c) determine peaks of the Fourier transformed voiced speech signal section;

(d) select partials by tracking the speech signal parameters of the determined peaks over a plurality of frames of the speech signal to determine trajectories; and

(e) calculate the pitch from the selected partials using a two-way mismatch error calculation, the two-way mismatch error calculation including;

setting a trial fundamental frequency (ƒ

_fund);

determining a plurality of predicted harmonics corresponding to the trial fundamental frequency;

for one of the plurality of predicted harmonics, determining if any of the selected partials is within (ƒ

_fund/2) of the predicted harmonic;

setting a harmonic frequency error equal to a frequency value of the predicted harmonic in response to determining that none of the selected partials is within (ƒ

_fund/2) of the predicted harmonic; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the harmonic frequency error.

40. The system as claimed in claim 39, wherein the Fast Fourier Transform operates on a frame of a windowed portion of the speech signal.

41. A system for determining a pitch of speech from a speech signal, comprising:

means for obtaining the speech signal;

means for distinguishing the speech signal into voiced, unvoiced or silenced speech signal sections using speech signal energy levels;

means for applying a Fourier Transform to the voiced speech signal section and obtaining speech signal parameters;

means for determining peaks of the Fourier transformed voiced speech signal section;

means for selecting partials by tracking the speech signal parameters of the determined peaks over a plurality of frames of the speech signal to determine trajectories; and

means for determining the pitch from the selected partials using a two-way mismatch error calculation, the two-way mismatch error calculation including;

setting a trial fundamental frequency (ƒ

_fund);

determining a plurality of predicted harmonics corresponding to the trial fundamental frequency;

for one of the plurality of predicted harmonics, determining if any of the selected partials is within (ƒ

_fund/2) of the predicted harmonic;

setting a harmonic frequency error equal to a frequency value of the predicted harmonic in response to determining that none of the selected partials is within (ƒ

_fund/2) of the predicted harmonic; and

determining whether to set the pitch equal to the trial fundamental frequency based at least in part on the harmonic frequency error.