METHOD AND SYSTEM FOR RECONSTRUCTING SPEECH FROM AN INPUT SIGNAL COMPRISING WHISPERS

• US 20120150544A1
• Filed: 08/25/2010
• Published: 06/14/2012
• Est. Priority Date: 08/25/2009
• Status: Abandoned Application

• Current Assignee: Nanyang Technological University
• Original Assignee: Nanyang Technological University
• Inventors: Sharifzadeh, Hamid Reza, Ahmadi, Farzaneh, McLoughlin, Ian Vince
• Application Number:
  US13/392,385
• Publication Number:
  US 20120150544A1
• Time in Patent Office: Days
• US Class Current: 704/262
• CPC Class Codes: G10L 2021/0135 Voice conversion or morphing
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters

Abstract

A system for reconstructing speech from an input signal comprising whispers is disclosed. The system comprises an analysis unit configured to analyse the input signal to form a representation of the input signal; an enhancement unit configured to modify the representation of the input signal to adjust a spectrum of the input signal, wherein the adjusting of the spectrum of the input signal comprises modifying a bandwidth of at least one formant in the spectrum to achieve a predetermined spectral energy distribution and amplitude for the at least one formant; and a synthesis unit configured to reconstruct speech from the modified representation of the input signal.

Claims

1. A system for reconstructing speech from an input signal comprising whispers, the system comprising:

an analysis unit configured to analyse the input signal to form a representation of the input signal;

an enhancement unit configured to modify the representation of the input signal to adjust a spectrum of the input signal, wherein the adjusting of the spectrum of the input signal comprises modifying a bandwidth of at least one formant in the spectrum to achieve a predetermined spectral energy distribution and amplitude for the at least one formant; and

a synthesis unit configured to reconstruct speech from the modified representation of the input signal.

2. A system according to claim 1, wherein the system further comprises:

a first pre-processing unit configured to detect speech activity in the input signal; and

a second pre-processing unit configured to classify phonemes in the input signal.

3. A system according to claim 2, wherein the first pre-processing unit comprises a plurality of detection mechanisms whereby an output of the first pre-processing unit is dependent on an output of each of the detection mechanisms.

4. A system according to claim 3, wherein the plurality of detection mechanisms comprise a first detection mechanism based on an energy of the input signal and a second detection mechanism based on a zero crossing rate of the input signal.

5. A system according to of claim 2, wherein the second pre-processing unit is configured to:

compare a power of the input signal in a first range of frequencies against a power of the input signal in a second range of frequencies, the first range of frequencies being lower than the second range of frequencies; and

classify the phonemes in the input signal based on the comparison.

6. A system according to claim 1, wherein the enhancement unit is further configured to locate formants according to the following steps:

obtaining roots of an equation formed by a plurality of Linear Prediction coefficients derived in the analysis unit;

calculating a bandwidth to peak ratio for each root of the equation; and

classifying a predetermined number of the roots lying on the imaginary axis and having smaller bandwidth to peak ratios as the located formants in the spectrum of the input signal.

7. A system according to claim 6, wherein the enhancement unit is further configured to extract the at least one formant from the located formants according to the following steps prior to modifying the bandwidth of the at least one formant:

deriving the probability of a formant occurring at each frequency in the spectrum using the located formants;

locating a plurality of standard frequency bands in the spectrum, each standard frequency band being a frequency band expected to comprise formants;

dividing each standard frequency band in the spectrum into a plurality of narrow frequency bands; and

for each standard frequency band in the spectrum, calculating a density for each narrow frequency band in the standard frequency band as a sum of the derived probabilities in the narrow frequency band and extracting the at least one formant as resonance peaks lying within the narrow frequency band having the highest density.

8. A system according to claim 7, wherein the enhancement unit is further configured to perform the following steps:

smoothing a trajectory of the at least one formant;

filtering the smoothed trajectory of the at least one formant; and

lowering frequencies of the at least one formant;

9. A system according to claim 7, wherein the modified representation of the input signal comprises a plurality of Linear Prediction coefficients derived from the at least one formant and the synthesis unit is configured to reconstruct speech using the plurality of Linear Prediction coefficients.

10. A system according to claim 9, wherein the analysis unit is configured to modify a Long Term Prediction transfer function for inserting pitch into the reconstructed speech based on the classifying of the phonemes in the input signal by the second pre-processing unit.

11. A system according to claim 1, wherein the predetermined spectral energy amplitude is derived based on an estimated difference between a spectral energy of whispered speech and a spectral energy of normally phonated speech.

12. A system according to claim 1, wherein the enhancement unit is configured to modify the bandwidth of the at least one formant while retaining a frequency of the at least one formant.

13. A method for reconstructing speech from an input signal comprising whispers, the method comprising:

analysing the input signal to form a representation of the input signal;

modifying the representation of the input signal to adjust a spectrum of the input signal, wherein the adjusting of the spectrum of the input signal comprises modifying a bandwidth of at least one formant in the spectrum to achieve a predetermined spectral energy distribution and amplitude for the at least one formant; and

reconstructing speech from the modified representation of the input signal.

14. A method according to claim 13, wherein prior to analysing the input signal, the method further comprises:

detecting speech activity in the input signal; and

classifying phonemes in the input signal.

15. A method according to claim 14, wherein the detecting of the speech activity in the input signal is performed using a plurality of detection mechanisms whereby an output of the detecting of the speech activity in the input signal is dependent on an output of each of the detection mechanisms.

16. A method according to claim 15, wherein the plurality of detection mechanisms comprise a first detection mechanism based on an energy of the input signal and a second detection mechanism based on a zero crossing rate of the input signal.

17. A method according to claim 14, wherein the classifying of the phonemes in the input signal comprises:

comparing a power of the input signal in a first range of frequencies against a power of the input signal in a second range of frequencies, the first range of frequencies being lower than the second range of frequencies; and

classifying the phonemes in the input signal based on the comparison.

18. A method according to claim 13, the method further comprising locating formants according to the following steps:

obtaining roots of an equation formed by a plurality of Linear Prediction coefficients derived from the analysing of the input signal;

calculating a bandwidth to peak ratio for each root of the equation; and

classifying a predetermined number of the roots lying on the imaginary axis and having smaller bandwidth to peak ratios as the located formants in the spectrum of the input signal.

19. A method according to claim 18, the method further comprising extracting the at least one formant from the located formants according to the following steps prior to modifying the bandwidth of the at least one formant:

deriving the probability of a formant occurring at each frequency in the spectrum using the located formants;

locating a plurality of standard frequency bands in the spectrum, each standard frequency band being a frequency band expected to comprise formants;

dividing each standard frequency band in the spectrum into a plurality of narrow frequency bands; and

for each standard frequency band in the spectrum, calculating a density for each narrow frequency band in the standard frequency band as a sum of the derived probabilities in the narrow frequency band and extracting the at least one formant as resonance peaks lying within the narrow frequency band having the highest density.

20. A method according to claim 19, wherein the adjusting of the spectrum of the input signal further comprises:

smoothing a trajectory of the at least one formant;

filtering the smoothed trajectory of the at least one formant; and

lowering frequencies of the at least one formant;

21. A method according to claim 19, wherein the modified representation of the input signal comprises a plurality of Linear Prediction coefficients derived from the at least one formant and the reconstructing of speech from the spectrally adjusted analysed input signal further comprises reconstructing speech using the plurality of Linear Prediction coefficients.

22. A method according to claim 21, wherein the analysing of the input signal further comprises modifying a Long Term Prediction transfer function for inserting pitch into the reconstructed speech based on the classifying of the phonemes in the input signal.

23. A method according to claim 13, wherein the predetermined spectral energy amplitude is derived based on an estimated difference between a spectral energy of whispered speech and a spectral energy of normally phonated speech.

24. A method according to claim 13, wherein the bandwidth of the at least one formant is modified while retaining a frequency of the at least one formant.