SPECTRAL REFINEMENT SYSTEM

• US 20080195382A1
• Filed: 11/30/2007
• Published: 08/14/2008
• Est. Priority Date: 12/01/2006
• Status: Active Grant

• Current Assignee: Cerence Inc., Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Schmidt, Gerhard Uwe, Krini, Mohamed
• Granted Patent: US 8,190,426 B2
• Time in Patent Office: Days
• US Class Current: 704/203
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/27 characterised by the analysis technique

Abstract

An audio enhancement refines a short-time spectrum. The refinement may reduce overlap between audio sub-bands. The sub-bands are transformed into sub-band short-time spectra. A portion of the spectra are time-delayed. The sub-band short-time spectrum and the time-delayed portion are filtered to obtain a refined sub-band short-time spectrum. The refined spectrum improves audio processing.

Claims

1. A method of processing an audio signal, comprising:

converting the audio signal from a continuous domain to a frequency domain and obtaining sub-band short-time spectra for a predetermined number of sub-bands of the audio signal;

delaying at least one of the sub-band short-time spectra to obtain a predetermined number of time-delayed sub-band short-time spectra for at least one of the predetermined number of sub-bands; and

filtering the sub-band short-time spectrum and the time-delayed sub-band short-time spectra to obtain a refined sub-band short-time spectrum for the at least one of the predetermined number of sub-bands.

2. The method of claim 1, where converting comprises:

windowing the audio signal to a windowed signal; and

discrete Fourier transforming the windowed signal to the sub-band short-time spectra.

3. The method of claim 2, where windowing comprises a Hann window function, a Hamming window function, or a Gaussian window function.

4. The method of claim 1, where filtering comprises selecting a portion of the sub-band short-time spectrum and time-delayed sub-band short-time spectra through a finite impulse response.

5. The method of claim 1, where filtering comprises multiplying filtering coefficients of a refinement matrix with the sub-band short-time spectrum and the time-delayed sub-band short-time spectra.

6. A method of processing an audio signal, comprising:

converting the audio signal from a continuous domain to a frequency domain and obtaining sub-band short-time spectra for a predetermined number of sub-bands of the audio signal;

delaying at least one of the sub-band short-time spectra to obtain a predetermined number of time-delayed sub-band short-time spectra for at least one of the predetermined number of sub-bands;

selecting neighbored sub-bands of the sub-band short-time spectra;

filtering, for each pair of neighbored sub-bands, the sub-band short-time spectrum and the time-delayed sub-band short-time spectra to obtain a first filtered spectrum and a second filtered spectrum; and

adding the first and second filtered spectra to obtain a refined sub-band short-time spectrum for each pair of neighbored sub-bands.

7. The method of claim 6, where filtering for each pair of neighbored sub-bands comprises multiplying filtering coefficients of a refinement matrix with the sub-band short-time spectrum and the time-delayed sub-band short-time spectra.

8. The method of claim 6, where converting comprises:

windowing the audio signal to a windowed signal; and

discrete Fourier transforming the windowed signal to the sub-band short-time spectra.

9. The method of claim 8, where windowing comprises a Hann window function, a Hamming window function, or a Gaussian window function.

10. The method of claim 6, where filtering for each pair of neighbored sub-bands comprises selecting a portion of the sub-band short-time spectrum and time-delayed sub-band short-time spectra through a finite impulse response.

11. A method of processing an audio signal, comprising:

determining a degree of stationarity of the audio signal;

filtering the audio signal to obtain filtered sub-band short-time spectra, if the degree of stationarity is below a predetermined threshold;

if the degree of stationarity is equal to or greater than the predetermined threshold;

converting the audio signal from a continuous domain to a frequency domain and obtaining sub-band short-time spectra for a predetermined number of sub-bands of the audio signal;

delaying at least one of the sub-band short-time spectra to obtain a predetermined number of time-delayed sub-band short-time spectra for at least one of the predetermined number of sub-bands;

filtering the sub-band short-time spectrum and the time-delayed sub-band short-time spectra to obtain a refined sub-band short-time spectrum for the at least one of the predetermined number of sub-bands; and

filtering the refined sub-band short-time spectrum to obtain the filtered sub-band short-time spectra;

converting the filtered sub-band short-time spectra from the frequency domain to the continuous domain and obtaining an intermediate audio signal; and

synthesizing the intermediate audio signal to obtain an output audio signal.

12. The method of claim 11, where the output audio signal comprises a noise reduced signal or an echo reduced signal.

13. The method of claim 11, where converting the filtered sub-band short-time spectra comprises inverse Fourier transforming the filtered sub-band short-time spectra to the intermediate audio signal.

14. The method of claim 11, where converting the audio signal comprises:

windowing the audio signal to a windowed signal; and

discrete Fourier transforming the windowed signal to the sub-band short-time spectra.

15. The method of claim 11, where filtering the sub-band short-time spectrum and the time-delayed sub-band short-time spectra comprises selecting a portion of the sub-band short-time spectrum and time-delayed sub-band short-time spectra through a finite impulse response.

16. A method of processing an audio signal, comprising:

converting the audio signal from a continuous domain to a frequency domain and obtaining sub-band short-time spectra for a predetermined number of sub-bands of the audio signal;

delaying at least one of the sub-band short-time spectra to obtain a predetermined number of time-delayed sub-band short-time spectra for at least one of the predetermined number of sub-bands;

filtering the sub-band short-time spectrum and the time-delayed sub-band short-time spectra to obtain a refined sub-band short-time spectrum for the at least one of the predetermined number of sub-bands;

determining a short-time spectrogram of the refined sub-band short-time spectrum; and

estimating a pitch of the audio signal, based on the short-time spectrogram.

17. A system for processing an audio signal, comprising:

transformation logic that converts the audio signal from a continuous domain to a frequency domain and generates sub-band short-time spectra for a predetermined number of sub-bands of the audio signal;

delay logic that time shifts at least one of the sub-band short-time spectra to obtain a predetermined number of time-delayed sub-band short-time spectra for at least one of the predetermined number of sub-bands; and

refinement logic that filters the sub-band short-time spectrum and the time-delayed sub-band short-time spectra to obtain a refined sub-band short-time spectrum for the at least one of the predetermined number of sub-bands.

18. The system of claim 17, where the transformation logic comprises:

windowing logic that selects portions of the audio signal to a windowed signal; and

conversion logic that discrete Fourier transforms the windowed signal to the sub-band short-time spectra.

19. The system of claim 18, where the windowing logic comprises a Hann window function, a Hamming window function, or a Gaussian window function.

20. The system of claim 17, where the refinement logic comprises a finite impulse response filter.

21. The system of claim 17, where the refinement logic comprises a first multiplication logic that multiplies filtering coefficients of a refinement matrix with the sub-band short-time spectrum and the time-delayed sub-band short-time spectra.

22. The system of claim 17, further comprising:

interpolation logic that filters the sub-band short-time spectrum and the time-delayed sub-band short-time spectra for each pair of selected neighbored sub-bands to obtain a first filtered spectrum and a second filtered spectrum; and

an adder that sums the first and second filtered spectra to obtain an additional sub-band short-time spectrum for each pair of the selected neighbored sub-bands.

23. The system of claim 22, where the interpolation logic comprises a second multiplication circuit that multiplies filtering coefficients of a refinement matrix with the sub-band short-time spectrum and the time-delayed sub-band short-time spectra.

24. The system of claim 17, further comprising:

change analysis logic that determines a degree of stationarity of the audio signal;

sub-threshold stationarity logic that filters the audio signal to obtain filtered sub-band short-time spectra, if the degree of stationarity is below a predetermined threshold;

super-threshold stationarity logic that filters the refined sub-band short-time spectrum to obtain the filtered sub-band short-time spectra, if the degree of stationarity is equal to or greater than the predetermined threshold; and

inverse conversion logic that transforms the filtered sub-band short-time spectra from the frequency domain to the continuous domain to obtain an output audio signal, the output audio signal comprising a noise reduced signal or an echo reduced signal.

25. The system of claim 17, further comprising:

frequency analysis logic that determines a short-time spectrogram of the refined sub-band short-time spectrum; and

sound analysis logic that estimates a pitch of the audio signal, based on the short-time spectrogram.