Voice activity detection method and method used for voice activity detection and apparatus thereof

• US 9,672,841 B2
• Filed: 06/30/2015
• Issued: 06/06/2017
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: ZTE Corporation
• Original Assignee: ZTE Corporation
• Inventors: Jiang, Dongping, Zhu, Changbao, Yuan, Hao
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/754,714
• Publication Number:
  US 20170004840A1
• Time in Patent Office: 707 Days
• Field of Search: None
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 21/0224 Processing in the time domain
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/81 for discriminating voice from music
• CPC Class Codes: G10L 25/84 for discriminating voice from noise

Abstract

The present document relates to a voice activity detection (VAD) method and methods used for voice activity detection and apparatus thereof, the VAD method includes: obtaining sub-band signals and spectrum amplitudes of a current frame; computing values of a energy feature and a spectral centroid feature of the current frame according to the sub-band signals; computing a signal to noise ratio parameter of the current frame according to a background noise energy estimated from a previous frame, an energy of SNR sub-bands and a energy feature of the current frame; computing a VAD decision result according to a tonality signal flag, a signal to noise ratio parameter, a spectral centroid feature, and a frame energy feature. The methods and apparatus of the present document can improve the accuracy of non-stationary noise (such as office noise) and music detection.

Claims

1. A voice activity detection (VAD) method applied in speech coding, wherein, the method comprises:

obtaining sub-band signals and spectrum amplitudes of a current frame;

computing values of a energy feature, a spectral centroid feature and a time-domain stability feature of the current frame by using the sub-band signals;

computing values of a spectral flatness feature and a tonality feature according to the spectrum amplitudes;

computing a signal to noise ratio (SNR) parameter of the current frame with a background energy estimated from a previous frame, the energy feature and an energy of SNR sub-bands of the current frame;

computing a tonality signal flag of the current frame with the energy feature, the spectral centroid feature, the time-domain stability feature, the spectral flatness feature and the tonality feature of the current frame;

computing a VAD decision result with the tonality signal flag, the signal to noise ratio parameter, the spectral centroid feature, and the energy feature.

2. The method of claim 1, wherein, before or after obtaining a VAD decision result, the method further comprises:

computing an initial background update flag of the current frame by using the energy feature, the spectral centroid feature, the time-domain stability feature, the spectral flatness feature, and the tonality feature of the current frame;

after obtaining the VAD decision result, the method further comprises;

updating the initial background update flag with the VAD decision, the tonality feature, the SNR parameter, the tonality signal flag, and the time-domain stability feature of the current frame;

obtaining the energy of background noise over individual sub-band and the energy of background noise over all sub-bands of the current frame with the updated flag of background noise, the energy feature of the current frame, and the energy of background noise over all sub-bands of the previous frame;

wherein, the energy of background noise of the current frame will be used to compute the SNR parameter of a next frame.

3. The method of claim 1, wherein:

the energy feature of the frame is a weighted or non-weighted sum of energies of all sub-bands;

the spectral centroid feature is a ratio of a weighted sum to a non-weighted sum of energies of all sub-bands or partial sub-bands, or a value obtained by applying a smooth filter to that ratio;

the time-domain stability feature is a ratio of a variance of a sum of amplitudes to an expectation of a square of amplitudes, or that ratio multiplied by a factor;

the spectral Flatness Feature is a ratio of a geometric mean to an arithmetic mean of certain spectrum amplitudes, or that ratio multiplied by a factor;

the tonality feature is obtained by computing a correlation coefficient of a spectrum amplitude difference of two adjacent frames, or by further smoothing that correlation coefficient.

4. The method of claim 1, wherein,

a step of computing a VAD decision result with the tonality signal flag, the signal to noise ratio parameter, the spectral centroid feature, and the energy feature, is as follows:

a. Obtaining a long-time SNR by computing a ratio of an average energy of long-time active frames to an average energy of long-time background noise for a previous frame;

b. Computing an average SNR of all sub-bands SNR2s for a plurality of recent frames to obtain an average SNR of all sub-bands SNR2_lt_ave;

c. Computing an SNR threshold snr_thr for making the VAD decision with the spectral centroid feature, the long-time SNR lt_snr, a number of previous continuous active frames continuous_speech_num, and a number of previous continuous noise frames continuous_noise_num;

d. Making an initial VAD decision with a VAD decision threshold and the SNR parameters SNR1 and SNR2 of the current frame;

e. Updating the initial VAD decision according to the tonality signal flag, the average SNR of all sub-bands SNR2_lt_ave, the spectral centroid, and the long-time SNR lt_snr;

f. Updating a number of hangover frames for active sound according to the decision result, the long-time SNR lt_snr, and the average SNR of all sub-bands SNR2_lt_ave for a plurality of previous frames, and the SNR parameter and the VAD decision for the current frame;

g. Adding an active-sound hangover according to the decision result and the number of hangover frames for active sound num_speech_hangover of the current frame to make the VAD decision.

5. The method of claim 4, wherein:

after the step d, the method further comprises computing the average energy of long-time active frames E_fg according to the initial VAD decision result;

after the step g, the method further comprises computing the average energy of long-time background noise E_bg with the VAD decision, and the computed value is used for making the VAD decision for the next frame.

6. A voice activity detection (VAD) apparatus applied in speech coding, wherein, the apparatus comprises hardware performing instructions stored in a non-transitory computer readable medium which executes steps in following units:

a filter bank, used to obtain sub-band signals of a current frame;

a spectrum amplitude computation unit, used to obtain spectrum amplitudes of the current frame;

a feature acquisition unit, used to compute values of a energy feature, a spectral centroid feature and a time-domain stability feature of the current frame according to the sub-band signals;

compute values of a spectral flatness feature and a tonality feature according to the spectrum amplitudes;

a flag computation unit, used to compute a tonality signal flag of the current frame according to the energy feature, the spectral centroid feature, the time-domain stability feature, the spectral flatness feature and the tonality feature of the current frame;

a signal to noise ratio computation unit, used to compute a signal to noise ratio parameter of the current frame according to a background noise energy estimated from a previous frame, a energy of SNR sub-bands and the energy feature of the current frame;

a VAD decision unit, used to compute a VAD decision result according to the tonality signal flag, the signal to noise ratio parameter, the spectral centroid feature and the energy feature.

7. A background noise detection method applied in speech coding, wherein, the method comprises:

obtaining sub-band signals and spectrum amplitudes of a current frame;

computing values of a energy feature, a spectral centroid feature and a time-domain stability feature of the current frame by using the sub-band signals;

computing values of a spectral flatness feature and a tonality feature according to the spectrum amplitudes;

detecting background noise by using the energy feature, the spectral centroid feature, the time-domain stability feature, the spectral flatness feature, and the tonality feature of the current frame, and judging whether the current frame is background noise or not.

8. The method of claim 7, wherein:

the energy feature of the current frame is a weighted or non-weighted sum of energies of all sub-bands;

the spectral centroid feature is a ratio of a weighted sum to a non-weighted sum of energies of all sub-bands or partial sub-bands, or a value obtained by applying a smooth filter to that ratio;

the time-domain stability feature is a ratio of a variance of a sum of amplitudes to an expectation of a square of amplitudes, or that ratio multiplied by a factor;

the spectral Flatness Feature is a ratio of a geometric mean to an arithmetic mean of certain spectrum amplitudes, or that ratio multiplied by a factor.

9. The method of claim 7, wherein, if any one of the following conditions is met, the current frame is not noise signal:

the time-domain stability feature lt_stable_rate0 is greater than a specified threshold

a smoothly-filtered value of the spectral centroid of a first section is greater than a specified threshold value, and the time-domain stability feature is also greater than a specified threshold value;

a value of the tonality feature or the smoothly-filtered tonality feature is greater than a specified threshold value, and the time-domain stability feature lt_stable_rate0 is also greater than a specified threshold value;

the spectral flatness of each sub-band or an average obtained by smoothing the spectral flatness is smaller than its specified threshold;

or, it is determined that the energy of the current frame E_t1 is greater than a specified threshold value E_thr1.

10. A tonality signal detection method applied in the field of speech coding, wherein, the method comprises:

obtaining sub-band signals and spectrum amplitudes of a current frame;

computing values of a energy feature, a spectral centroid feature and a time-domain stability feature of the current frame by using the sub-band signals;

computing values of a spectral flatness feature and a tonality feature according to the spectrum amplitudes;

with the tonality feature, the time-domain stability feature, the spectral flatness feature, and the spectral centroid feature of the current frame, determining whether the current frame is tonal signal.

11. The method of claim 10, wherein:

the spectral centroid feature is a ratio of a weighted sum to a non-weighted sum of energies of all sub-bands or partial sub-bands, or a value obtained by applying a smooth filter to that ratio;

the time-domain stability feature is a ratio of a variance of a sum of amplitudes to an expectation of a square of amplitudes, or that ratio multiplied by a factor;

the spectral Flatness Feature is a ratio of a geometric mean to an arithmetic mean of certain spectrum amplitudes, or that ratio multiplied by a factor;

the tonality feature is obtained by computing a correlation coefficient of a spectrum amplitude difference of two adjacent frames, or by further smoothing that correlation coefficient.

12. The method of claim 10, wherein:

when judging whether the current frame is a tonality signal or not, the following operations are performed;

A) Supposing the current frame is non-tonal signal and using a tonality frame flag tonality_frame to indicate whether the current frame is tonal;

B) If the tonality feature tonality_rate1 or its smoothed value tonality_rate2 is greater than a specified threshold, going to Step C), otherwise, going to Step D);

C) If the time-domain stability feature is smaller than a specified threshold, the spectral centroid feature is greater than a specified threshold, and the spectral flatness feature of each sub-band is smaller than its specified threshold thereof, then considering the current frame as a tonal frame, and setting the tonality frame flag, otherwise considering the current frame as a non-tonal frame, and setting the tonality frame flag, and going to step D);

D) Updating a tonal level feature tonality_degree according to the tonality frame flag, wherein an initial value of the tonal level feature tonality_degree is set when an active-sound detector begins to work;

E) Determining whether the current frame is tonal according to the updated tonal level feature tonality_degree and setting the tonality signal flag tonality_flag.

13. The method of claim 10, wherein:

if the current tonality frame flag indicates that the current frame is tonal, the following equation is used to update the tonal level feature tonality_degree;

tonality_degree=tonality_degree<sub>−

1</sub>·

td_scale_A+td_scale_B;

wherein, tonality_degree<sub>−

1</sub> is the tonal level feature of a previous frame whose initial value is set in a region of [0,1], td_scale_A is an attenuation coefficient, and td_scale_B is an accumulation coefficient.

14. The method of claim 10, wherein:

If the tonal level feature tonality_degree is greater than a specified threshold, the current frame is determined as tonal signal, otherwise, the current frame is determined as non-tonal signal.

15. A method applied in speech coding for updating a number of current active speech hangover frames in a VAD decision, wherein, the method comprises:

obtaining sub-band signals and spectrum amplitudes of a current frame;

Obtaining a long-time SNR lt_snr and an average SNR of all sub-bands SNR2_lt_ave by using the sub-band signals, updating a number of hangover frames for active sound according to a decision result, the long-time SNR lt_snr, and the average SNR of all sub-bands SNR2_lt_ave for several previous frames, and the VAD decision for the current frame.

16. The method of claim 15, wherein, the long-time SNR lt_snr is computed through a ratio of an average energy of long-time active frames to an average energy of long-time background noise for a previous frame;

the average SNR of all sub-bands SNR2s for a plurality of recent frames is computed to obtain the average SNR of all sub-bands SNR2_lt_ave.

17. The method of claim 15, wherein a precondition for updating the current number of hangover frames for active sound is that a flag of active sound indicates that the current frame is active sound.

18. The method of claim 15, wherein, when updating the number of current active speech hangover frames, if a number of previous continuous active frames is less than a specified threshold 1, and the long-time SNR lt_snr is less than a specified threshold 2, then the current number of hangover frames for active sound is updated by subtracting the number of previous continuous active frames from a minimum number of continuous active frames;

otherwise, if the average SNR of all sub-bands SNR2_lt_ave is greater than a specified threshold 3, and the number of previous continuous active frames is greater than a specified threshold 4, then the number of hangover frames for active sound is set according to a value of the long-time SNR, otherwise, that number of hangover frames is not updated.

19. A method applied in speech coding for adjusting a signal to noise ratio threshold in a VAD decision, wherein, the method for adjusting comprises:

obtaining sub-band signals and spectrum amplitudes of a current frame;

computing a spectral centroid feature of the current frame with the sub-band signals;

computing a long-time SNR through a ratio of an average energy of long-time active frames to an average energy of long-time background noise for a previous frame;

adjusting an SNR threshold for making the VAD decision with the spectral centroid feature, the long-time SNR, a number of previous continuous active frames continuous_speech_num, and a number of previous continuous noise frames continuous_noise_num.

20. The method of claim 19, wherein:

the spectral centroid feature is a ratio of a weighted sum to a non-weighted sum of energies of all sub-bands or partial sub-bands, or a value obtained by applying a smooth filter to that ratio.

21. The method of claim 19, wherein, the step of adjusting a signal to noise ratio threshold comprises:

setting an initial value of the SNR threshold snr_thr;

first adjusting the SNR threshold snr_thr with the spectral centroid feature;

further adjusting the snr_thr according to the number of previous continuous active frames continuous_speech_num, and the number of previous continuous noise frames continuous_noise_num, an average SNR of all sub-bands SNR2_lt_ave, and the long-time SNR lt_snr;

finally updating the SNR threshold snr_thr again according to the long-time SNR lt_snr to obtain the SNR threshold snr_thr of the current frame.

22. The method of claim 19, wherein,

an average SNR of all sub-bands SNR2s for a plurality of recent frames is computed to obtain the average SNR of all sub-bands SNR2_lt_ave.