METHOD AND APPARATUS FOR SPEECH ANALYSIS AND SYNTHESIS

• US 20080288258A1
• Filed: 04/03/2008
• Published: 11/20/2008
• Est. Priority Date: 04/04/2007
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Qin, Yong, Shuang, Zhi Wei, Meng, Fan Ping, Jiang, Dan Ning
• Granted Patent: US 8,280,739 B2
• Time in Patent Office: Days
• US Class Current: 704/261
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

The present invention provides a speech analysis method comprising steps of obtaining a speech signal and a corresponding DEGG/EGG signal; regarding the speech signal as the output of a vocal tract filter in a source-filter model taking the DEGG/EGG signal as the input; and estimating the features of the vocal tract filter from the speech signal as the output and the DEGG/EGG signal as the input, wherein the features of the vocal tract filter are expressed by the state vectors of the vocal tract filter at selected time points, and the step of estimating is performed using Kalman filtering.

Claims

1. A speech analysis method, comprising the steps of:

obtaining a speech signal and a corresponding DEGG/EGG signal;

providing the speech signal as the output of a vocal tract filter in a source-filter model taking the DEGG/EGG signal as the input; and

estimating the features of the vocal tract filter from the speech signal as the output and the DEGG/EGG signal as the input.

2. The speech analysis method according to claim 1, wherein the features of the vocal tract filter are expressed by the state vectors of the vocal tract filter at selected time points, and the step of estimating is performed using Kalman filtering.

3. The speech analysis method according to claim 2, wherein Kalman filtering is based on:

a state function

x_k=x<sub>k−

1</sub>+d_k and

an observation function

v_k=e_k^Tx_k+n_k,

wherein, x_k=[x_k(0), x_k(1), . . . , x_k(N−

1)]^T represents the state vector to be estimated of the vocal tract filter at time point k, wherein x_k(0), x_k(1), . . . , x_k(N−

1) represent N samples of the expected unit impulse response of the vocal tract filter at time k;

d_k=[d_k(0), d_k(1), . . . , d_k(N−

1)]^T represents the disturbance added to the state vector of the vocal tract filter at time k;

e_k=[e_k, e<sub>k−

1</sub>, . . . , e<sub>k−

N+1</sub>]^T is a vector, of which the element e_k represents the DEGG signal inputted at time k;

v_k represents the speech signal outputted at time k; and

n_k represents the observation noise added to the outputted speech signal at time k.

4. The speech analysis method according to claim 3, wherein Kalman filtering is a two-way Kalman filtering comprising a forward Kalman filtering and a backward Kalman filtering, wherein,

the forward Kalman filtering comprises the steps of:

forward estimation;

x_k^˜

=x<sub>k−

1</sub>*,

P_k^˜

=P<sub>k−

1</sub>+Q

correction;

K_k=P_k^˜

e_k[e_k^TP_k^˜

e_k+r]<sup>−

1</sup>

x_k*=x_k^˜

+K_k[v_k−

e_k^Tx_k^˜

]

P_k=[I−

K_ke_k^T]P_k^˜

forward recursion

k=k+1;

the backward Kalman filtering comprises the steps of;

backward estimation;

x_k^˜

=x_k+1*,

P_k^˜

=P_k+1+Q

correction;

K_k=P_k^˜

e_k[e_k^TP_k^˜

e_k+r]<sup>−

1</sup>

x_k*=x_k^˜

+K_k[v_k−

e_k^Tx_k^˜

]

P_k=[I−

K_ke_k^T]P_k^˜

backward recursion

k=k−

1;

wherein, x_k^˜

represents the estimated state value at time point k, x_k* represents the corrected state value at time point k, P_k^˜

represents the pre-estimated value of the covariance matrix of the estimation error, P_k represents the corrected value of the covariance matrix of the estimation error, Q represents the covariance matrix of disturbance d_k, K_k represents the Kalman gain, r represents the variance of the observation noise n_k, I represents the unit matrix; and

the estimation results of the two-way Kalman filtering are the combination of the estimation results of the forward Kalman filtering and those of the backward Kalman filtering using the following formula;

P_k=(P_k+⁻

+P_k−

<sup>−

1</sup>)<sup>−

1</sup>,

x_k*=P_k(P_k+<sup>−

1</sup>x_k+*+P_k−

<sup>−

1</sup>x_k−

*),

wherein, P_k+, x_k+ are the estimated state value and the covariance of the estimation obtained by the forward Kalman filtering respectively, and P_k−

, x_k−

represent the estimated state value and the covariance of the estimation obtained by the backward Kalman filtering respectively.

5. The speech analysis method according to claim 4, further comprising the step of selecting and recording the estimated state values of the vocal tract filter at selected time points obtained by the Kalman filtering, as the features of the vocal tract filter.

6. A speech synthesis method, comprising the steps of:

obtaining a DEGG/EGG signal;

obtaining the features of a vocal tract filter by;

obtaining a speech signal and a corresponding DEGG/EGG signal;

providing the speech signal as the output of a vocal tract filter in a source-filter model taking the DEGG/EGG signal as the input; and

estimating the features of the vocal tract filter from the speech signal as the output and the DEGG/EGG signal as the input; and

synthesizing speech based on the DEGG/EGG signal and the obtained features of the vocal tract filter.

7. The speech synthesis method according to claim 6, wherein the step of obtaining the DEGG/EGG signal comprises:

reconstructing a full DEGG/EGG signal using a DEGG/EGG signal of a single period based on a given fundamental frequency and time length.

8. A speech analysis apparatus, comprising:

a module for obtaining a speech signal;

a module for obtaining the corresponding DEGG/EGG signal; and

an estimation module for, by regarding the speech signal as the output of a vocal tract filter in a source-filter model with the DEGG/EGG signal as the input, estimating the features of the vocal tract filter from the speech signal as the output and the DEGG/EGG signal as the input.

9. The speech analysis apparatus according to claim 8, wherein the estimation module uses the state vectors of the vocal tract filter at selected time points to express the features of the vocal tract filter, and uses the Kalman filtering to perform the estimation

10. The speech analysis apparatus according to claim 9, wherein the Kalman filtering is based on:

a state function

x_k=x<sub>k−

1</sub>+d_k, and

an observation function

v_k=e_k^Tx_k+n_k,

wherein, x_k=[x_k(0), x_k(1), . . . , x_k(N−

1)]^T represents the state vector to be estimated of the vocal tract filter at time point k, wherein x_k(0), x_k(1), . . . , x_k(N−

1) represent N samples of the expected unit impulse response of the vocal tract filter at time k;

d_k=[d_k(0), d_k(1), . . . , d_k(N−

1)]^T represents the disturbance added to the state vector of the vocal tract filter at time k;

e_k=[e_k, e<sub>k−

1</sub>, . . . , e<sub>k−

N+1</sub>]^T is a vector, of which the element e_k represents the DEGG signal inputted at time k;

v_k represents the speech signal outputted at time k; and

n_k represents the observation noise added to the outputted speech signal at time k.

11. The speech analysis apparatus according to claim 10, wherein the Kalman filtering is a two-way Kalman filtering comprising a forward Kalman filtering and a backward Kalman filtering, wherein,

the forward Kalman filtering comprises the following steps:

forward estimation;

x_k^˜

=x<sub>k−

1</sub>*,

P_k^˜

=P<sub>k−

1</sub>+Q

correction;

K_k=P_k^˜

e_k[e_k^TP_k^˜

e_k+r]<sup>−

1</sup>

x_k*=x_k^˜

+K_k[v_k−

e_k^Tx_k^˜

]

P_k=[I−

K_ke_k^T]P_k^˜

forward recursion

k=k+1;

the backward Kalman filtering comprises the following steps;

backward estimation;

x_k^˜

=x_k+1*,

P_k^˜

=P_k+1+Q

correction;

K_k=P_k^˜

e_k[e_k^TP_k^˜

e_k+r]<sup>−

1</sup>

x_k*=x_k^˜

+K_k[v_k−

e_k^Tx_k^˜

]

P_k=[I−

K_ke_k^T]P_k^˜

backward recursion

k=k−

1;

wherein, x_k^˜

represents the pre-estimated state value at time point k, x_k* represents the corrected state value at time point k, P_k^˜

represents the pre-estimated value of the covariance matrix of the estimation error, P_k represents the corrected value of the covariance matrix of the estimation error, Q represents the covariance matrix of disturbance d_k, K_k represents the Kalman gain, r represents the variance of the observation noise n_k, I represents the unit matrix; and

the estimation results of the two-way Kalman filter are the combination of estimation results of the forward Kalman filter and those of the backward Kalman filtering using the following formula;

P_k=(P_k+<sup>−

1</sup>+P_k−

<sup>−

1</sup>)<sup>−

1</sup>,

x_k*=P_k(P_k+<sup>−

1</sup>x_k+*+P_k−

<sup>−

1</sup>x_k−

*),

wherein, P_k+, x_k+ are the estimated state value and the covariance of the estimation obtained by the forward Kalman filtering respectively, and P_k−

, x_k−

represent the estimated state value and the covariance of the estimation obtained by the backward Kalman filtering respectively.

12. The speech analysis apparatus according to claim 11 further comprising a selection and recording module for selecting and recording the estimated state values of the vocal tract filter at selected time points obtained by the Kalman filtering, as the features of the vocal tract filter.

13. A speech synthesis apparatus, comprising:

a module for obtaining a DEGG/EGG signal;

the speech analysis apparatus comprising;

a module for obtaining a speech signal;

a module for obtaining the corresponding DEGG/EGG signal; and

an estimation module for, by regarding the speech signal as the output of a vocal tract filter in a source-filter model with the DEGG/EGG signal as the input, estimating the features of the vocal tract filter from the speech signal as the output and the DEGG/EGG signal as the input; and

a speech synthesis module for synthesizing a speech signal based on the DEGG/EGG signal obtained by the module for obtaining a DEGG/EGG signal and the features of the vocal tract filter estimated by the speech analysis apparatus.

14. The speech synthesis apparatus according to claim 13, wherein the module for obtaining a DEGG/EGG signal is further configured to reconstruct a full DEGG/EGG signal using a DEGG/EGG signal of a single period based on a given fundamental frequency and time length.