Method and device for efficient frame erasure concealment in linear predictive based speech codecs

• US 7,693,710 B2
• Filed: 05/30/2003
• Issued: 04/06/2010
• Est. Priority Date: 05/31/2002
• Status: Active Grant

• Current Assignee: VoiceAge EVS LLC (FIG LLC (d/b/a Fortress Investment Group LLC))
• Original Assignee: VoiceAge Corporation
• Inventors: Jelinek, Milan, Gournay, Philippe
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/515,569
• Publication Number:
  US 20050154584A1
• Time in Patent Office: 2,503 Days
• Field of Search: 704/206, 704/207, 704/208, 704/212, 704/214, 704/219, 704/221, 704/225, 714/746, 714/747
• US Class Current: 704/207
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm

Abstract

The present invention relates to a method and device for improving concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder (106) to a decoder (110), and for accelerating recovery of the decoder after non erased frames of the encoded sound signal have been received. For that purpose, concealment/recovery parameters are determined in the encoder or decoder. When determined in the encoder (106), the concealment/recovery parameters are transmitted to the decoder (110). In the decoder, erasure frame concealment and decoder recovery is conducted in response to the concealment/recovery parameters. The concealment/recovery parameters may be selected from the group consisting of: a signal classification parameter, an energy information parameter and a phase information parameter. The determination of the concealment/recovery parameters comprises classifying the successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset, and this classification is determined on the basis of at least a part of the following parameters: a normalized correlation parameter, a spectral tilt parameter, a signal-to-noise ratio parameter, a pitch stability parameter, a relative frame energy parameter, and a zero crossing parameter.

Claims

1. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein;

conducting frame erasure concealment and decoder recovery comprises, when at least one onset frame is lost, constructing a periodic excitation part artificially as a low-pass filtered periodic train of pulses separated by a pitch period;

the method comprises quantizing a position of a first glottal pulse with respect to the beginning of the onset frame prior to transmission of said position of the first glottal pulse to the decoder; and

constructing the periodic excitation part comprises realizing the low-pass filtered periodic train of pulses by;

centering a first impulse response of a low-pass filter on the quantized position of the first glottal pulse with respect to the beginning of the onset frame; and

placing remaining impulse responses of the low-pass filter each with a distance corresponding to an average pitch value from the preceding impulse response up to the end of a last subframe affected by the artificial construction of the periodic part.

2. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter, and a phase information parameter related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein the concealment/recovery parameters include the phase information parameter and wherein determination of the phase information parameter comprises;

determining a position of a first glottal pulse in a frame of the encoded sound signal; and

encoding, in the encoder, a shape, sign and amplitude of the first glottal pulse and transmitting the encoded shape, sign and amplitude from the encoder to the decoder.

3. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter, and a phase information parameter related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein;

the concealment/recovery parameters include the phase information parameter;

determination of the phase information parameter comprises determining a position of a first glottal pulse in a frame of the encoded sound signal; and

determining the position of the first glottal pulse comprises;

measuring a sample of maximum amplitude within a pitch period as the first glottal pulse; and

quantizing a position of the sample of maximum amplitude within the pitch period.

4. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter, and a phase information parameter related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein;

the sound signal is a speech signal;

determining, in the encoder, concealment/recovery parameters comprises classifying successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

determining concealment/recovery parameters comprises calculating the energy information parameter in relation to a maximum of a signal energy for frames classified as voiced or onset, and calculating the energy information parameter in relation to an average energy per sample for other frames.

5. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter, and a phase information parameter related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein conducting frame erasure concealment and decoder recovery comprises;

controlling an energy of a synthesized sound signal produced by the decoder, controlling energy of the synthesized sound signal comprising scaling the synthesized sound signal to render an energy of said synthesized sound signal at the beginning of a first non erased frame received following frame erasure similar to an energy of said synthesized sound signal at the end of a last frame erased during said frame erasure; and

converging the energy of the synthesized sound signal in the received first non erased frame to an energy corresponding to the received energy information parameter toward the end of said received first non erased frame while limiting an increase in energy.

6. A method as claimed in claim 5, wherein:

the sound signal is a speech signal;

determining, in the encoder, concealment/recovery parameters comprises classifying successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

when the first non erased frame received after a frame erasure is classified as onset, conducting frame erasure concealment and decoder recovery comprises limiting to a given value a gain used for scaling the synthesized sound signal.

7. A method as claimed in claim 5, wherein:

the sound signal is a speech signal;

determining, in the encoder, concealment/recovery parameters comprises classifying successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

said method comprising making a gain used for scaling the synthesized sound signal at the beginning of the first non erased frame received after frame erasure equal to a gain used at an end of said received first non erased frame;

during a transition from a voiced frame to an unvoiced frame, in the case of a last non erased frame received before frame erasure classified as voiced transition, voice or onset and a first non erased frame received after frame erasure classified as unvoiced; and

during a transition from a non-active speech period to an active speech period, when the last non erased frame received before frame erasure is encoded as comfort noise and the first non erased frame received after frame erasure is encoded as active speech.

8. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter, and a phase information parameter related to the sound signal;

transmitting to the decoder concealment/recovery parameters determined in the encoder; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to the received concealment/recovery parameters;

wherein;

the energy information parameter is not transmitted from the encoder to the decoder; and

conducting frame erasure concealment and decoder recovery comprises, when a gain of a LP filter of a first non erased frame received following frame erasure is higher than a gain of a LP filter of a last frame erased during said frame erasure, adjusting an energy of an LP filter excitation signal produced in the decoder during the received first non erased frame to the gain of the LP filter of said received first non erased frame.

9. A method as claimed in claim 8 wherein:

adjusting the energy of the LP filter excitation signal produced in the decoder during the received first non erased frame to the gain of the LP filter of said received first non erased frame comprises using the following relation;

$E_q=E_1<br><br>\frac{E_{\mathrm{LP}<br><br><br><br>0}}{E_{\mathrm{LP}<br><br><br><br>1}}$

where E₁ is an energy at an end of the current frame, E_LPO is an energy of an impulse response of the LP filter of a last non erased frame received before the frame erasure, and E_LP1 is an energy of an impulse response of the LP filter of the received first non erased frame following frame erasure.

10. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein the concealment/recovery parameters include the phase information parameter and wherein determination of the phase information parameter comprises;

determining a position of a first glottal pulse in a frame of the encoded sound signal; and

encoding, in the encoder, a shape, sign and amplitude of the first glottal pulse and transmitting the encoded shape, sign and amplitude from the encoder to the decoder.

11. A method of concealing frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

determining, in the encoder, concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the concealment/recovery parameters include the phase information parameter;

determination of the phase information parameter comprises determining a position of a first glottal pulse in a frame of the encoded sound signal; and

determining the position of the first glottal pulse comprises;

measuring a sample of maximum amplitude within a pitch period as the first glottal pulse; and

quantizing a position of the sample of maximum amplitude within the pitch period.

12. A method for the concealment of frame erasure caused by frames erased during transmission of a sound signal encoded under the form of signal-encoding parameters from an encoder to a decoder, comprising:

determining, in the decoder, concealment/recovery parameters from the signal-encoding parameters, wherein the concealment/recovery parameters are selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal and are used for producing, upon occurrence of frame erasure, a replacement frame selected from the group consisting of a voiced frame, an unvoiced frame, and a frame defining a transition between voiced and unvoiced frames; and

in the decoder, conducting frame erasure concealment and decoder recovery in response to concealment/recovery parameters determined in the decoder;

wherein;

the concealment/recovery parameters include the energy information parameter;

the energy information parameter is not transmitted from the encoder to the decoder; and

conducting frame erasure concealment and decoder recovery comprises, when a gain of a LP filter of a first non erased frame received following frame erasure is higher than a gain of a LP filter of a last frame erased during said frame erasure, adjusting an energy of an LP filter excitation signal produced in the decoder during the received first non erased frame to a gain of the LP filter of said received first non erased frame using the following relation;

$E_q=E_1<br><br>\frac{E_{\mathrm{LP}<br><br><br><br>0}}{E_{\mathrm{LP}<br><br><br><br>1}}$

where E₁ is an energy at an end of the current frame, E_LPO is an energy of an impulse response of the LP filter of a last non erased frame received before the frame erasure, and E_LP1 is an energy of an impulse response of the LP filter of the received first non erased frame following frame erasure.

13. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to the concealment/recovery parameters received from the encoder;

for conducting frame erasure concealment and decoder recovery, the decoder constructs, when at least one onset frame is lost, a periodic excitation part artificially as a low-pass filtered periodic train of pulses separated by a pitch period;

the device comprises a quantizer of a position of a first glottal pulse with respect to the beginning of the onset frame prior to transmission of said position of the first glottal pulse to the decoder; and

the decoder, for constructing the periodic excitation part, realizes the low-pass filtered periodic train of pulses by;

centering a first impulse response of a low-pass filter on the quantized position of the first glottal pulse with respect to the beginning of the onset frame; and

placing remaining impulse responses of the low-pass filter each with a distance corresponding to an average pitch value from the preceding impulse response up to an end of a last subframe affected by the artificial construction of the periodic part.

14. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to the concealment/recovery parameters received from the encoder;

the concealment/recovery parameters include the phase information parameter;

to determine the phase information parameter, the determiner comprises a searcher of a position of a first glottal pulse in a frame of the encoded sound signal;

the searcher encodes a shape, sign and amplitude of the first glottal pulse and the communication link transmits the encoded shape, sign and amplitude from the encoder to the decoder.

15. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to the concealment/recovery parameters received from the encoder;

the concealment/recovery parameters include the phase information parameter;

to determine the phase information parameter, the determiner comprises a searcher of a position of a first glottal pulse in a frame of the encoded sound signal; and

the searcher measures a sample of maximum amplitude within a pitch period as the first glottal pulse, and the determiner comprises a quantizer of the position of the sample of maximum amplitude within the pitch period.

16. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to the concealment/recovery parameters received from the encoder;

the sound signal is a speech signal;

the determiner of concealment/recovery parameters comprises a classifier of successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

the determiner of concealment/recovery parameters comprises a computer of the energy information parameter in relation to a maximum of a signal energy for frames classified as voiced or onset, and in relation to an average energy per sample for other frames.

17. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to concealment/recovery parameters received from the encoder; and

for conducting frame erasure concealment and decoder recovery;

the decoder controls an energy of a synthesized sound signal produced by the decoder by scaling the synthesized sound signal to render an energy of said synthesized sound signal at the beginning of a first non erased frame received following frame erasure similar to an energy of said synthesized sound signal at the end of a last frame erased during said frame erasure; and

the decoder converges the energy of the synthesized sound signal in the received first non erased frame to an energy corresponding to the received energy information parameter toward the end of said received first non erased frame while limiting an increase in energy.

18. A device as claimed in claim 17, wherein:

the sound signal is a speech signal;

the determiner of concealment/recovery parameters comprises a classifier of successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

when the first non erased frame received following frame erasure is classified as onset, the decoder, for conducting frame erasure concealment and decoder recovery, limits to a given value a gain used for scaling the synthesized sound signal.

19. A device as claimed in claim 17, wherein:

the sound signal is a speech signal;

the determiner of concealment/recovery parameters comprises a classifier of successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

the decoder makes a gain used for scaling the synthesized sound signal at the beginning of the first non erased frame received after frame erasure equal to a gain used at an end of said received first non erased frame;

during a transition from a voiced frame to an unvoiced frame, in the case of a last non erased frame received before frame erasure classified as voiced transition, voice or onset and a first non erased frame received after frame erasure classified as unvoiced; and

during a transition from a non-active speech period to an active speech period, when the last non erased frame received before frame erasure is encoded as comfort noise and the first non erased frame received after frame erasure is encoded as active speech.

20. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the decoder conducts frame erasure concealment and decoder recovery in response to the concealment/recovery parameters received from the encoder;

the energy information parameter is not transmitted from the encoder to the decoder; and

when a gain of a LP filter of a first non erased frame received following frame erasure is higher than a gain of a LP filter of a last frame erased during said frame erasure, the decoder adjusts an energy of an LP filter excitation signal produced in the decoder during the received first non erased frame to a gain of the LP filter of said received first non erased frame.

21. A device as claimed in claim 20, wherein:

the decoder, for adjusting the energy of the LP filter excitation signal produced in the decoder during the received first non erased frame to the gain of the LP filter of said received first non erased frame, uses the following relation;

$E_q=E_1<br><br>\frac{E_{\mathrm{LP}<br><br><br><br>0}}{E_{\mathrm{LP}<br><br><br><br>1}}$

where E₁ is an energy at an end of a current frame, E_LPO is an energy of an impulse response of a LP filter of a last non erased frame received before the frame erasure, and E_LP1 is an energy of an impulse response of the LP filter of the received first non erased frame following frame erasure.

22. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the concealment/recovery parameters include the phase information parameter;

to determine the phase information parameter, the determiner comprises a searcher of a position of a first glottal pulse in a frame of the encoded sound signal; and

the searcher encodes a shape, sign and amplitude of the first glottal pulse and the communication link transmits the encoded shape, sign and amplitude from the encoder to the decoder.

23. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the concealment/recovery parameters include the phase information parameter;

to determine the phase information parameter, the determiner comprises a searcher of a position of a first glottal pulse in a frame of the encoded sound signal; and

the searcher measures a sample of maximum amplitude within a pitch period as the first glottal pulse; and

the determiner comprises a quantizer of the position of the sample of maximum amplitude within the pitch period.

24. A device for conducting concealment of frame erasure caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder, comprising:

in the encoder, a determiner of concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal; and

a communication link for transmitting to the decoder concealment/recovery parameters determined in the encoder;

wherein;

the sound signal is a speech signal;

the determiner of concealment/recovery parameters comprises a classifier of successive frames of the encoded sound signal as unvoiced, unvoiced transition, voiced transition, voiced, or onset; and

the determiner of concealment/recovery parameters comprises a computer of the energy information parameter in relation to a maximum of a signal energy for frames classified as voiced or onset, and in relation to an average energy per sample for other frames.

25. A device for the concealment of frame erasure caused by frames erased during transmission of a sound signal encoded under the form of signal-encoding parameters from an encoder to a decoder, wherein:

the decoder determines concealment/recovery parameters selected from the group consisting of a signal classification parameter, an energy information parameter and a phase information parameter related to the sound signal, for producing, upon occurrence of frame erasure, a replacement frame selected from the group consisting of a voiced frame, an unvoiced frame, and a frame defining a transition between voiced and unvoiced frames; and

the decoder conducts erased frame concealment and decoder recovery in response to determined concealment/recovery parameters;

wherein;

the concealment/recovery parameters include the energy information parameter;

the energy information parameter is not transmitted from the encoder to the decoder; and

the decoder, for conducting frame erasure concealment and decoder recovery when a gain of a LP filter of a first non erased frame received following frame erasure is higher than a gain of a LP filter of a last frame erased during said frame erasure, adjusts an energy of an LP filter excitation signal produced in the decoder during the received first non erased frame to a gain of the LP filter of said received first non erased frame using the following relation;

$E_q=E_1<br><br>\frac{E_{\mathrm{LP}<br><br><br><br>0}}{E_{\mathrm{LP}<br><br><br><br>1}}$

where E₁ is an energy at an end of a current frame, E_LPO is an energy of an impulse response of a LP filter of a last non erased frame received before the frame erasure, and E_LP1 is an energy of an impulse response of the LP filter to the received first non erased frame following frame erasure.