Speech content based packet loss concealment

• US 8,589,166 B2
• Filed: 09/21/2010
• Issued: 11/19/2013
• Est. Priority Date: 10/22/2009
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Zopf, Robert W.
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US12/887,353
• Publication Number:
  US 20110099014A1
• Time in Patent Office: 1,155 Days
• Field of Search: 704/262, 704/500, 704/270.1, 704/265, 704/233, 704/230, 704/228, 704/223, 704/22, 704/219, 704/211, 704/203, 704/201, 375/241, 375/240.27, 375/231, 370/514, 370/352, 370/270
• US Class Current: 704/262
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis
• CPC Class Codes: G10L 19/16 Vocoder architecture
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

Systems and methods are described for performing packet loss concealment (PLC) to mitigate the effect of one or more lost frames within a series of frames that represent a speech signal. In accordance with the exemplary systems and methods, PLC is performed by searching a codebook of speech-related parameter profiles to identify content that is being spoken and by selecting a profile associated with the identified content for use in predicting or estimating speech-related parameter information associated with one or more lost frames of a speech signal. The predicted/estimated speech-related parameter information is then used to synthesize one or more frames to replace the lost frame(s) of the speech signal.

Claims

1. A method for concealing the effects of one or more lost frames within a series of frames that comprise a speech signal, comprising:

composing an input vector that includes a computed value of a speech-related parameter for each of a number of frames that precede the lost frame(s);

comparing the input vector to at least one portion of each vector in a codebook, each vector in the codebook representing a different model of how the speech-related parameter varies over time;

selecting one of the vectors in the codebook based on the comparison;

determining a value of the speech-related parameter for each of the lost frame(s) based on the selected vector in the codebook; and

synthesizing one or more frames to replace the lost frame(s) based on the determined value(s) of the speech-related parameter.

2. The method of claim 1, wherein composing the input vector comprises composing an input vector that comprises a computed value of one of the following speech-related parameters for each of the number of frames that precede the lost frame(s):

a fundamental frequency;

a frame gain;

a voicing measure;

a spectral envelope; and

a pitch.

3. The method of claim 1, wherein the speech-related parameter comprises a scalar parameter.

4. The method of claim 1, wherein the speech-related parameter comprises a vector parameter.

5. The method of claim 1, wherein the speech-related parameter comprises a parameter that has been normalized for speaker independence.

6. The method of claim 1, wherein composing the input vector comprises:

composing an input vector that includes the computed value of the speech-related parameter for each of the number of frames that precede the lost frame(s) and a computed value of the speech-related parameter for each of a number of frames that follow the lost frame(s).

7. The method of claim 1, wherein comparing the input vector to at least one portion of each vector in the codebook comprises calculating a distortion measure based on the input vector and at least one portion of each vector in the codebook;

and

wherein selecting one of the vectors in the codebook based on the comparison comprises selecting a vector in the codebook having a portion that generates the smallest distortion measure.

8. The method of claim 1, wherein comparing the input vector to at least one portion of each vector in the codebook comprises calculating a similarity measure based on the input vector and at least one portion of each vector in the codebook;

and

wherein selecting one of the vectors in the codebook based on the comparison comprises selecting a vector in the codebook having a portion that generates the greatest similarity measure.

9. The method of claim 1, wherein comparing the input vector to at least one portion of each vector in the codebook comprises comparing the input vector to each of a series of overlapping portions of each vector in the codebook within an analysis window.

10. A method for concealing the effects of one or more lost frames within a series of frames that comprise a speech signal, comprising:

composing an input vector that includes a set of computed values of a plurality of speech-related parameters for each of a number of frames that precede the lost frame(s);

comparing the input vector to at least one portion of each vector in a codebook, each vector in the codebook jointly representing a plurality of models of how the plurality of speech-related parameters vary over time;

selecting one of the vectors in the codebook based on the comparison;

determining a value of each of the plurality of speech-related parameters for each of the lost frame(s) based on the selected vector in the codebook; and

synthesizing one or more frames to replace the lost frame(s) based on the determined value(s) of each of the plurality of speech-related parameters.

11. The method of claim 10, wherein composing the input vector comprises composing an input vector that comprises, for each of the number of frames that precedes the lost frame(s), a set of computed values corresponding to two or more of the following speech-related parameters:

a fundamental frequency;

a frame gain;

a voicing measure;

a spectral envelope; and

a pitch.

12. The method of claim 10, wherein at least one speech-related parameter in the plurality of speech-related parameters comprises a scalar parameter.

13. The method of claim 10, wherein at least one speech-related parameter in the plurality of speech-related parameters comprises a vector parameter.

14. The method of claim 10, wherein at least one speech-related parameter in the plurality of speech-related parameters comprises a parameter that has been normalized for speaker independence.

15. The method of claim 10, wherein composing the input vector comprises:

composing an input vector that includes the set of computed values of the plurality of speech-related parameters for each of the number of frames that precede the lost frame(s) and a set of computed values of the plurality of speech-related parameters for each of a number of frames that follow the lost frame(s).

16. The method of claim 10, wherein comparing the input vector to at least one portion of each vector in the codebook comprises calculating a distortion measure based on the input vector and at least one portion of each vector in the codebook;

and

wherein selecting one of the vectors in the codebook based on the comparison comprises selecting a vector in the codebook having a portion that generates the smallest distortion measure.

17. The method of claim 16, wherein calculating the distortion measure based on the input vector and at least one portion of each vector in the codebook comprises calculating a distortion measure that is a function of one or more characteristics of the speech signal or a bit stream that represents an encoded version of the speech signal.

18. The method of claim 16, wherein calculating the distortion measure based on the input vector and at least one portion of a vector in the codebook comprises:

calculating a parameter-specific distortion measure for each speech-related parameter in the plurality of speech-related parameters based on the input vector and a portion of a vector in the codebook; and

combining the parameter-specific distortion measures.

19. The method of claim 18, wherein combining the parameter-specific distortion measures comprises:

applying a weight to each of the parameter-specific distortion measures, wherein the weight applied to each of the parameter-specific distortion measures is selected based on one or more characteristics of the speech signal or a bit stream that represents an encoded version of the speech signal.

20. The method of claim 10, wherein comparing the input vector to at least one portion of each vector in the codebook comprises calculating a similarity measure based on the input vector and at least a portion of each vector in the codebook;

and

wherein selecting one of the vectors in the codebook based on the comparison comprises selecting a vector in the codebook having a portion that generates the greatest similarity measure.

21. The method of claim 20, wherein calculating the similarity measure based on the input vector and at least one portion of each vector in the codebook comprises calculating a similarity measure that is a function of one or more characteristics of the speech signal or a bit stream that represents an encoded version of the speech signal.

22. The method of claim 20, wherein calculating the similarity measure based on the input vector and at least one portion of a vector in the codebook comprises:

calculating a parameter-specific similarity measure for each speech-related parameter in the plurality of speech-related parameters based on the input vector and a portion of a vector in the codebook; and

combining the parameter-specific similarity measures.

23. The method of claim 22, wherein combining the parameter-specific distortion measures comprises:

applying a weight to each of the parameter-specific distortion measures, wherein the weight applied to each of the parameter-specific distortion measures is selected based on one or more characteristics of the speech signal or a bit stream that represents an encoded version of the speech signal.

24. The method of claim 10, wherein comparing the input vector to at least one portion of each vector in the codebook comprises comparing the input vector to each of a series of overlapping portions of each vector in the codebook within an analysis window.

25. A system for concealing the effects of one or more lost frames within a series of frames that comprise a speech signal, comprising:

at least one processor; and

at least one memory that stores software that is executed by the at least one processor, the software comprising;

a vector generation module that composes an input vector that includes a computed value of a speech-related parameter for each of a number of frames that precede the lost frame(s);

a codebook search module that compares the input vector to at least one portion of each vector in a codebook, each vector in the codebook representing a different model of how the speech-related parameter varies over time, selects one of the vectors in the codebook based on the comparison, and determines a value of the speech-related parameter for each of the lost frame(s) based on the selected vector in the codebook; and

a synthesis module that synthesizes one or more frames to replace the lost frame(s) based on the determined value(s) of the speech-related parameter.

26. A system for concealing the effects of one or more lost frames within a series of frames that comprise a speech signal, comprising:

at least one processor; and

at least one memory that stores software that is executed by the at least one processor, the software comprising;

an input vector generation module that composes an input vector that includes a set of computed values of a plurality of speech-related parameters for each of a number of frames that precede the lost frame(s);

a codebook search module that compares the input vector to at least one portion of each vector in a codebook, each vector in the codebook jointly representing a plurality of models of how the plurality of speech-related parameters vary over time, selects one of the vectors in the codebook based on the comparison, and determines a value of each of the plurality of speech-related parameters for each of the lost frame(s) based on the selected vector in the codebook; and

a synthesis module that synthesizes one or more frames to replace the lost frame(s) based on the determined value(s) of each of the plurality of speech-related parameters.