Method of and apparatus for processing at least one coded binary audio flux organized into frames

• US 6,807,526 B2
• Filed: 12/08/2000
• Issued: 10/19/2004
• Est. Priority Date: 12/08/1999
• Status: Expired due to Term

• Current Assignee: France Telecom SA (Orange S.A.)
• Original Assignee: France Telecom SA (Orange S.A.)
• Inventors: Mahieux, Yannick, Touimi, Abdellatif Benjelloun, Lamblin, Claude
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/731,776
• Publication Number:
  US 20010027393A1
• Time in Patent Office: 1,411 Days
• Field of Search: 704/205, 704/208, 704/222
• US Class Current: 704/222
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: H04M 3/561 by multiplexing
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

At least one coded binary audio flux organized into frames is created from digital audio signals which were coded by transforming them from the time domain into the frequency domain. Transform coefficients of the signals in the frequency domain are quantized and coded according to a set of quantizers. The set is determined from a set of values extracted from the signals. The values make up selection parameters of the set of quantizers. The parameters are also present in the frames. A partial decoding state decodes then dequantizes transform coefficients produced by the coding based on a set of quantizers determined from the selection parameters contained in the frames of the coded binary audio flux or of each coded binary audio flux. The partially decoded frames are subjected to processing in the frequency domain. The thus-processed frames are then made available for use in a later utilization step.

Claims

1. A method of processing at least one coded binary stream organized in the form of frames created from digital audio signals which were coded by at least an audio terminal in order to output processed frames to a subsequent using step, said coding of said digital audio signals including calculating transform coefficients by transforming the digital audio signals from the time domain to the frequency domain, then quantizing and coding said transform coefficients according to a set of quantizers determined by selection parameters extracted from said digital audio signals, said frames including said selection parameters and the thus-coded transform coefficients, the method comprising, for said at least one audio stream received from at least said terminal:

(1) obtaining said selection parameters from said frames of said audio stream and determining from said selection parameters the set of quantizers that was used during the quantizing step performed by said audio terminal;

(2) recovering the transform coefficients that were calculated by said audio terminal by partially decoding and dequantizing said frames, the recovering being performed by using the set of quantizers determined in step (1);

(3) producing processed frames by performing said specific process in the frequency domain on the dequantized transform coefficients obtained in step (2); and

(4) supplying said processed frames to a subsequent using step.

2. The method of claim 1 wherein said subsequent using step includes partially recoding the frames thus processed in a single step including requantization and recoding the processed transform coefficients.

3. The method of claim 2 wherein the specific process includes summing the dequantized transform coefficients obtained by the step (2) of partially decoding the frames belonging to streams of frames provided by a plurality of audio terminals.

4. Performing the method of claim 2 in a multi-terminal teleconferencing system wherein the specific process includes summing the dequantized transform coefficients obtained by the step (2) of partially decoding the frames belonging to streams of frames provided by a plurality of audio terminals, subtracting from said summed coefficients the dequantized transform coefficients obtained by the step (2) of partial decoding frames coming from one of said audio terminals, and supplying the thus obtained result to said subsequent using step concerning said audio terminal.

5. The method of claim 1 wherein said subsequent using step includes reconstructing an audio signal by transforming said processed frames from the frequency domain back in the time domain.

6. Performing the method of claim 5 in a multi-terminal teleconferencing system wherein the specific process includes summing the dequantized transform coefficients obtain by the step (2) of partially decoding the frames belonging to streams of frames provided by a plurality of audio terminals.

7. The method of claim 6 further including subjecting the values of the quantizer set selection parameters to said specific process of summing.

8. The method of claim 6 wherein the quantizer set selection parameters included in the audio frames of said at least one stream includes a set of values representing the energies of the audio signals in predetermined frequency bands, the set being called a spectral envelope, said method including summing the transformed coefficients produced by the step (2) of partially decoding each of the stream of frames and supplying the result of the summation to a subsequent using step of recoding, said method further including determining the total energy in each frequency band by summing the energies of the frames and supplying the result of the summing step to a subsequent using step of recoding.

9. Performing the method of claim 8 in a multi-terminal teleconferencing system, said method further including summing the transformed coefficients produced respectively by the step (2) of partially decoding the streams of frames coming from the said terminals and the recoding step associated with a terminal, supplying the result of said summation from which the transform coefficients produced by the step of partially decoding the stream of frames coming from said terminal are subtracted, determining the total energy in each frequency band by summing the energies of the frames coming from all the terminals, and, for the recoding step associated with a terminal, supplying the result of the summation from which the energy from the frame coming from the terminal is subtracted.

10. The method of claim 2 wherein the requantizing step of said subsequent using step is performed by a vectorial quantization with embedded dictionaries, further including setting the code word of an output band so it is the same as the codeword of an input band if the input dictionary corresponding to the band is included in the selected output dictionary;

or if the output dictionary is included in the input dictionary but the quantized vector, an element of the input dictionary, is also an element of the output dictionary, performing a process including reverse quantization then requantization in the output dictionary.

11. The method of claim 10 wherein the requantization step includes a vectorial quantization with embedded dictionaries, the dictionaries including a union of permutation codes, said method further including setting the code word for the output band so it is the same as the code word for the input band if the corresponding input dictionary for the band is included in the selected output dictionary, or in the opposite case where the output dictionary is included in the input dictionary but the quantized vector, an element of the input dictionary, is also an element of the output dictionary, otherwise reverse quantizing then requantizing the output dictionary, the requantization procedure being sped up if the closest neighbor of a leader of a vector of the input dictionary is a leader of the output dictionary.

12. The method of claim 1 wherein the audio frames of said at least one stream include voicing information on the corresponding audio signal, said method further including determining voicing information for the audio signal resulting from the specific process.

13. The method of claim 12 further including, in the step of determining voicing information for the audio signal resulting from said performing step (3) of said specific process, (a) if all the frames of all the streams are determined to have the same voicing state, considering this voicing state as the audio signal state resulting from step (3), and (b) (i) if all the frames of all the streams are determined not to have the same voicing state, determining the total energy of the set of audio signals of the voiced frames and the energy of the set of audio signals of the unvoiced frames, and (ii) then considering the voicing state of the set with the greatest energy as being the voicing state of the audio signal resulting from the said specific process.

14. The method of claim 1 wherein the audio frames of said at least one stream include tone information for the corresponding audio signal, said specific process further including, if all the frames are determined to be of the same kind, determining tone information for the audio signal resulting from said specific process as being the state of the signals of all the frames.

15. The method of claim 1 further including finding among all the frames the one with the most energy in a given band, and making the coefficients of the output frame equal to the coefficient of the frame with the most energy in a given band if the coefficients of the input frames other than the one with the most energy in a given band are masked by a masking threshold of the frame in the band.

16. The method of claim 15 further comprising equalizing the energies of the output frame in the band to the maximum energy of the input frame in said band.

17. Apparatus for performing a specific process on at least one coded binary stream organized in the form of frames created from digital audio signals which were coded by at least an audio terminal to output processed frames to a subsequent using step, said coding of said digital audio signals including first transforming the digital audio signals from the time domain to the frequency domain in order to calculate transform coefficients, then quantizing and coding said transform coefficients according to a set of quantizers determined by selection parameters extracted from said digital audio signals, said frames including said selection parameters and the thus-coded transform coefficients, the apparatus comprising:

(1) a first stage for obtaining said selection parameters from said frames of at least one audio stream received from said at least one terminal and for determining from said selection parameters the set of quantizers used during the quantizing step performed by said audio terminal;

(2) a second stage for partially decoding and dequantizing said frames in response to the set of quantizers determined by said first stage and for recovering the transform coefficients calculated by said audio terminal;

(3) a third stage for performing said specific process in the frequency domain on the dequantized transform coefficients obtained by said second stage for producing processed frames; and

(4) a fourth stage for supplying said frames processed by said third stage to a subsequent utilization stage.

18. The apparatus of claim 17 wherein said subsequent utilization stage includes a stage for partially recoding the frames thus supplied by said fourth stage in response to a requantization and a recoding of the processed transform coefficients.

19. The apparatus of claim 18 wherein the third stage includes a summing arrangement for summing the dequantized transform coefficients obtained by the second stage partially decoding the frames belonging to streams of frames derived at a plurality of audio terminals.

20. The apparatus of claim 18 for use in a multi-terminal teleconferencing system wherein the third stage includes a summing arrangement for summing the dequantized transform coefficients obtained by the second stage partially decoding the frames belonging to streams of frames derived at a plurality of audio terminals, a subtracting arrangement for subtracting from said summed coefficients the dequantized transform coefficients derived by the second stage partial decoding frames coming from one of said audio terminals, and an arrangement for supplying the thus obtained result to said subsequent utilization stage concerning said audio terminal.

21. The apparatus of claim 17 wherein said subsequent utilization stage includes a reconstructing arrangement for transforming said processed frames from the frequency domain back in the time domain.

22. The apparatus of claim 21 for use in a multi-terminal teleconferencing system wherein the third stage includes a summing arrangement for summing the dequantized transform coefficients derived by the second state partially decoding the frames belonging to streams of frames derived by a plurality of audio terminals.

23. The apparatus of claim 22 wherein the summing arrangement is arranged to process the values of the quantizer set selection parameters.

24. The apparatus of claim 22 wherein the set of quantizer selection parameters in the audio frames of said at least stream includes a set of values representing the energies of the audio signals in predetermined frequency bands, the set being called a spectral envelope, said apparatus including a summing arrangement for summing the transformed coefficients derived by the second stage partially decoding each of the stream of frames and for supplying the result of the summation to a subsequent recoding utilization stage, and an arrangement for determining the total energy in each frequency band in response to a sum of the energies of the frames and supplying the result of the summation to said subsequent recoding utilization stage.

25. The apparatus of claim 24 for use in a multi-terminal teleconferencing system, said apparatus further including a summing arrangement for summing the transformed coefficients derived respectively by the second state partially decoding the streams of frames coming from the said terminals and for supplying to a terminal the result of said summation from which the transform coefficients derived by partially decoding the stream of frames coming from said terminal are subtracted, an arrangement for determining the total energy in each frequency band in response to a sum of the energies of the frames coming from all the terminals and for providing to a terminal the result of the summation from which the energy from the frame coming from said terminals is subtracted.

26. The apparatus of claim 17 wherein the audio frames of said at least one stream include voicing information on the corresponding audio signal, said apparatus further including an arrangement for determining the voicing information for the audio-signal derived by said processing state.

27. The apparatus of claim 26 further including an arrangement (a) for determining voicing information for the audio signal derived by said third stage and an arrangement for considering this voicing state as the audio signal state if all the frames of all the streams are determined to have the same voicing state, or if all the frames of all the streams are determined not to have the same voicing state, (b) (i) for determining the total energy of the set of audio signals of the voiced frames and the energy of the set of audio signals of the unvoiced frames, and (ii) then for considering the voicing state of the set with the greatest energy as being the voicing state of the audio signal derived from the third stage.

28. The apparatus of claim 17 wherein the audio frames of said at least one stream include tone information for the corresponding audio signal, said third stage further including an arrangement for determining tone information for the audio signal derived by said third stage as being the state of the signals of all the frames if all the frames are determined to be of the same kind.

29. The apparatus of claim 17 further including an arrangement for finding among all the frames the frame with the most energy in a given band, and for making the coefficients of the output frame equal to the coefficient of the frame with the most energy in a given band if the coefficients of the input frames other than the one with the most energy in a given band are masked by a masking threshold of the frame in the band.

30. The apparatus of claim 29 further comprising an arrangement for equalizing the energies of the output frame in the band to the maximum energy of the input frame in said band.

31. The apparatus of claim 18 wherein the requantizing stage of said subsequent utilization stage is arranged to perform a vectorial quantization with embedded dictionaries, said apparatus further including means for either setting the code word of an output band so it is the same as the codeword of an input band if the input dictionary corresponding to the input band is included in the selected output dictionary, or if the output dictionary is included in the input dictionary but the quantized vector, an element of the input dictionary, is also an element of the output dictionary;

or for performing a process including reverse quantization then requantization in the output dictionary.

32. The method of claim 31 wherein the dictionaries include a union of permutation codes, said apparatus further including means for either setting the code word for the output band so it is the same as the code word for the input band if the corresponding input dictionary for the band is included in the selected output dictionary, or in the opposite case where the output dictionary is included in the input dictionary but the quantized vector, an element of the input dictionary, is also an element of the output dictionary, or otherwise for reverse quantizing then requantizing the output dictionary, the requantization procedure being sped up in the closest neighbor of a leader of a vector of the input dictionary that is a leader of the output dictionary.