Low-Complexity Packet Loss Concealment for Transcoded Audio Signals

• US 20190311723A1
• Filed: 04/09/2019
• Published: 10/10/2019
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: ULLMANN, Raphael Marc, DICKINS, Glenn N.
• Granted Patent: US 11,107,481 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: G10L 19/173 Transcoding, i.e. converting between two coded representations avoiding cascaded coding-decoding
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Systems and methods are described for concealing packet loss in a received audio stream. Packets of the audio stream may be received in a non-lapped transform domain format, where at least one packet is missing in the stream. The received packets are decoded, and each missing packet in the decoded stream is replaced by a reduced-energy signal block. Each reduced-energy signal block may also be modified at a beginning or ending boundary, and shifted such that a start or end of each missing packet does not coincide with a peak of a transform window of a lapped transform domain format. The raw audio signal may then be encoded into transform windows having the lapped transform domain format. Packet loss concealment may then be performed for selected transform windows that include modified reduced-energy blocks, either prior to transmission or after transmission by the receiving endpoint.

Claims

1. A method for concealing packet loss in a received audio stream, the method comprising:

receiving, by an audio communication system comprising one or more servers, packets of an audio stream, the received audio stream being in a non-lapped transform domain format, the packets having a sequential order, where a block of packets is missing in the sequence, the block of packets including at least one packet, each packet of the audio stream having a received window size;

decoding, by the audio communication system, the received packets to a sequence of blocks of a raw audio signal;

replacing, by the audio communication system, each missing packet in the sequence with a reduced-energy signal block, whereby the sequence of blocks of the raw audio signal comprises at least one reduced-energy signal block, wherein the reduced-energy signal block has a lower energy in comparison to remaining blocks of the raw audio signal, such that the reduced-energy signal block is distinguishable from remaining blocks of the received raw audio signal;

modifying, by the audio communication system, each reduced-energy signal block of the raw audio signal at one of a beginning boundary and an ending boundary;

shifting, by the audio communication system, the raw audio signal by a predetermined delay such that a start and end of each reduced energy block of the raw audio signal does not coincide with a peak of a transform window of a lapped transform domain format;

encoding, by the server, the raw audio signal into transform windows having the lapped transform domain format, wherein packet loss concealment is performed for a selected transform window of the encoded audio signal, the selected transform window having a modified reduced energy signal block located at a center portion; and

transmitting, by the audio communication system, the encoded audio signal to an endpoint.

2. The method of claim 1, wherein the transmitted audio signal, when received by a listener endpoint, is decoded to output raw audio, the decoding comprising overlapping the transform window containing the packet loss concealment with adjacent transform windows containing received audio packets.

3. The method of claim 1, the packet loss concealment comprising replacing the selected transform window that includes the modified reduced-energy signal block with a copy of a previous transform window of the encoded audio signal.

4. The method of claim 3, the at least one missing packet including multiple consecutive packets in the sequence, where the copy of the previous transform window is used to replace each transform window that includes the missing multiple consecutive packets.

5. The method of claim 3, the at least one missing packet including multiple consecutive packets in the sequence, where a copy of two transform windows preceding the multiple consecutive missing packets is used to replace one or more pairs of transform windows that include the multiple consecutive packets.

6. The method of claim 3, the at least one missing packet including multiple consecutive packets in the sequence, where the selected transform window has a decreased gain compared to a previous transform window.

7. The method of claim 1, the modifying each reduced-energy signal block comprising replacing a portion of each reduced-energy signal block near the ending boundary with a time-reversed copy of the raw audio signal packet immediately following each reduced-energy signal block.

8. The method of claim 7, wherein the portion of each reduced-energy signal block has a duration of ¼

a length of the transform window size.

9. The method of claim 1, the modifying each reduced-energy signal block comprising replacing a portion of each reduced-energy signal block near the beginning boundary with a time-reversed copy of a raw audio signal packet immediately preceding each reduced-energy signal block.

10. The method of claim 1, the modifying each reduced-energy signal block comprising replacing a portion of each reduced-energy signal block near the beginning boundary with a signal represented as g*(x0-x1_reversed), where g is a gain used in the packet loss concealment, x0 is a raw audio signal segment, having ¼

the transform window length, occurring ½

the transform window length before the block of silence, and x1_reversed is a time-reversed copy of a raw audio signal segment, having ¼

the transform window length, immediately preceding the reduced-energy signal block.

11. The method of claim 1, wherein each transform window of the lapped transform domain format comprises a plurality of audio samples, audio samples near boundaries of each transform window being encoded with a lower gain than audio samples near a center of each transform window.

12. The method of claim 1 the performing packet loss concealment further comprising at least one of dithering, decorrelation, and stochastic processing to reduce correlation and potential artifacts due to the encoding into the lapped transform domain format.

13. The method of claim 1, where the encoding the raw audio signal further comprises generating metadata to identify the presence of transform windows that include packet loss concealment-generated data, the method further comprising transmitting the generated metadata for downstream decisions about decoding or rendering the encoded audio signal.

14. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:

receive packets of an audio stream, the received audio stream being in a non-lapped transform domain format, the packets having a sequential order, where at least one packet is missing in the sequence, each packet having a received window size;

decode the received packets to a sequence of blocks of a raw audio signal;

replace each missing packet in the sequence with a reduced-energy signal block, whereby the sequence of blocks of the raw audio signal comprises at least one reduced-energy signal block, wherein the reduced-energy signal block has a lower energy in comparison to remaining blocks of the raw audio signal, such that the reduced-energy signal block is distinguishable from remaining blocks of the received raw audio signal;

modify each reduced-energy signal block of the raw audio signal at one of a beginning boundary and an ending boundary of the inserted blocks;

shift the raw audio signal by a predetermined delay such that a start and end of each reduced energy block of the raw audio signal does not coincide with a peak of a transform window of a lapped transform domain format;

encode the raw audio signal into transform windows having the lapped transform domain format, packet loss concealment being performed for a selected transform window, of the encoded audio signal, having a modified reduced energy signal block located at a center portion of the selected transform window; and

transmit the encoded audio signal.

15. A system for concealing packet loss in a received audio stream, the system comprising:

a communications transceiver that receives packets of an audio stream, the received audio stream being in a non-lapped transform domain format, the packets having a sequential order, where at least one packet is missing in the sequence, each packet having a received window size;

a decoder that decodes the received packets from the non-lapped format to a sequence of blocks of a raw audio signal and replaces each missing packet in the sequence with a reduced-energy signal block, whereby the sequence of blocks of the raw audio signal comprises at least one reduced-energy signal block, wherein the reduced-energy signal block has a lower energy in comparison to remaining blocks of the raw audio signal, such that the reduced-energy signal block is distinguishable from remaining blocks of the received raw audio signal;

a preprocessor that modifies each reduced-energy signal block of the raw audio signal at one of a beginning boundary and an ending boundary of the blocks of silence and shifts the raw audio signal by a predetermined delay such that a start and end of each reduced energy block of the raw audio signal does not coincide with a peak of a transform window of a lapped transform domain format; and

an encoder that encodes the raw audio signal into transform windows having the lapped transform domain format, the communications transceiver transmitting the encoded audio signal to one or more endpoints, packet loss concealment being performed for a selected transform window, of the encoded audio signal, having a modified reduced energy signal block located at a center portion of the selected transform window.

16. The system of claim 15, the encoder encoding blocks of the raw audio signal that include audio samples and marking any transform window, including the selected transform window, that includes reduced-energy signal blocks for replacement, the packet loss concealment comprising replacing, by the encoder or by the one or more endpoints, the marked transform windows with copies of previous transform windows of the encoded audio signal.

17. The system of claim 15, the decoder being modified to not attempt to compensate for any detected missing packets in the received audio stream.

18. The system of claim 15, the preprocessor being a part of the encoder, where the preprocessor receives the received window size of the audio stream as a sideband communication from the decoder and determines the predetermined delay based on the received window size and the transform window, the predetermined delay being determined such that the start and end of each block of the audio stream does not coincide with a center of the transform window.

19. The system of claim 18, where the reduced-energy signal block is a block of silence.