Scalable audio communications utilizing rate-distortion based end-to-end bit allocation

• US 7,283,966 B2
• Filed: 04/19/2002
• Issued: 10/16/2007
• Est. Priority Date: 03/07/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Qian, Zhu, Wenwu
• Primary Examiner(s): Knepper; David D.
• Application Number:
  US10/125,987
• Publication Number:
  US 20030171934A1
• Time in Patent Office: 2,006 Days
• Field of Search: None
• US Class Current: 704/500
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding

Abstract

A source encoder encodes audio signals into increasing quality layers defined in bit planes. Each bit plane has a data unit that includes a beginning partition having one or more contiguous refinement bits, a second partition having one or more contiguous coded significance bits, a third partition having one or more contiguous sign boundary mark bits, and a fourth partition having one or more contiguous coded sign bits. A channel encoder encodes the bit planes into respective columns containing multiple rows. Unequal error protection coding is provided according to the quality of each layer such that each row has row and column channel protection codes for the respective row and column that correspond to the respective quality layer. For the corresponding row and column, each row contains the row channel protection codes and either the compressed audio data from the respective layer or the column channel protection codes. A server machine can use a network feedback transmission to allocate bits to the source encoder and the channel encoder.

Claims

1. A method comprising:

encoding compressed audio data into increasing quality layers;

logically arranging each of the quality layers of the encoded compressed audio data into columns and the columns into rows; and

applying row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

2. The method as defined in claim 1, further comprising:

signal-processing input audio signals;

quantizing the signal-processing input audio signals into quantized data of weighted subbands; and

bit-plane coding the quantized data into the encoded compressed audio data logically arranged into increasing quality layers and being defined in an embedded audio bitstream of bit planes, wherein;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

3. The method as defined in claim 2, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

4. The method as defined in claim 2, wherein:

said quantizing quantizes using a variable length coding algorithm having a finite code;

the bit-plane coding executes a predetermined coding method; and

the predetermined coding method generates the third partition as an invalid codeword for the predetermined coding method.

5. The method as defined in claim 4, wherein the invalid codeword has a significant Hamming distance from valid codewords of the predetermined coding method.

6. A computer-readable medium encoded with computer-executable instructions, which when executed on a processor, direct a computer to perform claim 1.

9. The method as defined in claim 1, wherein, as compared to other said columns, the column including the base layer has:

more of said packets containing said column FEC symbols;

more of said row FEC symbols in the respective rows thereof; and

less of said the compressed audio data.

7. A method comprising:

source encoding audio data into compressed audio data logically arranged into a base layer and a plurality of increasing quality enhancement layers;

channel encoding each of the base and enhancement layers into a respective column logically arranged into a plurality of rows;

adding column Forward Error Correction (FEC) symbols to the respective column that corresponds to the respective base or enhancement layer; and

adding row FEC symbols to the respective row that corresponds to the respective base or enhancement layer, wherein;

each said row includes a packet;

each said column includes a plurality of said packets; and

each said packet includes;

the row FEC symbols for the respective row; and

one of;

the compressed audio data from one of the base and enhancement layers for the corresponding row and column;

or

the column FEC symbols for the corresponding row and column.

8. The method as defined in claim 7, wherein for any said column including one said layer that is of higher quality than that of another said column:

the number of said packets containing the column FEC symbols is lower;

the number of said row FEC symbols in the respective rows is lower; and

the compressed audio data in each of the respective packets is higher.

10. The method as defined in claim 7, wherein the source encoding audio data into compressed audio data logically arranged into a base layer and a plurality of increasing quality enhancement layers comprises:

signal-processing input audio signals;

quantizing the signal-processing input audio signals into quantized data of weighted subbands; and

bit-plane coding the quantized data into the encoded compressed audio data logically arranged into increasing quality layers and being defined in an embedded audio bitstream of bit planes, wherein;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

11. The method as defined in claim 10, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

12. The method as defined in claim 10, wherein:

said quantizing quantizes using a variable length coding algorithm having a finite code;

the bit-plane coding executes a predetermined coding method; and

the predetermined coding method generates the third partition as an invalid codeword for the predetermined coding method.

13. The method as defined in claim 12, wherein the invalid codeword has a significant Hamming distance from valid codewords of the predetermined coding method.

14. A computer-readable medium encoded with computer-executable instructions, which when executed on a processor, direct a computer to perform claim 7.

15. A scalable audio coding apparatus comprising:

a channel encoder for;

logically arranging encoded compressed audio data into increasing quality layers, each layer being logically arranged into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

16. The scalable audio coding apparatus as defined in claim 15, further comprising:

a signal processor for signal-processing input audio signals;

a quantizer for quantizing the signal processed input audio signals into quantized data of weighted subbands; and

a source encoder for bit-plane coding the quantized data into the encoded compressed audio data logically arranged into increasing quality layers and being defined in an embedded audio bitstream of bit planes, wherein;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

17. The scalable audio coding apparatus as defined in claim 16, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

18. The scalable audio coding apparatus as defined in claim 16, wherein:

the quantizer quantizes using a variable length coding algorithm having a finite code;

the bit-plane coding of encoder executes a predetermined coding method; and

the predetermined coding method generates the third partition as an invalid codeword for the predetermined coding method.

19. The scalable audio coding apparatus as defined in claim 18, wherein the invalid codeword has a significant Hamming distance from valid codewords of the predetermined coding method.

20. A computer usable medium having embodied thereon a computer program for coding audio signals, the computer program comprising:

a first code segment to;

logically arrange compressed audio data into increasing quality layers, each layer being arranged into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

21. The computer usable medium as defined in claim 20, wherein the computer program further comprises:

a second code segment for signal-processing input audio signals;

a third code segment for quantizing the signal processed input audio signals into quantized data of weighted subbands;

a fourth code segment to affect bit-plane coding the quantized data into an embedded audio bitstream of bit planes that are logically arranged into the increasing quality layers, wherein;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

22. The computer usable medium as defined in claim 21, wherein the computer program further comprises:

a fifth code segment to form a last partition of said data having dummy zeros, whereby the data unit is byte-aligned.

23. The computer usable medium as defined in claim 21, wherein:

a fifth code segment uses a variable length coding algorithm having a finite code to quantize the signal processed input audio signals into quantized data of weighted subbands;

a sixth code segment executes a predetermined bit-plane coding method; and

the predetermined bit-plane coding method generates the third partition as an invalid codeword for the predetermined bit-plane coding method.

24. The computer usable medium as defined in claim 23, wherein the invalid codeword has a significant Hamming distance from valid codewords of the predetermined bit-plane coding method.

25. A data structure, embodied on one or more computer-readable media, comprising compressed audio data logically arranged into increasing quality layers into respective columns, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein:

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

26. The data structure as defined in claim 25, wherein:

the increasing quality layers comprise a base layer and a plurality of enhancement layers;

the column protection codes comprise column FEC symbols that correspond to the respective base or enhancement layer; and

the row protection codes comprise row FEC symbols that correspond to the respective base or enhancement layer, wherein;

each said row includes a packet;

each said column includes a plurality of said packets; and

each said packet includes;

the row FEC symbols for the respective row; and

one of;

the compressed audio data from one of the base and enhancement layers for the corresponding row and column;

or

the column FEC symbols for the corresponding row and column.

27. The data structure as defined in claim 26, wherein for any said column including one said layer that is of higher quality than that of another said column:

the number of said packets containing the column FEC symbols is lower;

the number of said row FEC symbols in the respective rows is lower; and

the compressed audio data in each of the respective packets is higher.

28. The data structure as defined in claim 26, wherein, as compared to other said columns, the column including the base layer has:

more of said packets containing said column FEC symbols;

more of said row FEC symbols in the respective rows thereof; and

less of said the compressed audio data.

29. The data structure as defined in claim 25, wherein the compressed audio data logically arranged into increasing quality layers is defined in an embedded audio bitstream of bit planes, wherein:

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

30. The data structure as defined in claim 29, further comprising one or more zero in an end portion of the data structure, whereby the data structure is aligned with a respective byte.

31. The data structure as defined in claim 29, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

32. The data structure as defined in claim 29, wherein the third partition has an invalid codeword for a predetermined coding method used to form the compressed audio data logically arranged into increasing quality layers.

33. The data structure as defined in claim 32, wherein the invalid codeword has a significant hamming distance from valid codewords of the predetermined coding method.

34. A method comprising reconstructing packets of compressed audio data of increasing quality layers by arranging each said layer into a respective column and each said column into rows, wherein:

each said row has row and column protection codes for the respective row and column that correspond to the respective layer;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

35. The method as defined in claim 34, further comprising:

decoding the rows and columns, wherein;

the rows and columns define an embedded audio bitstream of bit-planes;

the decoding decodes the embedded audio bitstream of bit-planes into quantized data of weighted subbands;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits, wherein the third partition is between the second and fourth partitions; and

dequantizing the quantized data of weighted subbands into audio signals.

36. The method as defined in claim 35, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

37. The method as defined in claim 35, wherein the second code segment decodes using Reversible exponential Golomb (Exp-Golomb) codewords in a Reversible Variable Length Code (RVLC) algorithm.

38. The method as defined in claim 37, wherein the decoding:

decodes each said second partition having one or more contiguous coded significance bits using Reversible Exp-Golomb codewords that include a variable-length prefix part and a fixed-length suffix part;

performs error detection in the variable-length prefix of the coded significance bits in both forward and backward directions to detect an invalid codeword; and

identifies a location of the invalid codeword upon detection.

39. The method as defined in claim 38, wherein, upon identification of the location of the invalid codeword, the decoding:

compares a result of the error detection in the forward direction with a result of the error detection in the backward direction; and

accepts, for the decoding of the second partition, identical portions of the variable-length prefix of the coded significance bits as determined by the results of the error detection in the forward and backward directions.

40. A scalable audio decoding apparatus comprising:

a channel decoder to reconstruct packets of compressed audio data of increasing quality layers and to arrange each said layer into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

41. The scalable audio decoding apparatus as defined in claim 40, further comprising:

a source decoder to decode the rows and columns, wherein;

the rows and columns define an embedded audio bitstream of bit-planes;

the source decoder decodes the embedded audio bitstream of bit-planes into quantized data of weighted subbands;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits;

the third partition is between the second and fourth partitions; and

an inverse quantizer to dequantize the quantized data of weighted subbands into audio signals.

42. The scalable audio decoding apparatus as defined in claim 41, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

43. The scalable audio decoding apparatus as defined in claim 41, wherein the source decoder decodes using Reversible exponential Golomb (Exp-Golomb) codewords in a Reversible Variable Length Code (RVLC) algorithm.

44. The scalable audio decoding apparatus as defined in claim 43, wherein the source decoder:

decodes each said second partition having one or more contiguous coded significance bits using Reversible Exp-Golomb codewords that include a variable-length prefix part and a fixed-length suffix part;

performs error detection in the variable-length prefix of the coded significance bits in both forward and backward directions to detect an invalid codeword; and

identifies a location of the invalid codeword upon detection.

45. The scalable audio decoding apparatus as defined in claim 44, wherein, upon identification of the location of the invalid codeword, the source decoder:

compares a result of the error detection in the forward direction with a result of the error detection in the backward direction; and

accepts, for the decoding of the second partition, identical portions of the variable-length prefix of the coded significance bits as determined by the results of the error detection in the forward and backward directions.

46. A computer usable medium having embodied thereon a computer program for coding audio signals, the computer program comprising:

a first code segment to reconstruct packets of compressed audio data of increasing quality layers and to arrange each said layer into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater.

47. The computer usable medium as defined in claim 46, wherein the computer program further comprises:

a second code segment to decode the rows and columns, wherein;

the rows and columns define an embedded audio bitstream of bit-planes;

the second code segment decodes the embedded audio bitstream of bit-planes into quantized data of weighted subbands;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions; and

an inverse quantizer to dequantize the quantized data of weighted subbands into audio signals.

48. The computer usable medium as defined in claim 47, wherein each said data unit further comprises a last partition having dummy zeros, whereby the data unit is byte-aligned.

49. The computer usable medium as defined in claim 47, wherein the second code segment decodes using Reversible exponential Golomb (Exp-Golomb) codewords in a Reversible Variable Length Code (RVLC) algorithm.

50. The computer usable medium as defined in claim 49, wherein the second code segment:

decodes each said second partition having one or more contiguous coded significance bits using Reversible Exp-Golomb codewords that include a variable-length prefix part and a fixed-length suffix part;

performs error detection in the variable-length prefix of the coded significance bits in both forward and backward directions to detect an invalid codeword; and

identifies a location of the invalid codeword upon detection.

51. The computer usable medium as defined in claim 50, wherein, upon identification of the location of the invalid codeword, the second code segment:

compares a result of the error detection in the forward direction with a result of the error detection in the backward direction; and

accepts, for the decoding of the second partition, identical portions of the variable-length prefix of the coded significance bits as determined by the results of the error detection in the forward and backward directions.

52. A system comprising

a sender apparatus including:

a source encoder for coding compressed audio data into logically arranged increasing quality layers;

a channel encoder for logically arranging each layer into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater;

a sender transmitter element for sending a transmission of the rows and columns;

an interconnected network in communication with the sender apparatus; and

a receiver apparatus in communication with the interconnected network and including;

a receiver reception element for receiving the transmission of the rows and the columns in a plurality of packets over the interconnected network;

a channel decoder to reconstruct the plurality of packets into the logical arrangement of the rows and the columns; and

a source decoder to decode the rows and columns into audio signals.

53. The system as defined in claim 52, wherein:

the source encoder bit-plane codes quantized data of weighted subbands into the encoded compressed audio data logically arranged into increasing quality layers;

the rows and columns define an embedded audio bitstream of bit-planes;

the source decoder decodes the embedded audio bitstream of bit-planes into quantized data of weighted subbands; and

an inverse quantizer dequantizes the quantized data of weighted subbands into the audio signals.

54. The system as defined in claim 52, wherein:

the increasing quality layers are defined in an embedded audio bitstream of bit planes;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions.

55. The system as defined in claim 52, wherein:

the receiver apparatus further comprises;

a network monitor for monitoring the interconnected network; and

a receiver transmitter element for sending a transmission reflecting the monitoring of the interconnected network in a network feedback transmission addressed to the sender apparatus; and

the sender apparatus further comprises;

a sender reception element for receiving the network feedback transmission; and

means for using the network feedback transmission to allocate bits to the source encoder and the channel encoder.

56. The system as defined in claim 55, wherein:

the network monitor monitors the interconnected network for;

the Bit Error Rate (BER);

the fading depth;

the mobile speed of the receiver apparatus;

the transmission delay; and

the packet loss ratio of the packets;

the receiver transmitter element transmits;

the transmission of the network feedback transmission to the sender apparatus in an IP protocol over the interconnected network;

the BER, the fading depth, and the mobile speed of the receiver apparatus on the physical layer;

the transmission delay on the data link layer; and

the packet loss ratio on the application layer; and

the sender apparatus allocate bits to the source encoder and the channel encoder by deriving a status of the interconnected network and an estimate of the available bandwidth of the interconnected network from;

the Bit Error Rate (BER);

the fading depth;

the mobile speed of the receiver apparatus;

the transmission delay; and

the packet loss ratio of the packets.

57. The system as defined in claim 52, wherein:

the sender apparatus uses the network feedback transmission to allocate bits to;

the source encoder to bit-plane code the quantized data into encoded compressed audio data; and

the channel encoder to arrange the rows and the columns.

58. A system comprising:

a server machine including;

a signal processor for signal-processing input audio signals;

a quantizer for quantizing the signal processed input audio signals into quantized data of weighted subbands;

a source encoder for bit-plane coding the quantized data into encoded compressed audio data logically arranged into increasing quality layers and being defined in an embedded audio bitstream of bit planes, wherein;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions;

a channel encoder for;

logically arranging the encoded compressed audio data logically arranged into increasing quality layers, each layer being logically arranged into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater;

a sender reception element for receiving a network feedback transmission; and

a sender transmitter element for sending a transmission of the rows and columns;

an interconnected network in communication with the sewer machine; and

a client machine in communication with the interconnected network and including;

a network monitor for monitoring a status of the interconnected network;

a receiver transmitter element for sending a transmission of the status of the interconnected network in the network feedback transmission to the server machine;

a receiver reception element for receiving a transmission of a plurality of packets containing the rows and the columns;

a channel decoder to reconstruct the packets of compressed audio data of increasing quality layers and to arrange each said layer into a respective column, each column being logically arranged into rows, each row having row and column protection codes for the respective row and column that correspond to the respective layer, wherein;

for the corresponding row and column, each row contains the row protection codes and one of;

the compressed audio data from the respective layer;

or

the column protection codes; and

for any said column including one said layer that is of higher quality than that of another said column, the row and column protection codes are fewer and the compressed audio data is greater;

a source decoder to decode the rows and columns, wherein;

the rows and columns define an embedded audio bitstream of bit-planes;

the source decoder decodes the embedded audio bitstream of bit-planes into quantized data of weighted subbands;

the embedded audio bitstream includes binary data having bits;

each said bit-plane has a data unit that includes;

a beginning partition having one or more contiguous refinement bits;

a second partition having one or more contiguous coded significance bits;

a third partition having one or more contiguous sign boundary mark bits; and

a fourth partition having one or more contiguous coded sign bits; and

the third partition is between the second and fourth partitions; and

an inverse quantizer to dequantize the quantized data of weighted subbands into audio signals;

wherein the server machine uses the network feedback transmission to allocate bits to;

the source encoder to bit-plane code the quantized data into encoded compressed audio data; and

the channel encoder to logically arrange the rows and the columns.