Adaptive joint source channel coding

• US 20050031039A1
• Filed: 06/25/2004
• Published: 02/10/2005
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Tabatabai, Ali Jezu, Chou, Jim Chen
• Granted Patent: US 8,705,613 B2
• Time in Patent Office: Days
• US Class Current: 375/240.180
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/619 the transform being operated outside the prediction loop
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/94 Vector quantisation

Abstract

Adaptive joint source channel coding associates multiple predictors with a reference data unit, such as a macroblock or frame of video data. An encoder determines a sub-codebook in which each of the selected multiple predictors decodes to the reference data unit. An identifier for the sub-codebook is transmitted through a channel to a decoder for subsequent decoding of the reference data unit. The reference data unit itself does not need to be sent. The multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region. The decoder uses the identified sub-codebook and one of the predictors to decode the reference data unit. If none of the original predictors are correctly received, different types of error handling are employed based on the type of channel.

Claims

1. A computerized method comprising:

selecting multiple predictors for a reference unit of correlated data;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

2. The computerized method of claim 1 further comprising:

transmitting the reference unit.

3. The computerized method of claim 1, wherein the identifier is a compressed representation of the reference unit.

4. The computerized method of claim 1, wherein the multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region.

5. The computerized method of claim 4, wherein the decoding region is defined within an n-dimensional coefficient space.

6. The computerized method of claim 1, wherein determining a sub-codebook comprises:

partitioning a codebook until the sub-codebook is created.

7. The computerized method of claim 6, wherein the codebook comprises lattice partitions.

8. The computerized method of claim 1 further comprising:

determining that none of the multiple predictors were correctly received for decoding;

selecting an additional predictor; and

determining a new sub-codebook that decodes the additional predictor to the reference frame.

9. The computerized method of claim 8 further comprising:

sending an additional identifier for the new sub-codebook.

10. The computerized method of claim 9, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

11. The computerized method of claim 1, wherein the predictors are selected based on a maximum error calculation.

12. The computerized method of claim 1, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

13. The computerized method of claim 1, wherein the reference unit is a frame in a first video stream and the predictors are selected from frames in the first video stream and frames in a second video stream.

14. The computerized method of claim 1, wherein the reference unit is a frame in a first channel and the predictors are selected from frames in the first channel and frames in a second channel.

15. The computerized method of claim 1, wherein the reference unit is a frame at a first resolution and the predictors are selected from frames at a second resolution.

16. The computerized method of claim 1, wherein the reference unit is a delta frame in a video stream and the predictors are selected from an intraframe and other delta frames.

17. The computerized method of claim 1, wherein the reference unit is a delta frame in a video stream and one of the predictors is an intraframe.

18. The computerized method of claim 1, wherein the identifier is used in place of a prediction error in a lifting scheme for discrete wavelet transformation.

19. The computerized method of claim 1 further comprising:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

20. The computerized method of claim 1 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

21. The computerized method of claim 1 further comprising:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

22. The computerized method of claim 1 further comprising:

determining erroneous predictors based on decoding a forbidden symbol in a stream of units encoded using arithmetic coding.

23. The computerized method of claim 22, wherein the erroneous predictors are determined according to 1/ε

, where ε

represents a probability that the forbidden symbol will occur in the stream.

24. A computerized method comprising:

receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit; and

decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

25. The computerized method of claim 24 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

26. The computerized method of claim 24 further comprising:

receiving an identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

27. A machine-readable medium having instructions to cause a processor to execute a method comprising:

selecting multiple predictors for a reference unit of correlated data;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

28. The machine-readable medium of claim 27, wherein the method further comprises:

transmitting the reference unit.

29. The machine-readable medium of claim 27, wherein the identifier is a compressed representation of the reference unit.

30. The machine-readable medium of claim 27, wherein the multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region.

31. The machine-readable medium of claim 30, wherein the decoding region is defined within an n-dimensional coefficient space.

32. The machine-readable medium of claim 27, wherein determining a sub-codebook comprises:

partitioning a codebook until the sub-codebook is created.

33. The machine-readable medium of claim 32, wherein the codebook comprises lattice partitions.

34. The machine-readable medium of claim 27, wherein the method further comprises:

determining that none of the multiple predictors were correctly received for decoding;

selecting an additional predictor; and

determining a new sub-codebook that decodes the additional predictor to the reference frame.

35. The machine-readable medium of claim 34, wherein the method further comprises:

sending an additional identifier for the new sub-codebook.

36. The machine-readable medium of claim 35, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

37. The machine-readable medium of claim 27, wherein the predictors are selected based on a maximum error calculation.

38. The machine-readable medium of claim 27, wherein the method further comprises:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

39. The machine-readable medium of claim 27, wherein the method further comprises:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

40. The machine-readable medium of claim 27, wherein the method further comprises:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

41. A machine readable medium having instructions to cause a processor to execute a method comprising:

receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit; and

decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

42. The machine medium of claim 41, wherein the method further comprises:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

43. The machine medium of claim 41, wherein the method further comprises:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

44. A system comprising:

a processor coupled to a memory through a bus; and

an encoding process executed by the processor from the memory to cause the processor to select multiple predictors for a reference unit of correlated data, determine a sub-codebook in which each of the predictors decodes to the reference unit, and transmit an identifier for the sub-codebook to decode the reference unit from one of the predictors.

45. The system of claim 44, wherein the encoding process further causes the processor to transmit the reference unit.

46. The system of claim 44, wherein the identifier is a compressed representation of the reference unit.

47. The system of claim 44, wherein the multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region.

48. The system of claim 47, wherein the decoding region is defined within an n-dimensional coefficient space.

49. The system of claim 44, wherein the encoding process further causes the processor to partition a codebook until the sub-codebook is created when determining a sub-codebook.

50. The system of claim 49, wherein the codebook comprises lattice partitions.

51. The system of claim 44, wherein the encoding process further causes the processor to determine that none of the multiple predictors were correctly received for decoding, select an additional predictor, and determine a new sub-codebook that decodes the additional predictor to the reference frame.

52. The system of claim 51, wherein the encoding process further causes the processor to send an additional identifier for the new sub-codebook.

53. The system of claim 52, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

54. The system of claim 44, wherein the predictors are selected based on a maximum error calculation.

55. A system comprising:

a processor coupled to a memory through a bus; and

a decoding process executed from the memory by the processor to cause the processor to receive an identifier correspond to a sub-codebook in which each of multiple predictors decodes to a reference unit, and decode the reference unit from one correctly received predictor us the sub-codebook identified by the identifier.

56. The system of claim 55, wherein the decoding process further causes the processor to estimate the predictors if no predictors are correctly received, and decode the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

57. The system of claim 55, wherein the decoding process further causes the processor to receive an additional identifier for a new sub-codebook, and decode the reference unit from an additional predictor and the new sub-codebook.

58. An apparatus comprising:

means for selecting multiple predictors for a reference unit of correlated data;

means for determining a sub-codebook in which each of the predictors decodes to the reference unit; and

means for transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

59. The apparatus of claim 58 further comprising:

means for transmitting the reference unit.

60. The apparatus of claim 58, wherein the identifier is a compressed representation of the reference unit.

61. The apparatus of claim 58, wherein the multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region.

62. The apparatus of claim 61, wherein the decoding region is defined within an n-dimensional coefficient space.

63. The apparatus of claim 58, wherein the means for determining a sub-codebook comprises:

means for partitioning a codebook until the sub-codebook is created.

64. The apparatus of claim 63, wherein the codebook comprises lattice partitions.

65. The apparatus of claim 58 further comprising:

means for determining that none of the multiple predictors were correctly received for decoding;

means for selecting an additional predictor; and

means for determining a new sub-codebook that decodes the additional predictor to the reference frame.

66. The apparatus of claim 65 further comprising:

means for sending an additional identifier for the new sub-codebook.

67. The apparatus of claim 66, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

68. The apparatus of claim 58, wherein the predictors are selected based on a maximum error calculation.

69. An apparatus comprising:

means for receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit; and

means for decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

70. The apparatus of claim 69 further comprising:

means for estimating the predictors if no predictors are correctly received; and

means for decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

71. The apparatus of claim 69 further comprising:

means for receiving an additional identifier for a new sub-codebook; and

means for decoding the reference unit from an additional predictor and the new sub-codebook.