Adaptive joint source channel coding

• US 8,705,613 B2
• Filed: 06/25/2004
• Issued: 04/22/2014
• Est. Priority Date: 06/26/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Chou, Jim Chen, Tabatabai, Ali Jezu
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US10/877,589
• Publication Number:
  US 20050031039A1
• Time in Patent Office: 3,588 Days
• Field of Search: 375/240.12, 375/240.13, 375/240.16, 375/240.15, 375/240.25
• US Class Current: 375/240.01
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/619 the transform being operated outside the prediction loop
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/94 Vector quantisation

Abstract

Adaptive joint source channel coding associates multiple predictors with a reference data unit, such as a macroblock or frame of video data. An encoder determines a sub-codebook in which each of the selected multiple predictors decodes to the reference data unit. An identifier for the sub-codebook is transmitted through a channel to a decoder for subsequent decoding of the reference data unit. The reference data unit itself does not need to be sent. The multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region. The decoder uses the identified sub-codebook and one of the predictors to decode the reference data unit. If none of the original predictors are correctly received, different types of error handling are employed based on the type of channel.

Claims

1. A computerized method comprising:

selecting, by an encoding device, multiple predictors for a single reference unit of correlated data, the multiple predictors located within a decoding region centered on the single reference unit in an n-dimensional coordinate space and comprising other data units in the correlated data;

determining a sub-codebook for the decoding region in which each of the multiple predictors within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors, the sub-codebook having an identifier that further specifies the decoding region;

transmitting the multiple predictors to a separate decoder; and

transmitting the identifier for the sub-codebook as the compressed representation of the single reference unit, wherein the separate decoder recreates the entire single reference unit using the identified sub-codebook to decode any one of the multiple predictors within the specified decoding region that are correctly received by the separate decoder.

2. The computerized method of claim 1, wherein determining a sub-codebook comprises:

partitioning a codebook until the sub-codebook is created.

3. The computerized method of claim 2, wherein the codebook comprises lattice partitions.

4. The computerized method of claim 1 further comprising:

determining that none of the multiple predictors were correctly received for decoding;

selecting an additional predictor; and

determining a new sub-codebook that decodes the additional predictor to the reference frame.

5. The computerized method of claim 4 further comprising:

sending an additional identifier for the new sub-codebook.

6. The computerized method of claim 5, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

7. The computerized method of claim 1, wherein the predictors are selected based on a maximum error calculation.

8. The computerized method of claim 1, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

9. The computerized method of claim 1, wherein the reference unit is a frame in a first video stream and the predictors are selected from frames in the first video stream and frames in a second video stream.

10. The computerized method of claim 1, wherein the reference unit is a frame in a first channel and the predictors are selected from frames in the first channel and frames in a second channel.

11. The computerized method of claim 1, wherein the reference unit is a frame at a first resolution and the predictors are selected from frames at a second resolution.

12. The computerized method of claim 1, wherein the reference unit is a delta frame in a video stream and the predictors are selected from an intraframe and other delta frames.

13. The computerized method of claim 1, wherein the reference unit is a delta frame in a video stream and one of the predictors is an intraframe.

14. The computerized method of claim 1, wherein the identifier is used in place of a prediction error in a lifting scheme for discrete wavelet transformation.

15. The computerized method of claim 1 further comprising:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

16. The computerized method of claim 1 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

17. The computerized method of claim 1 further comprising:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

18. The computerized method of claim 1 further comprising:

determining erroneous predictors based on decoding a forbidden symbol in a stream of units encoded using arithmetic coding.

19. The computerized method of claim 18, wherein the erroneous predictors are determined according to 1/ε

, where ε

represents a probability that the forbidden symbol will occur in the stream.

56. The computerized method of claim 1, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

20. A computerized method comprising:

receiving, by an encoder from separate encoding device, an identifier as a compressed representation of a single reference unit, the identifier specifying a decoding region centered on the single reference unit within an n-dimensional coordinate space and further corresponding to a sub-codebook for the decoding region in which each of multiple predictors located within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors;

receiving the multiple predictors, wherein fewer than all of the multiple predictors within the decoding region are correctly received; and

recreating the entire single reference unit using the identified sub-codebook to decode any one of the correctly received multiple predictors.

21. The computerized method of claim 20 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

22. The computerized method of claim 20 further comprising:

receiving an identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

57. The computerized method of claim 20, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

23. A machine-readable non-transitory storage medium storing instructions which when executed by a processor cause the processor to execute a method comprising:

selecting, by an encoder, multiple predictors for a single reference unit of correlated data, the multiple predictors located within a decoding region centered on the single reference unit in an n-dimensional coordinate space and comprising other data units in the correlated data;

determining a sub-codebook for the decoding region in which each of the multiple predictors within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors, the sub-codebook having an identifier that further specified the decoding region;

transmitting the multiple predictors to a separate decoder; and

transmitting the identifier for the sub-codebook as the compressed representation of the single reference unit, wherein the separate decoder recreates the entire single reference unit using the identified sub-codebook to decode any one of the multiple predictors within the specified decoding region that are correctly received by the separate decoder.

24. The machine-readable non-transitory storage medium of claim 23, wherein determining a sub-codebook comprises:

partitioning a codebook until the sub-codebook is created.

25. The machine-readable non-transitory storage medium of claim 24, wherein the codebook comprises lattice partitions.

26. The machine-readable non-transitory storage medium of claim 23, wherein the method further comprises:

determining that none of the multiple predictors were correctly received for decoding;

selecting an additional predictor; and

determining a new sub-codebook that decodes the additional predictor to the reference frame.

27. The machine-readable non-transitory storage medium of claim 26, wherein the method further comprises:

sending an additional identifier for the new sub-codebook.

28. The machine-readable non-transitory storage medium of claim 27, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

29. The machine-readable non-transitory storage medium of claim 23, wherein the predictors are selected based on a maximum error calculation.

30. The machine-readable non-transitory storage medium of claim 23, wherein the method further comprises:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

31. The machine-readable non-transitory storage medium of claim 23, wherein the method further comprises:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

32. The machine-readable non-transitory storage medium of claim 23, wherein the method further comprises:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

58. The machine-readable non-transitory storage medium of claim 23, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

33. A machine-readable non-transitory storage medium storing instructions which when executed by a processor cause the processor to execute a method comprising:

receiving, by a decoder from a separate encoder, an identifier as a compressed representation of a single reference unit, the identifier specifying a decoding region centered on the single reference unit within an n-dimensional coordinate space and further corresponding to a sub-codebook for the decoding region in which each of multiple predictors located within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors;

receiving the multiple predictors, wherein fewer than all of the multiple predictors within the decoding region are correctly received; and

recreating the entire single reference unit using the identified sub-codebook to decode any one of the correctly received multiple predictors.

34. The machine-readable non-transitory storage medium of claim 33, wherein the method further comprises:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

35. The machine-readable non-transitory storage medium of claim 33, wherein the method further comprises:

receiving an additional identifier for a new sub-codebook; and

decoding the reference unit from an additional predictor and the new sub-codebook.

59. The machine-readable non-transitory storage medium of claim 33, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

36. A system comprising:

a processor coupled to a memory through a bus; and

an encoding process executed by the processor from the memory to cause the processor to execute as an encoder to select, multiple predictors for a single reference unit of correlated data, the multiple predictors located within a decoding region centered on the single reference unit in an n-dimensional coordinate space and comprising other data units in the correlated data, determine a sub-codebook for the decoding region in which each of the multiple predictors within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors, the sub-codebook having an identifier that further specifies the decoding region, transmit the multiple predictors to a separate decoder, and transmit the identifier for the sub-codebook as the compressed representation of the single reference unit, wherein the separate decoder recreates the entire single reference unit using the identified sub-codebook to decode any one of the multiple predictors within the specified decoding region that are correctly received by the separate decoder.

37. The system of claim 36, wherein the encoding process further causes the processor to partition a codebook until the sub-codebook is created when determining a sub-codebook.

38. The system of claim 37, wherein the codebook comprises lattice partitions.

39. The system of claim 36, wherein the encoding process further causes the processor to determine that none of the multiple predictors were correctly received for decoding, select an additional predictor, and determine a new sub-codebook that decodes the additional predictor to the reference frame.

40. The system of claim 39, wherein the encoding process further causes the processor to send an additional identifier for the new sub-codebook.

41. The system of claim 40, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

42. The system of claim 36, wherein the predictors are selected based on a maximum error calculation.

60. The system of claim 36, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

43. A system comprising:

a processor coupled to a memory through a bus; and

a decoding process executed from the memory by the processor to cause the processor execute as a decoder to receive, from a separate encoder, an identifier as a compressed representation of a single reference unit, the identifier specifying a decoding region centered on the single reference unit within an n-dimensional coordinate space and further corresponding to a sub-codebook for the decoding region in which each of multiple predictors located within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors, to receive the multiple predictors, wherein fewer than all of the multiple predictors are received correctly, and recreate the entire single reference unit using the identified sub-codebook to decode any one of the correctly received multiple predictors.

44. The system of claim 43, wherein the decoding process further causes the processor to estimate the predictors if no predictors are correctly received, and decode the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

45. The system of claim 43, wherein the decoding process further causes the processor to receive an additional identifier for a new sub-codebook, and decode the reference unit from an additional predictor and the new sub-codebook.

61. The system of claim 43, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

46. An apparatus comprising:

means for selecting, by an encoder means, multiple predictors for a single reference unit of correlated data, the multiple predictors located within a decoding region centered on the single reference unit in an n-dimensional coordinate space and comprising data units in the correlated data;

means for determining a sub-codebook for the decoding region in which each of the multiple predictors within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors, the sub-codebook having an identifier that further specifies the decoding region;

means for transmitting the multiple predictors to a separate decoder means; and

means for transmitting the identifier for the sub-codebook as the compressed representation of the single reference unit, wherein the separate decoder recreates the entire single reference unit using the identified sub-codebook to decode any one of the multiple predictors within the specified decoding region that is correctly received by the separate decoder.

47. The apparatus of claim 46, wherein the means for determining a sub-codebook comprises:

means for partitioning a codebook until the sub-codebook is created.

48. The apparatus of claim 47, wherein the codebook comprises lattice partitions.

49. The apparatus of claim 46 further comprising:

;

means for determining that none of the multiple predictors were correctly received for decoding;

means for selecting an additional predictor; and

means for determining a new sub-codebook that decodes the additional predictor to the reference frame.

50. The apparatus of claim 49 further comprising:

means for sending an additional identifier for the new sub-codebook.

51. The apparatus of claim 50, wherein the additional identifier combines with the identifier for the sub-codebook to identify the new sub-codebook.

52. The apparatus of claim 46, wherein the predictors are selected based on a maximum error calculation.

62. The apparatus of claim 46, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.

53. An apparatus comprising:

means for receiving, by a decoder means from a separate encoder means, an identifier as a compressed representation of a single reference unit, the identifier specifying a decoding region centered on the single reference unit within an n-dimensional coordinate space and further corresponding to a sub-codebook for the decoding region in which each of multiple predictors located within the decoding region is decodable to recreate the entire single reference unit without any of the other multiple predictors;

means for receiving the multiple predictors, wherein fewer than all of the multiple predictors are correctly received; and

means for recreating the entire single reference unit using the identified sub-codebook to decode any one of the correctly received multiple predictors.

54. The apparatus of claim 53 further comprising:

means for estimating the predictors if no predictors are correctly received; and

means for decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

55. The apparatus of claim 53 further comprising:

means for receiving an additional identifier for a new sub-codebook; and

means for decoding the reference unit from an additional predictor and the new sub-codebook.

63. The apparatus of claim 53, wherein the decoding region is a sphere and the identifier for the sub-codebook comprises an encoded value of a radius of the sphere.