Adaptive joint source channel coding

• US 20050129128A1
• Filed: 12/16/2004
• Published: 06/16/2005
• Est. Priority Date: 12/16/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Chou, Jim Chen, Tabatabai, Ali Jezu
• Granted Patent: US 8,582,640 B2
• Time in Patent Office: Days
• US Class Current: 375/240.180
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04L 1/0045 Arrangements at the receiver end
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

Adaptive joint source channel coding associates multiple predictors with a reference data unit, such as a macroblock or frame of video data. An encoder determines a sub-codebook in which each of the selected multiple predictors decodes to the reference data unit. An identifier for the sub-codebook is transmitted through a channel to a decoder for subsequent decoding of the reference data unit. The reference data unit itself does not need to be sent. The multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region. The decoder uses the identified sub-codebook and one of the predictors to decode the reference data unit. If none of the original predictors are correctly received, different types of error handling are employed based on the type of channel.

Claims

1. A computerized method comprising:

selecting multiple predictors for a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

2. The computerized method of claim 1 further comprising:

transmitting the reference unit.

3. The computerized method of claim 1 further comprising:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

4. The computerized method of claim 1, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

5. A machine-readable medium having instructions to cause a processor to execute a method comprising:

selecting multiple predictors for a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

6. The machine-readable medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the method further comprises:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

7. The machine-readable medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

8. A machine readable medium having instructions to cause a processor to execute a method comprising:

receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera; and

decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

9. The machine-readable medium of claim 8 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

10. A system comprising:

a processor coupled to a memory through a bus; and

an encoding process executed by the processor from the memory to cause the processor to select multiple predictors for a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera, determine a sub-codebook in which each of the predictors decodes to the reference unit, and transmit an identifier for the sub-codebook to decode the reference unit from one of the predictors.

11. The system of claim 10, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

12. A system comprising:

a processor coupled to a memory through a bus; and

a decoding process executed from the memory by the processor to cause the processor to receive an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera, and decode the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

13. The system of claim 12, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

14. An apparatus comprising:

means for selecting multiple predictors for a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera;

means for determining a sub-codebook in which each of the predictors decodes to the reference unit; and

means for transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

15. The apparatus of claim 14 further comprising:

means for receiving the identifier; and

means for decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

16. The apparatus of claim 14, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

17. An apparatus comprising:

means for receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data, wherein the reference unit is selected from data from a camera and the predictors for the reference unit are selected from data from a neighboring camera; and

means for decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

18. The apparatus of claim 17, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

19. A computerized method comprising:

selecting multiple predictors for a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

20. The computerized method of claim 19 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

21. The computerized method of claim 19, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

22. The computerized method of claim 21 further comprising:

decoding the descriptions to reconstruct the frames.

23. The computerized method of claim 22 further comprising:

combining the decoded descriptions to reconstruct the frames.

24. A machine-readable medium having instructions to cause a processor to execute a method comprising:

selecting multiple predictors for a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data;

determining a sub-codebook in which each of the predictors decodes to the reference unit; and

transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

25. The machine-readable medium of claim 24, wherein the method further comprises:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

26. The machine-readable medium of claim 24, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

27. The machine-readable medium of claim 26 further comprising:

decoding the descriptions to reconstruct the frames.

28. The machine-readable medium of claim 27 further comprising:

combining the decoded descriptions to reconstruct the frames.

29. A machine readable medium having instructions to cause a processor to execute a method comprising:

receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data; and

decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

30. The machine-readable medium of claim 29 further comprising:

estimating the predictors if no predictors are correctly received; and

decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

31. The machine-readable medium of claim 29, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

32. The machine-readable medium of claim 31 further comprising:

decoding the descriptions to reconstruct the frames.

33. The machine-readable medium of claim 32 further comprising:

combining the decoded descriptions to reconstruct the frames.

34. A system comprising:

a processor coupled to a memory through a bus; and

an encoding process executed by the processor from the memory to cause the processor to select multiple predictors for a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data, determine a sub-codebook in which each of the predictors decodes to the reference unit, and transmit an identifier for the sub-codebook to decode the reference unit from one of the predictors.

35. The system of claim 34, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

36. A system comprising:

a processor coupled to a memory through a bus; and

a decoding process executed from the memory by the processor to cause the processor to receive an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data from descriptions in a plurality of channels and decode the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

37. The system of claim 36 further comprising:

a decoding process executed from the memory by the processor to cause the processor to estimate the predictors if no predictors are correctly received, and decode the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

38. The system of claim 36, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

39. The system of claim 38 further comprising:

decoding the descriptions to reconstruct the frames.

40. The system of claim 39 further comprising:

combining the decoded descriptions to reconstruct the frames.

41. An apparatus comprising:

means for selecting multiple predictors for a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data;

means for determining a sub-codebook in which each of the predictors decodes to the reference unit; and

means for transmitting an identifier for the sub-codebook to decode the reference unit from one of the predictors.

42. The apparatus of claim 41 further comprising:

means for estimating the predictors if no predictors are correctly received; and

means for decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

43. The apparatus of claim 41, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

44. The apparatus of claim 43 further comprising:

means for decoding the descriptions to reconstruct the frames.

45. The apparatus of claim 44 further comprising:

means for combining the decoded descriptions to reconstruct the frames.

46. An apparatus comprising:

means for receiving an identifier corresponding to a sub-codebook in which each of multiple predictors decodes to a reference unit of correlated data from descriptions in a plurality of channels that transmit the correlated data; and

means for decoding the reference unit from one correctly received predictor using the sub-codebook identified by the identifier.

47. The apparatus of claim 46 further comprising:

means for estimating the predictors if no predictors are correctly received; and

means for decoding the reference unit from a weighted combination of the estimated predictors using the sub-codebook identified by the identifier.

48. The apparatus of claim of claim 46, wherein the reference unit is a frame in a video stream and the predictors are selected from past and future frames in the video stream.

49. The apparatus of claim of claim 48 further comprising:

means for decoding the descriptions to reconstruct the frames.

50. The apparatus of claim of claim 49 further comprising:

combining the decoded descriptions to reconstruct the frames.