Adaptive joint source channel coding

• US 8,582,640 B2
• Filed: 12/16/2004
• Issued: 11/12/2013
• Est. Priority Date: 12/16/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Chou, Jim Chen, Tabatabai, Ali Jezu
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US11/015,840
• Publication Number:
  US 20050129128A1
• Time in Patent Office: 3,253 Days
• Field of Search: 375/240.12, 375/240.13, 375/240.16, 375/240.15, 375/240.25
• US Class Current: 375/240.01
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04L 1/0045 Arrangements at the receiver end
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/89 involving methods or arrangements for detection of transmission errors at the decoder

Abstract

Adaptive joint source channel coding associates multiple predictors with a reference data unit, such as a macroblock or frame of video data. An encoder determines a sub-codebook in which each of the selected multiple predictors decodes to the reference data unit. An identifier for the sub-codebook is transmitted through a channel to a decoder for subsequent decoding of the reference data unit. The reference data unit itself does not need to be sent. The multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region. The decoder uses the identified sub-codebook and one of the predictors to decode the reference data unit. If none of the original predictors are correctly received, different types of error handling are employed based on the type of channel.

Claims

1. A computerized method comprising:

selecting, by an encoding device, a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;

determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;

transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;

determining that none of the predictors in the first set of multiple predictors was received; and

transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.

2. The computerized method of claim 1 further comprising:

transmitting the reference unit.

3. The computerized method of claim 1 further comprising:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

4. The computerized method of claim 1, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

5. A machine-readable non-transitory storage medium having instructions to cause a processor to execute a method comprising:

selecting a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;

determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;

transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;

determining that none of the predictors in the first set of multiple predictors was received; and

transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.

6. The machine-readable non-transitory storage medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the method further comprises:

receiving the identifier; and

decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

7. The machine-readable non-transitory storage medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

8. A machine-readable non-transitory storage medium having instructions to cause a processor to execute a method comprising:

receiving an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the predictors in the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;

determining that none of the predictors in the first set of multiple predictors was received;

receiving a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and

decoding the single reference unit from one correctly received predictor from the second set of multiple predictors using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.

9. The machine-readable non-transitory storage medium of claim 8 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

10. A system comprising:

a processor coupled to a memory through a bus;

an encoding process executed by the processor from the memory to cause the processor to select a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera, determine a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit, and transmit an identifier for the sub-codebook to decode the single reference unit from one of the first set of multiple predictors, wherein only one of the first set of multiple predictors is needed to correctly decode the single reference unit;

a determining process executed by the processor from the memory to cause the processor to determine that none of the predictors in the first set of multiple predictors was received; and

a transmitting process executed by the processor from the memory to cause the processor to transmit a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.

11. The system of claim 10, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

12. A system comprising:

a processor coupled to a memory through a bus; and

a first receiving process executed from the memory by the processor to cause the processor to receive an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;

a determining process executed from the memory by the processor to cause the processor to determine that none of the predictors in the first set of multiple predictors was received;

a second receiving process executed from the memory by the processor to cause the processor to receive a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and

a decoding process executed from the memory by the processor to cause the processor to decode the single reference unit from one correctly received predictor using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.

13. The system of claim 12, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

14. An apparatus comprising:

means for selecting a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;

means for determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;

means for transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;

means for determining that none of the predictors in the first set of multiple predictors was received; and

means for transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.

15. The apparatus of claim 14 further comprising:

means for receiving the identifier; and

means for decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.

16. The apparatus of claim 14, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.

17. An apparatus comprising:

means for receiving an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the predictors in the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;

means for determining that none of the predictors in the first set of multiple predictors was received;

means for receiving a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and

means for decoding the single reference unit from one correctly received predictor from the second set of multiple predictors using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.

18. The apparatus of claim 17, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.