Low-complexity error-resilient coder using a block-based standard

• US 6,141,448 A
• Filed: 02/02/1998
• Issued: 10/31/2000
• Est. Priority Date: 04/21/1997
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: HP Inc.
• Inventors: Khansari, Masoud, Bhaskaran, Vasudev
• Primary Examiner(s): Lee, Thomas D.
• Assistant Examiner(s): Chen, Wenpeng
• Application Number:
  US09/017,515
• Time in Patent Office: 1,002 Days
• Field of Search: 382/238, 382/232, 382/236, 382/245, 382/246, 348/420, 348/403, 348/415, 348/466, 386/111, 386/109, 364/737, 371/35, 371/37.1, 371/31, 371/37.7
• US Class Current: 382/236
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/895 in combination with error concealment

Abstract

Disclosed is an error-correction method and apparatus for transmission of block-based coding standard compliant video data, such as H.263, MPEG or JPEG. A picture is divided into slices wherein each slice of the picture is coded into a group of blocks (GOB) in which each macroblock in the GOB is encoded only with reference to other macroblocks in the same GOB. An erasure slice, which is also a GOB, is then formed wherein the data for each macroblock of the erasure slice is determined by summing a corresponding macroblock in each of the GOBs containing slices of the picture. The GOBs containing slices of the picture are then transmitted, along with the GOB containing the erasure slice, as block-based coding standard compliant data packets. The result is that if any one of the GOBs containing slices of the picture is lost, then the lost GOB can be reconstructed from the remaining GOBs and the erasure slice.

Claims

1. A method for transferring video data, the method comprising the steps:

receiving video data for a picture;

subdividing the video data into blocks;

transforming the blocks of video data into blocks of residual data;

quantizing the blocks of residual data;

comparing the video data for the picture with stored video data for a previous picture in order to generate motion vector data;

organizing the blocks of residual data into a plurality of macroblocks (MBs);

organizing the plurality of MBs into groups of blocks (GOBs), wherein each GOB corresponds to a slice of the picture and the motion vector data in each GOB is independent of any other GOBs;

constructing an erasure slice comprised of another GOB, wherein the GOB of the erasure slice is further comprised of another plurality of MBs, wherein each one of the plurality of MBs of the erasure slice corresponds to one of the plurality of MBs in each of a plurality of GOB of the picture and wherein the erasure slice is constructed with data in at least some of the plurality of MBs combined from corresponding MBs in the GOBs of the picture;

encoding each one of the GOBs of the picture data and the erasure slice into a predetermined standard compliant packet; and

transmitting each standard compliant packet.

2. The method of claim 1, wherein the step of constructing an erasure slice includes forming motion vector data for each one of the plurality of MBs of the erasure slice by linearly combining motion vector data of the corresponding one of the plurality of MBs in each GOB of the picture data.

3. The method of claim 2, wherein the step of forming motion vector data for each one of the plurality of MBs further comprises modulo summation performed using the equations:

##EQU3## wherein MV_x( (i,j) and MV_y (i,j) are the x and y motion vector components, respectively, for an ith one of the MBs of a jth GOB of picture data and MVE.sub.(x (i) and MVE_y (i) are the x and y motion vector components, respectively, for an ith one of the MBs of the GOB of the erasure slice.

4. The method of claim 1, wherein the step of constructing an erasure slice includes forming the residual data for each one of the plurality of MBs of the erasure slice by modulo summing the coefficients of the corresponding one of the plurality of MBs in each GOB of the picture data.

5. The method of claim 1, wherein the step of constructing an erasure slice includes forming a DQUANT value for each one of the plurality of MBs of the erasure slice by modulo summing the DQUANT value of the corresponding one of the plurality of MBs in each GOB of the picture data.

6. The method of claim 1, wherein the step of constructing an erasure slice includes forming a MTYPE value for each one of the plurality of MBs of the erasure slice by:

setting the MTYPE value for each one of the plurality of MBs of the erasure slice to INTER if the MTYPE value for each of the corresponding ones of the plurality of MBs in each GOB of the picture data is INTER; and

setting the MTYPE value for each one of the plurality of MBs of the erasure slice to INTRA if the MTYPE value is INTRA for any of the corresponding ones of the plurality of MBs in each GOB.

7. The method of claim 1, wherein the step of constructing an erasure slice includes:

determining an activity level of the picture; and

not forming an erasure slice if the activity level of the picture is less than a predetermined activity threshold level.

8. The method of claim 7, wherein the step of determining an activity level of the picture includes determining the activity level using the equation:

##EQU4## wherein MV_x( (i,j) and MV_y (i,j) are the x and y motion vector components, respectively, for an ith one of the MBs of a jth GOB of picture data.

9. The method of claim 1, wherein the step of constructing an erasure slice includes:

dividing the picture data into lossy and lossless reconstruction categories; and

allocating a variable number of bits in the erasure slice to at least one of the reconstruction categories.

10. The method of claim 9, wherein the step of allocating a variable number of bits in the erasure slice to at least one of the reconstruction categories includes varying the number of bits to the at least one of the reconstruction categories responsive to transmission channel characteristics.

11. The method of claim 1, the method further comprising:

receiving the predetermined standard compliant packets for the picture;

determining whether one of the GOBs of picture data has been lost in transmission;

reconstructing a lost one of the GOBs of picture data from the received GOBs of picture data and the erasure slice GOB.

12. The method of claim 11, wherein the step of reconstructing a lost one of the GOBs of picture data includes:

reconstructing motion vector data for each one of the plurality of MBs of the lost one of the GOBs of picture data by linearly combining motion vector data of the corresponding one of the plurality of MBs in each received GOB of the picture data and the corresponding one of the plurality of MBs in the erasure slice GOB.

13. The method of claim 12, wherein the step of reconstructing motion vector data for each one of the plurality of MBs of the lost one of the GOBs of picture data further comprises modulo summation performed using the equations:

##EQU5## wherein MVL_x( (i) and MVL_y (i) are the x and y motion vector components, respectively, for an ith one of the MBs of the lost GOB of picture data, MV_x( (i,j) and MV_y (i,j) are the x and y motion vector components, respectively, for an ith one of the MBs of a jth GOB of picture data, excluding the lost GOB of picture data, and MVE_x (i) and MVE_y (i) are the x and y motion vector components, respectively, for an ith one of the MBs of the GOB of the erasure slice.

14. The method of claim 11, wherein the step of reconstructing a lost one of the GOBs of picture data includes:

reconstructing residual data for each one of the plurality of MBs of the lost one of the GOBs of picture data by summing residual data of the corresponding one of the plurality of MBs in each received GOB of the picture data and the corresponding one of the plurality of MBs in the erasure slice GOB.

15. The method of claim 11, wherein the step of reconstructing a lost one of the GOBs of picture data includes:

reconstructing a DQUANT value for each one of the plurality of MBs of the lost one of the GOBs of picture data by modulo summing the DQUANT value of the corresponding one of the plurality of MBs in each received GOB of the picture data and the corresponding one of the plurality of MBs in the erasure slice GOB.

16. The method of claim 11, wherein the step of reconstructing a lost one of the GOBs of picture data includes:

setting an MTYPE value for each one of the plurality of MBs of the lost one of the GOBs of picture data to the MTYPE value of the corresponding one of the plurality of MBs in the erasure slice GOB.

17. The method of claim 11, wherein:

the step of constructing an erasure slice comprised of another GOB further comprises constructing multiple erasure slices, each one of the multiple erasure slices being comprised of a respective plurality of MBs, and wherein each one of the respective plurality of MBs for each of the multiple erasure slices corresponds to one of the plurality of MBs in each GOB of a subset of the picture data; and

the step of encoding each one of the GOBs of the picture data and the erasure slice into a predetermined standard compliant packet further comprises encoding each one of the GOBs of the picture data and each one of the multiple erasure slices into standard compliant packets.

18. A block-based coding standard compliant encoder comprising:

a subtractor having first and second input terminals and an output terminal, the subtractor being configured to subtract a signal received at the second input terminal from a signal received at the first input terminal in order to produce a difference signal at the output terminal, wherein the first input terminal of the subtractor is configured to receive video data corresponding to a picture;

a transformer having input and output terminals, the input terminal of the transformer being coupled to the output terminal of the subtractor, wherein the transformer is configured to transform the difference signal received at the input terminal of the transformer into residual data output at the output terminal of the transformer;

a quantizer having input and output terminals and a control terminal, the input terminal of the quantizer being coupled to the output terminal of the transformer, wherein the quantizer is configured to quantize the residual data received from the transformer responsive to a QUANT signal received at the control terminal of the quantizer to produce quantized data output at the output terminal of the quantizer;

a variable length encoder and multiplexor having an input terminal coupled to the output terminal of the quantizer, an output terminal, an INTER/INTRA terminal configured to receive an INTER/INTRA signal, a PTYPE terminal configured to receive a PTYPE signal, a QUANT terminal configured to receive the QUANT signal, and a motion vector terminal configured to receive motion vector data, wherein the input terminal of the variable length encoder and multiplexor is coupled to the output terminal of the quantizer, and wherein the variable length encoder and multiplexor is configured to encode the quantized data and form block-based coding standard compliant data packets which include the INTER/INTRA, PTYPE and QUANT signals and the motion vector data and wherein each group of blocks of the block-based coding standard compliant data packets corresponds to a slice of picture data;

a buffer configured to receive and store the block-based coding standard compliant data packets formed by the variable length encoder and multiplexor; and

an erasure slice constructor configured to receive the block-based coding standard compliant data packets stored in the buffer and, responsive thereto, generate an erasure slice corresponding to the picture from which one of the block-based coding standard compliant data packets can be reconstructed from the remaining block-based coding standard compliant data packets for the picture and the erasure slice for the picture.

19. The encoder of claim 18, wherein the erasure slice constructor is further configured to form motion vector data for each one of a plurality of macroblocks of the erasure slice by linearly combining motion vector data of the corresponding one of a plurality of macroblocks in each group of blocks of the picture data.

20. The encoder of claim 19, wherein the erasure slice constructor is further configured to form the motion vector data -for each one of the plurality of macroblocks using the equations:

##EQU6## wherein MV_x( (i,j) and MV_y (i,j) are the x and y motion vector components, respectively, for an ith one of the macroblocks of a jth group of blocks of picture data and MVE_x (i) and MVE_y (i) are the x and y motion vector components, respectively, for an ith one of the macroblocks of the erasure slice.

21. The encoder of claim 18, wherein the erasure slice constructor is further configured to form the residual data for each one of the plurality of MBs of the erasure slice by summing the coefficients of the corresponding one of the plurality of MBs in each GOB of the picture data.

22. The encoder of claim 18, wherein the erasure slice constructor is further configured to form a DQUANT value for each one of the plurality of MBs of the erasure slice by modulo summing the DQUANT value of the corresponding one of the plurality of MBs in each GOB of the picture data.

23. The encoder of claim 18, wherein the erasure slice constructor is further configured to form a MTYPE value for each one of the plurality of MBs of the erasure slice by:

setting the MTYPE value for each one of the plurality of MBs of the erasure slice to INTER if the MTYPE value for each of the corresponding ones of the plurality of MBs in each GOB of the picture data is INTER; and

setting the MTYPE value for each one of the plurality of MBs of the erasure slice to INTRA if the MTYPE value is INTRA for any of the corresponding ones of the plurality of MBs in each GOB.

24. The encoder of claim 18, wherein the erasure slice constructor is further configured to:

determine an activity level of the picture; and

only form an erasure slice if the activity level of the picture is greater than a predetermined activity threshold level.

25. The encoder of claim 24, wherein the erasure slice constructor is further configured to determine the activity level using the equation:

##EQU7## wherein MV_x( (i,j) and MV_y (i,j) are the x and y motion vector components, respectively, for an ith one of the MBs of a jth GOB of picture data.

26. The encoder of claim 18, wherein the erasure slice constructor is further configured to:

divide the picture data into lossy and lossless reconstruction categories; and

allocate a variable number of bits in the erasure slice to at least one of the reconstruction categories.

27. The encoder of claim 26, wherein the erasure slice constructor is further configured to vary the number of bits to the at least one of the reconstruction categories responsive to transmission channel characteristics.

28. A block-based coding standard decoder comprising:

an input buffer configured to receive and store block-based coding standard compliant data packets corresponding to a picture along with an erasure slice corresponding to the picture;

a variable length decoder and demultiplexor configured to receive and decode the block-based coding standard compliant data packets, the variable length decoder and demultiplexor being configured to output quantized data decoded from the block-based coding standard compliant data packets at an output terminal and extract an INTER/INTRA signal, a PTYPE signal, a QUANT signal and motion vector data from the block-based coding standard compliant data packets which are output at an INTER/INTRA terminal, a PTYPE terminal, a QUANT terminal and a motion vector terminal, respectively, and where the variable length decoder and demultiplexor is further configured to extract and output group of blocks header information from the block-based coding standard data packets at an header information terminal;

an inverse quantizer having input and output terminals and a control terminal, the input terminal of the quantizer being coupled to the output terminal of the variable length decoder and demultiplexor, wherein the inverse quantizer is configured to inverse quantize the quantized data received from the variable length decoder and demultiplexor responsive to the QUANT signal received at the control terminal of the inverse quantizer to produce residual data output at the output terminal of the inverse quantizer;

an inverse transformer having input and output terminals, the input terminal of the inverse transformer being coupled to the output terminal of the inverse quantizer, wherein the inverse transformer is configured to inverse transform the residual data received at the input terminal of the inverse transformer into a difference signal at the output terminal of the inverse transformer;

a motion compensation predictor configured to receive the INTER/INTRA and PTYPE signals and the motion vector data and, responsive thereto, generate a reference signal;

an adder configured to receive the reference signal and the difference signal and, responsive thereto, generate a video signal corresponding to the picture; and

an erasure slice reconstructor configured to receive the block-based coding standard data packets and the erasure slice stored in the buffer, where the erasure slice reconstructor is further configured to receive the group of blocks header information and, responsive thereto, determine whether there is a lost block-based coding standard data packet and reconstruct the lost packet from the remaining block-based coding standard compliant data packets for the picture and the erasure slice for the picture.

29. The decoder of claim 28, wherein the erasure slice reconstructor is further configured to reconstruct motion vector data for each one of a plurality of macroblocks of the lost block-based coding standard compliant data packet by linearly combining motion vector data of a corresponding one of a plurality of macroblocks in each received block-based coding standard compliant data packet and a corresponding one of a plurality of macroblocks in the erasure slice.

30. The decoder of claim 29, wherein the erasure slice reconstructor is further configured to reconstruct the motion vector data for each one of the plurality of macroblocks of the lost block-based coding standard compliant data packet by performing the following equations:

##EQU8## wherein MVL_x( (i) and MVL_y (i) are x and y motion vector components, respectively, for an ith one of the macroblocks of the lost block-based coding standard compliant data packet of picture data, MV_x( ((i,j) and MV_y (i,j) are x and y motion vector components, respectively, for an ith one of the macroblocks of a jth block-based coding standard compliant data packet of picture data, excluding the lost block-based coding standard compliant data packet of picture data, and MVE_x (i) and MVE_y (i) are x and y motion vector components, respectively, for an ith one of the macroblocks of the erasure slice.

31. The decoder of claim 28, wherein the erasure slice reconstructor is further configured to reconstruct residual data for each one of a plurality of macroblocks of the lost one of the block-based coding standard compliant data packets of picture data by summing residual data of a corresponding one of the plurality of macroblocks in each received block-based coding standard compliant data packet of the picture data and a corresponding one of the plurality of macroblocks in the erasure slice.

32. The decoder of claim 28, wherein the erasure slice reconstructor is further configured to reconstruct a DQUANT value for each one of a plurality of macroblocks of the lost one of the block-based coding standard compliant data packets of picture data by modulo summing the DQUANT value of a corresponding one of the plurality of macroblocks in each received block-based coding standard compliant data packet of the picture data and a corresponding one of the plurality of macroblocks in the erasure slice.

33. The decoder of claim 28, wherein the erasure slice reconstructor is further configured to set an MTYPE value for each one of a plurality of macroblocks of the lost one of the block-based coding standard compliant data packets of picture data to the MTYPE value of a corresponding one of the plurality of macroblocks in the erasure slice.