Video encoding apparatus that adjusts code amount by skipping encoding of image data

• US 6,819,714 B2
• Filed: 04/19/2001
• Issued: 11/16/2004
• Est. Priority Date: 04/20/2000
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Morishita, Masahiro, Yamada, Nobuhiko, Isomura, Tsuyoshi
• Primary Examiner(s): Philippe, Gims
• Assistant Examiner(s): WONG, ALLEN C
• Application Number:
  US09/838,432
• Publication Number:
  US 20010046263A1
• Time in Patent Office: 1,307 Days
• Field of Search: 375/240, 375/240.01, 375/240.03, 375/240.12, 375/240.17, 375/240.25, 382/236, 348/414.1
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server

Abstract

A comparing/judging unit 125 of a video encoding apparatus compares, prior to encoding each image data, the predictive amount of codes accumulated in a receiver buffer of a decoding apparatus with a threshold that is set for a picture type of a picture to be obtained by encoding the image data. The comparing/judging unit 125 judges, when the predictive amount in the receiver buffer is below the threshold, that encoding the image data is to be skipped, and makes the DCT unit 113 cancel the encoding process. The comparing/judging unit 125 then outputs an all-skip picture in which all macroblocks except a first macroblock and a last macroblock in each slice layer are skipped macroblocks from a SKIP picture storage memory 126.

Claims

1. A video encoding apparatus that encodes a sequence of image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding apparatus comprising:

comparing means for comparing, prior to encoding of each image data, a predictive accumulation amount with a predetermined threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded; and

skipping means for (a) canceling the encoding of the image data and (b) using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the predetermined threshold, the proxy code indicating to display image data that is identical to previously decoded image data.

2. The video encoding apparatus of claim 1, further comprising

threshold setting means for setting a threshold for a picture type of a picture that is obtained by encoding the image data, the picture type being one of an I-picture, a P-picture, and a B-picture.

3. The video encoding apparatus of claim 2,

wherein the skipping means (a) uses an all-skip B-picture as the proxy code when the picture type is a B-picture, the all-skip B-picture being a B-picture in which macroblocks in all slice layers except a first macroblock and a last macroblock in each slice layer are skipped macroblocks, and (b) uses an all-skip P-picture as the proxy code when the picture type is an I-picture or a P-picture, the all-skip P-picture being a P-picture in which macroblocks in all slice layers except a first macroblock and a last macroblock in each slice layer are skipped macroblocks.

4. The video encoding apparatus of claim 3,

wherein the threshold is a predictive code amount of the picture.

5. The video encoding apparatus of claim 3,

wherein the threshold setting means calculates a variance of pixel values of the image data, and sets a higher threshold for a higher calculated variance.

6. The video encoding apparatus of claim 3,

wherein the threshold setting means sets a higher threshold for a higher activity ACT of an original image, the activity ACT being a sum of activities act of all macroblocks included in the original image, an activity act being expressed by the equation

7. The video encoding apparatus of claim 3,

wherein the threshold setting means sets a predictive code amount of each of an I-picture and a P-picture as the threshold for each of an I-picture and a P-picture, and sets a value larger than a predictive code amount of a B-picture as the threshold for a B-picture.

8. The video encoding apparatus of claim 7,

wherein when M≧

2, M representing an appearance cycle of an I-picture or a P-picture, the threshold setting means sets;

a threshold Ti of an I-picture as Ti=Ei;

a threshold Tp of a P-picture as Tp=Ep;

a threshold Tb(i) of a B-picture immediately preceding an I-picture in an encoding order as Tb(i)=Eb+(Ei−

R) when (Ei−

R)≧

0, and Tb(i)=Eb when (Ei−

R)<

0; and

a threshold Tb(p) of a B-picture immediately preceding a P-picture in the encoding order as Tb(p)=Eb+(Ep−

R) when (Ep−

R)≧

0, and Tb(p)=Eb when (Ep−

R)<

0,

where Ei is the predictive code amount of an I-picture, Ep is the predictive code amount of a P-picture, Eb is the predictive code amount of a B-picture, and R is a transmission bit amount during each decoding time interval.

9. The video encoding apparatus of claim 7,

wherein when M≧

3, M representing an appearance cycle of an I-picture or a P-picture, the threshold setting means sets;

a threshold Ti of an I-picture as Ti=Ei;

a threshold Tp of a P-picture as Tp=Ep;

a threshold Tb2(i) of a B-picture B2(i) immediately preceding an I-picture in an encoding order as Tb2(i)=Eb+(Ei−

R) when (Ei−

R)≧

0, and Tb2(i)=Eb when (Ei−

R) <

0;

a threshold Tb1(i) of a B-picture B1(i) immediately preceding a B-picture B2(i) in the encoding order as Tb1(i)=Eb+(Tb2(i)−

R) when (Tb2(i)−

R)≧

0, and Tb1(i)=Eb when (Tb2(i)−

R)<

0;

a threshold Tb2(p) of a B-picture B2(p) immediately preceding a P-picture in the encoding order as Tb2(p)=Eb+(Ep−

R) when (Ep−

R)≧

0, and Tb2(p)=Eb when (Ep−

R)<

0; and

a threshold Tb1(p) of a B-picture B1(p) immediately preceding a B-picture B2(p) in the encoding order as Tb1(p)=Eb+(Tb2(p)−

R) when (Tb2(p)−

R)≧

0, and Tb1(p)=Eb when (Tb2(p)−

R)<

0,

where Ei is the predictive code amount of an I-picture, Ep is the predictive code amount of a P-picture, Eb is the predictive code amount of a B-picture, and R is a transmission bit amount during each decoding time interval.

10. The video encoding apparatus of claim 7,

wherein when M≧

3, M representing an appearance cycle of an I-picture or a P-picture, the threshold setting means sets the threshold of a B-picture B2 immediately preceding an I-picture in an encoding order, higher than the threshold of a B-picture B1 immediately preceding the B-picture B2 in the encoding order.

11. The video encoding apparatus of claim 10,

wherein when M≧

3, M representing an appearance cycle of an I-picture or a P-picture, the threshold setting means sets;

a threshold Ti of an I-picture as Ti=Ei;

a threshold Tp of a P-picture as Tp=Ep;

a threshold Tb2(i) of a B-picture B2(i) immediately preceding an I-picture in an encoding order as Tb2(i)=Eb+(Ei−

R) when (Ei−

R)≧

0, and Tb2(i)=Eb when (Ei−

R)<

0;

a threshold Tb1(i) of a B-picture B1(i) immediately preceding a B-picture B2(i) in the encoding order as Tb1(i)=Dbskip+(Tb2(i)−

R) when Dbskip+(Tb2(i)−

R)≧

Eb, and Tb1(i)=Eb when Dbskip+(Tb2(i)−

R)<

Eb;

a threshold Tb2(p) for a B-picture B2(p) immediately preceding a P-picture in the encoding order as Tb2(p)=Eb+(Ep−

R) when (Ep−

R)≧

0, and Tb2(p)=Eb when (Ep−

R)<

0; and

a threshold Tb1(p) of a B-picture B1(p) immediately preceding a B-picture B2(p) in the encoding order as Tb1(p)=Dbskip+(Tb2(p)−

R) when Dbskip+(Tb2(p)−

R)≧

Eb, and Tb1(p)=Eb when Dbskip+(Tb2(p)−

R)<

Eb

where Ei is the predictive code amount of an I-picture, Ep is the predictive code amount of a P-picture, Eb is the predictive code amount of a B-picture, R is a transmission bit amount during each decoding time interval, and Dbskip is a code amount of an all-skip B-picture.

12. A video encoding apparatus that encodes a sequence of image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding apparatus comprising:

threshold setting means for setting, after encoding each image data, a threshold according to a picture type of the encoded image data in a case where M≧

2, M representing an appearance cycle of an I-picture or a P-picture, as the following,

a threshold Ti in a case where the image data is encoded to be an I-picture is set as Ti=Di, where Di is a predictive code amount of the I-picture,

a threshold Tp in a case where the image data is encoded to be a P-picture is set as Tp=Dp, where Dp is a predictive code amount of the P-picture,

a threshold Tb(i) in a case where the image data is encoded to be a B-picture immediately preceding an I-picture in an encoding order is set as Tb(i)=Dp+(Ei−

R) when (Ei−

R)≧

0, and Tb(i)=Dp when (Ei−

R)<

0, where Db is a predictive code amount of the B-picture, Ei is a predictive code amount of the I-picture, and R is a transmission bit amount during each decoding time interval, and

a threshold Tb(p) in a case where the image data is encoded to be a B-picture immediately preceding a P-picture in the encoding order is set as Tb(p)=Dp+(Ep−

R) when (Ep−

R)≧

0, and Tb(p)=Dp when (Ep−

R)<

0, where Db is a predictive code amount of the B-picture, Ep is a predictive code amount of the P-picture, and R is a transmission bit amount during each decoding time interval;

comparing means for comparing, after encoding the image data, a predictive accumulation amount with the threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded; and

skipping means for using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the threshold, the proxy code indicating to display image data that is identical to previously decoded image data.

13. A video encoding apparatus that encodes a sequence of image data in a frame structure, each image data forming one frame, comprising:

comparing means for comparing, prior to encoding of each image data, one of (a) a predictive accumulation amount of a receiver buffer in a decoding apparatus and (b) an accumulation amount of an output buffer, with a predetermined standard value, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer;

skipping means for (a) canceling the encoding of the image data in the frame structure and (b) substituting a proxy code indicating to display two fields that each are identical to one of a top field and a bottom field of previously decoded image data, for data that is obtained by encoding a top field and a bottom field of the image data, if the amount of data is below the predetermined standard value.

14. The video encoding apparatus of claim 13,

wherein the skipping means uses the proxy code indicating to display the two fields that each are identical to the field that is the nearest, in a display order, to each of the top field and the bottom field of the image data which has been canceled to be encoded in the frame structure, the field being selected out of the top field and the bottom field of the previously decoded image data.

15. The video encoding apparatus of claim 14,

wherein when the image data which has been canceled to be encoded in the frame structure is encoded to be a B-picture, the skipping means uses two all-skip B-pictures that each are a B-picture in which macroblocks in all slice layers except a first macroblock and a last macroblock in each slice layer are skipped macroblocks as the proxy code, and when the image data which has been canceled to be encoded in the frame structure is encoded to be an I-picture or a P-picture, the skipping means uses two all-skip P-pictures that each are a P-picture in which macroblocks in all slice layers except a first macroblock and a last macroblock in each slice layer are skipped macroblocks as the proxy code.

16. The video encoding apparatus of claim 15,

wherein (a) each all-skip P-picture used by the skipping means when the image data which has been canceled to be encoded in the frame structure is encoded to be an I-picture or a P-picture uses a bottom field of a previously displayed I-picture or P-picture as a reference field, (b) each all-skip B-picture used by the skipping means when the image data which has been canceled to be encoded in the frame structure is encoded to be a B-picture B1 uses a top field of a previously displayed I-picture or P-picture as a reference field, and (c) each all-skip B-picture used by the skipping means when the image data which has been canceled to be encoded in the frame structure is encoded to be a B-picture B2 uses a top field of a successively displayed I-picture or P-picture as a reference field.

17. The video encoding apparatus of claim 16,

wherein when the image data which has been canceled to be encoded in the frame structure is encoded to be an I-picture or a P-picture, the skipping means cancels encoding of image data encoded to be a B-picture immediately following the I-picture or the P-picture in the display order, and substitutes two all-skip B-pictures that each use a bottom field of a previously displayed picture as a reference field, for data obtained by encoding the top field and the bottom field of the image data.

18. A video encoding apparatus that encodes a sequence image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding apparatus comprising:

comparing means for comparing, prior to encoding of each image data, a predictive accumulation amount with a threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded, the threshold being set for a picture type of a picture that is obtained by encoding the image data; and

controlling means for (a) canceling the encoding of the image data and (b) using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the threshold and the picture type is a B-picture, the proxy code indicating to display image data that is identical to previously decoded image data.

19. A video encoding apparatus that encodes a sequence of image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding apparatus comprising:

comparing means for comparing, prior to encoding of each image data, a predictive accumulation amount with a threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded, the threshold being set for a picture type of a picture that is obtained by encoding the image data; and

controlling means for (a) canceling the encoding of the image data and (b) using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the threshold and the picture type is a B-picture or a P-picture, the proxy code indicating to display image data that is identical to previously decoded image data.

20. A video encoding method for encoding a sequence of image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding method comprising:

comparing step for comparing, prior to encoding of each image data, a predictive accumulation amount with a predetermined threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded; and

skipping step for (a) canceling the encoding of the image data and (b) using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the predetermined threshold, the proxy code indicating to display image data that is identical to previously decoded image data.

21. A video encoding method for encoding a sequence of image data in a frame structure, each image data forming one frame, comprising:

comparing step for comparing, prior to encoding of each image data, one of (a) a predictive accumulation amount of a receiver buffer in a decoding apparatus and (b) an accumulation amount of an output buffer, with a predetermined standard value, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer;

skipping step for (a) canceling the encoding of the image data in the frame structure and (b) substituting a proxy code indicating to display two fields that each are identical to one of a top field and a bottom field of previously decoded image data, for data that is obtained by encoding a top field and a bottom field of the image data, if the amount of data is below the predetermined standard value.

22. A video encoding program for encoding a sequence of image data while predicting an accumulation amount of a receiver buffer in a decoding apparatus, each image data forming one frame or one field, the video encoding program comprising:

comparing step for comparing, prior to encoding of each image data, a predictive accumulation amount with a predetermined threshold, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer by the time when data obtained by encoding the image data is decoded; and

skipping step for (a) canceling the encoding of the image data and (b) using a proxy code as data that is fetched from the receiver buffer at the decoding time, if the amount of data is below the predetermined threshold, the proxy code indicating to display image data that is identical to previously decoded image data.

23. A video encoding program for encoding a sequence of image data in a frame structure, each image data forming one frame, comprising:

comparing step for comparing, prior to encoding of each image data, one of (a) a predictive accumulation amount of a receiver buffer in a decoding apparatus and (b) an accumulation amount of an output buffer, with a predetermined standard value, the predictive accumulation amount being an amount of data predicted to be accumulated in the receiver buffer;

skipping step for (a) canceling the encoding of the image data in the frame structure and (b) substituting a proxy code indicating to display two fields that each are identical to one of a top field and a bottom field of previously decoded image data, for data that is obtained by encoding a top field and a bottom field of the image data, if the amount of data is below the predetermined standard value.