Adaptive coding method for interlaced scan digital video sequences

• US 5,347,308 A
• Filed: 10/05/1992
• Issued: 09/13/1994
• Est. Priority Date: 10/11/1991
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Wai, Lucas H. Y.
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): LEE, RICHARD J
• Application Number:
  US07/956,954
• Time in Patent Office: 708 Days
• Field of Search: 358/133, 358/136, 348/394, 348/416, 348/420
• US Class Current: 375/240.23
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00

Abstract

An efficient coding system for interlaced scan video sequences. An input picture is partitioned into smaller blocks of pixel data where the two fields in each blocks are rexamined for correlation. If the difference between the two fields in the block is low (or the correlation is high), the block will be coded with a frame coding method that assumes all lines in the block are progressive scanned, and therefore, with efficient inter-frame motion compensation method and chrominance data intra-frame sub-sampling method designed. On the other hand, if the difference between the two fields in the block is high (or the correlation is low), the block will be coded with a field coding method where more emphasis is made on exploring correlation within each field (intra-field) and correlation of each field and its neighboring fields (inter-field); furthermore, intra-field sub-sampling of the chrominance data is better performed in such blocks.

Claims

1. A method of coding an interlaced scan digital video signal comprising the steps of:

(a) partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) determining a difference of pixel values of two fields within each of said blocks of pixel data;

said step (b) of determining a difference of pixel values of two fields comprising

(i) calculating a mean square error between a line pair comprising an even and an adjacent odd line of a respective block of said blocks of pixel data,

(ii) calculating a mean square error between consecutive odd line pairs and consecutive even line pairs of said respective block, and

(iii) comparing the two calculated mean square errors; and

(c) subjecting each of said blocks of pixel data, according to said difference, (i) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (ii) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction.

2. A method of coding an interlaced scan digital video signal comprising the steps of:

(a) partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) determining a difference of pixel values of two fields within each respective block of said blocks of pixel data; and

(c) subjecting each of said blocks of pixel data, according to said difference, (1) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (2) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction, said frame coding process comprising the steps of;

(i) determining whether or not said respective block of said blocks of pixel data needs to be subjected to an inter-frame motion compensation process and generating a first determined result,

(ii) subjecting said respective block, according to the first determined result, to said inter-frame motion compensation process where an inter-frame correlation is used to reduce an amount of information which needs to be coded in said respective block,

(iii) subjecting (1) said respective block or (2) a result of the inter-frame motion compensation process, according to the first determined result, to a frame restructuring process so as to spatially partition said respective block or said result into smaller sub-blocks, and

(iv) subjecting said sub-blocks to a block coding process where information in each sub-block is compressed.

3. A method according to claim 2, wherein the inter-frame motion compensation process comprises the steps of:

determining whether to subject said respective block (i) to an inter-frame motion prediction process or (ii) to an inter-frame motion interpolation process and generating a second determined result;

selectively subjecting said respective block, according to the second determined result, to said inter-frame motion prediction process where a forward motion vector is detected with reference to a previous coded frame, wherein motion compensation is performed on said respective block using the detected forward motion vector so as to generate a motion compensated block, and comparing the motion compensated block with said respective block; and

selectively subjecting said respective block, according to the second determined result, to said inter-frame motion interpolation process where a forward motion vector is detected with reference to a previous coded frame, a backward motion vector is detected with reference to a future coded frame, including performing three motion compensations respectively using the detected forward motion vector, the detected backward motion vector, and both the detected forward and backward motion vectors, and performing comparisons between results of the three motion compensation together with said respective block.

4. A method according to claim 3, wherein the block coding process comprises the steps of:

discrete cosine transforming each of said sub-blocks into DCT coefficients;

quantizing the DCT coefficients;

zig-zag scanning the quantized DCT coefficients;

variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

5. A method according to claim 4, further comprising a block decoding process comprising the steps of:

demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

inverse zig-zag scanning said sub-blocks followed by inverse quantizing of said sub-blocks to reproduce the DCT coefficients;

inverse DCT processing said coefficients and producing a result;

subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of said encoded information and (ii) a coded result of said step (b) to reproduce the sub-blocks; and

inverting the frame restructuring process according to the coded result of said step (b) to reproduce the respective block.

6. A method according to claim 2, wherein the frame restructuring process further comprises the step of subjecting said sub-blocks that contain chrominance information to an intra-frame sub-sampling process to reduce resolution.

7. A method according to claim 2, wherein the block coding process comprises the steps of:

discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

quantizing the DCT coefficients;

zig-zag scanning the quantized DCT coefficients;

variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

8. A method according to claim 7, further comprising a block decoding process comprising the steps of:

demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

inverse zig-zag scanning said sub-blocks, followed by inverse quantizing of said sub-blocks to reproduce the DCT coefficients;

inverse DCT processing said coefficients and producing a result;

subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of said encoded information and (ii) a coded result of said step (b) to reproduce the sub-blocks; and

inverting the frame restructuring process according to the coded result of said step (b) to reproduce the respective block.

9. A method according to claim 7, wherein said step (b) comprises:

(i) determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) comparing said inter-field difference and said intra-field difference to obtain said difference of said step (b).

10. A method according to claim 2, wherein said step (b) comprises:

(i) determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) comparing said inter-field difference and said intra-field difference to obtain said difference of said step (b).

11. A method of coding an interlaced scan digital video signal comprising the steps of:

(a) partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) determining a difference of pixel values of two fields within each respective block of said blocks of pixel data; and

(c) subjecting each of said blocks of pixel data, according to said difference, (1) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (2) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction, said field coding process comprising the steps of;

(i) determining whether or not said respective block of said blocks of pixel data needs to be subjected to an inter-field motion compensation process and generating a first determined result,

(ii) subjecting said respective block, according to the first determined result, to said inter-field motion compensation process where an inter-field correlation is used to reduce the amount of information needed to be coded in said respective block,

(iii) subjecting (1) said respective block or (2) a result of the inter-field motion compensation process, according to the first determined result, to a field restructuring process where even lines and odd lines of said respective block are separated into smaller sub-blocks, and

subjecting said sub-blocks to a block coding process where information in each sub-block is compressed.

12. A method according to claim 11, wherein the inter-field motion compensation process comprises the steps of:

determining whether to subject said respective block (i) to an inter-field motion prediction process or (ii) to an inter-field motion interpolation process and generating a second determined result;

selectively subjecting said respective block, according to the second determined result, to said inter-field motion prediction process where a first forward motion vector is detected with reference to a previous coded field for even lines of said respective block, a second forward motion vector is detected with reference to a previous coded field for odd lines of said respective block, a motion compensation is performed using the detected first and second forward motion vectors so as to generate a motion compensated block, and the motion compensated block is compared with said respective block;

selectively subjecting said respective block, according to the second determined result, to said inter-field motion interpolation process where first and second forward motion vectors are detected with reference to previous coded fields for the even and odd lines of said respective block, respectively, first and second backward motion vectors are detected with reference to future coded fields for the even and odd lines of said respective block, respectively, three motion compensations are performed by respectively using the detected first and second forward motion vectors, the first and second backward motion vectors, and the first and second forward and backward motion vectors, and including performing comparisons between results of the three motion compensation together with said respective block.

13. A method according to claim 12, wherein the block coding process comprises the steps of:

discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

quantizing the DCT coefficients;

Zig-zag scanning the quantized DCT coefficients;

variable length coding (VLC) the result of the zig-zag scanning to generate encoded information.

14. A method according to claim 13, further comprising a block decoding process comprising the steps of:

demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

inverse zig-zag scanning said sub-blocks followed by inverse quantizing of the sub-blocks to reproduce the DCT coefficients;

inverse DCT processing said coefficients and generating result;

subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of the encoded information and (ii) a coded result of said step (b) to reproduce the sub-blocks; and

inverting of the field restructuring process according to the coded result of said step (b) to reproduce the respective block.

15. A method according to claim 11, wherein the field restructuring process further comprises the step of:

subjecting said sub-blocks that contain chrominance information to an intra-field sub-sampling process to reduce resolution.

16. A method according to claim 11, wherein the block coding process comprises the steps of:

discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

quantizing the DCT coefficients;

zig-zag scanning the quantized DCT coefficients;

variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

17. A method according to claim 16, further comprising a block decoding process comprising the steps of:

demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

inverse zig-zag scanning said sub-blocks followed by inverse quantizing of the sub-blocks to reproduce the DCT coefficients;

inverse DCT processing said coefficients and producing result;

subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) and (ii) a coded result of said step (b) to reproduce the sub-blocks; and

inverting of the field restructuring process according to the coded result of said step (b) to reproduce the respective block.

18. A method according to claim 11, wherein said step (b) comprises:

(i) determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) comparing said inter-field difference and said intra-field difference to obtain said difference of said step (b).

19. An apparatus for coding an interlaced scan digital video signal comprising:

(a) means for partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) means for determining a difference of pixel values of two fields within each of said blocks of pixel data;

said means (b) for determining a difference of pixel values of two fields comprising

(i) means for calculating a mean square error between a line pair comprising an even and an adjacent odd line of a respective block of said blocks of pixel data,

(ii) means for calculating a mean square error between consecutive odd line pairs and consecutive even line pairs of said respective block, and

(iii) means for comparing the two calculated mean square errors; and

(c) means for subjecting each of said blocks of pixel data, according to said difference, (i) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (ii) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction.

20. An apparatus for coding an interlaced scan digital video signal comprising:

(a) means for partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) means for determining a difference of pixel values of two fields within each respective block of said blocks of pixel data; and

(c) means for subjecting each of said blocks of pixel data, according to said difference, (1) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (2) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction, said frame coding process comprising;

(i) determining whether or not said respective block of said blocks of pixel data needs to be subjected to an inter-frame motion compensation process and generating a first determined result,

(ii) subjecting said respective block, according to the first determined result, to said inter-frame motion compensation process where an inter-frame correlation is used to reduce an amount of information which needs to be coded in said respective block,

(iii) subjecting (1) said respective block or (2) a result of the inter-frame motion compensation process, according to the first determined result, to a frame restructuring process so as to spatially partition said respective block or said result into smaller sub-blocks, and

(iv) subjecting said sub-blocks to a block coding process where information in each sub-block is compressed.

21. An apparatus according to claim 20, wherein said (c) means comprises means, for performing the inter-frame motion compensation process, comprising:

means for determining whether to subject said respective block (i) to an inter-frame motion prediction process or (ii) to an inter-frame motion interpolation process and generating a second determined result;

means for selectively subjecting said respective block, according to the second determined result, to said inter-frame motion prediction process where a forward motion vector is detected with reference to a previous coded frame, wherein motion compensation is performed on said respective block using the detected forward motion vector so as to generate a motion compensated block, and comparing the motion compensated block with said respective block; and

means for selectively subjecting said respective block, according to the second determined result, to said inter-frame motion interpolation process where a forward motion vector is detected with reference to a previous coded frame and a backward motion vector is detected with reference to a future coded frame, including performing three motion compensations respectively using the detected forward motion vector, the detected backward motion vector, and both the detected forward and backward motion vectors, and performing comparisons between results of the three motion compensation together with said respective block.

22. An apparatus according to claim 21, wherein said (c) means comprises means, for performing the block coding process, comprising:

means for discrete cosine transforming each of said sub-blocks into DCT coefficients;

means for quantizing the DCT coefficients;

means for zig-zag scanning the quantized DCT coefficients;

means for variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

23. An apparatus according to claim 22, further comprising means for performing a block decoding process comprising:

means for demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

means for inverse zig-zag scanning said sub-blocks followed by inverse quantizing of said sub-blocks to reproduce the DCT coefficients;

means for inverse DCT processing said coefficients and producing a result;

means for subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of said encoded information and (ii) a coded result of said (b) means to reproduce the sub-blocks; and

means for inverting the frame restructuring process according to the coded result of said (b) means to reproduce the respective block.

24. An apparatus according to claim 20, wherein the frame restructuring process performed by said (c) means further comprises the step of subjecting said sub-blocks that contain chrominance information to an intra-frame sub-sampling process to reduce resolution.

25. An apparatus according to claim 20, wherein said (c) means comprises means, for performing the block coding process, comprising:

means for discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

means for quantizing the DCT coefficients;

means for zig-zag scanning the quantized DCT coefficients;

variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

26. An apparatus according to claim 25, further comprising means for performing a block decoding process comprising:

means for demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

means for inverse zig-zag scanning said sub-blocks, followed by inverse quantizing of said sub-blocks to reproduce the DCT coefficients;

means for inverse DCT processing said coefficients and producing a result;

means for subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of said encoded information and (ii) a coded result of said (b) means to reproduce the sub-blocks; and

means for inverting the frame restructuring process according to the coded result of said (b) means to reproduce the respective block.

27. An apparatus according to claim 25, wherein said (b) means comprises:

(i) means for determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) means for comparing said inter-field difference and said intra-field difference to obtain said difference of said (b) means.

28. An apparatus according to claim 20, wherein said (b) means comprises:

(i) means for determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) means for comparing said inter-field difference and said intra-field difference to obtain said difference of said (b) means.

29. An apparatus for coding an interlaced scan digital video signal comprising:

(a) means for partitioning each frame of said interlaced scan digital video signal into blocks of pixel data;

(b) means for determining a difference of pixel values of two fields within each respective block of said blocks of pixel data; and

(c) means for subjecting each of said blocks of pixel data, according to said difference, (1) to a field coding process where at least one of an intra-field correlation or an inter-field correlation is used to achieve information compaction, or (2) to a frame coding process where at least one of an intra-frame correlation or an inter-frame correlation is used to achieve information compaction, said field coding process comprising;

(i) determining whether or not said respective block of said blocks of pixel data needs to be subjected to an inter-field motion compensation process and generating a first determined result,

(ii) subjecting said respective block, according to the first determined result, to said inter-field motion compensation process where an inter-field correlation is used to reduce the amount of information needed to be coded in said respective block,

(iii) subjecting (1) said respective block or (2) a result of the inter-field motion compensation process, according to the first determined result, to a field restructuring process where even lines and odd lines of said respective block are separated into smaller sub-blocks, and

(iv) subjecting said sub-blocks to a block coding process where information in each sub-block is compressed.

30. An apparatus according to claim 29, wherein said (c) means comprises means, for performing the inter-field motion compensation process, comprising:

means for determining whether to subject said respective block (i) to an inter-field motion prediction process or (ii) to an inter-field motion interpolation process and generating a second determined result;

means for selectively subjecting said respective block, according to the second determined result, to said inter-field motion prediction process where a first forward motion vector is detected with reference to a previous coded field for even lines of said respective block, a second forward motion vector is detected with reference to a previous coded field for odd lines of said respective block, a motion compensation is performed using the detected first and second forward motion vectors so as to generate a motion compensated block, and the motion compensated block is compared with said respective block;

means for selectively subjecting said respective block, according to the second determined result, to said inter-field motion interpolation process where first and second forward motion vectors are detected with reference to previous coded fields for the even and odd lines of said respective block, respectively, first and second backward motion vectors are detected with reference to future coded fields for the even and odd lines of said respective block, respectively, three motion compensations are performed by respectively using the detected first and second forward motion vectors, the first and second backward motion vectors, and the first and second forward and backward motion vectors, and including performing comparisons between results of the three motion compensation together with said respective block.

31. An apparatus according to claim 30, wherein said (c) means comprises means, for performing the block coding process, comprising:

means for discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

means for quantizing the DCT coefficients;

means for zig-zag scanning the quantized DCT coefficients;

means for variable length coding (VLC) the result of the zig-zag scanning to generate encoded information.

32. An apparatus according to claim 31, further comprising means for performing a block decoding process comprising:

means for demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

means for inverse zig-zag scanning said sub-blocks followed by inverse quantizing of the sub-blocks to reproduce the DCT coefficients;

means for inverse DCT processing said coefficients and generating a result;

means for subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) of the encoded information and (ii) a coded result of said (b) means to reproduce the sub-blocks; and

means for inverting of the field restructuring process according to the coded result of said (b) means to reproduce the respective block.

33. An apparatus according to claim 29, wherein the field restructuring process further comprises:

means for subjecting said sub-blocks that contain chrominance information to an intra-field sub-sampling process to reduce resolution.

34. An apparatus according to claim 29, wherein said (c) means comprise means, for performing the block coding process, comprising:

means for discrete cosine transforming (DCT) each of said sub-blocks into DCT coefficients;

means for quantizing the DCT coefficients;

means for zig-zag scanning the quantized DCT coefficients;

means for variable length coding (VLC) the result of the zig-zag scanning to produce encoded information.

35. An method according to claim 34, further comprising means for performing a block decoding process comprising:

means for demultiplexing/VLC decoding the encoded information to reproduce the zig-zag scanned sub-blocks and included side information describing the sub-blocks;

means for inverse zig-zag scanning said sub-blocks followed by inverse quantizing of the sub-blocks to reproduce the DCT coefficients;

means for inverse DCT processing said coefficients and producing a result;

means for subjecting the result of the inverse DCT processing to one of inverse inter-frame motion compensation processing and inverse inter-field motion compensation processing, according to (i) a coded picture type (I, P or B-picture) and (ii) a coded result of said (b) means to reproduce the sub-blocks; and

means for inverting of the field restructuring process according to the coded result of said (b) means to reproduce the respective block.

36. An apparatus according to claim 29, wherein said (b) means comprises:

(i) means for determining an inter-field difference and an intra-field difference of pixel values of two fields within each respective block of said blocks of pixel data before subjecting said blocks of pixel data to coding processing, and

(ii) means for comparing said inter-field difference and said intra-field difference to obtain said difference of said (b) means.