Three-dimensional image coding by merger of left and right images

• US 5,767,898 A
• Filed: 06/20/1995
• Issued: 06/16/1998
• Est. Priority Date: 06/23/1994
• Status: Expired due to Fees

• Current Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Hamamoto, Yasuhachi, Kobayashi, Satoko, Urano, Takashi, Sugimoto, Etsuko, Kodama, Hideo
• Primary Examiner(s): LIN, BRYAN T
• Application Number:
  US08/492,908
• Time in Patent Office: 1,092 Days
• Field of Search: 348/43, 348/42, 348/46, 348/47
• US Class Current: 348/43
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0077 Colour aspects
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

A method is disclosed, for encoding a signal with a three dimensional image sequence using a series of left and right images. Each image in the left image series is a picture formed by non-interlaced or interlaced scanned left line images, and each image in the right image series is a picture formed by non-interlaced or interlaced scanned right line images. The left line images contained in the left picture are merged with the right line images contained in the right picture to produce an alternately arranged left and right line merged picture. The merged picture is encoded using an MPEG-2 compliant encoder.

Claims

1. A method of encoding a signal with a three dimensional image sequence using left and right video image sequences, each of the video images being a picture constituted by image lines obtained by scanning, said method comprising the steps of:

merging left line images contained in a left picture with right line images contained in a right picture continuously to produce a plurality of alternately arranged left and right line merged pictures;

encoding one of said merged pictures as a first merged picture to produce an I-picture;

predictively encoding one of said merged pictures as a second merged picture by using said first merged picture, producing a P-picture; and

predictively encoding one of said merged pictures as a third merged picture located between said first and second merged pictures, by using at least one of said first and second merged pictures, producing a B-picture,

wherein each of said predictively encoding steps for the P-picture and B-picture comprises the steps of;

separating the corresponding merged picture into a plurality of 16×

16 pixel macroblocks;

calculating a first value indicative of the predictive encoding efficiency performed on each of said 16×

16 pixel macroblocks by using a target macroblock and one of said first and second merged pictures;

separating each of said plurality of 16×

16 pixel macroblocks into left and right 16×

8 pixel macroblock halves, each of said left halves containing left line images, and each of said right halves containing right line images;

separating at least one of said first and second merged pictures into a right field and a left field, said right field containing right line images and said left field containing left line images;

calculating a second value indicative of the predictive encoding efficiency performed on each of the left and right 16×

8 pixel macroblock halves, by using one of said right and left 16×

8 pixel macroblock halves in combination with said right and left fields, wherein said first value is higher than said second value when the correlation between the left and right line images in each of said 16×

16 pixel macroblocks is higher than the correlation between one of said left and right 16×

8 pixel macroblock halves and one of said left and right fields;

selecting the most efficient predictive encoding process between said first and second values, wherein the predictive encoding process of each of said 16×

16 pixel macroblocks is selected when said first value is higher than said second value; and

predictively encoding the corresponding merged picture to produce the P-picture or B-picture, based on the most efficient of the predictive encoding processes.

2. The method according to claim 1, wherein said left picture serves as a left frame including the left line images obtained by non-interlaced scanning, and wherein said right picture serves as a right frame including the right line images obtained by non-interlaced scanning.

3. The method according to claim 1, wherein said left picture includes two fields, each of which is constituted by the left line images obtained by interlaced scanning, and wherein said right picture includes two fields, each of which is constituted by the right line images obtained by interlaced scanning.

4. The method according to claim 1, wherein each of said 16×

16 pixel macroblocks is one of a two-dimensional 16×

16 pixel macroblock and a three-dimensional 16 ×

16 pixel macroblock, wherein the correlation between the left and right line images in said two-dimensional 16×

16 pixel macroblock is higher than the correlation between the left and right line images in said three-dimensional 16×

16 pixel macroblock, and

wherein said selecting step includes the steps of;

selecting the predictive encoding process of each of said 16×

16 pixel macroblocks when each of said 16×

16 pixel macroblocks is said two-dimensional 16×

16 pixel macroblock; and

selecting the predictive encoding process of each of said left and right macroblocks when each of said 16×

16 pixel macroblocks is said three-dimensional 16×

16 pixel macroblock.

5. An apparatus for encoding a signal with a three dimensional image sequence using left and right video image sequences, each of the video images being a picture constituted by line images obtained by scanning, said method comprising the step of using an encoding apparatus, wherein said encoding apparatus includes:

a mixer for merging left line images contained in a left picture with right line images contained in a right picture to produce an alternately arranged left and right line merged picture, wherein said merged picture includes a plurality of 16×

16 pixel macroblocks;

an encoder for executing the encoding on each of said macroblocks of the merged picture, wherein said encoder includes;

first encoder for performing inter-frame predictive encoding as well as motion compensation on each of said macroblocks;

second encoder for separating each macroblock into left and right 16×

8 pixel macroblock halves, each of said left halves containing the left line images and each of said right halves containing the right line images, and for performing inter-field predictive encoding as well as motion compensation on each of said 16×

8 macroblock halves; and

selector for selecting one of encoded data produced by said first encoder and data produced by said second encoder, wherein said selector selects said inter-frame predictive encoding as well as motion compensation of said first encoder when the correlation between the left and right line images in each of said 16×

16 pixel macroblocks is higher than one of the correlation between the adjacent left line images in said left 16×

8 pixel macroblock half and the correlation between the adjacent right line images in said right 16×

8 pixel macroblock half.

6. The apparatus according to claim 5, wherein each of said 16×

16 pixel macroblocks is one of two-dimensional 16×

16 pixel macroblock and a three-dimensional 16×

16 pixel macroblock, wherein the correlation between the left and right line images in said two-dimensional 16×

16 pixel macroblock is higher than the correlation between the left and right line images in said three-dimensional 16×

16 pixel macroblock, and

wherein said selector selects the predictive encoding process of each of said 16 ×

16 pixel macroblocks when each of said 16×

16 pixel macroblocks is said two-dimensional 16×

16 pixel macroblock and selects the predictive encoding process of each of said left and right 16×

8 macroblock halves when each of said 16×

16 pixel macroblocks is said three-dimensional 16×

16 pixel macroblock.

7. An apparatus for encoding a signal with a three dimensional image sequence using left and right video image sequences, each of the video images being a picture constituted by image lines obtained by scanning, said apparatus comprising:

a mixer for merging left line images contained in a left picture with right line images contained in a right picture continuously to produce a plurality of alternately arranged left and right line merged pictures; and

an encoder for encoding one of said merged pictures as a first merged picture to produce an I-picture, for predictively encoding one of said merged pictures as a second merged picture by using said first merged picture by using said first merged picture to produce a P-picture, and for predictively encoding one of said merged pictures as a third merged picture located between said first and second merged pictures by using at least one of said first and second merged pictures to produce a B-picture,

wherein said encoder includes;

a macroblock converter for separating a merged picture into a plurality of 16×

16 pixel macroblocks;

a first value calculating circuit for calculating a first value indicative of the predictive encoding efficiency performed on each of said 16×

16 pixel macroblocks by using a target macroblock and one of said first and second merged pictures;

a first separator for separating each of said plurality of 16×

16'"'"' pixel macroblocks into left and right 16×

8 pixel macroblock halves, each of said left halves containing left line images, and each of said right halves containing right line images;

a second separator for separating at least one of said first and second merged pictures into a right field and a left field, said right field containing right line images and said left field containing left line images;

a second value calculating circuit for calculating a second value indicative of the predictive encoding efficiency performed on each of the left and right macroblocks, by using one of said right and left macroblock halves in combination with said right and left fields, wherein said first value is higher than said second value when the correlation between the left and right line images in each of said 16×

16 pixel macroblocks is higher than the correlation between one of said left and right 16×

8 pixel macroblock halves and one of said left and right fields; and

a value comparator for selecting the most efficient predictive encoding process between said first and second values, wherein said value comparator selects the predictive encoding process of each of said 16×

16 pixel macroblocks when said first value is higher than said second value, whereby said encoder predictively encodes a merged picture to produce the P-picture or B-picture, based on the most efficient of the predictive encoding processes.

8. The apparatus according to claim 7, wherein said encoder further includes an inserter for inserting at least one of information indicative of encoded data of a three dimensional motion picture, information indicating which a merged picture to be encoded is image obtained by non-interlaced scanning or image obtained by interlaced scanning and information indicative of data formats of a merged picture and a macroblock, into data of said I-picture, data of said P-picture and data of said B-picture.

9. The apparatus according to claim 7, wherein said left picture serves as a left frame including the left line images obtained by non-interlaced scanning, and wherein said right picture serves as a right frame including the right line image s obtained by non-interlaced scanning.

10. The apparatus according to claim 7, wherein said left picture includes two fields, each of which is constituted by the left line images obtained by interlaced scanning, and wherein said right picture includes two fields, each of which is constituted by the right line images obtained by interlaced scanning.

11. The apparatus according to claim 7, wherein each of said 16×

16 pixel macroblocks is one of a two-dimension 16×

16 pixel macroblock and a three-dimensional 16 ×

16 pixel macroblock, wherein the correlation between the left and right line images in said two-dimensional 16×

16 pixel macroblock is higher than the correlation between the left and right line images in said three-dimensional 16×

16 pixel macroblock, and

wherein said value comparator selects the predictive encoding process of each of said 16 ×

16 pixel macroblocks when each of said 16×

16 pixel macroblocks is said two-dimensional 16×

16 pixel macroblock and selects the predictive encoding process of each of said left and right macroblocks when each of said 16×

16 pixel macroblocks is said three-dimensional 16×

16 pixel macroblock.

12. A method of encoding a signal with a three dimensional image sequence using left and right video image sequences, each of the video images being a picture constituted by image lines obtained by scanning, said method comprising the steps of:

merging the left line images contained in said left picture with the right line images contained in said right picture to produce an alternately arranged left and right line merged blocks for use in a first encoding process;

producing first blocks, each constituted by left line images and second blocks, each constituted by right line images for use in a second encoding process;

estimating a first predictive encoding efficiency of said first encoding process and a second predictive encoding efficiency of said second encoding process;

comparing the first and second predictive encoding efficiencies;

selecting the most efficient predictive encoding process based on the comparing result; and

predictively encoding the corresponding blocks to produce an encoded picture based on the most efficient of the predictive encoding processes.

13. The method according to claim 12, wherein said each merged block is one of a two-dimensional merged block and a three-dimensional merged block, wherein the correlation between the left and right line images in said two-dimensional merged block is higher than the correlation between the left and right line images in said three-dimensional merged block, and

wherein said selecting step includes the steps of selecting the predictive encoding process of said merged blocks when each said merged block is said two-dimensional merged block and selecting the predictive encoding process of said first and second blocks when said each said merged block is said three-dimensional merged block.

14. The method according to claim 12, wherein said predictive encoding process includes the steps of:

preforming discrete cosine transform (DCT) processing on said each merged block when the correlation between the left and right line images in said each merged block is higher than one of the correlation between the adjacent left lines in said first block and the correlation between the adjacent right lines in said second block; and

performing separately discrete cosine transform (DCT) processing on said each first and second blocks when the correlation between the left and right line images in said each merged block is lower than one of the correlation between the adjacent left lines in said each first block and the correlation between the adjacent right lines in said each second block.

15. A method of encoding a signal with a three dimensional image sequence using left and right video image sequences, each of the video images being a picture constituted by image lines obtained by scanning, said method comprising the steps of:

merging the left line images contained in said left picture with the right line images contained in said right picture to produce an alternately arranged left and right line merged blocks for use in a first encoding process;

producing first blocks, each constituted by left line images and second blocks each constituted by right line images for use in a second encoding process;

estimating a first predictive encoding efficiency of said first encoding process and a second predictive encoding efficiency of said second encoding process, wherein said first predictive encoding efficiency is higher than said second predictive encoding efficiency when the correlation between the left and right line images in said each merged block is higher than one of the correlation between the adjacent left lines in said each first block and the correlation between the adjacent right lines in said each second block;

comparing the first and second predictive encoding efficiencies;

selecting the most efficient predictive encoding process based on the comparing result, wherein the predictive encoding process of said merged blocks is selected when said first predictive encoding efficiency is higher than said second predictive encoding efficiency; and

predictively encoding the corresponding blocks to produce an encoded picture based on the most efficient of the predictive encoding processes.

16. The method according to claim 15, wherein each said merged block is one of a two-dimensional merged block and a three-dimensional merged block, wherein the correlation between the left and right line images in said two-dimensional merged block is higher than the correlation between the left and right line images in said three-dimensional merged block, and

wherein said selecting step includes the steps of:

selecting the predictive encoding process of said merged blocks when said each merged block is said two-dimensional merged block; and

selecting the predictive encoding process of said first and second blocks when said each merged block is said three-dimensional merged block.

17. The method according to claim 15, wherein said predictive encoding process includes the steps of:

performing discrete cosine transform (DCT) processing on said each merged block when the correlation between the left and right line images in said each merged block is higher than one of the correlation between the adjacent left lines in said each first block and the correlation between the adjacent right lines in said each second block; and

performing separately discrete cosine transform (DCT) processing on said each first and second blocks when the correlation between the left and right line images in said each merged block is lower than one of the correlation between the adjacent left lines in said each first block and the correlation between the adjacent right lines in said each second block.