Apparatus for encoding a multi-view moving picture

• US 6,999,513 B2
• Filed: 09/17/2002
• Issued: 02/14/2006
• Est. Priority Date: 04/20/2002
• Status: Active Grant

• Current Assignee: Korea Electronics Technology Institute (Government of South Korea)
• Original Assignee: Korea Electronics Technology Institute (Government of South Korea)
• Inventors: Sohn, Kwang Hoon, Kim, Je Woo, Choi, Byeong Ho, Jung, Hyeok Koo, Lim, Jeong Eun
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US10/246,262
• Publication Number:
  US 20030202592A1
• Time in Patent Office: 1,246 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

A disparity prediction stage and a motion prediction stage predict a disparity vector and a motion vector by extending a MPEG-2 structure into a view axis and using spatial/temporal correlation. A disparity/motion compensation stage compensates an image reconstructed by the disparity prediction stage and the motion prediction stage by using a sub-pixel compensation method. A residual image encoding stage performs an encoding to provide a better visual quality and a three-dimensional effect of an original image and the reconstructed image. A bit rate control stage controls a bit rate for assigning an effective amount of bit to each frame on the reconstructed image according to a bit rate. An entropy encoding stage generates a bit stream on multi-view video source data according to the bit rate.

Claims

1. An apparatus for encoding multi-view video source data, comprising:

preprocessing stage, when multi-view video source data is input, for eliminating from the input data a noise and solving an imbalancing problem;

disparity prediction stage and motion prediction stage for estimating a disparity vector and a motion vector, respectively, on the basis of a moving picture axis including an I picture;

disparity/motion compensation stage for compensating an image predicted by the disparity and the motion estimation stages by using a sub-pixel compensation technique;

residual image encoding stage for receiving an original picture from the preprocessing stage and a reconstructed picture provided from the disparity/motion compensation stage and performing a residual image encoding process in order to improve a picture quality and enhance a three-dimensional effect of the picture;

bit rate control stage for controlling a bit rate for each frame in a recovered picture encoded by the residual image encoding stage; and

entropy encoding stage for generating bit streams for the multi-view moving picture source data according to the bit rates controlled by the bit rate control stage.

2. The apparatus of claim 1, wherein the preprocessing stage removes problems such as noise elimination and imbalancing by revising a reference image and an image to be revised through the mean and variance thereof.

3. The apparatus of claim 1, wherein the disparity/motion estimation stage sets up an estimated direction by using a multi-view disparity vector and a motion vector generated when the multi-view disparity vector is extended into moving picture.

4. The apparatus of claim 1, wherein the bit rate control controlled by the bit rate control stage uses a MPEG-2 bit rate control method, wherein a structure having one I picture inside a GGOP regards GGOP as one GOP and performs a bit rate control;

a structure having two I pictures inside the GGOP performs a bit rate control by dividing GGOP into two parts including each one of I pictures;

a structure having a C number of I pictures inside the GGOP is considered to have the C number of MPEG-2 moving picture and controls a bit rate on each separately, the C is positive integer.

5. The apparatus of claim 1, wherein the bit streams are constituted with a main bit stream and an auxiliary bit stream, the main bit stream having a same structure with MPEG-2 bit stream for the moving picture on the time axis having the I picture in order to be compatible with MPEG-2, and the auxiliary bit stream including data for processing multi-view moving picture on the time axis having no I picture.

6. The apparatus of claim 5, wherein the auxiliary bit stream includes a position and the number of I pictures, the number of views, a disparity vector and a motion vector of a view axis having no I picture, residual image encoding information and additional information.

7. The apparatus of claim 5, wherein the main bit stream is divided into a sequence layer that is a structure of MPEG-2, a GOP layer, a picture layer, a slice layer, a macro block layer and a block layer for compatibility with MPEG-2.

8. The apparatus of claim 6, wherein the auxiliary bit stream is inserted a sequence header, a sequence extension header, a picture header and a picture extension header with maintaining a structure of MPEG-2 to thereby distinguish data corresponding to multi-view moving picture source from data suitable for a MEPG-2 specification by applying a constrained parameters flag to a sequence header of MPEG-2.

9. The apparatus of claim 8, wherein the constrained parameters flag is a flag for distinguishing MPEG-1 from MPEG-2, but in multi-view moving picture encoder the constrained parameters flag becomes a flag for determining whether data to be processed is a multi-view moving picture signal or MPEG-2 moving picture.

10. The apparatus of claim 9, if the flag is set up as ‘

0’

, the data to be processed is the multi-view moving picture and if the flag is set up as ‘

1’

, the data to be processed is MPEG-2 moving picture and a reference frame number of the flag indicating a number of multi-view moving picture is inserted into the sequence extension header.

11. The apparatus of claim 10, wherein the reference frame number indicates the number of I pictures.

12. The apparatus of claim 11, wherein the reference frame position that is a flag for designating a position corresponding to a view and a time axis is inserted into a position of the I picture and, when a picture encoding type of the picture is a P picture and a B picture, information for distinguishing a picture encoding type using a disparity vector or a motion vector is added.

13. The apparatus of claim 12, in case of the P picture, other values are assigned to distinguish the disparity vector or the motion vector and a search range on the disparity vector is set up by additionally inserting the search range of the disparity vector into the motion vector search range in the picture extension layer;

a disparity horizontal code that is a flag for designating a horizontal search range of the disparity vector and a disparity vertical code for designating the vertical search range of the disparity vector are added.

14. The apparatus of claim 1, a structure having one I frame inside the GGOP and a structure having two I frames inside the GGOP are used in order to effectively apply multi-view moving picture according to a size of the disparity vector generated by a camera parameter for obtaining the multi-view moving picture, wherein the multi-view moving picture structure having a small disparity vector is encoded by using the structure having one I frame and the multi-view moving picture having a big disparity vector is encoded by the structure having two I frames so that an error caused by the disparity vector estimation process is reduced and when generating a certain number of MPEG-2 structure bit streams, apart from the disparity vector, I frame uses the certain number of structures.