VIRTUAL VIEW IMAGE SYNTHESIS METHOD AND APPARATUS

• US 20110255592A1
• Filed: 10/27/2009
• Published: 10/20/2011
• Est. Priority Date: 10/27/2008
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Jeon, Byeong Moon, Lim, Jae Hyun, Jeon, Yong Joon, Sung, Jae Won
• Granted Patent: US 8,913,657 B2
• Time in Patent Office: Days
• US Class Current: 375/240.02
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/24 Systems for the transmission of television signals using pulse code modulation H04N21/00 takes precedence

Abstract

The virtual view image synthesis method according to the present invention comprises: generating an intermediate picture from a first viewpoint and an intermediate picture from a second viewpoint by swapping a reference picture from a first viewpoint and a reference picture from a second viewpoint; determining whether there is a hole in said intermediate picture from a first viewpoint; if there is a hole in said intermediate picture from a first viewpoint, obtaining the pixel value of said intermediate picture from a second viewpoint based on the location of said hole; and allocating the pixel value of said intermediate picture from a second viewpoint to said hole. A virtual picture is synthesized using a reference picture, thereby reducing the quantity of information due to encoding a virtual viewpoint picture.

Claims

1. A method for processing a video signal, the method comprising:

generating a first viewpoint intermediate picture and a second viewpoint intermediate picture by warping a first viewpoint reference picture and a second viewpoint reference picture;

determining whether a hole is present in the first viewpoint intermediate picture;

specifying a position of the hole when the hole is present in the first viewpoint intermediate picture;

obtaining a pixel value of the second viewpoint intermediate picture on the basis of the hole position; and

allocating a pixel value of the second viewpoint intermediate picture to the hole,

wherein the hole is a specific region in which no pixel value is allocated to the first viewpoint intermediate picture.

2. The method according to claim 1, wherein the first viewpoint reference picture and the second viewpoint reference picture are distinguished from each other by viewpoint information, wherein the viewpoint information is used to identify a viewpoint to which the reference picture pertains.

3. The method according to claim 1, wherein the pixel value of the second viewpoint intermediate picture is obtained from a pixel located at the same position as the hole position.

4. The method according to claim 1, wherein the allocating of the pixel value of the second viewpoint intermediate picture to the hole further includes:

obtaining a difference in pixel value between a pixel contiguous to the hole and a pixel of the second viewpoint intermediate picture that is located at the same position as the pixel contiguous to the hole,

wherein one value is allocated to the hole using the difference in pixel value.

5. The method according to claim 4, wherein the pixel contiguous to the hole includes pixels which are located at a left side, a right side, an upper side, a lower side, a left upper side, a left lower side, a right upper side, and a right lower side of the hole.

6. The method according to claim 4, wherein the hole is assigned the sum of an average of differences in pixel value and a pixel value of the second viewpoint intermediate picture.

7. The method according to claim 1, wherein the generating of the first viewpoint intermediate picture and the second viewpoint intermediate picture by warping the first viewpoint reference picture and the second viewpoint reference picture includes:

if pixels of one or more first viewpoint reference pictures are mapped to a pixel of the first viewpoint intermediate picture, warping an average of pixel values of the one or more first viewpoint reference pictures using a pixel value of the first viewpoint intermediate picture.

8. The method according to claim 1, wherein the generating of the first viewpoint intermediate picture and the second viewpoint intermediate picture by warping the first viewpoint reference picture and the second viewpoint reference picture includes:

if a pixel of at least one first viewpoint reference picture is mapped to a pixel of the first viewpoint intermediate picture, comparing depth information of pixel of the first viewpoint reference picture; and

warping a pixel of the first viewpoint reference picture having the smallest depth information from among depth information using a pixel of the first viewpoint intermediate picture.