Virtual view image synthesis method and apparatus

• US 8,913,657 B2
• Filed: 10/27/2009
• Issued: 12/16/2014
• Est. Priority Date: 10/27/2008
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Sung, Jae Won, Jeon, Yong Joon, Jeon, Byeong Moon, Lim, Jae Hyun
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): EDWARDS, TYLER B
• Application Number:
  US13/126,088
• Publication Number:
  US 20110255592A1
• Time in Patent Office: 1,876 Days
• US Class Current: 375/240.02
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 7/24 Systems for the transmission of television signals using pulse code modulation H04N21/00 takes precedence

Abstract

The virtual view image synthesis method according to the present invention comprises: generating an intermediate picture from a first viewpoint and an intermediate picture from a second viewpoint by swapping a reference picture from a first viewpoint and a reference picture from a second viewpoint; determining whether there is a hole in said intermediate picture from a first viewpoint; if there is a hole in said intermediate picture from a first viewpoint, obtaining the pixel value of said intermediate picture from a second viewpoint based on the location of said hole; and allocating the pixel value of said intermediate picture from a second viewpoint to said hole. A virtual picture is synthesized using a reference picture, thereby reducing the quantity of information due to encoding a virtual viewpoint picture.

Claims

1. A method of processing a video signal, the method comprising:

generating a first viewpoint intermediate picture and a second viewpoint intermediate picture by warping a first viewpoint reference picture and a second viewpoint reference picture wherein, when a plurality of pixels of the first viewpoint reference picture are mapped to a pixel of the first viewpoint intermediate picture, generating the first viewpoint intermediate picture comprises;

obtaining position information in the first viewpoint intermediate picture corresponding to the plurality of pixels of the first viewpoint reference picture;

obtaining a pixel value of the pixel of the first viewpoint intermediate picture based on pixel values of the plurality of pixels of the first viewpoint reference picture and the position information; and

modifying the pixel value of the pixel of the first viewpoint intermediate picture based on a number of the plurality of pixels of the first viewpoint reference picture and the pixel of the first viewpoint intermediate picture and a sum of the pixel values of the plurality of pixels of the first viewpoint reference picture and the pixel value of the pixel of the first viewpoint intermediate picture;

determining whether a hole is present in the first viewpoint intermediate picture;

based on a determination that a hole is present in the first viewpoint intermediate picture, specifying a position of the hole;

obtaining a pixel value of the second viewpoint intermediate picture on the basis of the position of the hole in the first viewpoint intermediate picture;

obtaining differential pixel values between pixels of the first viewpoint intermediate picture that are contiguous to the hole and pixels of the second viewpoint intermediate picture that are located at positions in the second viewpoint intermediate picture corresponding to positions of the pixels in the first viewpoint intermediate picture that are contiguous to the hole; and

allocating, to the hole, a sum of an average of the differential pixel values and the pixel value of the second viewpoint intermediate picture,

wherein the hole is a specific region in which no pixel value is allocated to the first viewpoint intermediate picture.

2. The method according to claim 1, wherein the first viewpoint reference picture and the second viewpoint reference picture are distinguished from each other by viewpoint information, wherein the viewpoint information is used to identify a first viewpoint to which the first viewpoint reference picture pertains and a second viewpoint to which the second viewpoint reference picture pertains.

3. The method according to claim 1, wherein the pixel value of the second viewpoint intermediate picture is obtained from a pixel located in the second viewpoint intermediate picture at a position corresponding to a position of the hole in the first viewpoint intermediate picture.

4. The method according to claim 1, wherein the pixels contiguous to the hole include pixels which are located at a left side, a right side, an upper side, a lower side, a left upper side, a left lower side, a right upper side, and a right lower side of the hole.