Method of and an apparatus for 3-dimensional structure estimation

• US 6,049,625 A
• Filed: 10/15/1997
• Issued: 04/11/2000
• Est. Priority Date: 10/15/1996
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Sakamoto, Shizuo
• Primary Examiner(s): Kelley, Christopher S.
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US08/950,599
• Time in Patent Office: 909 Days
• Field of Search: 382/106, 382/154, 382/278, 382/190, 382/194, 382/199, 382/201, 382/202, 382/203, 356/239.2, 356/239.3, 356/239.8, 356/12, 356/3.11, 356/11, 250/559.23, 250/559.31, 250/559.19, 250/559.2, 250/559.22, 345/419, 345/425
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method of 3-dimensional structure estimation of the invention, making use of a plurality of stereo-pictures, repeats, for each pixel of a first picture, a step of extracting corresponding small regions (4) corresponding to the concerning pixel according to a depth estimation (68); a step of calculating a neighboring correspondence value for each of the corresponding small regions representing correspondence among neighboring corresponding small regions of picures taken by neighboring cameras; a step of obtaining a sum (61) of the neighboring correspondence values; and selecting a value of the depth estimation (68) which gives a singular value of the sum (61) representing correspondence among the corresponding small regions.

Claims

1. An apparatus for 3-dimensional structure estimation for estimating a 3-dimensional structure of an object from image data of a plurality of pictures of the object each taken from each viewing position arranged on a straight line by a camera with an optical axis parallel to a direction perpendicular to the straight line;

said apparatus comprising;

small region extraction means for extracting corresponding small regions each from the image data of each of the plurality of pictures, a position of each of said corresponding small regions in corresponding each of the plurality of pictures being defined by a focal distance of the camera, a distance between a viewing position wherefrom said corresponding each of the plurality of pictures is taken and a viewing position wherefrom a first of the plurality of pictures is taken, a position of a reference pixel of image data of the first of the plurality of pictures, and a variable representing a depth of a point of the object corresponding to said reference pixel;

neighboring small region average calculation means for calculating, for each of the plurality of pictures, an average pixel value of said corresponding small regions of neighboring certain of the plurality of pictures;

differential calculation means for calculating differential of pixel values of each of said corresponding small regions to said average pixel value corresponding to said each of said corresponding small regions;

sum calculation means for outputting a sum of said differential of all of said corresponding small regions as a differential sum;

depth counting means for varying a value of said variable in a range;

reference pixel position counting means for revising said position of said reference pixel; and

minimum cost depth selecting means for obtaining 3-dimensional structure information of the object by finding a value of said variable which makes said differential sum minimum for each pixel of image data of the first picture by controlling said depth counting means and said reference pixel position counting means.

2. An apparatus for 3-dimensional structure estimation for estimating a 3-dimensional structure of an object from image data of a plurality of pictures of the object each taken from each viewing position arranged on a straight line by a camera with an optical axis parallel to a direction perpendicular to the straight line;

said apparatus comprising;

corresponding pixel extraction means for extracting corresponding pixels each from the image data of each of the plurality of pictures, a position of each of said corresponding pixels in corresponding each of the plurality of pictures being defined by a focal distance of the camera, a distance between a viewing position wherefrom said corresponding each of the plurality of pictures is taken and a viewing position wherefrom a first of the plurality of pictures is taken, a position of a reference pixel of image data of the first of the plurality of pictures, and a variable representing a depth of a point of the object corresponding to said reference pixel;

neighboring pixel average calculation means for calculating, for each of the plurality of pictures, an average pixel value of said corresponding pixels of neighboring certain of the plurality of pictures;

differential calculation means for calculating differential of a pixel value of each of said corresponding pixels to said average pixel value corresponding to said each of said corresponding pixels;

sum calculation means for outputting a sum of said differential of all of said corresponding pixels as a differential sum;

depth counting means for varying a value of said variable in a range;

reference pixel position counting means for revising said position of said reference pixel; and

minimum cost depth selecting means for obtaining 3-dimensional structure information of the object by finding a value of said variable which makes said differential sum minimum for each pixel of image data of the first picture by controlling said depth counting means and said reference pixel position counting means.

3. An apparatus for 3-dimensional structure estimation for estimating a 3-dimensional structure of an object from image data of a plurality of pictures of the object each taken from each viewing position arranged on a straight line by a camera with an optical axis parallel to a direction perpendicular to the straight line;

said apparatus comprising;

small region extraction means for extracting corresponding small regions each from the image data of each of the plurality of pictures, a position of each of said corresponding small regions in corresponding each of the plurality of pictures being defined by a focal distance of the camera, a distance between a viewing position wherefrom said corresponding each of the plurality of pictures is taken and a viewing position wherefrom a first of the plurality of pictures is taken, a position of a reference pixel of image data of the first of the plurality of pictures, and a variable representing a depth of a point of the object corresponding to said reference pixel;

neighboring small region variance calculation means for calculating, for each of the plurality of pictures, a variance of pixel values in said corresponding small regions of neighboring certain of the plurality of pictures;

sum calculation means for outputting a sum of said variance of all of said corresponding small regions as a variance sum;

depth counting means for varying a value of said variable in a range;

reference pixel position counting means for revising said position of said reference pixel; and

minimum cost depth selecting means for obtaining 3-dimensional structure information of the object by finding a value of said variable which makes said variance sum minimum for each pixel of image data of the first picture by controlling said depth counting means and said reference pixel position counting means.

4. An apparatus for 3-dimensional structure estimation for estimating a 3-dimensional structure of an object from image data of a plurality of pictures of the object each taken from each viewing position arranged on a straight line by a camera with an optical axis parallel to a direction perpendicular to the straight line;

said apparatus comprising;

corresponding pixel extraction means for extracting corresponding pixels each from the image data of each of the plurality of pictures, a position of each of said corresponding pixels in corresponding each of the plurality of pictures being defined by a focal distance of the camera, a distance between a viewing position wherefrom said corresponding each of the plurality of pictures is taken and a viewing position wherefrom a first of the plurality of pictures is taken, a position of a reference pixel of image data of the first of the plurality of pictures, and a variable representing a depth of a point of the object corresponding to said reference pixel;

neighboring pixel variance calculation means for calculating, for each of the plurality of pictures, a variance of values of said corresponding pixels of neighboring certain of the plurality of pictures;

sum calculation means for outputting a sum of said variance of all of said corresponding pixels as a variance sum;

depth counting means for varying a value of said variable in a range;

reference pixel position counting means for revising said position of said reference pixel; and

minimum cost depth selecting means for obtaining 3-dimensional structure information of the object by finding a value of said variable which makes said variance sum minimum for each pixel of image data of the first picture by controlling said depth counting means and said reference pixel position counting means.