Method and apparatus for reconstructing three-dimensional objects

• US 5,475,422 A
• Filed: 06/20/1994
• Issued: 12/12/1995
• Est. Priority Date: 06/21/1993
• Status: Expired due to Term

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Mori, Takeaki, Yasuno, Takayuki, Suzuki, Satoshi
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/262,541
• Time in Patent Office: 540 Days
• Field of Search: 348/48
• US Class Current: 348/48
• CPC Class Codes: G06T 11/006 Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

One of the images of an object captured by at least three cameras is defined as a basic image, and a basic backprojection line is defined which passes through a feature point in the basic image corresponding to a noted three-dimensional feature point on the object and the optical center of the camera that captured the basic image. Reference backprojection lines are defined with respect to respective feature points on an epipolar line, obtained by projecting the basic backprojection line onto the image by each of the other cameras, and the optical center of said each camera. The coordinates of intersection points of the basic backprojection line and the reference backprojection lines are calculated and the number of the reference backprojection lines intersecting at each intersection point on the basic backprojection line is counted. The point having the maximum number of intersections is determined to be the position of the noted three-dimensional feature point. Similar processing is performed for all of the feature points in the basic image, by which the positions of corresponding three-dimensional feature points of the object are determined.

Claims

1. A method of obtaining three-dimensional information of an object by capturing images of said object by a plurality of cameras and making three-dimensional feature points on said object to correspond to feature points in said images, said method comprising the steps:

(a) imaging said object by n cameras to obtain n frames of images and outputting said images as image data, said n being an integer equal to or greater than 3;

(b) receiving said image data and extracting feature points of said object on said n frames of images respectively;

(c) defining one of said n cameras as a basic camera, defining the image picked up by said basic camera as a basic image, defining the other remaining cameras as reference cameras, defining the images captured by said reference cameras as reference images, and defining a backprojection line, which passes through the optical center of said basic camera and a feature point in said basic image chosen in correspondence to a noted three-dimensional feature point on said object, as a basic backprojection line;

(d) projecting said basic backprojection line onto said reference images to define thereon epipolar lines, respectively;

(e) defining backprojection lines, which pass through said feature points on said epipolar lines on said reference images and optical centers of said reference cameras, as reference backprojection lines;

(f) calculating the coordinates of intersection points of said basic backprojection line and said reference backprojection lines, counting the numbers of intersections of said basic backprojection line and said reference backprojection lines at their respective intersection points, and determining the intersection point of the maximum count value to be the position of said noted three-dimensional feature point; and

(g) repeating said steps (c) through (f) for each of said feature points on said basic image to obtain the positions of said three-dimensional feature points on said object as its three-dimensional information.

3. The method of claim 1 or 2, wherein a position on said basic backprojection line is defined every predetermined unit length of said basic backprojection line lengthwise thereof and the number of intersections at each of said intersection points is the number of intersections in said unit length at the position on said basic backprojection line through which said reference backprojection lines pass.

4. The method of claim 3, wherein said reference backprojection lines intersecting with said basic backprojection line are reference backprojection lines lying within a predetermined range of distance from said basic backprojection line.

5. The method of claim 1, wherein said step of counting said number of intersections is a step of filtering the distribution of said intersections along said basic backprojection line by convolution to emphasize the concentration of said distribution of said intersections and determining the position where said filtered distribution of said intersections is maximum to be the position of said noted three-dimensional feature point.

10. The method of claim 1 or 2 wherein said step (g) includes a step of removing, after said feature point corresponding to said noted three-dimensional feature point is identified by performing said series of steps (c) through (f), said identified feature point from candidates for processing in the subsequent repetition of said steps (c) through (f).

11. The method of claim 1 or 2, wherein the number n of said cameras is in the range of from 5 to 8.

12. The method of claim 1 or 2, further comprising a step of newly repeating, upon each completion of said step (g), said step (g) for every one of said cameras, using one of said reference cameras as a new basic camera and said basic camera as a new reference camera.

13. The method of claim 1 or 2, further comprising a step of obtaining upon each repetition of simultaneous capturing of images of said object by said n cameras at a fixed period, three-dimensional information of said object following said series of steps.

2. A method of obtaining three-dimensional information of an object by capturing images of said object by a plurality of cameras and making three-dimensional feature points on said object to correspond to feature points in said images, said method comprising the steps:

(a) imaging said object by n cameras to obtain n frames of images and outputting said images as image data, said n being an integer equal to or greater than 3;

(b) receiving said image data and extracting feature points of said object on said n frames of images, respectively;

(c) defining one of said n cameras as a basic camera, defining the other remaining cameras as reference cameras, defining the images captured by said reference cameras as reference images, and defining a backprojection line, which passes through the optical center of said basic camera and a feature point in said basic image chosen in correspondence to a noted three-dimensional feature point on said object, as a basic backprojection line;

(d) projecting said basic backprojection line onto said reference images to define thereon epipolar lines, respectively;

(e) defining backprojection lines, which pass through said feature points on said epipolar lines on said reference images and optical centers of said reference cameras, as reference backprojection lines;

(f) calculating the coordinates of intersection points of said basic backprojection line and said reference backprojection lines, filtering the distribution of said intersection points along said basic backprojection line by convolution to emphasize the concentration of said distribution of said intersection points, and determining the position where said filtered distribution of the intersection points is maximum to be the position of said noted three-dimensional feature point; and

(g) repeating said steps (c) through (f) for each of said feature points on said basic image to obtain the positions of said three-dimensional feature points on said object as its three-dimensional information.

6. The method of claim 2 or 5, wherein said step of filtering is performed by a Laplacian-Gaussian filter.

7. The method of claim 2 or 5, wherein said step of filtering is performed by a square filter.

8. The method of claim 2 or 5, wherein said step of filtering is performed by a triangular filter.

9. The method of claim 2 or 5, wherein said step of filtering is performed by a Gaussian filter.

14. A three-dimensional information reconstructing apparatus comprising:

n camera means disposed at different positions, for capturing images of an object and for outputting said images as image data, said n being an integer equal to or greater than 3; and

image information processing means supplied with said image data from said n camera means, for acquiring three-dimensional information of said object from feature points in said images;

said image information processing means comprising;

feature extracting means for extracting said feature points in said images;

basic backprojection line generating means wherein one of said n camera means is defined as a basic camera, the image captured by said basic camera means as a basic image, the other remaining camera means as reference camera means, the images captured by said reference camera means as reference images, and whereby a backprojection line, which passes through the optical center of said basic camera means and a feature point in said basic image chosen in correspondence to a noted three-dimensional feature point on said object, is obtained as a basic backprojection line;

epipolar line generating means for projecting said basic backprojection line onto said reference images to obtain epipolar lines;

reference backprojection line generating means for obtaining, as reference backprojection lines, backprojection lines which pass through feature points on said epipolar lines on said reference images and the optical centers of said reference camera means; and

three-dimensional feature extracting means for calculating the coordinates of intersection points of said basic backprojection line and said reference backprojection lines, for counting the number of intersections of said basic backprojection line and said reference backprojection lines at each of said intersection points and for determining that one of said intersection points which has the maximum count value is the position of said noted three-dimensional feature point;

whereby the positions of respective three-dimensional feature points on said object, corresponding to respective feature points on said basic image, are obtained as said three-dimensional information of said object.

16. The apparatus of claim 14, wherein said feature extracting means includes filter means for emphasizing the concentration of said distribution of intersections by convolution along said basic backprojection line and the position where said filtered distribution of intersections is maximum is determined to be the position of said noted three-dimensional feature point.

21. The method of claim 14 or 15, wherein said number n of cameras means is in the range of from 5 to 8.

22. The apparatus of claim 14 or 15, further comprising camera control means for repeating simultaneous image capturing of said object by said n camera means at a fixed period and for supplying said image data to said image information processing means upon each generation of said image data.

15. A three-dimensional information reconstructing apparatus comprising:

n camera means disposed at difference positions, for capturing images of an object and for outputting said images as image data, said n being an integer equal to or greater than 3; and

image information processing means supplied with said image data from said n camera means, for acquiring three-dimensional information of said object from feature points in said images;

said image information processing means comprising;

feature extracting means for extracting feature points in said images;

basic backprojection line generating means wherein one of said n camera means is defined as a basic camera means, the image captured by said basic camera means as a basic image, the other remaining camera means as reference cameras, the images captured by said reference camera means as reference images, and whereby a backprojection line, which passes through the optical center of said basic camera means and a feature point in said basic image chosen in correspondence to a noted three-dimensional feature point on said object, is obtained as a basic backprojection line;

epipolar line generating means for projecting said basic backprojection line onto said reference images to obtain epipolar lines;

reference backprojection line generating means for obtaining, as reference backprojection lines, backprojection lines which pass through feature points on said epipolar lines on said reference images and the optical centers of said reference camera means;

filtering means which calculates the coordinates of intersection points of said basic backprojection line and said reference backprojection lines, respectively, and filters the distribution of said intersection points along said basic backprojection line by convolution to emphasize the concentration of said distribution of said intersection points; and

three-dimensional feature extracting means whereby the position where said filtered distribution of said intersection points is maximum is determined to the position of said noted three-dimensional feature point;

whereby the positions of respective three-dimensional feature points on said object, corresponding to respective feature points on said basic image, are obtained as said three-dimensional information of said object.

17. The method of claim 15 or 16, wherein said filter means is a Laplacian-Gaussian filter.

18. The apparatus of claim 15 or 16, wherein said filter means is square filter.

19. The apparatus of claim 15 or 16, wherein said filter means is a triangular filter.

20. The apparatus of claim 15 or 16, wherein said filter means is a Gaussian filter.