Image system for three dimensional, 360 DEGREE, time sequence surface mapping of moving objects

• US 5,852,672 A
• Filed: 06/09/1997
• Issued: 12/22/1998
• Est. Priority Date: 07/10/1995
• Status: Expired due to Term

• Current Assignee: Lawrence Livermore National Security LLC (Government of the United States of America)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Lu, Shin-Yee
• Primary Examiner(s): Moore, David K.
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US08/871,557
• Time in Patent Office: 561 Days
• Field of Search: 382/103, 382/107, 382/106, 382/108, 382/152, 382/153, 382/154, 382/285, 382/291, 382/294, 434/4, 434/257, 345/427, 345/419, 345/358, 345/953, 348/47, 348/48, 348/94, 348/139, 348/136, 352/60, 901/47, 250/559.23, 250/559.31, 395/94, 356/12, 356/3.11, 356/376
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2441 using interferometry
• CPC Class Codes: G01B 11/245 using a plurality of fixed, simultaneously operating transducers G01B11/2408 - G01B11/2425, G01B11/255 take precedence
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

A three-dimensional motion camera system comprises a light projector placed between two synchronous video cameras all focused on an object-of-interest. The light projector shines a sharp pattern of vertical lines (Ronchi ruling) on the object-of-interest that appear to be bent differently to each camera by virtue of the surface shape of the object-of-interest and the relative geometry of the cameras, light projector and object-of-interest Each video frame is captured in a computer memory and analyzed. Since the relative geometry is known and the system pre-calibrated, the unknown three-dimensional shape of the object-of-interest can be solved for by matching the intersections of the projected light lines with orthogonal epipolar lines corresponding to horizontal rows in the video camera frames. A surface reconstruction is made and displayed on a monitor screen. For 360° all around coverage of theobject-of-interest, two additional sets of light projectors and corresponding cameras are distributed about 120° apart from one another.

Claims

1. A system for detecting and measuring the three-dimensional positions occupied by the surfaces of an object-of-interest, comprising:

a first video camera and a second video camera separated from one another along a baseline and aimed in tandem at a target area and providing for respective first and second video images;

a light projector disposed between the first and second video cameras for projecting a first set of parallel light line segments at said target area; and

three-dimensional surface generation means connected to receive said first and second video images and a camera pair calibration matrix and for translating the respective observations of said light lines distorted by their falling on an object-of-interest into a plurality of three-dimensional coordinates that estimate a surface of said object-of-interest and that conform to a plurality of measurements taken for said camera pair calibration matrix;

wherein, the three-dimensional surface generation means include frame grabbing means for synchronizing respective first and second video image outputs of the first and second video cameras and for storing a sequence of video frames in memory that represent image columns parallel to said first set of parallel line segments of light and image rows orthogonal to said first set of parallel line segments of light;

wherein;

the three-dimensional surface generation means includes image warping means connected to the first and second video cameras for electronically warping the images from each camera to have a common plane of observation, wherein the horizontal and vertical linear scales of each image coincide with the other, and providing for an aligned image output for each of said first and second video images;

wherein, the three-dimensional surface generation means includes stereo image registration means for inputting said first and second video images line-by-line and taking separate second order derivatives of each to find a first and a second plurality of zero-crossings (i₁, i₂, . . . ,i_n)^left and (a₁, a₂, . . . ,a_p)^right, wherein connections are plotted between corresponding zero-crossings of adjacent first video image lines, and connections are also plotted between corresponding zero-crossings of adjacent second video image lines, and wherein said first image zero-crossings are matched with said second image zero-crossings, line-by-line;

wherein, said plotting of corresponding zero-crossings of adjacent first image lines, said plotting of said corresponding zero-crossings of adjacent second image lines, and said matching of first and second images are all consistent with one another, and said matched pixel pairs are adjusted for consistency between epipolar lines, as indicated by said light lines, in both said first and second video images, and wherein said matched pixel pairs are also adjusted for consistency between successive image frames (t-3), (t-2), (t-1), (t), with said matches that are between zero-crossings being interpolated, and matched pairs being output at sub-pixel resolution.

2. A system for detecting and measuring the three-dimensional positions occupied by the surfaces of an object-of-interest, comprising:

a first video camera and a second video camera separated from one another along a baseline and aimed in tandem at an object-of-interest and that for respective first and second video images from different perspectives;

a holographic monochromatic infrared light projector disposed between the first and second video cameras for projecting a first set of parallel light line segments at said object-of-interest;

three-dimensional surface generation means connected to receive said first and second video images and a camera pair calibration matrix and for translating the respective observations of said light lines distorted by their falling on an object-of-interest into a plurality of three-dimensional coordinates that estimate a surface of said object-of-interest and that conform to a plurality of measurements taken for said camera pair calibration matrix;

frame grabbing means connected to the three-dimensional surface generation means for synchronizing respective first and second video image outputs of the first and second video cameras and for storing a sequence of video frames in memory that represent image columns parallel to said first set of parallel line segments of light and image rows orthogonal to said first set of parallel line segments of light;

image warping means connected to the first and second video cameras for electronically-warping the images from each camera to have a common plane of observation, wherein the horizontal and vertical linear scales of each image coincide with the other, and providing for an aligned image output for each of said first and second video images; and

stereo image registration means connected to the three-dimensional surface generation means for inputting said first and second video images line-by-line and taking separate second order derivatives of each to find a first and a second plurality of zero-crossings (i₁, i₂, . . . ,i_n)^left and (a₁, a₂, . . . ,a_p)^right, wherein connections are plotted between corresponding zero-crossings of adjacent first video image lines, and connections are also plotted between corresponding zero-crossings of adjacent second video image lines, and wherein said first image zero-crossings are matched with said second image zero-crossings, line-by-line, wherein said plotting of corresponding zero-crossings of adjacent first image lines, said plotting of said corresponding zero-crossings of adjacent second image lines, and said matching of first and second images are all consistent with one another, and said matched pixel pairs are adjusted for consistency between epipolar lines, as indicated by said light lines, in both said first and second video images, and wherein said matched pixel pairs are also adjusted for consistency between successive image frames (t-3), (t-2), (t-1), (t), with said matches that are between zero-crossings being interpolated, and matched pairs being output at sub-pixel resolution.

3. A three-dimensional motion camera system, comprising:

a plurality of stereo-imaging units for distribution around an object-of-interest, wherein each includes a first video camera and a second video camera separated from one another along a baseline and aimed in tandem at said object-of-interest wherein each video camera provides for respective first and second video images;

a laser light projector included in each of the stereo-imaging units and disposed between said first and second video cameras for projecting interfering collimated monochromatic beams of light to paint finely spaced parallel light lines on the surfaces of said object-of-interest;

three-dimensional surface generation means connected to receive each pair of said first and second video images and a respective camera pair calibration matrix and for translating the respective observations of said light lines distorted by their falling on an object-of-interest into a plurality of three-dimensional coordinates that estimate a surface of said object-of-interest and that conform to a plurality of measurements taken for said camera pair calibration matrix;

frame grabbing means connected to the three-dimensional surface generation means for synchronizing respective first and second video image outputs of said first and second video cameras and for storing a sequence of video frames in memory that represent image columns parallel to said first set of parallel line segments of light and image rows orthogonal to said first set of parallel line segments of light;

image warping means connected to said first and second video cameras for electronically-warping the images from each camera to have a common plane of observation, wherein the horizontal and vertical linear scales of each image coincide with the other, and providing for an aligned image output for each of said first and second video images; and

stereo image registration means connected to the three-dimensional surface generation means for inputting said first and second video images line-by-line and taking separate second order derivatives of each to find a first and a second plurality of zero-crossings (i₁, i₂, . . . ,i_n)^left and (a₁, a₂, . . . ,a_p)^right, wherein connections are plotted between corresponding zero-crossings of adjacent first video image lines, and connections are also plotted between corresponding zero-crossings of adjacent second video image lines, and wherein said first image zero-crossings are matched with said second image zero-crossings, line-by-line, wherein said plotting of corresponding zero-crossings of adjacent first image lines, said plotting of said corresponding zero-crossings of adjacent second image lines, and said matching of first and second images are all consistent with one another, and said matched pixel pairs are adjusted for consistency between epipolar lines, as indicated by said light lines, in both said first and second video images, and wherein said matched pixel pairs are also adjusted for consistency between successive image frames (t-3), (t-2), (t-1), (t), with said matches that are between zero-crossings being interpolated, and matched pairs being output at sub-pixel resolution.