Method for real time correlation of stereo images

• US 6,516,087 B1
• Filed: 01/10/2000
• Issued: 02/04/2003
• Est. Priority Date: 01/10/2000
• Status: Expired due to Term

• Current Assignee: Sensar, Inc. (Advent International Corporation)
• Original Assignee: Sensar, Inc. (Advent International Corporation)
• Inventors: Camus, Theodore A.
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US09/480,371
• Time in Patent Office: 1,121 Days
• Field of Search: 382/154, 382/278, 382/284, 345/419, 345/420, 345/421, 345/422, 345/423, 345/424, 345/426, 345/427, 345/428, 348/47, 356/12
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

In a method for correlating two stereo images, the images are subjected to a Laplacian operator and further processed to produce reduced gray scale Laplacian images in which the pixels have a value of +1, 0 or −1. Then the two images are overlapped to produce pairs of overlapping pixels. The values of the two overlapping pixels are summed in a manner so that if both pixels are +1 or both pixels are −1 the summed value is +1, if one pixel is +1 and the other pixel is −1, the resulting sum is −1 and if one or both pixel are zero, the resulting sum is zero. All of the sums or correlation values in regions about each pixel in the two overlapping images are added together to get a new correlation value for each pixel in the overlap resulting in a correlation image. Then, the two Laplacian images are shifted relative to one another and correlation values are again computed for this new overlap. This process is repeated several times resulting in correlation values for each overlap. For each pixel, the overlap which has the highest correlation value is the best match. Having determined the best match one can then determine the location of an object or point in the field of view using standard stereo processing techniques.

Claims

1. A method for stereo imaging correlation comprising:

a. taking a reference image of a subject with a first stereo camera the reference image comprised of a first set of pixels, each pixel having a unique coordinate and gray scale value;

b. taking a shift image of the subject with a second stereo camera, the shift image comprised of a second set of pixels, each pixel having a unique coordinate and gray scale value, the second camera spaced from the first camera in one dimension;

c. performing Laplacian transforms on the reference image and the shift image to form a reference gray scale Laplacian image and a shift gray scale Laplacian image;

d. forming a reduced gray scale Laplacian reference image and a reduced gray scale Laplacian shift image such that each pixel in the reduced gray scale Laplacian images are assigned a value of +1, 0 or −

1;

e. performing a correlation between the reference reduced gray scale Laplacian image and the shift reduced gray scale Laplacian image images comprising the steps of;

i) overlapping the reference reduced gray scale Laplacian image and the shift reduced gray scale Laplacian image to create an overlap image having pairs of overlapping pixels;

ii) combining the assigned values of selected pairs of overlapping pixels to form a reduced gray scale correlation image of correlation values, one correlation value for each pair of overlapping pixels;

iii) combining regions of the reduced gray scale correlation image so that a continuous correlation function exists for the image;

iv) shifting the shift reduced gray scale Laplacian image relative to the reference reduced gray scale Laplacian image; and

v) repeating steps i) through iv) to create a set of gray scale overlap correlation images;

f. using the gray scale overlap correlation images to select an overlap per pixel in the reference image; and

g. using the selected overlap to determine a position of at least one point on the subject.

2. The method of claim 1 wherein the selected pairs of overlap pixels correspond to a single region of adjacent pixels from a first correlation image and a single region of adjacent pixels from a second correlation image.

3. The method of claim 1 wherein the selected pairs of overlap pixels correspond to pixels within at least two distinct regions of a first correlation image and to pixels from at least two distinct regions of a second correlation image.

4. The method of claim 3 wherein the combining of assigned values is performed using a non-uniform weighted kernel.

5. The method of claim 1 wherein a stereo processing algorithm is used to determine at least one point on the subject.

6. The method of claim 1 wherein the pixels have one of three values {+1, 0, −

1} and the pixel values are summed in a manner such that if both pixels are +1 or both pixels are −

1, the sum is +1, if one pixel is −

1 and the other pixel is +1, the value is −

1 and if at least one pixel is zero, the sum is zero.

7. The method of claim 1 wherein the combining of assigned values is performed using a non-uniform weighted kernel.